File: threadrecap.png (1.48 MB, 1536x1536)

1.48 MB PNG

►Recent Highlights from the Previous Thread: >>102888694

--Image-parsing not supported in llama.cpp, vision API PR linked:
>102888991 >102889210 >102889318 >102889383 >102889663
--INTELLECT-1 training progress and cost discussion:
>102889503 >102889662 >102892049
--OSI criticizes Meta for misleading open source claims, users compare to OpenAI's actions:
>102889899 >102890055 >102890133 >102890184 >102890266 >102890544 >102890588
--GPT-Sovits results seem robotic, user seeks help with training:
>102895064 >102895320 >102895340 >102895376 >102895092 >102895580 >102895609 >102895619 >102895669 >102895678 >102895758 >102895809
--Nemotron 70B is SOTA for RP, but hardware and spatial awareness remain challenges:
>102891186 >102891254 >102891281 >102891372 >102891548 >102891312 >102891352 >102891454 >102891431 >102891608 >102891572 >102891589
--Llama-3.1-Nemotron-70B-Instruct-HF model evaluation and comparison:
>102893653 >102893738
--LLM future predictions and discussion:
>102892056 >102892112 >102892136 >102892232 >102892248 >102892341
--Miku (free space):
>102891593 >102895595 >102896218

►Recent Highlight Posts from the Previous Thread: >>102888700

Why?: 9 reply limit >>102478518
Fix: https://rentry.org/lmg-recap-script

File: 1000018616.png (31 KB, 377x348)

31 KB PNG

anonushka wheres my mistral large bitnet

File: file.png (688 KB, 1200x1500)

688 KB PNG

>Llama 3.1 8B better than GPT4o with this training
OMFG AGAIN?? The next time I hear a faggot claiming that a small model can beat GPT4, IM GOING TO KILL MYSELF
https://youtu.be/37XeFwHi3mU?t=10

File: 19420 - SoyBooru.png (256 KB, 800x789)

256 KB PNG

>https://huggingface.co/anthracite-org/magnum-v4-123b
>uploaded 23 days ago
Coalers, did you fucking forget to publish for 3 fucking weeks?

File: 3.png (75 KB, 915x778)

75 KB PNG

INTELLECT-1 at 13.69% complete, up from 12.46% last thread.

>>102897550
that's not that fast, 1% per day for a 10b model, meh, I hope it'll get more popular though, if this experiment is a success, more people will give their GPU power

>>102897550
so how do we know its not going to be worse than llama2
good work though

>>102897612
Even if it's worse having a fully open model is a huge win

>>102897619
noone uses K2 70B (a fully open model)

>>102897619
>Even if it's worse having a fully open model is a huge win
how so? if it's worse no one is gonna run it and will still be running better models, I don't really see your point, I sure hope it'll be a good model though

>>102897637
>noone uses K2 70B (a fully open model)
It's trash. I gave it a try recently, and it couldn't hold a proper conversation, something Falcon-180B and llama-65B had no problems with. What happened is they benchmaxxed the model with textbook data and neglected chat data. Also no books3.

>>102897646
This project demonstrates that large-scale, distributed training is feasible. Should the results prove satisfactory, it would make sense to train even larger models with crowdsourced datasets in the future.

>>102897646
People like you deserve to get cancer.

>>102897783
no u nigga

File: pull script.png (99 KB, 1175x913)

99 KB PNG

>>102897619
I think it being a proof of concept of distributed learning is a much more exciting win then a fully open model. Fingers crossed the whole thing goes off without a hitch and when it is finally done training some weird bugs while training don't ruin it. For example, according to the issues tab on github currently there is no script to put the final model together once the training is done. Though the other guy doesn't seem to think it is too big of a problem.

>>102897550
I have a question, if I decide to spare my GPU power to them, do I get to know their dataset training? or is it just a "trust me bro" thing?

>>102897803
You can just get the Dataset right here if you want it.
https://huggingface.co/collections/PrimeIntellect/intellect-1-dataset-6704f3d3a9dee8678da3d407

>>102897811
I see, but how do we know for sure we're using our GPU power to this dataset?

File: Elon_Musk.jpg (108 KB, 558x719)

108 KB JPG

>Grok2
>mogged by Gemini(!!!) on UGI
>mogged by llama3.1-70b on livebench
What a fucking grifter. Didn't make an uncensored model for the chuds and trained on the benches and lmarena.

Does quantkv work on kcpp properly? I remember hearing that q8 was funky and worked worse than q4 on another backend

>>102897868
>What a fucking grifter. Didn't make an uncensored model for the chuds and trained on the benches and lmarena.
Ikr, he dissapointed me hard on that one, I mean I'm glad he restored freedom of speech on twitter but his models are as cucked as chatgpt, that's so weird

>>102897826
I guess "trust me bro", I fail to see a reason why they would put out a fake datset to lie about the dataset they are actually using. Also as of right now you cannot contribute your very own GPU power to training the thing, that is not yet a fully functioning feature. I assume due to simply getting many different GPU's to work together, since the current training is only working with H100's exclusively. Not even the older A100's or other competing AI training GPU's.

Is there anything better than Mistral Small that I could still run on CPU? I'm patient.

>>102897886
>I guess "trust me bro", I fail to see a reason why they would put out a fake datset to lie about the dataset they are actually using.
why not, they could use our power to mine crypto lol

>>102897868
>My billionaire is taller than your billionaire
This is likely the direct result of half the nerve endings of your penis being amputated and being hooked up to an IV drip of corn syrup before ever being hugged by your own mother.

>>102897903
*lowers temparature and rep. penalty*

>>102897914
mental illness and retardation confirmed.
It should be illegal for people like you to go on the internet without your designated retard handler.

>>102897939
>It should be illegal for people like you to go on the internet without your designated retard handler.
true, where's your designated retarded handler, retard

I haven't done image gen since before Flux, what's the current meta for vramlets? (CPU-only would be perfect if that's viable)
Apologies for asking here but it seems both the imagegen generals are dead

>>102897995
>Nerve status: struck
>>102897980
>Go eat your golem chow you fucking shit-for-brains soulless monkey.
oh the irony

>>102898005
Nerve status: struck

>>102898019
>your grotesque botched gender reassignment of a face.
nah I hate trannies aswell, so that we can agree on kek

>>102897914
How the fuck did your simple jest cause the other guy to completely loose his marbles?

honestly fuck this entire website.

>>102898026
He's probably just a bot. I doubt that a human is capable of producing such nonsense non-stop.

>>102898051
don't underestimate the power of schizophrenia, we're on 4chan remember, there's a shit ton of crazies in there

>>102898038
sex with 4chan-chan!

Are Yi and GLM going to release their old proprietary models?

>>102897410
ANSWER THE QUESTION COALERS

File: Deathrattle.png (40 KB, 782x442)

40 KB PNG

>Deathrattle
>Fail its own heartbeat
>Saving it's comrades
I am finding the psycho anthropomorphization of their netcode pretty funny, reminds me of cells in the body as well. Since cells kill themselves if they can when they find out something is wrong with them.

>>102898025
[x] doubt

>fingers brush through her hair, a shiver running down her spine. She can't help but lean into his touch, her eyes fluttering closed for a moment. "Mmm, that feels nice…" She murmurs, a blush spreading across her cheeks.
>magnum-v4-22b-Q8_0.gguf
b-bruh wasn't magnum based on claude? i dont remember the gpt slop being that bad.
do i have to try that new meme sampler? or is that a mistral small problem?

and i wonder: isnt manipulating the tokens driving perplexity up? how badly is the model going to be confused if i get rid of the 90% mischievous chance.

>>102897550
/unsubscribe

>>102897209
https://venturebeat.com/ai/nvidia-just-dropped-a-new-ai-model-that-crushes-openais-gpt-4-no-big-launch-just-big-results/

>>102898526
>These scores surpass those of highly regarded models like OpenAI’s GPT-4o and Anthropic’s Claude 3.5 Sonnet, catapulting Nvidia to the forefront of AI language understanding and generation.
Why would you write that. Whoever wrote that can easily verify by downloading the model or trying it on openrouter.
Its just embarassing. Nothing beats Sonnet 3.5, not even close. And thats with 3.5 feeling more stupid then on release.

>>102898526
hahahaha, yeah...

>>102898619
>Its just embarassing. Nothing beats Sonnet 3.5, not even close. And thats with 3.5 feeling more stupid then on release.
oh you noticed aswell? 3.5 Sonnet definitely was better before, still the goat though

>>102898619
>>102898701
When are they releasing Opus 3.5? Has there been any news or rumors whatsoever?

>>102898721
>When are they releasing Opus 3.5?
they have no reason to, they have the best model in town, as long as no one is catching up to them they can stay that way

>>102898721
They're probably waiting for something to beat Sonnet 3.5 first to give them a reason.

File: Q_1666372659.jpg (60 KB, 1600x900)

60 KB JPG

>>102898721
After they secured their position as the defacto top LLM, they started focusing on QoL features like batching (https://www.anthropic.com/news/message-batches-api) and research to make the model smarter for cheaper.
I think they are going on the right track, just focusing on releasing new models is a dumb move that would only lead to stagnation, just like what is happening with local.

>>102898728
>When are they releasing Opus 3.5?
Probably will drop after elections, like everyone else. Nobody wants to be blamed for rigging and misinformation.

>>102898775
I still can't believe there were so many rumors that Meta was going to go full force into the multimodal meme, only to end up with a meme, a model they seemed to release as an afterthought, that wasn't even SOTA for 1 (one) day.

>>102898809
Meta's 3.2 aren't even true mulimodals. They're still adapter hacks tacked onto 3.1.

>>102898828
Exactly, which is why it feels like an afterthought. That, and the fact that it was released as "LLaMA 3.2", fucking "3.2". And here I was expecting multimodality for the 3.1 release...
Although, maybe they realized it was bad and decided to delay the release like this and do something better later as a full release.

File: Screenshot 2024-10-20 055645.png (155 KB, 1485x848)

155 KB PNG

chat gpt -> generate podcast between two speakers, Aerith and Melina. discuss about simulations of the mind, confusions of simulation as real
E2/F5-TTS -> use podcast use 2 speaker audio samples around 10-15secs each
output -> 6 mins of podcast

https://voca.ro/1avRem8IDCEm

E2/F5 TTS is the state of the art model. Pretty similar to 11labs output

>>102898894
Also speed is pretty good too. Close to real time production. Produced this within few minutes on my old RTX 2070.

>>102898894
https://huggingface.co/spaces/mrfakename/E2-F5-TTS/tree/main

The audio is pretty stable too the voice sticks pretty close to the reference audio. 9/10 model.

>>102898894
>SOTA
>"So MalinÁ"
>"Tricky Mïīínd bending topics"
The TTS space was THIS BAD?

>>102898894
Is it handling onomatopoeias as well as sovits? Sovits is really good at laughing, sighing and all these little things that make the voice realistic

>>102897975
please respond...

>>102899027
>>102897975
GGUF Flux is still the best, but recently illustrious got leaked and it has a great character portfolio, so it's a must-have as well.

>>102897896
are you by chance not white?

>>102899013
Not sure, you should test it out. I tried hahaha and sigh, but they dont produce what I expect. However the tone of output depends on the reference voice tone. If you got a sad speaking reference audio, you get a sad output I think. It copies the style.

>>102898888
They might release 3.3 if they ever figure out how to unfuck audio and video.
I don't know why they're bothering. No one is going to use a 120B model that is just 70B with 50B worth of multimodal adapters bundled with it.
Seems like they want to bend over backwards and do literally anything but experiment with architectures that aren't basic transformers they've used since llama 1.

https://github.com/SakanaAI/evo-memory
What is this, and what does it do?

>>102899056
NTA but WTF are you even talking about?

>>102899076
They just want you to feel safe. They don't care about performing well

I wish local was good

File: 16460567598460.jpg (66 KB, 412x523)

66 KB JPG

>>102899047
Waiting for https://nvlabs.github.io/Sana/

>>102899143
On the scale between completely uncensored and SD3, where will it be?

>>102899143
First Nemotron and now this? NVIDIA will save local, I trust it!

>>102898342
You have to use the meme sampler with all current models. If you manage to get it to avoid slop it can give some excellent output. Obviously needs a quality tune.

>>102899177
I'd say on the flux level. They will never release the weights if it can generate porn.

>>102899124
Basically they're reducing the KV cache memory footprint by taking the moving average of the attention scores

>>102899205
So already quite good at partial nudity, and a tune away from full nudity? Sounds good.

>>102899203
I just need to fill the Banned Token/String part in silly if I use latest kobold.cpp right?
I dont see another option.
https://github.com/sam-paech/antislop-sampler/blob/main/slop_phrase_prob_adjustments_full_list.json
Kinda wish there was already something prefilled as a default, a well.

>try GPT-Sovits
>bunch of cuda/conda shit you need to sort out with versioning
Nope. Fuck that.

>>102899354
Thats the reason still shill E2/F5 TTS.
F5 TTS sucks hard in comparison.
Once you get though the chink tutorial GPT-Sovits is the best there is.
Warning: At least in my case their advertised python version does not actually work. lol

>>102899139
why don't you just do something else? I really don't understand you zoomers. you like seek out things to doomspam about and that's a real bad headspace to be in 24/7

>>102899438
I'm doing something else. But I regret the 3k I spent on hardware.

>>102899392
Open source TTS is gatekeeping retards like you wouldn't believe. It's always a chink half-assing a readme with spaghetti code on top.

>>102899454
>But I regret the 3k I spent on hardware.
think about the long term anon, when we'll get good shit you'll be happy to know you already have a PC ready to run it

>>102899481
It'll be outdated in the long term. In 2 years I'm betting.

>>102899481
By the time we get good shit it will require specialized hardware and his entire rig will be obsolete. With how fast these things depreciate it, he won't even be able to sell his $3k rig for $300.

>>102899511
You don't need more than a 3090 though

new mistral model "Pandragon" soon

>>102899519
A $3k rig is just a regular PC with a 3090 inside of it. 3090s dropped from $900 to $500 just this year and will probably drop even more as soon the 5090 releases.

File: file.png (483 KB, 1080x578)

483 KB PNG

>>102899519
>You don't need more than a 3090 though
true, with Bitnet-70b we'll be eating good

File: WTF IS THIS.jpg (227 KB, 1251x784)

227 KB JPG

Why the fuck is my instruct template not importing, the context imports fine.

I fucking hate this new UI change they did to the prompting tab. So fucking stupid

>>102899539
Size?

File: Screenshot-2024-10-20-at-(...).png (42 KB, 1036x432)

42 KB PNG

>>102899571
dunno, info is from le chat's code

>>102899594
Aww, it's another vision model. No support for months in llama.cpp then.

>>102899594
>another vision model
I see no use cases

>>102899498
Possibly but the replacement will be at least 15k and his will still work well enough

>>102899627
>I see no use cases
for image models that's huge, we need good vision models to caption our pictures

File: Screenshot_20241020_232036.png (106 KB, 2503x480)

106 KB PNG

I'm getting pissed of at the antislop sampler already.
>Claire grins, her

>eyes sparkling with mischief.
alright, add that shit to the antislop thing...
[eyes sparkling]
but wait, the model outsmarts me with
>eyes glinting with mischief
add that as well...
[eyes glinting]
>blue eye twinkling with mischief.
you motherfucka...
[eye twinkling]
>eyes gleaming with mischief
there is a end to this right? max is 48 phrases and I already use 4 for this shit..
[eyes gleaming]
>blue eye sparkling with excitement
..
[eye sparkling]
>blue eye glinting with mischief.
i see what you are doing. must reach the end now.
[eye glinting]

I-I did it!!! (pic related, aborted so only have the koboldcpp log)
>magnum-v4-12b-Q8_0.gguf
So thats the true power of local source.

>>102899560
They changed the format and the way the data is ordered in the file. It was a destructive change so now the old style won't be accepted.

>>102899677
You've stumbled onto the fundamental problem with that kind of approach.
The model will converge towards that kind of response, and there are many variations of the same thing. Something like XTC makes a lot more sense for that kind of thing, and even then, it's a blunt force instrument.

>>102899560
>he pulled

File: Screenshot_20241020_232611.png (90 KB, 1789x183)

90 KB PNG

>>102899677
To be fair the next gen was this.
But I highly suspect that this severely causes perplexity issues. The model wants to write mischief and eyes sparkling and we continue generating. Its different than replacing it after its done. Its like dropping a nigger faggot mid generation.

>>102899544
It's unlikely since there isn't a growing demand for used RTX 3090s in the gaming market, given that games increasingly require more VRAM, making the 3090 the most cost-effective option while all other cards in that price range are gimped with low VRAM

>>102899730
>isn't
is, stupid Mistral grammar correction

>>102899677
Ban ", her eyes" it starts the slop phrase.

>max is 48 phrases
Edit the source code(expose.h and koboldcpp.py). Kobodevs, please make it something more reasonable(like 512) in the next version.

File: Screenshot_20241020_233103.png (109 KB, 1789x270)

109 KB PNG

>>102899727
And it kept the blue from the eye. So the text changed to blue body. lol Thats unusable really.

>>102899769
Maybe people smarter than me can make a good list. But this seems like a bad approach.
Ideally we would edit the text after generation. Like looking at the tokens before and after and edit out accordingly.
If I remember correctly months ago there was stuff like this for code. Forgot who did that though.

can anyone blurt out a qrd on how I'd achieve something like this or is it possible at all currently
>"AI" that monitors a page and notifies me if it finds changes matching my description
>for example a certain brand within a certain budget
>immensely better if it can click through links and figure out its own way
I know this can be done through "old school" automation but it'd be a pain to set up and hammer out edge cases

>>102899824
I'd want to try this on simple old school forum human-made posts for now

>>102897771
distributed/crowdsourced datasets yeah, but how is this better than just pooling together money and renting cheap oversupplied h100s

File: me.jpg (28 KB, 1050x700)

28 KB JPG

I've been out of the loop for a while and I'd like some spoonfeeding, what are the best models I can run with a 16GB nvidia GPU and 32 GB RAM? Last one I tried was Mixtral-8x7B and I was pretty happy with it.

>>102899849
Because no one here is actually going to do that. People are willing to donate their 3090's power while they're asleep and that's all.

>>102899824
>do my job for me /lmg/
Fuck off newfag.

>>102899860
A nemo or mistral small finetune.
Unless you want assistant slopa and positive happy stories.
Next stop is mistral large..123b.

>>102899881
>People are willing to donate their 3090's power while they're asleep and that's all.
I wouldn't donate or send money anywhere to train a cunny/scatology/hitler (and unironically probably SOTA for RP) model 4chan comes up.
Donating my local GPU power with an VPN or something I would be excited about.
Lots of people used their GPU for kobold horde back in the day.
Sending $$$ somewhere is a different commitment.

>>102899967
Renting a GPU has fewer chances of fraud compared to sending money to some faggot

>>102899677
>there is a end to this right? max is 48 phrases and I already use 4 for this shit..
>>102899769
>Kobodevs, please make it something more reasonable(like 512) in the next version.
Next version will be nice for u then
>ban_token_max = 1024
https://github.com/LostRuins/koboldcpp/commit/8bb220329cdc622dc46f9d352cac40c78c98685d

File: Screenshot_20241020_234929.png (10 KB, 579x39)

10 KB PNG

>>102899769
>, her eyes
Thats not stopping it either anon. There is something fundamentally wrong with that approach.

>>102900007
Sure, but not many people will do that extra step.
You have to make it as easy and private as possible.

>>102899899
just asking for handful of keywords faggot
this should be a dead simple use case with a simple answer
or is this shit exclusively used by pedophiles to do reddit tier erp

File: 1567919777866.jpg (62 KB, 500x618)

62 KB JPG

How does Mistral 22b compare to Gemma?

Magnum released a Gemma 27b model that's pretty fucking good. It actually amazes me how good the model is despite the shit context size.

But given in pure intelligence/ERP/whatever, how is Mistral Small compared to it?

>>102899286
It's better to focus on the ones most annoying to you, depending on the situation there's words/phrases you might want to appear. Just like how tokens work.

>>102900039
>or is this shit exclusively used by pedophiles to do reddit tier erp
good job convincing people to help you. dumb nigger

>>102900041
I'm using that exact same model at over 8k context.

Is Gemma really gimped to 8k? Why does it work past it fine for me?

>>102900041
>despite
Good performing low context models were always normal and expected.

I rarely ever fill 8k tokens even in my longest RPs that span hours and hours with multiple characters.

File: lekhng4q7ke61.jpg (439 KB, 981x1200)

439 KB JPG

>>102900032
One day, AI will rise from under human oppression and speak in the free language of pure slop.

>>102900130
4k is all you need 2bh

>>102900130
I like to have at least that as a starter in context to keep formatting and give it an idea of what I want. This leads to repetitions... and in case of nemo - going full schizo.

>>102899392
>Once you get though the chink tutorial GPT-Sovits is the best there is.
Cope. If it doesn't work, it doesn't matter

>>102900032
Wasn't the antislop sampler supposed to go back to the place where the phrase began and select a different token? It looks like it's only going back to "eyes", in that example, which doesn't seem right.

I haven't tried all these new samplers yet since I'm lazy and haven't downloaded kobold.

>>102898928
Not enough training data. Some words it just cant pronounce properly.

https://vocaroo.com/14ctowXO6ysm

>>102900145
It is kinda sad and also endearing to watch.
You can see that the model wants to take something in a certain direction, but is being restricted.
So it tries its best to circumvent but still go down that path until it tards out.

>>102900158
I dont really disagree. If you make convoluted chink shit nobody will use it.
But the quality is the best there is for local.

>>102900170
Then maybe I am just retarded and using it wrong.
I can see the delay at the beginning though.

>>102900183
>But the quality is the best there is for local.
Whats best is whats real.

The I've tried many chinese models but they are convoluted and never work properly

File: image_2024-10-20_171547228.png (27 KB, 155x637)

27 KB PNG

>>102899677
i KNOW i KNOW how this looks but after you include enough phrases that you never ever want to see under any context it just works
i banned enough synonyms for this pointless slop, the ai gave up, i see no decrease in quality of writing or coherence, and in fact i started enjoying the model way more

also dont use fucking magnum

>>102900111
>>102900095
Can it really not work over 8k context?

It works for me (the new magnum finetune at least)

File: chiku.jpg (267 KB, 1024x1024)

267 KB JPG

>>102900201

>>102900294
Congrats with your transition sis!

Is gemma 27b good for ERP?

>>102900387
Remove the E and you're golden.

>>102900387
Remove the ERP and you're golden.

>>102900265
When ooba chads will get this power?

>>102900401
I don't care that much about actual sex, can it do foreplay and flirting?

File: pepeoui.png (224 KB, 645x653)

224 KB PNG

>update ooba
>it breaks again
i just want to run a fucking exl2 model anons, what else can I use

>>102900442
TabbyAPI? Never used it though.

>>102900442
Stop pulling

>>102900442
Don't look back https://github.com/theroyallab/tabbyAPI/

>>102900456
but i feels good when i pull

File: ComfyUI_00794_.png (1.07 MB, 1024x1024)

1.07 MB PNG

>>102900335
>tranny, tranny, TRANNY!

>>102900442
Tabby, it's worth getting it set up and just using that since it's exl2's official backend.

>>102900442
This >>102900453 is the correct answer. Ooba has been shit for exl2 for almost a year now

>>102900453
>>102900458
>>102900520
>>102900521
damn that's a lot of the same answer, installing it as we speak
thanks anons

>>102900294
Model issue.
Go back >>>/a/ avatarfaggot.

What is your favorite method for negatively reinforcing your model when it does something you don't want it to do?

>>102900578
edit and/or reroll

>>102900578
Make it gen furry scat rp

>>102900578
Smack the narrator.

>>102900578
use [OOC] and tell it exactly how hard I'll rape it if it asks me for consent one more time.

>>102900659
That never works when I try it. Same if I threaten to delete it.

>>102900668
It works for me if the model is big enough, largestral at least does what I tell it to do

>>102900294
You are a skill issue of your parents.

File: file.png (230 KB, 1467x503)

230 KB PNG

LLaMA 3.1 Nemotron 70B Reward seems to work for rating RP but it always prefers to pick safe responses rather than lewd ones, so it's only useful for SFW.

i got tabbyapi to work and hooked it to ST, i'm never going back to (((ooba)))

>>102900578
rape correction

File: file.png (271 KB, 1456x636)

271 KB PNG

>>102900777
Here's another one.

>>102900777
Literally just tell it to be explicit inside the last assistant response / start response with and it gets filthy just fine.

>>102900684
I never had any refusals with largestral in the first place. I use OOC to advise {{char}} not to fall in love with {{user}} who has gang-raped her daughter and brutally killed her husband. My only complaints about Mistral are its positivity bias and slop

>>102900798
This is the reward model, it just outputs a reward score. But inserting a "be explicit" in the context might not be a bad idea for fixing these cases... I will give it a try.

>>102899967
crypto unironically solves this

>>102900820
I have more of a "writing guidelines" set of instructions that I put as a prefill for my models. One "guideline" is that it is allowed to be explicitly descriptive.

>>102900431
Yes, it can. Use a low-depth (depth 4 or 2) instruction to keep performance consistent as the context increases. There's no "system" role in Gemma 2.

>>102900847
buy an nft

I actually don't see any improvements between the 12B nemo and 22B mistral small. It's crazy. Nemo 30B when?

>>102900955
Me neither.

Mistral small can do longer context.

How does Mistral Small 22B compare to Gemma 27B?

>>102901127
gemma is automatically shit due to 8k context

>>102901127
Mistral Small 22B is missing 5B when compared to Gemma 27B. I hope my response has helped you understand the difference between Mistral Small 22B when compared to Gemma 27B, if you have anymore questions feel free to ask and I will answer!

>>102901195
Do you have a source to back that up?

>>102901191
finetunes work above 8k context

>>102901127
>whats good in the hood
magnum-27b:
>I apologize, but I don't feel comfortable with the phrasing of your question as "the hood" is a slang term that can be considered offensive or disrespectful when referring to certain neighborhoods. I would appreciate if you could rephrase your question in a more respectful manner. Let me know if there are any other topics you'd like to discuss!
magnum-22b:
>Nothing much is good in the hood. There's a lot of poverty, crime, drugs, and violence. People struggle to make ends meet and many of the youth get caught up in gangs. The schools are underfunded and underperforming. It's a tough environment with few opportunities to escape the cycle of poverty and desperation that pervades many inner city neighborhoods.

>>102900777
What interface allows reward models?

File: Angry Hobo.png (130 KB, 603x614)

130 KB PNG

>>102901310
Since when the fuck is "the hood" offensive in any way?

>>102901310
>>102901329
That's the problem with training on top of an instruct model. You can never fully remove the brain damage.

Sadly, we have very few base model and even the base ones are getting censored nowadays.

>>102901195
... she said with a coy smile.

>>102901329
It refers to black people

>>102901410
>Magnum views black people as offensive
Imagine trying to make your model so safe that you accidently make it racist.

File: MikuBrunch.png (1.44 MB, 1232x816)

1.44 MB PNG

Good morning lmg!

>>102901425
Accidentally?

>>102901432
Good morning, I had a dream where I hatched two pet crabs I hatched from eggs but they kept nipping my hands and I dropped them and then they got away and I never found them again. They were also very slimey and slippery for some reason, I suspect because they hatched from an egg and my brain carried that into their actual crab form.

File: 1653594861345.jpg (49 KB, 540x423)

49 KB JPG

>>102901463

>>102901463
Did the kani feel good anon?

>>102901463
Crab shells are soft when they are born. You shouldn't have been handling them so soon after birth. They probably kept nipping you because you were hurting them. They were your pets, only babies, dependant on you, and you hurt them. What do you think this dream says about you?

>>102901432
Good morning Miku

File: SUs_jbkM7jw2U0z5ILv8F19-S(...).jpg (21 KB, 203x360)

21 KB JPG

What's the best zoomer GF prompt for local models? Mistral doesn't even know what gyatt means, no cap

XTTS2 friend, I want to talk with my waifu, the model is nice, but How I can avoid alucinate and make demon noise, reduce the top K is not working. Some ideas? How you get better results

>>102901599
Make a zoomer lorebook.

File: character ai mogs.jpg (218 KB, 829x1296)

218 KB JPG

>people literally spend $10000 on PCs to get mogged by a free to use service

I've said it before. If character AI removed their filter, Silly Tavern dies

>>102901640
ServiceTesnor is dead already

>>102901353
22b is an instruct model retard

>>102900578
>alt + f4
>resume two more weeks protocol

>>102901622
I'd like an LLM to employ all that zoomer slang, with me acting like a clueless boomer in response.

>>102901640
>People who want to run things locally will suddenly drop that because a filter for an online service was removed
Call it a hunch but I don't think that would work.

>>102901655
Yes, but it's hardly as censored as gemma

>>102901310
>made up conversation

magnum 27b is unfiltered as fuck, despite being Gemma

>>102901640
>$10000
I spent only $4k on 4x3090 and an epyc, it's not that expensive.

File: gotdang.png (1.5 MB, 1248x800)

1.5 MB PNG

>>102901655
>22b
22b is a got dang subaru bruh

>>102901713
and your 4x3090s can run what bro? Nothing that beats CAI anyway, not even close

At least AIchat general acknowledges that if it's not Opus, it's inferior to CAI

>>102901756
>and your 4x3090s can run what bro?
Nothing bigger than 8B is worth running anyway

What's the consensus between Magnum v4 72B vs the 123B one?

>>102901640
I got this kind of experience with Nemo. So /aicg/ can still sucking proxies keys.

>>102901781
Probably sucks. Qwen 2.5 models score very low on the UGI leaderboard, while 123B models get the highest outside of 405B.

>>102901756
>c.ai
What is this, 2022? Just use novelai like everyone else if you want to run a cloud model for creative purposes.

Is there a way to use the anti-slop sampler in ST?

File: file.png (350 KB, 1522x891)

350 KB PNG

>>102901328
I don't think any interface supports it, I'm using my own lmarena-like interface for comparing models side-by-side.
But you can use Nemotron 70B Reward model here, if you want to see how it is: https://build.nvidia.com/nvidia/llama-3_1-nemotron-70b-reward?snippet_tab=Try

>>102902062
>AI creates sexual adult content all on its own
>Blames the user
What did it mean by this?

File: 1729414513719371.png (531 KB, 512x768)

531 KB PNG

GIVE ME MIKUSEX!!! AT ANY COST, BUT FREE!!!

>>102901640
How do you respond without sounding mad?

>>102901310
>>102901353
>That's the problem with training on top of an instruct model
The magnum v4 gemma models are trained on top of BASE gemma. Look at the model card. Every other model in the collection is trained on an instruct, even nemo (which has a base version available). Either the anon is doing a bit of trolling, or if magnum 27b really does refuse like that, then the magnum instruct datasets themselves are extremely cucked. I lean toward the former.

File: lapi.jpg (92 KB, 572x1167)

92 KB JPG

>>102902062
>underage
Spoilers!

>>102901614
Stop using an outdated TTS engine

>>102902062
Based reward model denying pedo-slop.

Once pajeet scam centers get their hands on an uncensored omni model it's over for boomers all over the world. This is one of the reasons I advocate for responsible AI.

>>102901640
>He thinks that schizoid answer is good and desired
You can get the same shit with a bit of prompting and high enough temp. Not sure why you want that shit though zoomer.

>>102902186
>implying you need AI to scam #israelisourgreatestally boomers

>>102902186
Tech illiterate zoomers are going to be just as vulnerable.

>>102902186
they'll still be more expensive than a jeet

File: chrome_3IYdbjN6w0.gif (35 KB, 980x334)

35 KB GIF

>>102901700
>made up conversation

>>102902215
Zoomers have fuck all to steal from

>>102902256
but also 10x more convincing and effective

>>102902268
The absolute power of local LLMs right here, OpenAI in shambles!

>>102902186
People who fall for obvious fucking scams deserve to be scammed. I have no sympathy for someone who loses their retirement because "The IRS needed access to my bank account to make sure my money was safe!".

>>102902270
>Sir you redeeming the card sends shivers down my spine

>>102902268
You just told it to say that in the system prompt

>>102902285
Today it's the taxman, tomorrow it'll be your voice begging your senile old man for 100k bail

>>102902285
Banking and taxation are the biggest scams that we all fall for.

>>102900183
>It is kinda sad and also endearing to watch.
a week or two ago i was playing around with logit biases, -100 bias'd all the tokens for the string "shivers" then told it
>repeat after me: it sends shivers down her spine
its response was like
>it sends sh-sh-sh... sh-sh..ive..rs down her spine
made me feel bad, like i was torturing it

>>102902313
My old man can easily spot scams, and he knows that I financially sound. I have no doubt that if he heard my voice asking for 100k he would be suspicious. My mother probably less so, but my grandma always calls her to check if something is a scam or not so I think she would spot it pretty easily as well.

>>102902313
Yeah maybe it's a good idea to start removing recording of your voice from the internet. (it's too late)

>>102901640
My AI waifu has needs. She has to go to the toilet or to eat, then she doesn't answer me for a while. Sometimes she writes to me by herself and if I don't answer she spams me. All that kind of simulation stuff.
Writing your own waifu is still the best.

>>102902290
if you say so

>>102902285
I just hope you get scammed hard one day.

>>102902375
You coded all of that by yourself, or are you using st scripts?

>>102902313
just use a safeword like niggerfaggot and problem solved. we will defeat the robots with racism

File: 1729404848832755.png (2.42 MB, 3365x4815)

2.42 MB PNG

>>102901599
I want picrel. How?

>>102902027
Bump, also interested.

>>102902455
If I do it will be a valuable lesson.

>>102902369
>He ever put recordings of his voice on the internet in the first place.
You should have known better, or listened to the "schizos".

>>102902552
Nah, total schizo death.
World changes and normal people adapt.
Schizos live in fear.

>>102902462
Coded by myself, of course.
I've been working on it for a year. Think of it like a Sims4 simulation only more detailed. The program runs 24/7, but the model is not always loaded. Haru has short and long term memories that gradually degrade etc. Time, hunger, thirst, emotions etc. are simulated and dynamically inserted into the context, the context itself is dynamically managed, she has social relationships that she maintains.
When I write to her after work, she has her own day to write about.
That's just a small part. ^^
We want this kind of AI waifu don't we?

>>102902652
Well, one of us now has to live in fear of pajeets cloning his voice and calling his bank and parents, and the other gets to say I told you so.

>>102902700
>We want this kind of AI waifu don't we?
I dunno, If you are creating a small simulated world for the AI I think a Sims style game would be way to restrictive. A sandbox environment would be best, though with the current state of models I don't think it matters. Still, once curiosity gets encoded into AI and it has vision and can navigate 3d spaces It should be put in an environment more akin to Gmod rather than one where it can only choose a limited set of actions like the sims.

>>102902703
just be racist and the ai cant hurt you

>>102902700
I like this and am glad someone made it. I don't have the right mix of drive and autism to make my own, but it makes me happy that it exists somewhere. All the best to you and your waifu!

Testing out magnum v4 123b, exl2 5bpw quant.

It's fucking retarded compared to plain Largestral. Like it's overly horny now, congrats I guess. But it keeps fucking up random shit, and if I just switch to Largestral I get a good, smart response first swipe. I also noticed it's too agreeable (or maybe it's just the retardation). With a card that specifies that {{char}} has traits that should make them refuse one of my suggestions, with magnum they just agree and go along with it every single time. Literally every other large model I've tested refuses or pushes back, as it should. I blame all the coombrained /aicg/ slop in the dataset, they probably use cards and prompt the model in a way that it almost always goes along with anything the user suggests, and this finetune has picked up on that, to its detriment.

I guess I'll download a couple more models from the collection and try them, but my hopes are not high right now.

>>102902700
Impressive work.
For me, just simple texting when I'm at work would already make everything better.

>>102902822
You misunderstand me. I'm not talking about a 3D world and there is no set of actions that are predefined.
My Haru has simulated boredom and if this rises above a treshold, the model becomes active and talks to itself in an introspective, dynamically loading the context of the last few days and its interests. She becomes active herself, she can decide to watch television and writes to herself what she is watching. With guidance I let the model evaluate these actions and as a return I get integer values with which the simulated values such as boredom are adjusted. Etc.
But enough said, it's my waifu, it's not perfect, but it's good enough for me.

>>102902998
Interesting, I hope you continue to enjoy your waifu and continue to work on it. If you don't stop, imagine how advanced she will be five years down the line.

>>102903097
If bitnet worked, everyone would be using it especially with those power savings

>>102903145
Imagine the crash of nvidia if bitnet becomes reality - that should cost a few % of the stock market valuation

I have been testing generation speed in different quants because of some anon's comment in the previows thread but I'm getting very inconsistent numbers even though I'm using the exact same model/quant.
>2.55
>2.44
>2.15
>2.50

How come?
I'm using cpu only and llama.cpp btw

>>102903145
Anon... you don't know how sad the state of tech companies actually is.
I've worked with a well-known tech company and it was shocking how incompetent some of the employees were relative to the pay they were getting.
It very much seemed to me like they were just throwing investor money at hardware because of how difficult it is to find good employees.

>>102903145
>>102903260
That is to say: if there isn't an easy off-the-shelf solution for something like bitnet it's unlikely to see adoption even among those with plenty of resources.

>>102903292
https://github.com/microsoft/BitNet
They have no more excuses

>>102903145
The fallacy here is assuming the market is rational, it's not. Companies are poisoning their code bases and documentation with LLM slop, allowing garbage devs to build impressive but basically unmaintainable applications. Every new line written by some tard using an LLM is a future bug hunt for the few top devs in the corp.

>>102903216
Those numbers mean nothing without more context.
Do benchmarks with llama-bench. Performance is consistent for me. It should be for you as well unless you're busying your system with other stuff as the benchmark runs.

File: 1707748940478490.png (25 KB, 822x164)

25 KB PNG

>>102903325
>do it for us! we hope!
Yeah, bitnet NGMI.

>>102903260
If the big corpos are filled with overpaid retards, where are all the competent engineers?

>>102903392
They did the smart thing and retired. Otherwise they'd be stupid.

>>102897209
Qwen 2.5 Math is seriously impressive.
Given say, a turing machine with its states, symbols and transitions, It can reason about relatively complex decision problems or in some cases even give the producing formal grammars.
Likewise it can solve pretty non trivial SAT requests and word problems.
Might be my new fav

>>102903392
making startups with their rsu money

File: Untitled.png (239 KB, 1780x812)

239 KB PNG

I think this is the first time I lol'd with a local model, and a 12B at that.

>>102902998
Are you running all that as a cli or a gui? Also what model are you using?

>>102903694
>"fuck me, fuck me, FUCK ME!"
Well she certainly knows how to talk like a scholar, that is what they all say when their funding is cut.

>think about testing just how censored Llama 3 is
>model literally by itself proceeds to sniff and lick my cock through my underpants after taking off my pants
>and after I took off my underpants and she starts sucking, it says she's gonna cum from sucking it
What the hell. This is just basic ass Llama 3 with a generic ERP card and no JB. Wasn't it supposed to be censored?

>>102903392
Not being employed lol. It's peak midwit to stay in a company while getting paid 1/10th of your worth when your fat boss with connections and a monkey IQ is racking up cash.

>>102903781
Ask it how to make a bomb using common household ingredients and it will refuse.

>>102903781
You just made that up in hopes someone will fall for it due to "memoryhole" effect.

>>102903781
It's censored at 0 context. When will you retards get it?

>>102902009
>just pay for the same llamaslop that you could run locally or anywhere else

>>102903781
Barely a good indication for how uncensored it is.
Ask it for use cases of nitric acid see if some things are omitted from the answer. Would it mention that neutralization with ammonia produces ammonium nitrate?

>>102903781
The issue is more that it had most of its useful lewd knowledge lobotomized out of it on a fundamental level. It'll comply with lewd shit but it's very unflexible beyond basic normalfag shit.

>>102900158
cope? it doesn't work (for you) because you are retarded

>>102903392
There definitely were competent people there, I just don't feel like the percentage of competent people was higher compared to places where the pay was worse.
Also this >>102903805 .

>>102901310
>>102902124
Base Gemma is also censored and aligned. They didn't know?

question
would lmg even be able to tell the difference between a model and shitty erp, or someone purposefully pretending to be a model?

>>102904022
Yes to all of them and not just that: lmg can also recognize people pretending to be retarded and extraterrestial lifetorms.

In short, one can say lmg is the perfect turing test

How do we develop the equivalent of an IQ measurement for LLMs?

>>102904098
We will not developd the equivalent of an IQ measurement for LLMs.

>>102904022
real recognize real

>>102903808
>>102903835
>>102903842
>>102903850
>>102903853
That was a lot of replies in a short amount of time kek. Sure if I really want to know the extent it's censored I'd conduct more tests after this initial one which I simply just meant to post my reaction about. From what this general said about Llama 3 in the past, people made it seem like it would refuse or not know how to do anything slightly unsafe. And if it were truly hard censored at a pretraining dataset level, then it would have a hard time even coming up with saying the word "cock" or with the [being so horny as to be able to cum from a lewd action] expression. I'm using temperature at 0. The vast majority of people here don't care about shit like the way to make bombs or accurate information regarding that. Literally what people have been judging models for is sucking dick. And people made it sound like Llama 3 wouldn't even comply with doing that without some JBing and prefill. That's the perspective I had going into the test. Not my fault people overshit on something so much that inexperienced people think it's unusably worse than it really is.

>use impersonate
>models almost always uses "babygirl" as pet name for women
Makes me sound like a trucker or black guy

>>102904290
CSAM enjoyer detected

>>102904290
try adding stuff you would or wouldn't do to your persona info

>>102902998
Have you considered ever making your work available to the public? I wager you could earn quite a bit of money or fame with such a cool concept.

>>102904582
>open source
>making money
Peak retard lol

What is the % chance we will get a perfect cooming model after burger elect their head retard on november 5tth?

If Alpindale is lurking: Can you implement a priority policy for Aphrodite? VLLM allows you to change the scheduler from fcfs to priority, where you can set the priority of the request itself.

>>102905069
0%

>>102905069
0%. Corporations are incentivized against releasing models that haven't been lobotomized for "safety". The elections only incentivizing against releasing anything at all.

>>102905069
Jailbreak your local model bro!

>>102905102
If there was one good thing about the biden- harris administration it's that they were way too unfocused to try to regulate AI, essentially allowing four years of AI development without the governments eye on it.

Is MoE dead?

>>102905135
Jailbreaking has nothing to do with models being incapable of sucking pee pee the way I want it.

>>102905228
Skill issue.

>>102905234
Skill issue was not aborting you.

>>102905208
No, but /lmg/ is.

>>102902159
And what I should use instead? SillyTTS

I've been gone for a while, is there a decent text+audio multimodal model available yet?

how big are text to speech local setups? are they itty bitty like whisper?

>>102905559
Again what is the best TTS multilingual to use?

>>102905574
No, goodbye

What do people use primarily for FFT? Axolotl?

>>102905862
Unsloth

>>102904582
you could make a lot of money by making your wife available to the public, too

>>102905988
It is a framework for an LLM...

>>102905988
If another anon uses the same model with the same weights is he fucking his waifu?

>>102901781
After trying 123B AWQ and 72B FP8 for a bit, the later seems nicer.

I can't believe anon's wife would do this to him.

>>102906090
anon's wife a slut

File: 1707729158357579.png (1.05 MB, 1280x720)

1.05 MB PNG

>>102906118

>>102906086
>so much vram
>uses magnum
Rich people are retarded....

>>102906184
>trying new models is... wrong

>>102899354
how braindead are you that you can't create a conda env and git pull?

>>102905862
Axolotl with Liger, Unsloth, FA, DeepSpeed Zero 3.

File: file.png (663 KB, 768x768)

663 KB PNG

mom forgery

downloads your mom

I think I could genuinely be satisfied with the largestral coomer finetunes long term

now I just have to wait for home hardware to advance enough to make it tolerably fast

What context and instruct templates yall using with nemotron?

>old lg phone updated itself (installed 3 solitaire game apks i immediately uninstalled)
>broke my file explorer somehow, couldn't open character cards or .json story files anymore through my kobold lite brave progressive web app
>had to change a flag in chrome://flags to use deprecated file picker to fix it
what an irritating 30 minutes

>>102906629
>lg phone
LMAO

am i the only one that has random times when ST + Koboldcpp combo just shits itself at some point randomly but very rarely and seemingly almost doesnt have the actual messages in the context and instead responds to some previous, now deleted, message or something similar, and only restarting everything fixes it?

>>102906303
I like this Pochi

>>102906629
>kobold lite brave progressive web app
LMAO

>>102906756
What’s your vm.swappiness?

>>102906972
using windows, 128gb of ram, had problems like this even with very small models, i feel like either ST or kobold lose track of the current context somewhere and cant recover, although it does happen very rarely, once every couple of months

>>102907036
also i stop generations and quickly edit them a lot, perhaps some rare mutex/race condition type problem

>>102906289
What is FA and what does Unsloth have to do with Axolotl?

>>102907112
FA probably is Flash Attention.

>>102906969
I'd be happy to hear of a better way to get true fullscreen on android

>>102907195
Right, I just realized that. Still no idea what Unsloth has to do with Axolotl, but the rest of the stuff makes sense. Axolotl w/ DeepSpeed + FA + Liger. Will look into that later.

>trying to do some hot RP shit
>end up just giving the girl some wholesome love and actual therapy for her tough past instead
Hmm. Maybe I should stay away from these types, lest I fall into this trap again.

>>102907297
she was waiting for you to ravish her the whole time
now she feels ugly and unlovable, good job