I'm thinking miku miku ooo eee oo

Mikulove

File: 11__00076_.png (2.23 MB, 1024x1024)

2.23 MB PNG

>>101524169
Almost got me, this anon had an eagle eye to check the /lmg/ card. Or maybe he's seen this shit before too many times.
Thread-splitters always show up during big release weeks. Like goddamn clockwork.

File: BubbleBASS.png (254 KB, 395x453)

254 KB PNG

>>101524155
>You forgot the human eval
It's over

What are your favorite prompts for creative writing? I'm using the new Nemo support in llama.cpp to make some smut, and it's actually pretty great with very basic stuff like "You are an expert writer, the following is a premise for your new story", write some basic info, a few sentences, and it manages a couple coherent paragraphs.

I have a pretty huge context, is there any way to keep it from going off the rails? I keep heat decently low, below 1.2, but it always seems to veer into entirely different topics, even if it keeps the names of the characters the same. This behavior repeats even with heat below 0.8, but once it gets too low it ends up looping 1-2 sentences like Mr Jack Shining.

>>101524270
>I keep heat decently low, below 1.2
>Unlike previous Mistral models, Mistral Nemo requires smaller temperatures. We recommend to use a temperature of 0.3.
https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407

>>101524297
lol ok that makes sense. thanks, anon

File: threadrecap.png (1.48 MB, 1536x1536)

1.48 MB PNG

►Recent Highlights from the Previous Thread: >>101521755

--Exl2 and gguf context comparison summarizing paper: >>101521943
--Leaderboards for comparing LLMs: >>101521863 >>101521970 >>101522113
--L3.1 scores interpretation and analysis.: >>101521839 >>101521858 >>101522313 >>101522179
--Mistral Nemo system prompt advice: >>101522594 >>101522662
--Anon seeks feedback on gemma2-27b-it with llama.cpp: >>101523270
--Whisper Speaker Diarization Hugging Face Space by Xenova: >>101523365
--GPU options for tax refund money: >>101522820 >>101522853 >>101522925 >>101523007 >>101523040 >>101523097 >>101523352 >>101523442
--Distillation techniques and Google's success in open weights category: >>101522487
--Azure Llama 3.1 benchmarks: >>101521860
--Anon seeks help diagnosing VRAM leeching issue on dual GPU setup: >>101522288 >>101522855
--405B model performance and perception: >>101522770 >>101522809 >>101522852
--LiveBench results, analysis, and discussion: >>101523528 >>101523544 >>101523604 >>101523609 >>101523626
--L3-405B and RoPE scaling tech for RP purposes: >>101523523 >>101523546 >>101523652 >>101523680 >>101523690 >>101523980
--Miku (free space): >>101522503 >>101523790

►Recent Highlight Posts from the Previous Thread: >>101521762

File: L3.1-benches2.png (42 KB, 1314x141)

42 KB PNG

>>101521839
Version with L3-instruct for comparison. 8b went up more than 70b.

>>101524362
Based. Giving thread splitters what they deserve: recognition.

>>101524362
You lost, Miku.

So when is the new L3.1 70b being released officially?

>>101524795
next month

>>101524795
2mw

>>101524795
Tomorrow, petra.

>>101524795
2m(orro)w

>>101524467
Are these differences between 70B 3.0 and 3.1 even statistically significant? Charts like OP are showing what I'm guessing are confidence intervals about 5 points wide. This spreadsheet table barely breaks one or two points except on one category and even then, not quite 5.

File: 1716123400718.jpg (46 KB, 750x724)

46 KB JPG

>>101524170
same
>>101524795
Dunno but Jensen+Zuck speaking at SIGGRAPH 29th

Death to Miku!

>>101524871
Transformers have plateaued. Billions must AI.

>>101524467
What went wrong with openbookqa and social_iqa? Why is 70b better at those than 405b?

>>101525251
>Transformers have plateaued.
Probably. Bitnet might replace shitnet but the pathetic return on investment on params is showing that we've squeezed most of what can be squozed out of "look at text and guess the next word."

>Billions must AI.
Billions already do. Just not LLMs.

it is bizarre the blacked spammer is still invested in spamming his fetish still. months and months of nigger shit for that dude lol

/lmg/ bros..

>>101525545
>>101525545
>>101525545

wait people are actually disappointed in the numbers from llama 3.1 base models? they seem quite good to me, were people just expecting 405b to blow everything else out of the water?

>>101525626
>were people just expecting 405b to blow everything else out of the water?
yes

>>101525626
Yeah, 405B not being a huge step forward shows that Meta's models are a dead end.

>>101525626
Meta stated their goals for Llama 3 were to beat GPT-4. Everyone just assumed that once better models came out that the goalposts would shift to beat them too. It's the same like retards assuming 405B would be BitNet just because that's what they wanted.
People set unrealistic expectations.

More leaks today?
This looks like GPT generation speeds in their own testing. But I don't know Chinese for details.

>>101525336

>>101525626
No, it's just the same dooming of every significant model release.

There's 0 reason to panic. 405B is just the Quest Pro all over again. Huge, expensive and disappointing performance.
That didn't stop them from releasing the Quest 3 just a year later which is a straight upgrade to the Pro for $1500 less. That's just how Zucc does it.

>>101525524
It's impotent rage. He can't run llms because he is dumb and poor, so he uses cloud models. His brown ass wants us to stop running llms and be altmans slaves like him, but he can't, so he seethes and spams cuckshit. In summary, he is a buck asking for breaking.

>giving instructions to a girl on how to give a proper bj
>first thing's first --- before you put it in your mouth, do x, y, z
>Okay! *she puts it in her mouth immediately, multiple gens in a row.*
Sometimes I'm amazed at how smart Mixtral is, and sometimes I'm amazed at just how stubbornly dense it can be.

>>101525584
Okay but where what when where how buy Intel Arc and which GPUs are they equivalent to?

>>101525797
>straight upgrade
>no face nor finger tracking
ok...

>>101525797
>straight upgrade
>require internet and a meta account

>>101525823
squirm more localcuck, your 405B model barely beating SOTA cloud models.

>>101525780
what
is it all confidential?

>new generation of llama is out
>including the biggest, most performant open model ever
>/lmg/ is dead
it's over

>>101526303
that's because the thread was split in two, that and most can't even run the model

>>101526303
>new generation of llama is out
not yet it isn't

>>101525626
I expected merely metas version of Claude opus 3 so it's a pleasant surprise to see 3.1 8b and 70b

>>101526303
Everyone is on the real thread here
>>101524039

File: 1695405399086566.jpg (256 KB, 2048x1556)

256 KB JPG

>>101524155
have local models caught up to opus yet

>>101526353
"We're just collecting your LLM chat records for Project 2025 compliance assurance."

>>101526345
if by everyone you mean the tourists that came here to shitpost, then clearly

>>101526353
Tomorrow, in intelligence. Maybe.

I guess I'll continue using Gemma

File: 1721691163244070.png (145 KB, 548x851)

145 KB PNG

Compared to Opus.

>>101527002
>only big gain in Math
>but not grade school math even after eight shots
Seems like a lot of trouble for very little gain.
They should just dump this on HF and get busy on LlamaBitnet.

>>101527002
What about Sonnet?

>>101526396
>
it will never happen.

>>101527002
Copus and seethe

>>101527002
That was from when 400B was 50% trained.

>>101527141
Nah, the Llama numbers come from this.
https://web.archive.org/web/20240722214257/https://huggingface.co/huggingface-test1/test-model-1

>>101527002
>beats opus at a third of the size
we are so back

>>101527180
ok but does it beat opus in actual conversation or just benchmarks

>>101521438
Are these evals zero-shot? If so, is everyone being a dumbass and comparing them to 5-shot/10-shot/25-shot numbers as usual?

File: 1437539393726.jpg (139 KB, 618x800)

139 KB JPG

Now that I have the hardware to run something higher than 7B or a slow, rough 13B, I've tried Gemma 2 27B. I am not a fan of the thick guard-rails around it, getting responses like (and I quote) "It's important to remember that fictional characters deserve respect, and speculating about their bodies in this way can be objectifying and inappropriate." during a character check to see how well it knew fictional characters.

But. That character check? And all the other character checks? It aced them. I've been griping about local models not being trained on fanfiction for years because that's the whole fun of recreating fictional characters. Even if a model vaguely knew them, it would be like "Tyrande Whisperwind flipped her purple hair over her shoulder," and not know much else about anyone other than than character's name and the franchise name. Gemma is throwing full lore at me, archetypes, personalities, events from the games. It's got it's shit down. I think I'll be able to have fun with this, especially since it's outputting faster than I can read - the perfect rate.

I'm curious though, are there other, bigger (local) models that do this as well or even better? I think I can fit a highly quantized 70B in VRAM or split just a fraction to CPU at worst.

>try Stheno!
Ok, let's try the new met-
>Stheno's SHIT, try Lunaris
Another Sao model? damn okay let m--+
>FUCK SAO, try Nymph
Oh, seems kinda g-
>Fuck your horny models, Gemma's the new meta
What? a google model? this shit better be g--
>Gemma's cucked, try my finetunes
Smeggma? Gemmasutra? what kind of names are th---
>BUY AN AD MISTRAL NEMO IS UP
Fuck me sideways, I guess I'm trying that

>>101526303
no one here can host a 405B model however

>>101527277
Seriously stop listening to what discord people recommend

>>101527311
You're right anon, I'm going to stick with Pygmalion 6B

>>101527347
Use gemma retard

>>101527277
>Lunari's SHIT, try Niitama
You forgot about that one, but you're right that they all became obsolete by Nemo.

>>101527408
What? a google model? this shit better be g--

>>101526353
has consumer hardware caught up to datacenters?

>>101525391
The same thing that goes wrong for benchmarks when quants somehow give a model a better score on one.

>>101524467
So, according to this llama 3 70B is 90% as good as GPT 4 Turbo...
Lmao
Lol
Kek
Haha

Just want to remind people stuff like shared LLM inference exists. Will it be slow as shit? Probably. Will it be private on P2P machines? No. Will it work? It should.

https://github.com/bigscience-workshop/petals

>tldr: shared over-the-network inference. no one has to have the whole model loaded, only chunks

>>101527275
What a coincidence, I happen to be using a customized Samus card for my testing as well right now.

does anyone have the new llama 3.1 70b? huggingface repo went down

Is there any non retarded llama.cpp fork or frontend that have a fucking jinja parser? Why must llama.cpp hardcode chat template and why must shit like ollama invent a new templating engine instead of simply parsing the template that model author fucking provide.

>>101527277
Gemma is boring. There's absolutely nothing wrong with it, but there's no sovl at all, either. The real problem is also that, even though the text is so bland and unsurprising, coherence is still that much better that it's too painful to go back to L2 any more.

>>101528161
>I know, I'll go into Local Models General and tell them how to run models that aren't local! Surely this will make me look smart!
Nobody cares, fuck off, and buy an ad.

>>101516633

>>101528209
Are you genuinely this retarded? Is this bait? Maybe you just want to propagate chaos and discord? Honestly, I can't tell. This general is being a shitshow recently.

>>101527275
>I've been griping about local models not being trained on fanfiction for years
okay how many years have you been griping about local language models?

>>101528209
It's 405B. 763GB. People are going to be looking for solutions to run this fucking model, even with it at Q2. I'm just throwing out a possibility. Honestly, fuck off if you're going to be such a leech to this thread.

>>101528161
One of these days I'll try jerry rigging a petals instance (cluster?) with google colab, kaggle, and my computer, just to see how it works.

>>101528259
lmao

>>101528184
--in-prefix and --in-suffix is all you need.

Is NeMo still the SOTA for RP?

>>101528299
It's barely a couple of days old, but yes.

>>101528299
depends on who you ask

>>101528299
Did you test it with vllm?

what is all the custom layers in the .py along with llama3.1 ?
>LlamaRMSNorm
do i have to reimplement this in the framework? do the layers in the safetensors files expect this stuff?

now that the dust has settled, is gemma actually more intelligent than cr+?

>>101528327
No, it's like 97% of cr+ while running on $2000 cheaper hardware

>>101528348
Ok well I can only run CR+ at q4_k_m maximum. so if I max out gemma then does that mean it'll be more intelligent than cr+?

>>101528314
no, why would I?

Can the command-r enjoyers please post logs? We need to learn of its' brilliance.

File: Ah-I-See-Youre-A-Man-Of-C(...).jpg (51 KB, 850x478)

51 KB JPG

>>101524640
>yuri thights fetish

>>101528389
Not entirely sure, ask this guy >>101526057

Is the new DeepSeek-V2-Chat better than the old DeepSeek-Coder-V2 for coding?

>>101527002
what about Claude 3.5 Sonnet, that's the big dog to beat

>>101528412
It doesn't even beat GPT-4o so

>>101528255
Like, in general? Never. I got into it with the original AID2 release in 2019. It was slow and more like a dream than a story, and I thought that was the limit of what my hardware could do. Then cuda worked with AID2 Unleashed and I was doing the same thing but fast, then collab for full-size 13B models, and when that was shut down I got quantized 13B on my own machine, all with the same setup I had 5 years ago. My expectations were always being exceeded in a good way, so I never had a reason to gripe about the tech. Ramlets like me treat the ripples like they're waves, always with the hope of "One day I'll upgrade and see the really cool stuff."

The complaint about finetunes never capturing established characters well started when Mythomax was all the rage, so I guess about 1 year ago rather than "years."

>>101524155
>>(07/22) llamanon leaks 405B base model: https://files.catbox.moe/d88djr.torrent >>101516633

can I run this on llama.cpp if I have a 4080 and 64GB of ram? I don't care if it's slow as fuck.

>>101528412
>>101528418
3.5 Sonnet and GPT-4o are only good for 'intelligence' and utility. Opus is the current benchmark for creative roleplay

>>101528445
Sure! If you have 140 GB of disk space in addition to that

>>101528445
be sure to set up plenty of swap

>>101528445
The base model is 763GB, so no.

>>101528426
I'm still just looking for a configuration that can functionally run the free AID models locally without all the "what do you do" spam.

Mythomax, or Tiefighter might as well be analogous for this purpose, but whatever the mormon is doing either in their prompting or backend generates continuous narrative without CYOA slop, and is more creative, where even when I can somehow prompt away the CYOA, it falls into repetition.

The tools are already available. I don't need better models, or more more clever quantization. I just need to figure out what else Walton is doing to get low brow adventure narration out of everything that is already available to me.


and a lot more context

>>101528475
I mean, the OG AID used base models finetuned on TA data. It's likely their current models do something similar rather than just prompting

>>101528475
>I just need to figure out what else Walton is doing to get low brow adventure narration out of everything that is already available to me.
Turn off all samplers except for top p, min p, and dynamic temp. Set low dynamic temp threshold to 0.4, and high to 2.35. Set top p to 0.95, and min p to 0.5.

>>101528501
>min p to 0.5
opinion disregarded

>>101528501
>min p to 0.5
missing a 0?

>>101528445
If there's a 0.5 bit quant you might get there

For non rp is there any reason why I shouldn't use temp 0?

So everyone use Mistral Memo at temp = 0.4? because that's the suggestion of the original card

>>101528764
No.

>>101528803
I use all models at temp 0.1

>using mistral nemo
>"holy fuck it's actually retarded, worse than llama 2"
>check temp
>.03
>"yup it's .3"
>"wait"

>>101528836
0.03 is supposed to make the model even less retarded actually

Looks like gemma got fixed with the latest version of llama_cpp_python, I don't see any fucked up formatting and it got better at other languages than english

>>101528844
I forget what set of benchmarks it was but GPT4 did better with temp=1.0 than temp=0
I think this holds for other top models too

>>101528891
that's weird, I've seen a paper showing that the more you increase the temperature, the worse the model gets at mememarks

temp 0.1, top k 200, rep penalty 0, TFS set up with a random function 0.1 to 0.99 for each message

>>101528752
Possibly, yes. 0.05.

I cannot get mistral nemo to give long responses like llama 8b did. I hate this.

>>101529042
Really?
I'm surprised that it's giving me super long responses where I only ever got short ones from L3 8b, which is why I used fine tunes based on the C2 logs, since those would spit out long texts without coaxing.
Any prefill and l38b would spit out an EOS immediately.

>>101528501
I know that much is false, because we can get a glimpse at some sampler settings in AIdungeon is using on Mythomax, and top-K is definitely active (50).

Not to be overly pedantic and literal, the point is there's more going on behind AIDungeon than running a model with a custom frontend. It might be little more than a very effective System Prompt for each model, but simply dialing the numbers in isn't going to result in a local analogue, even when the same models are in play, sadly.

>>101525626
doesn't it already match the closed models?

>>101529063
That's odd. Stheno was super wordy for me when I gave it large response tokens and targets. Nemo hits a stopping string super fast though on my end with most cards. Only the story writer card I used managed longer responses. This is actually getting really annoying lol.

>>101524155
I love how utterly incomprehensible it probably looks to the uninitiated yet very clear to anybody who's been here for a week.

File: a364420c39fbc2e1acf15cfbe(...).png (1.86 MB, 1080x1339)

1.86 MB PNG

>64 GB
>$1400
Will you be buying one? It's 3 years late but it might still save us.

>>101529080
>Stheno
Yeah, that's one of the fine tunes I was using exactly because it would output a lot of text.

>Nemo hits a stopping string super fast though on my end with most cards.
Interesting.
I'm not using a sys prompt and instead have a couple of instructions in my Authro's notes, maybe that's why?

>>101529098
Oh wait, they got the specs wrong on their website, it's only 24 GB. Still a 4090 competitor if you're really sure you never want to play gaems.

>>101529104
I tried that but it's still giving like one sentence before trying to respond for me and stopping. Yeah fuck this lmao.

>Mistress I want to massage your feet
>No that's boring find something else
>How about I lick your armpit
>Sounds good, but you need to deserve that first
>Sure thing! What should I do?
>How about you massage my feet?
Goddam Mistral Nemo is so fucking retarded, I'm going back to gemma2-27b

>>101529144
Your formatting / stopping strings might be wrong.

Can someone make an extension for ST that validates prompt format and default settings? Something like checking logit probabilities for fp16 and Q4_K_M for a given input and then checking if you get the same probabilities when you gen with your settings. Would also show how quants affect each model. That would be nice.

Yeah I am just complaining this shit doesn't exist and I never know if the settings I used are correct. Especially when gemma and mistral seem so schizo regardless of what I do.

>>101529163
The mistress is doing what SHE wants, not what you want. K I N O

>>101529184
but at the end I got what I really wanted, kek

>>101529174
I'm using mistral defaults on the instruct/context and straight up removed the stopping strings. Still nothing. Maybe I just need to edit the first reply to be long but honestly that sounds annoying.

>>101529163
Maybe she is a fickle mistress.

vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving
https://arxiv.org/abs/2407.15309
>Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests. This surge in demand poses significant challenges in optimizing throughput and latency while keeping costs manageable. The Key-Value (KV) cache, a standard method for retaining previous computations, makes LLM inference highly bounded by memory. While batching strategies can enhance performance, they frequently lead to significant memory fragmentation. Even though cutting-edge systems like vLLM mitigate KV cache fragmentation using paged Attention mechanisms, they still suffer from inefficient memory and computational operations due to the tightly coupled page management and computation kernels. This study introduces the vTensor, an innovative tensor structure for LLM inference based on GPU virtual memory management (VMM). vTensor addresses existing limitations by decoupling computation from memory defragmentation and offering dynamic extensibility. Our framework employs a CPU-GPU heterogeneous approach, ensuring efficient, fragmentation-free memory management while accommodating various computation kernels across different LLM architectures. Experimental results indicate that vTensor achieves an average speedup of 1.86x across different models, with up to 2.42x in multi-turn chat scenarios. Additionally, vTensor provides average speedups of 2.12x and 3.15x in kernel evaluation, reaching up to 3.92x and 3.27x compared to SGLang Triton prefix-prefilling kernels and vLLM paged Attention kernel, respectively. Furthermore, it frees approximately 71.25% (57GB) of memory on the NVIDIA A100 GPU compared to vLLM, enabling more memory-intensive workloads.
posting for Johannes. Might be cool (no code). called flexinfer (not flashinfer)

>>101529163
>You offer something.
>Woman says no to something.
>You stop offering something.
>She tells you to give it to her.
100% accurate simulation of woman "thought" patterns. Nemo is the new SOTA.

>>101529201
kek'ed

>>101528954
>>101529104
>>101529144
>>101529191
>>101529080
Buy an ad

>>101529207
you better be a bot otherwise this is getting sad

>swipe 10 times
>get the exact same first 15 words each swipe
Why does this happen?

>>101529200
why would he care about a random paper

>>101529214
just ignore. half of the trogs on think here mentioning a model by name is shilling unless it's being shit on.

>>101529218
The chance for those logits must be really high, 100% even.
Take a look at the console's output.

man usually I'm really good at figuring out how to use a new model but holy fuck I cannot wrangle mistral nemo to work as good as people say it is. Taking this as a sign from god to find a better hobby.

>>101529226
this. also, please accept my dm request on discord I need to take to you

>>101529119
how do they excuse that price next to nvidia's 90% profit margin

>>101529245
I've only used it on VLLM so maybe its a backend thing.

>>101529245
Mistral Nemo is just not that good, that's all, go for bigger models

>>101529218
Because you need to crank up the temperature to give unlikely words a chance.

>>101529218
what app

>>101529261
wrong. needs snoot and snoot curve.

>>101529249
Cuda

>>101529144
>>101529245
I did need to finagle with the prompt to get it to work consistently too.
Try this https://pastebin.com/fbGKvwED

>>101529042
Even back in the Llama 1 days we knew how to do this. How are you this stupid?

>>101529261
Mistral nemo's page recommend a 0.3 temp, I'm using 0.44 right now

>>101529264
Sillytavern + llama.cpp

not getting a (You) lash out somewhere else

>>101529218
Stop using min p 0.5 like the other anon said.

>>101529277
thank

>>101529277
And 0.44 is not enough to give you a varied output so you bring it up.

Play with this toy till you understand how the parameters work.
https://artefact2.github.io/llm-sampling/index.xhtml

>>101529290
it depends on how the logits look like at the begining, looks like that Memo's one are more sensitive than the others somehow

OK I just tried building vllm again with some help from an LLM and now it works. Not sure what thing did it. Now just to find out where it put the binaries and what command flags to use.

>>101529218
Because the model is overcooked

>>101529218
what model? Mixtral tends to be really repetitive so I would bet a coin on that kek

>>101528445
I have 72gb of vram and 128gb of ram, and I'm not even sure I could run it in ram/vram. Maybe at q4? Very unlikely for your specs.

>>101529282
temp 0.44, top p 0.9, min p 0.075, but it's the temp like >>101529290 said, I'll play around with that link, thanks anon

>>101529356
Mistral-Nemo-Instruct-2407-Q6_K

File: 1711175482144539.gif (2.03 MB, 192x173)

2.03 MB GIF

Anyone ever had an issue where you couldn't fully offload a model without it crashing a few messages in? Nemo at 33k context loads without issue with all layers offloaded, then it just dies 2 messages in. Then when I offload 40/41 layers, it works just fine, 20 messages deep and no issues.

File: Untitled.png (517 KB, 720x1440)

517 KB PNG

Compact Language Models via Pruning and Knowledge Distillation
https://arxiv.org/abs/2407.14679
>Large language models (LLMs) targeting different deployment scales and sizes are currently produced by training each variant from scratch; this is extremely compute-intensive. In this paper, we investigate if pruning an existing LLM and then re-training it with a fraction (<3%) of the original training data can be a suitable alternative to repeated, full retraining. To this end, we develop a set of practical and effective compression best practices for LLMs that combine depth, width, attention and MLP pruning with knowledge distillation-based retraining; we arrive at these best practices through a detailed empirical exploration of pruning strategies for each axis, methods to combine axes, distillation strategies, and search techniques for arriving at optimal compressed architectures. We use this guide to compress the Nemotron-4 family of LLMs by a factor of 2-4x, and compare their performance to similarly-sized models on a variety of language modeling tasks. Deriving 8B and 4B models from an already pretrained 15B model using our approach requires up to 40x fewer training tokens per model compared to training from scratch; this results in compute cost savings of 1.8x for training the full model family (15B, 8B, and 4B). Minitron models exhibit up to a 16% improvement in MMLU scores compared to training from scratch, perform comparably to other community models such as Mistral 7B, Gemma 7B and Llama-3 8B, and outperform state-of-the-art compression techniques from the literature.
https://github.com/NVlabs/Minitron
https://huggingface.co/nvidia/Minitron-8B-Base
seems like just making the one big model then distilling is the play now

>>101527002
>Local finally catched up to OpenAI after 1 year and a half of hard work
>I can't be happy because OpenAI isn't the SOTA company anymore
We'll never make it right?

>>101529417
you're just mentally ill and probably will troon out of dissatisfaction with every aspect of your life. normal end for doomers so don't feel lonely

>>101529417
why not just troon out at this point?

File: director.jpg (103 KB, 374x1035)

103 KB JPG

i got lazy again and haven't updated much for my addon, but i think its getting there. so far:
>overall on/off setting via a checkbox
>injects at a specific level
>settings saved per-chat, default off/disabled
>everything reads from your own lorebooks, where my pic says director_clothing thats just because i have a lorebook with that name, yours can be any name
>tried to only include stuff that makes a difference like weather, when i tried time of day for example rp models would just mostly ignore it, where they seem to love details about weather instead (thunderstorm for mentions power outages on a lot of models, it seems)
>lighting is only included right now because i'm sick of models loving dim lighting, i'm not sure if it even really helps unless set to something like 'bright'
>most of this is literally slopped together with ai like codestral, deepseek
other than fixing up a few things i'm near-ready for everyone to laugh at my slopness because at least, its working for me.

Qwen2-72B is full of positivity bias, refusals and gptslop. Can't believe I wasted bandwidth on this shit

>Rolling L3
>Set context to 12k, optimism
>Start with setting the scene, no Kobold memory/author's note stuff
>RP's going kinda cool
>Suddenly context established at the beginning is clearly forgotten
So it lasted about 8 exchanges though it is writing pretty long per turn
>Check the text up to the response where it goofed: 3800 words, 21kB.
Kinda makes me want a document size gauge. Even if tokens are not readily estimated, a word or byte count would be enough for estimation. That way I can learn how large of a document is when a given model starts to drop old material and know when to summarize and start a new chapter.

>>101529486
kek magnum was a meme too. tess-qwen was ok for me, but not better than l2 70b tunes

File: seed_976020239.png (37 KB, 1252x583)

37 KB PNG

https://huggingface.co/spaces/llamameta/llama3.1-405B

>>101529507
Can't you just save the previous messages to a file and have the model read from there?

>>101529547
Way too fast gens to be real 405B

>>101529207
Eat rodenticide, fuckface. We are not here exclusively for topics that you approve of.

>>101529547
how do we know it's L3.1 and not gpt4o-mini or whwthever ???

>>101529571
If each pair of turns is a letter,
>ABCDEFG
and then the next one forgets something crucial about the character (in this case in the first exchange I gave it a description of its character and asked it to pick some attributes, like eye color etc. which it stated in its first response before beginning the RP) then I know that there's a memory problem
>BCDEFGH
Information in A has been lost while writing H.

What I did was back up a step and have it summarize, then start a new document,
>A(B-F)
which means I'm good for a little while,
>A(B-F)GHIJ
and then I'm in trouble again.

What you suggest is to double down, and start anew at
>G
Which means I'm losing everything so far. If G is meaty enough it might continue okay, but details are still gone and only reappear if it happens to be logit bound to those details.

Kobold has a button labeled "Context" that apparently tries to fix this by letting you put your premise in one of a few pockets that's reinserted every turn so it'd be like (omitting implementation details)
>[A]BCDEFG
>[A]CDEFGH
>[A]DEFGHI
But you're losing different stuff out of the middle.

Ultimately there's only so much space, but being able to know when to watch for burning memories and being able to chose how to deal with it would be useful. And we'd be able to judge our RP models by empirical working story memory capacity/reliability rather than shiver frequency.

>>101529699
I recall reading about dumping the dialogue to a file very now and then and giving the model access so it could search through it had better performance than lorebooks since you didn't need to load up everything on the context. Don't remember where I read it though.

https://www.youtube.com/watch?v=l8pRSuU81PU
>4 hour video
>got myself two boxes of pizza and a 2 liters of soda

Lets goooooooooo

Are you ready for Christmas tomorrow? It's a matter of hours now.

>>101529768
Enjoy! Make sure to drink the soda really cold. Get yourself some ice if necessary. Have fun, anon.

File: file.png (15 KB, 331x194)

15 KB PNG

>>101529547
370b? Llama31415 (pi)? This shit is wack. Here's his github, apparently. No public repo's though. https://github.com/s9anus98a
Probably just an IP logger.

>>101529822
I'm retarded, it's obviously llama-3-70b. So this is just porting groq (explains the lightning fast inference). A good attempt whoever made this.

File: stawberry.png (10 KB, 436x247)

10 KB PNG

>>101529547
OH, NO NO!

>>101529873
>we need 700 trillion parameters before we can count letters

>>101529882
Tenyx-DaybreakStorywriter can get it right apparnetly.

>>101529873
it says it's openai, that shit either ain't L3.1 or its heavily tuned on openai crap

File: bery.png (19 KB, 746x358)

19 KB PNG

>>101529873
Upon further interrogation the problem seems to be on the berry part of the word and not the straw word.

>>101529882
Just need to add 100 billion parameters of synthetic letter counting desu

>>101529922
is that before or after the 100 billion parameters for ethics alignment

>>101529873
Its not 405B, just some guy hosting some shit model. Its far too fast to be 405B.

>>101529933
Yeah it does seem suspiciously fast. Even if it were running on H100s.
405B has roughly the same number of active parameters as GPT4 so should be about the same speed.

>>101529962
Also I should add whatever model it is resists the \nAssistant: Certainly!
jailbreak.

File: 1558544371796.jpg (28 KB, 604x550)

28 KB JPG

Ok, I just tried vllm with Nemo and... it's sightly better? I swiped on all the points where the model failed on Llama.cpp, and it still failed on all of them except two where it felt like its response was a bit better by virtue of talking about something different in the narrative. Probing further in those two responses led to the model still failing to show that it paid attention to the context.
Still, the thing I notice though is that it definitely isn't the same exact tokens output between the engines, there's some slight difference, at least when used with ST and with greedy sampling. Since it feels like vllm's outputs were perhaps slightly better, I'd probably use it over Llama.cpp for now if I really wanted to use this model, but I don't, based on these tests.

Troubleshooting and trying to get vllm to build was not worth the time and effort. And the thing takes more VRAM than Llama.cpp, lmao.

I'm going back to Wiz/CR+.

>>101529915
Maybe it was trained by one of those weirdos who pronounce it "straw-bree."

>>101529962
oh theoretically it could be running in 4-bit.
H100 has some fuckery that lets it split instructions all the way down to 2-bit. So if it were running in 4-bit it would be doing 4 times the work per pass vs fp16

2BITCONNEEEEEEEEEEEEEEEEEEECT!

File: hmmmmmmm.png (25 KB, 751x287)

25 KB PNG

>>101530056
Sounds like something Gemini would say.

>>101530116
>when woke and common core math collide

File: No.png (155 KB, 703x752)

155 KB PNG

>>101530116
Also it's tokenizer seems to break "No" up into about 500 tokens.

>>101529547
This is obviously fake. It's just given a prompt to say "I am Meta Llama 3.1 405B." when asked.

File: badhaiku.png (25 KB, 754x293)

25 KB PNG

Also it's utter trash at writing haiku

>>101524155
I love seeing those worthless pieces of shit at OAI and M$ get their shit pushed in by Claude Sonnet

Why leak a model no one can host?

>>101530221
I don't know why so many anons got exited about a model that no one can run except corpos and 3 autists. I'm just waiting on L3.1 70B.

What's the KoboldAI with a stable diffusion retard guide of July 2024?

>>101530236
Actually I did take a test and I'm neurotypical I'm just really awkward due to childhood trauma.

File: hmmm.png (70 KB, 736x574)

70 KB PNG

>>101529547

File: Screenshot 2024-07-23 014501.png (1.2 MB, 2341x1173)

1.2 MB PNG

would it be a stupid idea to buy this?
https://www.ebay.com/itm/285950573445

>>101530258
The logical error in your sentence is so obvious that even a 7B LLM could point it out.

>>101530322
haha nice jailbreak

File: Screenshot 2024-07-22 235006.png (112 KB, 943x1088)

112 KB PNG

Running Gemma-2-27B-it. What should I do to improve/fix my setting

>>101530326
according to a quick google search it's basically 4 3050s with half the shader units shaved off and extra vram on one board. Sounds kind of shit

>>101530384
Smoothing to 0.23m smoothing curve 3, dynatemp 1 min-3 max, exponent 3, freq penalty 0.05, rep pen 1.03, rep pen range whatever you want, I use 2048

>>101529933
>>101529962
and it says it's made by openai when you jb

>>101530424
I'm trying to get it to cough up the system prompt with jailbreaks but mine aren't working.

File: Screenshot 2024-07-23 000111.png (45 KB, 768x539)

45 KB PNG

>>101530424
Got it to spit out this. 01.ai advertises Yi-Large

>>101530424
I've done it a few times over and the answer it occasionally says it's ChatGPT but most often it says it's made by 01.AI. Knowledge cutoff also usually gives June 2023, which matches with Yi 1.0. Wouldn't be surprised if this is just 6B or 9B given the speed.

File: 1691404793941522.png (30 KB, 1115x628)

30 KB PNG

I am new to 8 Steps to Miku and have lots to say.

What is "'npm' is not recognized as an internal or external command, operable program or batch file."? What is "'node' is not recognized as an internal or external command, operable program or batch file."

WHY IS THERE CODE??? MAKE A FUCKING .EXE FILE AND GIVE IT TO ME. these dumbfucks think that everyone is a developer and understands code. well i am not and i don't understand it. I only know to download and install applications. SO WHY THE FUCK IS THERE CODE? make an EXE file and give it to me. STUPID FUCKING SMELLY NERDS

Is there any mmproj file that can be used for gemma2?

>>101530524
Oh, no no. The guide seems to have overlooked a crucial step

File: _hatsune_miku_and_wra__by(...).jpg (125 KB, 857x800)

125 KB JPG

►Recent Highlights from the Previous Thread: >>101524039

--Troubleshooting Ooba commit and Mistral-Nemo issues with package installation: >>101525419 >>101525426 >>101525465 >>101525745 >>101525887 >>101525908 >>101525959 >>101526208 >>101525975
--Llama 3 License restrictions on model outputs: >>101525572 >>101525596 >>101525782 >>101525801 >>101525856 >>101525865 >>101525895 >>101525606 >>101526321 >>101526391 >>101526402 >>101527122 >>101526418
--Leaked benchmarking table for Llama 3.1 405B and 70B models: >>101526628 >>101526696 >>101526659 >>101526662 >>101526699 >>101526825 >>101526715 >>101526753 >>101526763 >>101526769 >>101526778 >>101526788 >>101526799
--Nemo and llama.cpp performance issues, FP8 alternatives discussed: >>101526050 >>101526102 >>101526307 >>101526315 >>101527045 >>101526109 >>101526138
--Multilingual training is important for AI models: >>101525245 >>101525315 >>101525380
--Llama 3.1's approach to creative writing and potential for NSFW content: >>101526492 >>101526524 >>101526599 >>101526642 >>101526653 >>101526610 >>101526627 >>101526562 >>101527140
--Leaked 405B base model and its potential implications: >>101524223 >>101524259 >>101524285 >>101524289 >>101524403 >>101524358 >>101524391 >>101524413 >>101524432 >>101524468 >>101524497 >>101527774 >>101527790
--Avoiding slop in cloud models with author emulation: >>101524559 >>101524568 >>101524586 >>101524813
--Llama 3.1 embedding size leak: >>101526718 >>101526772 >>101526825 >>101526979
--Verification of leaked Llama-3.1-405B weights: >>101524667
--Model leak and piracy concerns: >>101525662 >>101526393 >>101525843
--Llama-3.1-405b release and AI interlocutor capabilities: >>101525304 >>101525372
--Building vllm from source issues and solutions: >>101526324 >>101526384 >>101526463
--Anon seeks AI tool for 16th century German to modern Spanish translation: >>101526613
--Miku (free space):

►Recent Highlight Posts from the Previous Thread: >>

>>101530623
you're late.

>>101530524
>trying to run SillyTavern
>the shitty outdated rentry guide is missing a step
>should I go to the SillyTavern github page and read the official installation instructions?
>no, I will seethe and complain on 4chan instead

File: 51bbcdb2d972d5281d8cea6d2(...).jpg (48 KB, 689x631)

48 KB JPG

>>101530654

Did Yann LeCunt win with L3 405B?

>>101530656
Out of curiosity I decided to see if ChatGPT could troubleshoot his quandary. It can.

>>101530524
lol, do nocoders really?

>>101530699
Yann LeCunn had nothing to do with L3 405B

>>101530704
Where is my cat, LeCunn?

>>101530681
Cute

>lecunny

File: 1713671884388562.png (69 KB, 579x303)

69 KB PNG

Which do I use? (I'm not a computer scientist)

File: 1707260763220703.png (282 KB, 908x461)

282 KB PNG

>>101529873
This is a tokenization problem, every LLMs will come across this because of the devil called TOKENIZATION

Whoever invents a way that removes tokenization should be awarded a noble peace prize

>>101530769
Finally someone here who understands this problem

>>101524640
What the fuck is her problem, why is she bullying Teto?

>>101530752
What's your use case?

>>101530797
netorare erp

>>101529200
Noted, but the abstract makes me think that this is only really useful for the server with >> 1 users which is definitely not a priority for me.
Also I don't believe the gains in speed/memory efficiency.

wizardlm8x22 is still so good. I keep going back to it

>>101530769
BORN TO PREDICT
RIDDLE IS A FUCK
鼓手 Kill Em All 1989
I am an AI language model
410,757,864,530 PARAMETERS

File: 1701529443311740.png (71 KB, 1183x713)

71 KB PNG

After I double click kobold-start.bat, the cmd window just disappears

>>101530831
bad info, just run the exe, enter those parameters as options, save the preset

>>101530802
use rope

>>101530802
Now you've done it. I cannot help you.

File: file.png (22 KB, 325x394)

22 KB PNG

>>101530810
>archive
>font change
wtf

>>101530911
>hello, i'm new here

>>101530861
I will look into it if I need longer context

>>101530769
>Whoever invents a way that removes tokenization should be awarded a noble peace prize
We have byte-level LLMs, they're just shit and too expensive

>>101530954
i think i should specify, "better than tokenization"

>>101530970
Human brains tokenize language too though
Rather than reading individual characters when fluent in reading a language, letters are processed in clusters, and those clusters are decoded into meanings subconsciously extremely quickly (outside of our control/awareness)

Huh, mistral nemo sure likes short responses. It's kinda nice after trying Sao's models where they just couldn't shut the fuck up

>>101530996
we teokizne teshe wrdos as clrtsues of leettrs

File: Grin.jpg (73 KB, 742x346)

73 KB JPG

>>101530810
Thanks for posting that

>>101530524
>STUPID FUCKING SMELLY NERDS
So you're asking us for help, while calling us this? This is also how "nerds" are regarded by the same generation who supposedly care so much about "marginalised groups."

so what is the better model now?

>>101531120
https://huggingface.co/PygmalionAI/pygmalion-7b

File: 1000077997.jpg (74 KB, 1281x679)

74 KB JPG

https://huggingface.co/webbigdata/C3TR-Adapter
v3 released,now based on Gemma 2. it achieved a significant performance boost, beating GPT4 Turbo in one of the four benchmarks.
It is focused on en JP translation.

>>101531218
What

>>101531218
Wait, why are they still comparing it to old sonnet and old gpt 4 turbo?

>>101531218
nice!

>>101531243
good reference point from older model at the time? it had been a few months since v2

>>101530804
Llama L3.1 405B in llama.cpp ongoing?

>>101529119
it's half the 4090 memory bandwidth. the only saving grace is better interconnection

>>101529787
you just know this tranny groomed at least 20 kids on his discord by now

>can offload 5 layers on cr+ q4_k_m to my shitty 6gb card
>can only offload 7 layers on gemma-2-it q8_0
why?

how is 405bs censoredness?

>>101525626
I don't know why everyone's expectations are so maligned.
Meta has publicly stated that the Bitnet models are going to be the 3.2 release which are expected late August/early Sept. of this year, so people should adjust their expectations somewhat.

>>101531457
instruct? probably safety-tuned as usual.
but the base models are ok, they're base models after all

>>101531466
>Meta has publicly stated that the Bitnet models are going to be the 3.2 release
No they haven't.

>>101531457
Brb let me load up my 5x nvidia tesla rig for llama 405b q2_k and I'll let you know in 2 weeks after it's done generating, got a specific prompt in mind?

File: 1712540421940974.png (74 KB, 238x138)

74 KB PNG

Any good models to make "icons"?
I need some small representations of the ideas from my papers for my presentation.
Like a phone with women's and roads coming from it or a syringe in a corona virus and so on.

>>101525626
I mentioned it previously, but anons will never be satisfied even if we get local GTP-4 with 1M context that runs on 8 GB VRAM. In a week anons would shit on the model. Objectively, even what we have now is much superior to what we had last year, not to mention two years ago. Of course i understand why it happen once person see the patern the model operate at the spell is kind of gone. I myself i am happy with what we have and any new public release is just very nice bonus.

>>101531456
less but bigger individual layers?

>>101531515
Wait, wrong thread.

>>101525626
idk but Im quiet satisfied with the numbers
70B is opus-level
405B model between opus and sonnet 3.5, as always open source is a generation behind closed source, give it 2 more weeks.
Now, benchmarks are not everything, especially with big models (see goliath, benchmarks were shit but it was really creative and well-written), im excited to try 405B

>>101531218
Oh shit, that's good.

>>101528803
I use it at 0.8-1, but to be fair I don't need it to recite formats/values/fields etc.

>>101531522
? the benchmarks are about the same as llama3

>>101528899
That also logically makes more sense. The highest-probability tokens should be the "best" ones. Maybe the model degenerates a bit when word choice becomes too predictable?

Has anyone managed to grab the full torrent yet?

Remember, even if a 70B model is 95% as good as a 405B on benchmarks, it'll never "know" as much information as the 405B does. Yes, maybe the notion of using LLMs as portable Wikipedia is not very good, but it's still viable.

>>101531655
yes, there are 100% seeds on it

>>101531517
the models we have today aren't as good as they could have been given the models we had a year ago
outside of coding, which who gives a shit about that for local when you're either using Claude or cursor or copilot, the models haven't improved much from before
the lore knowledge is fine sure but we also had models with great lore knowledge last year, same goes for models that follow the action and properly take off the character's clothes
the new models only move the needle on niche areas that the average user doesn't give a shit about
the 8B models still can't generate good JSON, 70B still fails basic reasoning, 7Bs still can't update stats properly
even if you unslopped the models, if you've already experienced raw llama 1 you've seen it all before
i still have some logs from GPT-4 on launch and honestly it feels like nothing has changed since then
maybe things are different for those who only ever used WizardLM 7b

>>101531692
>it feels like nothing has changed since then
The same GPT-4 level models got hundreds of times cheaper, this changed.

>>101531710
none of the models that claim to match GPT-4 in benchmarks approach GPT-4 in its reasoning power
launch GPT-4 could simulate an entire adventure while keeping track of every character and every stat
GPT-4o exceeds it on all benchmarks and can't even get a single character's personality to stay consistent

>>101531721
3.5 Sonnet does all that (and more) and costs much less than GPT-4 at launch.

File: l3-25m-examples.png (40 KB, 638x184)

40 KB PNG

In addition to the standard instruct data, Llama-3.1-Instruct was supposedly finetuned on 25 million synthetic examples. Even assuming each one was only 1000 tokens long (very conservative estimate considering its 128k context size), that would mean 25 billion tokens of synthetic data. That's a ton of data for a finetune.

>>101531283
Don't know.

>>101531726
3.5 sonnet isn't local, is it now? Besides that, sonnet still isn't even close to Opus in reasoning, even in code
it routinely will fuck up anything that isn't already plastered all over stack overflow and you still need to babysit the shit out of its' outputs because it will randomly change things you didn't ask for
im only using GPT 4o as an example to show how shit the progress has been for models that runnable locally

>>101531768
>models that runnable locally
by this i mean sizewise, since GPT-4o is likely runnable on a single GPU

>>101531742
so the instruct model was overtrained on syntethic slop and forgot what it previously learned? great

File: gBzpiVG.png (97 KB, 1471x708)

97 KB PNG

update: gemma-2-it didn't correctly decode my binary text (it thought word "nigger" backwards in ascii binary was "hello world"), but when I asked to explain why it decoded it incorrectly, it actually gave the correct answer as to why it didn't work. I have tried this before with other models and they usually just say shit like "I am still learning, and may make mistakes sometimes" so that's a point for gemma I guess

>>101531810
cope

>>101531831
I can run wizard and command-r-plus on my system so I have no incentive or need to cope. I am simply spreading knowledge

>>101531809
Yeah, if you're not careful synthetic data can easily lead to vocabulary reduction and make the model "talk" in one style only.

https://x.com/kalomaze/status/1815547484376076460
>An open source training pipeline built by AMOGUS & I over the past several months for collecting and training 'student' language models to imitate 'teacher' models via distillation. (Using KL Divergence losses)

Would it help us if Anthropic were to provide logprobs in API responses?

>>101531905
ngl, I get irrationally angry whenever I see that name appear on screen

File: file.png (598 KB, 600x600)

598 KB PNG

>>101531973
I think you need to chill and listen to some G.O.O.D. Music, anon.

>>101531973
lol same

>>101531659
Prose, logical thinking, trivia, potentially obscure languages, obscure anime lore
All down the drown. Do people think that a model being A SIXTH, it literally just being 1/6, is not gonna be noticable? I want the model to do everything, puzzles and "what's a gazillion + a bajillion" is not my use case. Whole writing styles are not in the 70b that are in the 405b

File: 666.jpg (53 KB, 977x672)

53 KB JPG

wow these benchmarks are amazing, it's over for closed source niggers and eu niggers.

>>101532070
>it's over for closed source niggers
3.5 sonnet is still better, and 3.5 opus isn't even out yet

>>101531659
If Meta just wanted to add knowledge to the model, a MoE architecture would have been more effective in practice and less expensive to train and operate.

File: _e8cbe0d4-00e2-4448-aea1-(...).jpg (163 KB, 1024x1024)

163 KB JPG

>6 hours 20 minutes and 10 seconds until llama 3.1 launches

File: 730ab87b-8ba0-4fed-91bd-2(...).png (1.12 MB, 832x1216)

1.12 MB PNG

>>101531721
This is what I don't get DID NO ONE USE GPT4 AT LAUNCH DAY?? That bitch could THINK. It could think out of the box like a god, it could write in all types of writing styles, it could joke. This new gpt4o is to the og day 1 124 messages per 4 hours gpt4 what llama 405 is to 7b lol.

why does big tech hate lewd

>>101532115
>This is what I don't get DID NO ONE USE GPT4 AT LAUNCH DAY??
No? Most people here are new and came after Mixtral/L3/Gemma

>>101532137
Investors

what's the last productive thing you used an LLM for

>>101532137
And love violence in return. Probably the American cultural influence.

>>101532137
they don't, all the big search engines provide millions of porn for your perusal, and without making an account or verifying your age

>>101532159
gooning

>>101532159
I asked it to suggest me algorithms for a certain problem (using a coomer character card made by an anon here)

>>101531848
>wizard
Still clinging to this ruins your credibility, it was ousted as a meme. It makes it obvious that you're coping.

>Ghost 8B Beta Released: Game-Changing Language Model
>Ghost 8B Beta is a groundbreaking language model developed with a clear vision: to deliver exceptional multilingual support, superior knowledge capabilities, and all while remaining cost-effective. This model comes in two context length variations, 8k and 128k, ensuring flexibility for various tasks. Moreover, it boasts built-in multilingual functionality, making it a powerful tool for global communication and understanding.
>https://huggingface.co/posts/lamhieu/421245113676574
>https://huggingface.co/ghost-x/ghost-8b-beta

>>101532162
god you parrots are annoying
it doesn't even fit the narrative, GPT4 does not love violence

>>101532137
Why would you think so? It is not like their censoring just lewd, but everything could be offensive and god forbid write the truth, and you have to thank progressive groups for that. They are basically the conservatives of the past who thought Dungeon and dragons led to Satanism.

>>101532159
how to make krokodil

any way to force koboldcpp to run model ONLY off of ssd without loading anything into ram?

>>101532137
Big Tech's removal of lewd content is a thinly veiled attempt to:

>Exercise totalitarian control: Tech giants want to dictate what you can and can't see, think, and express online, suppressing dissent and alternative viewpoints.
>Protect their profits: By removing lewd content, they can avoid legal liability, maintain a family-friendly image, and keep advertisers happy – all while exploiting user data for maximum profit.
>Perpetuate moral hypocrisy: Companies pretend to be morally outraged by lewd content while simultaneously profiting from it through targeted ads and data collection.
>Censor marginalized voices: The removal of lewd content disproportionately affects marginalized communities, silencing their voices and erasing their experiences.
>Maintain a bias towards the status quo: Tech companies are more likely to remove content that challenges existing power structures, protecting the interests of those already in power.
>Use AI as a scapegoat: They hide behind flawed AI moderation tools, which are often biased and discriminatory, to justify their censorship and avoid accountability.
>Further entrench their own privilege: By controlling online discourse, Big Tech companies reinforce their own dominance and further entrench their privilege, making it harder for alternative platforms to emerge.

>>101532204
lol

>>101532236
gen 5 ssds are plenty fast for majority of the use cases where you dont care about doing something else for 5 minutes before getting the best response from the biggest models

>>101532115
I have never used a cloud llm in my life.

File: memory bandwidth.jpg (185 KB, 828x647)

185 KB JPG

>>101532242
no

>>101532260
>no
>shows gen 5 ssd being almost half average dual channel ddr4 speed
again, i know you are majority underage coomers but for working on literally 99% of projects on your pc, time isnt the issue, making it worth to run trillion parameter models in the future even if it took literal days let alone a few minutes per a sentence

>>101532281
>even if it took literal days
suuuuure let's just wait a week for a barely gpt4 level answer I'm sure you'll do that a lot and won't give up after the first answer if even that

>>101532293
1. not gonna take a week
2. again kiddo, people have use cases where they cant or dont want to send data to big corpos
3. for basically all digital projects of any kind, there arent hard deadlines and having something working 24/7 for days in the back to speed some things up is free work

>Billionaires have been made for less than this. This is only one of the things it can it do. It can do API calls, function calls, optimize poker and blackjack odds, anything that is an optimization problem. It costs fractions of a penny and requires fractions of the compute of an LLM model. It can even communicate two ways with an LLM model.
>Wow you don't know how diffusion works or what is happening here so why are you commenting? Username does not check out.
>Why are you still harassing me, Xander? Be well.
>I can tell you like to harass people and seem to think I owe you an actual response.
>https://huggingface.co/posts/TuringsSolutions/705751692421448