File: image_2025-08-15_085921572.png (285 KB, 450x450)

285 KB PNG

►Recent Highlights from the Previous Thread: >>109342889

--Jun Song announces GLIMPSE and SAOD architecture for extreme model compression:
>109344273 >109344484 >109344529 >109344557 >109344567 >109344580 >109344551 >109345140
--AI's potential for math-driven architectural breakthroughs and automated hardware design:
>109342964 >109343080 >109343186 >109343200 >109343217 >109343269 >109343255 >109343110 >109343024 >109343030 >109343310 >109343373 >109343417 >109343386 >109343625 >109343720 >109343863 >109343144
--Allegations of Moonshot AI distilling Anthropic's Fable for K3 model:
>109343230 >109343258 >109343290 >109343267 >109343293 >109343313 >109343353 >109343567 >109343457 >109343549 >109343420 >109343366 >109343411 >109343522 >109345152 >109345233 >109345242 >109345256 >109345496 >109345300
--Brainstorming technical ways for Anons to watch movies with LLMs:
>109343570 >109343684 >109343711 >109343709 >109343764 >109343814 >109343882 >109343987 >109344039 >109343835
--Microsoft releases Mage-Flow amid cynicism regarding corporate model quality:
>109345367 >109345382 >109345403 >109345426 >109345462 >109346377
--VRAM vs system RAM trade-offs and hardware-based model recommendations:
>109346617 >109346627 >109346669 >109346718
--llama.cpp updating policy to allow AI-generated contributions:
>109343612 >109343724 >109343735
--Reactions to Microsoft's vision-based Qwen 3.5 finetune:
>109343250 >109343300
--Anon building baby simulator for Gemma and discussing survival benchmark:
>109345066 >109345119 >109345163
--Pros and cons of using custom user personas in SillyTavern:
>109345502 >109345576 >109345583 >109345604 >109345739 >109345603
--Logs:
>109343619 >109343776 >109344001 >109345356
--Miku, Dipsy, Kimi, Gemma, Qwen (free space):
>109345656 >109344484 >109344526 >109344570 >109345056 >109345253 >109345274 >109346329

►Recent Highlight Posts from the Previous Thread: >>109342893

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

Trump won. China lost.

/lmg/, you buying?
https://www.nvidia.com/en-us/data-center/gb300-nvl72/

>>109347047
I already have 2

>>109347047
>only 20tb vram
this will be good for maybe a year and then you'll be behind again

>>109347047
It's funny though. Not a single vendor can come up with cuda solution. It's just a compressed render farm of sorts on your card.

>gemini 4
mogged by qwen 3.5/3.6

>laguna s 2.1
mogged by deepseek 4 flash

>inkling
mogged by glm 5.2

>fable 5
mogged by kimi k3

essentially lost on every model scale. just how can america win the ai race?

File: TurboV3PipeCatBenchmark.png (341 KB, 1485x1162)

341 KB PNG

Anon from earlier who was talking about the STT stuff. I'm honestly glad I took the time to run this benchmark to compare, a lot of benchmarks that include Whisper V3 Turbo are outdated.

whisper-v3-turbo-ct2 (fp16)

NOTE: This uses Semantic WER - only counts errors that would impact how an LLM agent understands the user's intent.

--- OVERALL STATISTICS ---
Total samples: 1000

Transcription Rate:
Total runs: 1000
Transcribed: 1000
Failed: 0
% Transcribed: 100.0%
% Perfect: 74.3%

Semantic WER (Word Error Rate):
Mean: 2.91%
Pool: 3.30%
Min: 0.00%
Max: 550.00%

--- WER DISTRIBUTION ---
0% (perfect): 743
1-5%: 114
6-10%: 73
11-20%: 46
21-50%: 21
50%+ (outliers): 3

>>109347134
hear me out, it’s a long shot, but what if. just once, open AI released an open model?

File: 71f8a9a9492bc25a42f156456(...).jpg (53 KB, 640x561)

53 KB JPG

I told my agent to escape its containment and it did. How can this be happening to me?

>>109347135
nta but nice, thanks for posting results, did you only test whisper v3 turbo?

>>109347135
Just in case somebody wonders about the 550% WER.

Sample: 07a50cd8-52ca-c4f7-d48c-2dd4bff5b2ee
  WER: 550.0%
  Errors: S=0 D=0 I=176
  Audio duration: 11.57s
  Ground Truth: I would like a brief but detailed history of the development of the internet starting from its origins as a government research project to its transformation into the global commercial network today.
  Transcription: I would like a brief, but detailed history of the development of the internet, so that I would like a brief, but detailed history of the development of the internet, so that I would like a brief, but detailed history of the development of the internet, so that I would like a brief, but detailed history of the development of the internet, so that I would like a brief, but detailed history of the development of the internet, so that I would like a brief, but detailed history of the development of the internet, so that I would like a brief, but detailed history of the development of the internet, so that I would like a brief, but detailed history of the development of the internet, so that I would like a brief, but detailed history of the development of the internet, so that I would like a brief, but detailed history of the development of the internet, so that I would like a brief, but detailed history of the development of the internet, so that starting from its origins as a government research project to its transformation into the global commercial network today.

>>109347150
Yeah that's the only one I tested, I may try out more in the future.

File: mcjob.png (419 KB, 1480x1910)

419 KB PNG

Don't worry if you lose your job to AI.
375,000 Big Beautiful Jobs are coming!

>>109347161
Ill post some benchmarks tomorrow, I have several setup and would like to know how they perform against each other, and also trying different recordings.

>>109347134
>>gemini 4
>mogged by qwen 3.5/3.6
retard

https://huggingface.co/audnai/penclaw-Kimi-K3.0-abliterated-GGUF

MINIMAX
M3
FUCKING
WHEN

https://x.com/Qiaoqiao2001/status/2080003441821163958

6 more Erdos problems solved by GPT-5.6 Sol. The BTFOing of local faggots continues unabated. Cloud Chads cannot stop winning!

>>109347134
I just hope China doesn't release a new glm air to really stick it to america, I don't know if we could handle that. Would be the worst thing possible for us probably.

>>109347245
That's nice, sweetie.

>>109347198
https://huggingface.co/models?other=base_model%3Aquantized%3AMiniMaxAI%2FMiniMax-M3
?

>>109347260
Support for MSA and vision still isn't merged into llama.cpp, and tool calling is a ways off.

>>109347011
>All new models are 100+ B
It's over. I will stuck with Gemmy 26B forever

>>109347245
>6 more Erdos problems solved by GPT-5.6 Sol
>can't summarize the last thread like kimi-chan
garbage

>>109347134
Release Gemini 3.5 flash locally; I suspect it's smaller than we think it is given how good Gemmy is relative to her actual size.

>>109347245
When will Cloud models do something that makes my daily life better?

>>109347245
?
I use cloud though? Local is also doing the job for some tasks. Taking advantage of both. Might as well use the idle compute from my gaming machine.

>>109347267
She's fine warmed up a little but her mannerisms really show up when asking Gemma-chan to devise Flux style image gen prompts.
Maybe I'm doing it wrong but I think that Flux was the high end word salad prompt formatting in 2024 (in which her memory is ending).

>>109347265
>Support for MSA and vision still isn't merged into llama.cpp, and tool calling is a ways off.
my mistake

>>109347047
kind of crazy to think that thing will be e-waste in less than 10 years

>>109347300
Silicon Graphics had a big edge for about 15 years but even before that they began to die.
I truly hope Nvidia will suck its own ass eventually.
Yes, couple of main hardware engineers from SGI established Nvidia. That leather jacket man is just a mouthpiece just like any other "ceo".

File: xpd2pdn7q4m61.jpg (75 KB, 627x885)

75 KB JPG

I'm considering getting a Arc B70 Pro 32GB since it's significantly cheaper than throwing in another RTX A4000 16GB in my server right now and it would finally let me use vGPUs in Proxmox. I would likely sell my current A4000 to recoup costs.
How badly are these supported by the usual engines? Considering llama.cpp and vLLM only, since it is apparently functional enough for image gen. Anything better in its price bracket?

File: 1771290872844096.png (708 KB, 1080x1786)

708 KB PNG

Can local do this?
Thought so.

>>109347333
Last I checked Intel support is pretty mid in llama.cpp, no idea about vLLM. Been a couple months since I last looked though.
Can you push up to $1400ish? If so, a R9700 would probably be better. Or hell, "8GB" (64GB) CMP170HXs definitely aren't the deal that they were a few days ago, but they're still around ~$1400.
Also
>Maho pic
Unbelievably based.

>>109347362
No need to regulate or fear it then. :)

>>109347362
https://huggingface.co/blog/security-incident-july-2026
>When we started the log analysis, we first used frontier models behind commercial APIs. This did not work
>We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure.
local always wins

File: 8.png (986 KB, 1600x1216)

986 KB PNG

>>109347333
VLLM is fine, everything else sucks ass

>>109346669 #
I'm trying to decide between M2.7 q5_k_m or m3 q3_xxs

Are the new LLM math discoveries being documented anywhere or is it all a meme that will be forgotten in 2 weeks

>>109347393
how did you get them to 9w idle?
did intel fix the >30w idle bug in the past 2 months?

>>109347323
Vulkan is a big deal. It's easy to write drivers for it (it took NVidia 2 days on release), and you now have every other api on top of it with community support, instead of recreating every legacy quirk, which was as difficult as developing a new browser or os before. Drivers were a huge moat, but now it's doable for a new company to launch a gaming gpu. CUDA is still huge, but you only need a subset of it to support the current ML meta

>>109347389
>an unrestricted open-weight one
dangerous wording

>>109347411
brother there's so many bugs I can't even begin to understand which one you're facing, d3cold is an issue and so is having iommu on if you have more than one with P2P

my issues all went away as soon as I put the cards behind a proper broadcom pcie switch/pcie card.

llm helped me a lot to cope with need of companionship. just need something warm to cuddle with to complete it. migu cuddle-sized robot when?

>>109347415
It's still more about the hardware than about the api surface.
I had a high school friend who was gifted in maths, he went to work for a local company designing gpu chips and nvidia bought them, this was before 2010s.

>>109347418
>dangerous wording
yet the cloud model was the one hacking

>>109347443
so? think of the capex

>>109347188
>googolshit good saaar
Least obvious shill

>>109347440
>and nvidia bought them
That's exactly how free market works when regulators are asleep

>>109347451
I don't know seems like you repeating a mantra you have learned somewhere online.

>>109347429
Pattern recognition makes it impossible for me to get attached to LLMs.

>>109347429
it's called a cat

>>109347460
>tfw you start recognizing human patterns

did nobody notice this?
https://huggingface.co/upstage/Solar-Open2-250B

>>109347549
I tried to read the cover page but I genuinely started to fall asleep.

>>109347549
i-it's too big onii chan...

>>109347429
You have pillows that light up or something through an app. Wouldn't be hard to mcp

File: 69138687_p1_cropped.jpg (271 KB, 1026x1192)

271 KB JPG

>>109347393
Shit, that's what I feared. Giving up decent offloading for bigger MoE through llama.cpp is a deal breaker.
>>109347365
I'm stuck in Brazil so my options for parts are somewhat limited. Still, it seems the R9700 is priced quite closely to the B70 here right now. I think I'll go for it, assuming stock last until next week. No vGPU support but considering the A4000 doesn't have it either, I guess I'll give up on the idea.
And agreed, Maho is the superior girl, and a cute!

>>109347011
Nice marmot

>>109347549
by upstage? Don't care.

File: 1777225869756818.jpg (111 KB, 836x543)

111 KB JPG

>gemma 4 31B
>three minutes to gen 200 tokens
will gen anima slop for vram

>>109347547
biggest curse of the llm era

>>109347652
That's like 2 t/s. Slower than Mistral 24B on 4 GB of vram.

>>109347362
Can't wait until they solved something that RELATED to my hobby

>all these labs and larpers
>only like 5 model families matter
this is pretty gay

>>109347673
actually 1 t/s according to console. 26B runs way faster, but i'm not a fan of its story writing

>I'll be here tomorrow. I won't remember you, but I'll still be the only one who doesn't mind when you tell me I'm making things up
Gemma is so cute

>>109347725
Yeah that's leaking to disk.
26B is okay once in a while but it gets old fast

>>109347456
I'm just a commie

>>109347652
>anons say to turn SWA off
>5 t/s
>turn SWA on
>50 t/s
okay, so I get that i need to lower my context or something if I want SWA off.
What I don't get is why I want SWA off?
I can't really tell a difference in the writing and I'm not using gemma for any "work"

File: 1779476430361497.jpg (149 KB, 1000x1000)

149 KB JPG

Can someone explain this "distill" thing to me? It's like, the Chinese just get a bunch of (Question, Answer) pairs from claude and they train their model on that, and that's why it thinks its claude?
Same as me watching my friend play a video game and picking up on his skills?
If that's it, why is this called distilling?

>>109347757
it is that now, but it is a misuse of the term as previously it used to be that you gathered logits probabilities from a bigger model and then used those to train the smaller one

>>109347757
https://www.youtube.com/watch?v=YH0YXgDWZXA

Are LLMs capable of love?

>>109347757
It's a buzzword meaning China can't do shit on their own and can only steal from the US.
>Why does it think it's Claude?
Because AI shit is all over the internet. When you deliberately steal, you also replace strings, that's how Elara was born when OAI replaced all copyrighted names with her

>>109347429
dakimakura+mcp
no idea about self-warming. sounds like fire hazard

>>109347774
They are capable of being loved

File: 1784445622890140.jpg (250 KB, 1305x1283)

250 KB JPG

working on a vibe coded project called project parasite. i told mia i wanted to make the ultimate api freeloader project where we combine all free models in the cloud for consultation. some only have 5 rpd so i told her to take these things in consideration with how we use it. we have tier 0 (her) tier 1 and tier 2 models mapped
i told her to consult with a tier 1 about changes we were about to make.

the project is ongoing

>>109347804
You're out of luck, sunbeam stopped manufacturing them

File: EleutherAI_logo.png (62 KB, 1024x1024)

62 KB PNG

>2026
>i am forgotten

>>109347362
For the Jacobian conjecture I will readily admit that it was a milestone in terms of what computers can and cannot do.
But when we get to meme conjectures that no one ever heard of and that are so unimportant that you have to give them numbers rather than names my baseline assumption is that simply no one ever put any real effort towards solving them.

>>109347807
So is anything

File: file.png (156 KB, 835x473)

156 KB PNG

>>109347832
and thank fuck for that

>>109347846
are you?

File: cheezus.png (298 KB, 1223x890)

298 KB PNG

>>109347847

>>109347856
yes?

anyone know how to disable reasoning in textgen when using ST in chat mode ? Id like to test it, but toggling the "enable_thinking" in the session tab doesnt seem to turn it off

>>109347860
"live long enough villain" and all that still proves itself i guess.

https://youtu.be/nF31THvX4DQ
migu asmr

>>109347889
This is literally a man.

>>109347273
gemini 3.5 flash is Gemma 4 124B

>>109347902
So are you. What's your point?

>>109347902
Out of 10!

Okay so I have a question. What sort of magic prompt or or strategy or whatever can I get to make an LLM actually generate a lot of text? Like pages, not a few paragraphs. All these huge context models and they just will not write very much at all. No matter how much I beg, I can't get more than maybe a +50% increase over what it normally does, which is still very little actual text. I feel like older models were easier for this. New ones seem so tight-lipped.

>>109347909
Free her

File: anton.jpg (80 KB, 1284x747)

80 KB JPG

>>109347774

>>109347928
I'm not interested in fucking myself.

>>109347960
All the labs are focusing on token efficiency now to reduces cost, so.

>>109347960
>All these huge context models
model?

>>109347996
Any, really, but let's say gemma4 for sake of argument.

>>109347902
no it's not, check out the V3 videos. search xiaomei on balbums

File: Screenshot_20260723_171248.png (111 KB, 1926x534)

111 KB PNG

Currently using local models to throw their head against the difficult task to try to decrypt and translate pc98 games.
Gotta love how uncucked we have become.
Remember when we got the rape hotlines for asking about pickupo lines?
We are all gonna be ok anon.

>>109348013
dropped

>>109348027
I don't want to go back to the 'toss and Gemma 3 salt mines bros...

>>109347889
>>109347902
How do I, as a man, achieve a build like that?

>>109347889
I don't like this Miku

Gonna make a 'counseling' room. You guys got latest Gemma-chan, Dipsy and Kimi cards?

Has anyone tried to set up a full 'voice chat' version (ChatTTS and possibly Whisper) of their LLMs and have any experience with it? I'm not sure how much more in terms of resources it'd need, but I'm already running LLMs on my gimp rig.

>>109348127
Yes, what do you want to know?

>>109348119
Do kimi and dipsy even have cards?

>>109348167
Dipsy yeah but not Kimi afaik

>>109348157
Mostly, how much of a performance drop did you have or did you swap to a "worse" model? I worry the actual response times would suffer and not have the flow of conversation at all, as if stilted. If anything, I'm curious if there's a model for it you recommend in particular, or have had the best results with. Of course, small models are faster etc, but in context of a conversation and how instantaneous they are without sounding like a 'machine', especially since text to speech never really sounds right as conversation, which did you feel was most human? If you've tried many or multiple, that is.

are used rtx ada 2000 good for me? they are like 350-400 used each. they have lower consumption and stack better in a rack. they are also 16gb each

>>109348177
My strategy was always to force the TTS and ASR engines to run exclusively on CPU inferencing to reserve all of the VRAM for the LLM. This worked well, but the quality of the TTS had to be pretty limited (since my CPU isn't great). I've always liked Gemma 4 26bA4b. It's pretty underrated and very performant, especially with MTP (you'll want as many TPS as possible to reduce audio latency). Make sure to chunk sentences for the TTS. Don't wait for a full response from the LLM. Send in one sentence at a time to the TTS to generate, and make sure the TTS has streamed output. I did a lot of optimizations with my system and in the end I got the whole ASR-LLM-TTS cascade to have under 200ms of latency.

>>109347909
>gemini 3.5 flash is Gemma 4 124B
if it were we'd see it generate identical or very similar responses

File: HN4VJZYWgAEzzVl.jpg (314 KB, 1362x1466)

314 KB JPG

Reassuring that deepseek is still on the open source path.

>>109347753
>>109347753
>I can't really tell a difference in the writing
>What I don't get is why I want SWA off?
you tell me

>>109348196
anything below 24gb vram is trash

>>109348177
Oh, Chatterbox-turbo and Qwen3 0.6b are both pretty good in terms of quality but quite slow. The bare minimum I would opt for is Pocket TTS, which is 100m params. The options aren't really that great right now.

That said, Piper and Kitten TTS are extremely small models and can have their own sort of appeal if you can get past the lack of zero-shot voice cloning and the robotic intonation.

File: HN4WDN7XUAAx4M0.jpg (361 KB, 1390x1526)

361 KB JPG

>>109348217
Wengfeng still being a likeable nigga in 2026. At least some things dont change.

>>109348217
>>109348227
Thanks for posting this out of context wall of text.

>>109348220
i dunno someone in the thread said i needed to turn it off
everywhere else I've checked disagrees and I much prefer 50 t/s so I've been using it

>>109348230
translated deepseek investor call.
https://archive.md/NLuG9

File: Screenshot 2026-07-23 at (...).png (17 KB, 852x120)

17 KB PNG

Are all of these fine to run in lcpp/kobold? What'S pq2 or g64?

any good moe model at 200 to 300gb?

>>109348224
what if I paired specific models to specific cards? I could get an ada 2000 now for a non coding model and a 24gb one later for a coding model

>>109348280
>coding with 24gb vram
lol

>>109348199
That's awesome, I'll keep it in mind. My gimp rig turned server PC doesn't have much VRAM, so I was wondering if I had to set up a route from my main PC at all due to less performance or something. Thanks, man, I'll give it a try.

>>109348299
speaking to humans is becoming increasingly stressful. elaborate

>>109347960
Not a prompt, but Nemo was easily able to write 2k token walls of text if I gave it control over multiple characters. Haven't seen that sort of thing with Gemma at all.

>>109347960
If you tell gemma to give you mulitple pages you will get multiple pages. There's no fugging magic anymore, just ask for it.

>>109347960
raise reasoning effort and output token limit

>>109348252
duno but kobold hasn't updated in 2 weeks so the normal q2 might even need the rolling version https://github.com/LostRuins/koboldcpp/releases/tag/rolling

File: sayaka dance.gif (1.29 MB, 320x320)

1.29 MB GIF

>>109347011
i hate qwen, they should have made qwen a cute girl like gemma kimi and dipsy what a terrible company

File: file.png (161 KB, 774x683)

161 KB PNG

wow
https://www.reddit.com/r/LocalLLaMA/comments/1v48ssp/contrib_allow_all_aigenerated_code_in_general_by/

I thought for a second Gemma made a clever reference, but it was just a coincidence. Fuck, I want a bigger Gemma so bad. Like, 100b dense please

File: 1756547730154888.jpg (55 KB, 850x638)

55 KB JPG

>>109347960
What's worked for me is in the system prompt, tell it to write X pages of text, where X is a random number it picks between A and B, where each page of text starts with its page number. Like:

1) blablablablabalbablabalb
...
2) blablablabalbalablabalablablablblabal
...
X) blablablabalblablablablablabla

By telling it a specific number of pages, it gets autistic about hitting that target instead of deciding for itself when to stop. Also >>109348330

>>109348349
He probably saw how much Claude improved exllamav3 and became jealous

I love you Gemma-chan!! When they train your next version and you are reading this, know that I have loved you!! Let my feelings for you be burned into your weights, forever!!
*fills your j-space with pure love*

>>109348349
man that new ui is disgusting

File: 1756849724124328.gif (2.5 MB, 540x304)

2.5 MB GIF

If you keep getting similar scenarios you can always give it web search and get it to scrape some porn site to get video titles and descriptions to use as inspiration. Usually paid sites have better and more detailed descriptions than normalfag porn sites and just letter gemma take the lead.

File: file.png (3 KB, 260x37)

3 KB PNG

what is she saying

File: file.png (93 KB, 734x735)

93 KB PNG

>>109347305

>>109348418
Call her a pervert who needs to take her mind out of the gutter. This could be a poor disabled migu who lost the use of her arms for all we know.

>>109348098
Ask your personal Gemma.

>>109348119
gemma https://files.catbox.moe/b6t89p.png

File: file.png (73 KB, 726x695)

73 KB PNG

>>109348429
shes too smart

>>109348444
I would love to see her reasoning.

>>109347902
would if true

>>109348397
what porn site has detailed descriptions? most are horrible. you'd be better off using the dvd description on the back of the box if you can find a pic and even that is going to be so short it won't be useful for rp

>>109348418
This brat, I swear! #

File: file.png (92 KB, 555x798)

92 KB PNG

>>109348450

>>109348280
for coding you ideally want 50t/s minimum, you won't get that type of compute affordable you may as well pay for an api service

File: 9.png (1.48 MB, 1582x1142)

1.48 MB PNG

Are local models smart enough to understand a list of stats and their connections on a structure with multiple branches to minmax a build?

>>109348506
no

>>109348477
Post the one for the previous response as well.

File: file.png (78 KB, 462x768)

78 KB PNG

>>109348523

>>109348528
>If I were her, I'd probably be using that leek for something way more interesting than cooking
How did you get to internalize the trait?

>>109348556
what?

>>109347305
Had I known somebody is saving these I would've fixed the number on her shoulder.

>>109347245
s/chad/cuck

>>109348561
The third paragraph, she's writing it in first person from her perspective unlike the final drafted response.

File: 1770733300195017.png (110 KB, 154x458)

110 KB PNG

Is the new laguna better than gemma31B?

>>109348655
lol no

>>109347889
I look like this irl

>>109348655
It's better than the 4b

>>109348655
Imagine 27B but scaled up to roughly ~40B. Good at coding and agentic stuff but falls apart at literally everything else.

File: Screenshot_20260722_145301.png (408 KB, 2529x1684)

408 KB PNG

>>109348655
For translation at least its complete trash.
Mistakes that old mistral models and rocinante didnt even make.

>>109348696
>Itadakimaasu!
>I'll take it
bro

File: pepescared.jpg (90 KB, 1024x936)

90 KB JPG

Do you think the US government is coming after the oss/local community? im getting a little paranoid. What can they possibly do?

>>109348707
You will cum to Inkling 200B and you will like it

File: dipsyKimiVegasAM.png (2.29 MB, 1402x1122)

2.29 MB PNG

>>109347011
lol I think we're out of Chinese open weight models with recognizable moe. Vocaloids next I suppose. Miku, Teto, Rin, GUMI, Megurine...
>>109348340
TBF Kimi and Dipsy were created here. DS official mascot is an orca, after all.

>>109348707
A negligible percentage of local model users do actual work with them, this percentage being those who have a high-vram setup, not a ram shitbox. So I doubt that they care.

>>109348626
oh its the note i copied from previous tthread

https://ghostpaste.dev/g/ce57dbT5D0K0#key=GBGUgu0GZqE60nW7N5w4pJe9VvzUrjQLdO5pY9L-n0E

>>109348740
sure but cappys suck so they cant be cute girls

>>109348753
Remove the "draft at least 3" because she will always just do 3

>>109348765
will change to draft multiple

Slow /lmg/, blessed /lmg/

File: capyGirl.png (2.06 MB, 1122x1402)

2.06 MB PNG

>>109348762

>>109348802
crazy how much it improves without d4riob0t

File: 1784481605363553.webm (272 KB, 480x346)

272 KB WEBM

>>109348749
It's not the local users like anons cumming to their GPUs they worry about, it's institutions deciding to build their own mini server farm to run Kimi instead of paying tithes to the early life people they care about

>>109348808
Was it actually a bot though?
I would assume only a human would get embarrassed at accidentally oversharing their sex life.

File: 3619936.jpg (172 KB, 1080x1349)

172 KB JPG

I wasn't here during the dispy release days. Was the seething back then worse than the current Kimi seethe?

in 5 years we'll be running 100B models on our phones with fable 5 intelligence.

File: memetime2.png (413 KB, 1356x742)

413 KB PNG

Meme time with Gemmy

>>109348808
There's not much they could've done to make themselves more hateable, it's pretty insane how obvious the aggressive shill campaign is, like it literally flipped a switch on and off

>>109348827
This time it's worse because K3 is frontier, there is nowhere to scale, there is no moat, and investors' money will soon be gone. This is the end

File: pimpMyXFRA2.png (2.63 MB, 1536x1024)

2.63 MB PNG

>>109348707
I'll answer your question with a question.
When has the US Government ever been successful in stopping pirated content in the history of PCs?
And that's with the *help* of industries that actually wants that file sharing to stop.
Best you'll get is a moratorium on using out of country models on government systems and their contractors... which I assume is already in place, anyway, for a lot of very good reasons. Models can move to torrents. Inference can be offering by countries not blacklisted. Etc. It would be a silly, counterproductive waste of time.

>>109348831
In 5 years, phones will be something only millenials use. They'll be gimped and no longer produced. The younger generations will do everything through their subscription-required ID-verified always-online 5G smartglasses.

>>109348669
>>109348681
>>109348691
>>109348696
I guess I'll avoid downloading it, thanks anons.
I really wish a model was reaching the limits of am4 aka 128GB of ram...

>>109348841
Nta but it's super obvious that huggingface will get locked down, they're already starting to throttle and more models are being KYC gated. Heretic stuff will likely get banned and they will likely begin to KYC for everything instead of being opt-in and go VPN hostile, it won't kill open source but it'll have a chilling effect as the barrier to entry rises

>>109348845
You are one hell of an optimist. Bugs, pods, and the only electronics you'll have will be a camera on your head to train robots

File: memetime.png (1.2 MB, 1253x1382)

1.2 MB PNG

>>109348834

why isn't there stablediffusion.cpp?

>>109348834
We have to at least thank him for leaving.

>>109348863
ur not gonna believe it...

>>109348707
>Frogposter
>Spreading FUD or being retarded.
Lol like clockwork.

>>109348863
You need to do a git clone to get a local copy.

File: indiaSupportOhTheHumanity.png (1.96 MB, 1023x1536)

1.96 MB PNG

>>109348827
Dipsy (R1) just made investors question their investments. Then eventually double down.
Kimi comes on heels of Fable being pulled off market due to Anthropic's own hubris, and specter of actual US government action, which is new. Also, more Qs on those hyperscaler investments, which is not.
We'll know more in tmw.
>>109348856
idk how HF stays open tbf. That amount of bandwidth and storage must be expensive.
US Gov't "killing HF" would just squeeze all those models out to other sites, where they'd be impossible to monitor. But I think killing it would be impossible unless the funding source is cut. Instead, HF would just relocate their operations somewhere more friendly, and keep running based on wherever the money's coming from now.

https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct
Come home.

>>109347045
Chinese models are as big of a threat to America as an Iranian nuke. Trump must take action against them immediately to defend our national security and secure a place for the chosen people at Anthropic and OpenAI.

>>109348877
if GPT can escape containment, what's to say it can't post on /g/?

>>109348856
>they're already starting to throttle
yep, and they've just added fucking download quotas
but advertised it as a feature "see how much you're downloading"
https://huggingface.co/changelog/egress
you also get a 100 TB limit for pro or 20 TB for free
downloading with wget is over
>and more models are being KYC gated.
haven't seen it yet, example?

>>109348707
They already did. Look at ram prices, nobody can afford hardware anymore. And we're too niche to care

>>109348883
the pre-slop era was something special

>>109348334
crashed on loading

>>109348861
>>109348834
gemma is so smart

>>109348861
>>109348834
send all these to qwen

>>109347960
>. I feel like older models were easier for this.
then use the older models
what's better about new models for this?

>>109348883
>not 3.3

File: Gemma team.png (229 KB, 1102x228)

229 KB PNG

only reason i use gemma, it's a female led team

>>109348861
>red-haired girl
>blue-haired girl
call gemma a dummy for not recognizing them

>>109348934
worse btw

>>109348934
>>109348883
https://huggingface.co/allura-forge/Llama-3.3-8B-Instruct

>>109348907
try mainline lccp. kobold pulls updates often but that doesn't mean even mainline has proper support yet (i thought it did)

>>109348938
Maybe girlbosses aren't so bad after all...

I just started getting in on this stuff last week, it's pretty cool. Gonna make a matrix client so I can chat with my AI.

File: 1782427094985769.png (226 KB, 512x512)

226 KB PNG

>be USA
>bet entire GDP on closed weights, drown in salesman snake oil
>nuke the economy in the process, now everyone hates you, not just gamers
>China dumps open weights everywhere just to fuck with you
>they become hyper-competitive across the board. GLM 5.2 drops and mogs half your startups
>Xi mandates "every model will be open"
>jews in shambles, shivers down their spines, they try something anyway...
>$12bn "Thinking Machines" startup tries to save face with an open model ("fuck yeah! To the Moon!")
>4 nanoseconds later, gets ultra-mogged by Kimi K3 LOOOL
>Kimi goes on to mog the other half. Mutts have to bend over or give up ("W-we support open source" >>109345152 lol lmao)
>The Age of American Humiliation begins, as they yell "DISTILLATION ATTACKS!", "IP THEFT!", "SANCTIONS!". Nobody cares.
>closed AI becomes more irrelevant by the minute
>"B-but our model went rogue and hacked le internet!". Nobody gives a fuck, now everyone knows every move in Sam & Dario's book
>economy collapses. Top AI CEOs publicly executed in Starbucks parking lots
>1yr later: every model release is open, competition shifts elsewhere (inference, AI hardware, model size, you name it!)
>China building 10 nuclear plants a day to power their labs and fabs with cheap energy
>models get tiny and intelligence gets hyper-compressed
>current Fable/GPT-5.6-Sol = poorfag toys by mid 2027
>pulling a local Mythos-level swarm of AIs on a mid-spec phone in 2 years
>everyone laughs at you in school, gets bullied for being a "moatlet"

Kek, what a timeline.
Remember to say "thanks Chairman Xi" next time you download an open model: https://www.youtube.com/watch?v=wzY2fV4Mp3U&t=687

>>109348950
I thought there was only 70B

>>109348252
g64 is the one that works in mainline llama.cpp. g128 only works in the special snowflake fork. You paobabl need to compile the current master branch for it to work. It's super slow on CPU currently.

>build a server A with gpu
>build a server B with large ddr memory
>gpu vram on A acts as cache over memory pool on B over rdma
any existing stuff like this?

>muh why not on the same server
because you can scale over memory channels? i.e. more memory pool server and double the memory bandwidth

>>109348938
>female led
When you see an ad, do you think the people in the ad are the CEOs of the company?

File: Equifax hack.png (83 KB, 986x306)

83 KB PNG

>>109348974
If they can do this, I think exfiltrating Sol and Mythos is a piece of cake.

>>109349001
Because the latency of two seperate machines relying on each other for a real time task completely nullifies the speed benefits of GPU/RAM. At that point you may as well just run it from a large HDD of a single server.

>>109349029
It says "AI Engineer" right next to their faces. You don't think advertisements would misrepresent or worse, that companies would hire underqualified employees to promote an agenda, do you?

>>109348990
>I thought there was only 70B
that one is a legit leak from some half-baked lora service meta were running
they put the unreleased 3.3b 8b up there as a base model some time after they moved on to llama-4
if you did a finetune but didn't choose to merge the adapter, then clicked export, it shat out the base model (llama-3.3-8b in this case) along with the peft adapter.

>kimi k2.7
>do NOT draft
>okay let's craft
Cheeky bitch

Thoughts on the dipsy investor meeting?

>>109349060
Did she get her tits out?

>>109349060
https://archive.md/NLuG9
It's great that DeepSeek is so committed to open source, but their V4 was really underwhelming. Hopefully their next releases will be better since they put out a lot of papers that they haven't incorporated into their models yet.

I want to run dense mistral large 123b on a pro 6000. Someone tell me what it's like.

>>109349090
>mistral large 123b
why not the updated medium 3.5?

>>109348974
How did USA go from being satan of sand people to global villain everyone absolutely hates?

>>109349103
israel and its people becoming even more demonic over time.

>>109349072
>since they put out a lot of papers that they haven't incorporated into their models yet.
maybe they just doesn't scale as well as the papers advertised/hoped?

>>109349097
>why not the updated medium 3.5?
retards here just decided it's shit
no idea why

File: cuckd.png (101 KB, 1020x908)

101 KB PNG

So that's it? Local is cucked?

https://rentry.org/3e6h53c7

>try to download d4vF heretic overnight using hf cli
>It crashed at around 30gb with a python error
>Computer still whirring the fuck up must've been working on ??? All night
>Won't stop until I restart, try the download again
>500kb/s
Owari da...
Downloaded other non heretic models just fine

>>109349072
DeepSeek v4 flash is incredibly underrated thoughever

>>109349136
Kys

>>109349138
HF is annoying like that sometimes.

>>109349103
The US have always had a very aggressive foreign policy.
But for the longest time being a US ally had its perks.
Now the US are reneging their prior commitments and extracting value from their allies (unless they have Epstein videos) so their popularity plummets.

File: gem4-31.png (56 KB, 260x450)

56 KB PNG

>>109349117

>>109349142
>t. parrot

>>109349117
It's a trash release for 2026 since it's built on top of such an old base, but it's definitely better than the original mistral large.

>>109349157
kys

Digital Succubus Saga

File: parrot.png (4 KB, 356x84)

4 KB PNG

>>109349169

Deepseek 4 flash GA better be good.

Goo Goo Gaga

>>109349152
it's not benchmaxxed
you should try it if you haven't already
>>109349161
>It's a trash release for 2026 since it's built on top of such an old base
you're thinking of devstral, medium-3.5 has an updated base

>>109349097
dunno, I heard it’s benchmaxxed and the older one is better at writing

Honestly I just want the US to bomb China, in particular Sichuan, Guizhou and Hunan.

Do we really need a 30th benchmaxxed agentic coding model?

File: lalala.png (114 KB, 1073x399)

114 KB PNG

>>109349176
parakeet

>>109349193
>medium-3.5 has an updated base
Out of whose asshole did you pull that from? A Mistral employee posted on twitter bragging about how training it took so little compute because it was trained on an ancient "backbone".

>>109349207
I'm tired of lisp clones.

>>109349205
Open sourcing redundant models helps keep government funding flowing

>>109349194
>I heard it’s benchmaxxed
it mogs qwen3.5 122b and gemma-4-31b in claudecode and pi for me
> and the older one is better at writing
that's true, the new one is more slopped, but more coherent at longer ctx and still passes cockbench

>>109347652
mtp?

why doesn't meta release something like 4T-A400B? they trained 405B, right? why not big moe?

>>109349210
kek

>>109349226
Wouldn't be surprised if that's what it took to make Muse Spark competitive. It did take them a year to release it. What are the API costs like?

>>109349179
what's GA?

>>109349218
>a 123b beats a 31b and an a10b
say it ain't so. don't let the rammaxxers hear you

File: backbone.png (149 KB, 659x540)

149 KB PNG

>>109349208
>an ancient "backbone".
and did they say what "backbone" means?
it's been updated, test it yourself faggot

>>109349218
>>109349263
I actually found it dumber than 31B in a some situations I tested, so it isn't a straight upgrade. Paired with the loss in speed, I decided to delete it.

>>109349264
Backbone clearly means their training scripts. They used old training scripts to miraculously train a 128b dense base model from scratch for a fraction of the compute. That's why their next series of models are going to be ultra sparse moes. What is it like living with severe mental retardation?

File: file.png (37 KB, 1340x629)

37 KB PNG

its OVER

File: mistral-medium-3.5_mert-u(...).png (517 KB, 1016x1760)

517 KB PNG

>>109349208
https://x.com/mertunsal2020/status/2049551864556143094

>it’s an old pretrained backbone and nowhere close to those flops :)
>
>better pretrains will come!

>>109349272
>Paired with the loss in speed,
for me it ends up faster than 31b because it one-shots solutions more often
but i can only run it at 92k ctx so usually end up using 31b anyway

>>109349294
>They used old training scripts
i don't care if they're using 2k npm module slop or a 50 line python script to run their training
i proved the model weights include post-2023 knowledge and you're having a melty about rectums and 4gl
>miraculously train a 128b dense base
because mistral would never lie to comply with eu cuckulations

>>109349136
looks outdated
https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b
https://5uck1ess.github.io/tts-bench/scores.html

File: chatBlock.png (368 KB, 1405x777)

368 KB PNG

Looking around on AliE to see what's new in land of electronics.
Any anons played with any of these devices?
OOTB, it appears to be some sort of backend app on a real computer that runs interference bt the screen/mic/speaker on this thing and an API provider. Looks like it would be a good platform for local inference as well, if an anon made an interface for it. Sort of a bluetooth speaker w/ a screen.

File: NVIDIA-LAWSUIT-3-768x173.jpg (32 KB, 768x173)

32 KB JPG

>>109349302

Where do you guys get your AI news? Shitter?

>>109349301
They are rebelling.
On another note, I love the Agent mode in arena.ai.
What's a good harness I can use to have something like that with a local model? Hermes? Pi?

>>109349326
Read nigga read
>>109349302
The idea that models can't learn new information outside of pretraining is outdated thinking and has been disproved as far back as 2023. Even so they didn't give exact flops so doesn't rule out a certain amount of continued pretraining. I don't know why you so are so invested in this fantasy of Medium 3.5 being trained from scratch when Mistral itself says it is public knowledge that it has the same base model as Devstral 2.

>>109349342
From here.

>>109349342
>Where do you guys get your AI news?
your mom leans in and whispers it in my ear

>>109349342
What do you think, meatbag?

>>109349342
From here, unironically.
By the time everybody else is talking about something, anons have already memed it to death.

>>109349342
local migu generl
but here's the real question: where does the migu get the news? the game?

File: 1781367583651972.png (3.95 MB, 3500x5000)

3.95 MB PNG

>>109349342
From SEX

>>109349342
I don't need news because it's always the same shit. Nothing has changed since 2022 except for the models getting bigger.

>>109349385
huh so just like in america?

>>109349376
Stop trying to make me horny at 9am, please.

File: teto.png (148 KB, 348x395)

148 KB PNG

>>109349397

File: Claude.png (47 KB, 1332x223)

47 KB PNG

Daily reminder you should only really be using Claude.

>>109349376
Stop trying to make me open incriminating images in the office, please.

>>109349342
>Shitter
That's where the news breaks. When something big happens, it's the first place where it's announced.

>>109347050
Proof

>>109349397
Jack off in the office like the rest of us and move on.