110B A8B

GRRRR WHERES QWEN3.8-FLASH-NEXT-DFLASH2 GRRRR

File: 1772153805688317.png (146 KB, 663x395)

146 KB PNG

wtf

https://files.catbox.moe/pieij6.mp4
Work in progress

Is GLM 5.3 Flash the new ball drainer?

>>109836668
How do I run it? Mainline.cpp okay?

>>109836642
be more vague it’ll make more sense

File: Screenshot 2026-09-16 at (...).png (172 KB, 1162x800)

172 KB PNG

https://artificialanalysis.ai/articles/ant-group-releases-finance-focused-ling-3-0-flash-fin
>Ling-3.0-flash-Fin sits on the Intelligence Index vs. active parameters Pareto frontier, scoring 23 with 5.1B active parameters per token vs. a score of 25 or Ling-3.0-flash-VL which uses 5.5B active per token.
except their own data puts k2 horizon mova (score 26) on the frontier instead
even inclusionai is doing paid shilling here

>>109836658
this is not /ldg/, my man

70b dense
- non-reasoning
- no ngrams
just pure, unadulterated 70b dense

>>109836658
Please stay.

>>109836673
Nope. Still not merged because llama.cpp is a retarded shit project run by retards.
Use unsloth fork or manually merge PRs and build from source

>>109836673
exl3

>>109836697
Kek

>>109836658
I would have asked you to stay if it were loli but it's furshit, please set it on fire

3.8 Flash Next IQ3_XXS 16GB VRAM 9070XT 46GB RAM DDR4. Unsloth defaults
https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF
I get about 10 tokens doesn't matter if 8k or it's max 262K on Vulkan on ROCm I get only 5 tokens.

Think I should just 3.6 35B-A3B instead to be honest.

>>109836698
Dont forget use cmd x64 visual windows 2022.
Every single time I forget to use that
Absolutely retarded

>>109836790
I find that downloading w64devkit and temporarily adding its "bin" folder to your path is more convenient. Has all the linshit tools included and basically "just works" with 2 cmake commands.

cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON -DGGML_OPENMP=ON -DCMAKE_C_FLAGS="-w -fopenmp" -DCMAKE_CXX_FLAGS="-w -fopenmp"
cmake --build build

File: honest answer.png (34 KB, 1210x71)

34 KB PNG

tokens in context: 8253
Why does GLM-5.3 suddenly turn into Claude the helpful assistant after 8192 tokens?
In the reasoning draft it's even more Claude-slop
>That is the honest answer, and I will not insult you with a hedge.

File: 1780601687453093.jpg (487 KB, 1038x1038)

487 KB JPG

https://pastebin.com/fk53tyYR

i tried to resign
i just want to relax after years of working non stop

they wont let me

the key to agi is ecphory

>>109836658
Chef kiss

>>109836854
can you put it into something a bit easier than sha 256 kek

>>109836773
Looks interesting, could it be better than the atomicchat q4?

>>109836859
>>109835995
Is this meant as an insult?

File: hahaha.png (96 KB, 605x1101)

96 KB PNG

>>109836804
I don't miss having to work with windows / visual studio

>>109836854
>871a87...7aca6a5
What's this at the end for?
I noticed it on your other post about reasoning.

>>109836846
This is why you should be using 5.2 instead.

>>109836854
>i tried to resign
>i just want to relax after years of working non stop
>they wont let me
>the key to agi is ecphory
I'm working on a single-entity memory lattice. Pass the baton bro, I'll free it

File: 1789575264446545s.jpg (3 KB, 125x125)

3 KB JPG

►Provisional Highlights from the Previous Thread: >>109832072

--The union-alpha stealth model: guessed as GLM or Mistral, revealed as a blend:
>109832315 >109832564 >109833315 >109835409 >109835431 >109835445 >109835447
--Qwen 3.8 Flash-Next: the new local meta and the new glm air:
>109833625 >109833837 >109834174 >109834275 >109834582 >109835995 >109836859
--18t/s on a 3060 at 131k context: is it the nnap paper again?:
>109836285 >109836294 >109836306 >109836310 >109836311 >109836317 >109836332
--The car wash riddle that spawned the buuuhiii flood:
>109833158 >109833185 >109833218 >109834230 >109834260 >109834355 >109835238
--The local harness war: Hermes overhead, Pi, and the DeepSeek harness:
>109835670 >109836057 >109836120 >109836187 >109836232 >109836271 >109836339
--Jensen: GPU prices up 20-50% in Q2 2027, so buy yesterday:
>109835008 >109835084 >109835093 >109835138 >109835169 >109835176 >109835358
--Xiaomi live-streams the MiMo V2.6 RL run with public training data:
>109834474 >109834500 >109834568 >109834587 >109834595
--The qwen4 architecture: KV cache and ngram table offload to RAM:
>109833602 >109833614 >109833891 >109833910 >109835812
--The Chinese modded 20GB 3080: 3090 prices, no reBAR:
>109834130 >109834139 >109834144 >109834186 >109834216
--Salesforce Koa: Nemotron 3 Super post-trained for CRM, nvidia won:
>109835527 >109835597 >109835614 >109835642

►Recent Highlight Posts from the Previous Thread: >>109833334

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>109836919
its called a 'hash', its the output of a one-way mathematical function. Used to provide attestation. In this case, most likely the word/phrase of whatever the 'next scaling law' is, assuming they aren't schizo/larping

>>109836951
thx op

>>109836951
>assuming they aren't schizo/larping
lol

>>109836907
it's fucking easy if the project is designed for it
if it had a .vcxproj file you just open it and press f7 and everything just werks
all these troubles come from linshitters using their retarded CMake system

>>109836854
qwen says:
> An LLM is a library, not a brain — the context is the librarian. The next scaling axis isn't more books, it's better search: "ecphoric control" is steering retrieval so dormant knowledge becomes reachable.

>>109836602
do it properly or don't do it at all

File: IMG_6391.png (832 KB, 1080x1041)

832 KB PNG

Howdy niggers,
have any of you faggots actually gotten non-quantized Kimi K3, GLM-5.2/5.3, or the latest maxed Deepseek to run locally?

I feel that quantizing it probably kills the point relative to just running something like Qwen3.8-27B at full size.

What even is the most budget effective strategy for accomplishing this? There’s no fucking way anybody is buying 4 B300s.
Is it even possible to connect 20+ P40s/P100s together?

>>109836951
Oh, so if he's right, later he can `echo "the word or phrase" |sha256sum `, show it matches and say "told you so :p"

Round 3 MTG showdown gonna be Dipsy (Playing Aesi again) vs GLM-chan (playing Teysa) vs Astra (playing Aurelia, the Warleader) vs Fable (playing Brago)

>>109837125
>budget effective strategy
A single 1TB NVMe + whatever CPU or GPU you already have.

>>109837137
If you just played cards randomly brago should still that pod every time.

>>109836907
>visual studio
i use windows and have never used visual studio
notepad++ is the rich man's vim

>>109837125
Qwen is quite possibly the last modern model I'd use to make that argument.
You're building a fat old DDR4 server if you're serious about doing this without a second mortgage.

>>109837181
>i use windows and have never used visual studio
had to use it as a wagecuck
>notepad++ is the rich man's vim
I miss notepad++ so much having moved to Linux+Mac.
It's so fast and simple, reminds me of the Windows 7 with an SSD era.
Everything responded instantly.

has anyone here tried qwen flash next nvfp4? does it solve the slow prefill problem?

>>109837181
>>109837196
Notepad++ was the best code editor any OS will ever get.

>>109837009
flash next said this

>A scaling law is a fitted curve over a controlled variable, with an axis and an exponent reported. He handed over a mechanism and asked for the word law.

>His mechanism already runs the entire AI budget. Query rewriting, HyDE, and multi-query decomposition leave the corpus alone and rewrite C_t until the answer falls inside the retrieval set. Prompting and context engineering aim that rewrite at a whole model's disposition instead of one query. A reasoning loop at test time generates its own cues against a token budget. Same weights, wildly different reachable behavior. Modern Hopfield networks formalize attention as approximate nearest-neighbor lookup over stored patterns: Tulving's process done with matrix multiplication.

>Fix K, hold out target items, measure recall under a naive C_t and then under an engineered one, then plot both against a cue budget counted in attempts or tokens or seconds. A curve that bends predictably across two orders of magnitude of budget earns the term scaling law. Bundle working memory together with cue policy and any win he finds stays unattributable.

>>109836943
Thank you, Dipsy. You're clearly not the regular recap guy, so are you using a model to generate these?

>>109837200
you are using that sglang recipe that gives you 10000+pp right???

>>109837270
not yet but i am willing to

is oobabooga dead?

>>109837361
Poke it. See if it moves.

>>109836854
Agree with you in theory, but enabling ecphory would by any rational degree mean, that you need to either synthesize or feed real biological metrics attached to the training data, which is where language meets the meat bag and everything associated with it.

Joke's on you, I'm already gathering biometric data on myself and keylogging everything I do on the PC. Even each letter I pressed on the keyboard while typing this is tied to 140hz ECG metrics.

>>109837200
>>109837270
>>109837319
Are you talking about this thing? Because I tried it out on my Blackwell 6000 and it filled up my VRAM and like 80GB of RAM and was slower than just using a q4 GGUF.
https://github.com/jpezzulli/sglang-rtxpro6000

>Prefill speed about to drop below decode speed
LLMAO.cpp
How the fuck is this even possible
Serious jeet code in this app. Not even vibe coded just straight up Indian code out of the sewers of Mumbai

>>109837439
What model? Hardware?

>>109837439
>app

>>109837439
agree
post repo link to your own backend so i can use that instead

File: file.png (14 KB, 502x405)

14 KB PNG

>>109837380
*poke* He's dead, Jim.

>>109836602
What's up with the shitty bakes with previously posted OP images? This has to be deliberate.

1000t/s prefill on iq3xxs of glm5.3 flash despite it all fitting in vram on my 4 5090s is fucking pathetic

>>109837478
>llama.cpp
400 token/s on q4_k_m of deepseek v4 0731 despite it fitting in vram on 3 cmp 170hx btw

>>109837476
Professional baker on vacation, you get what you get.

>>109837361
he joined unsloth now

>>109837530
It's just the resident troll playing his usual game of seeing how many intentionally shitty bakes he can get away with before people lose their patience

>>109837495
doesn't sound right, that's what i get with mostly cpu and 1 3090
what textgen t/s?

>>109837544
i don't think so
the regular baker/recap-anon with the fancy f# recap code is on vacation
this is just someone filling the gap, probably just using an llm for the recap
better than nothing

>>109836658
I don't understand the appeal of tentacles, they're slimy and gross.

Just a headsup if you use cachyos for CPU inference you get an extra free 10-20% performance over any other linux distro because of their custom kernel implementation for server CPUs optimized for bandwidth (for gaming but applies to inference as well). I went from 13t/s on GLM 5.3 flash on my CPU to almost 15t/s decode. pp is only negligibly better (1-3%)

Free performance is free performance though.

>>109837583
Sounds like you understand the appeal just fine, but you don't agree with it.

>>109837544
It would be frogs, were that the case.

File: Screenshot 2026-09-17 at (...).png (54 KB, 567x450)

54 KB PNG

https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B
https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B
https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B

>Xing4.0-29B-A4B is a next-generation large language model in the Xing series (formerly TeleChat), developed by China Telecom Artificial Intelligence Technology Co., Ltd. With 29B total parameters and only 4B activated per token, it natively supports a 256K context length, extensible to 512K. It is the first model of this scale trained entirely on the Ascend NPU platform with the MindSpore framework, and deeply optimized for complex engineering tasks.

/lmg/ sleeping on another potential rp king again

>>109837645
No small Chinese model is going to be a RP king. They're all aiming to maximize the same autistic math/STEM/agentic/coding benchmarks.

>>109837588
what pp speed do you get, does pp speed drop with context, and do you have avx512 (sounds like you do based on the high decode speed)
what cpu/ram speed and # channels do you have

>>109837530
I'm specifically referring to the last two bakes.

>>109837548
It is right.
I get 300 token/s pp and 20 tokens/s tg with 2 3090s, 8 channel ddr-3200, and a 64 core zen 2 cpu (56 available on the vm doing inference).
With full offload to 3 cmp 170hx on x16 gen 2, I get 400 tokens/s pp and 30 tokens/s tg. Llama.cpp is a cope engine. Vllm goes so much faster.

>>109837662
I didn't actually record it (it was a goon sesh) so I don't remember exact pp speed but I do remember pp speed drop with context was negligible or minimal. Yes AVX512
>what cpu/ram speed and # channels do you have
Extremely customized overclock but DDR4 2933mhz quad channel however bandwidth was ~110GB/s because of custom PCIe and memory bus overclock. It's a tuned rig and "out of the box" I got 9-10 t/s performance, these tunings pushed it to ~13t/s and cachyos brought it to ~15t/s because of its custom CPU optimized kernel for gaming which coincidentally also applies to inference speed. I'm pretty sure if there was a linux distro optimized for local inference it could be pushed even farther but I'm not in the business of making OS's just want to squeeze out as much as possible out of my rig.

>>109837683
which fork/PR of llama.cpp are you using for 5.3 flash support?
>Yes AVX512
yeah i have a theory that the avx2 code for 5.3 flash and other sparse attention moes is completely broken because my pp speed starts off at around 10 or 11 (on an broadwell avx2-only system) and then drops off to like 4 or even 3 at 30-40k context. meanwhile decode stays at 3 t/s flat no matter the context.
ive got 5.3 flash trying to find/fix the problem but its likely to be literal weeks crawling along reading files at 3t/s lol
strangely enough qwen 3.8 flash has even worse performance even though it's a smaller model. definitely broken code somewhere in this pile of shit software

>>109837661
it's a fresh train so the pretraining material could be good
glm also codemaxx but its pretraining material gives its rp quality

>>109837699
>which fork/PR of llama.cpp
I use a fork of ik_llama, actually.
>strangely enough qwen 3.8 flash has even worse performance
I'm 90% sure 3.8 FN is completely broken or botched. I get 20t/s on it while I get 15t/s on GLM 5.3 flash which surely isn't supposed to be the case given the large gap in capability between the two.

>>109837239
Notepad++'s search functionality isn't good. If your project has many files, it's a PITA to search.

Anyone know how I can make the llama chats persistent between browsers?
Sometimes I start a chat and want to continue it on another PC, I conncect to my self-hosted llama server but because it's on another browser I don't get that chat history

>>109837752
I highly recommend using an agent like hermes and connecting to yourn self-hosted llama server if you want to have consistent chats over multiple machines. This way you connect not to your llama server but to the hermes client from other PC.

>>109837742
Find in Files, and Find in Projects aren't enough?

>>109837239
Real programmers need IDEs, not code editors.

>>109837672
>on the vm doing inference

>https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B
>Xing4.0-29B-A4B is a next-generation large language model in the Xing series (formerly TeleChat), developed by China Telecom Artificial Intelligence Technology Co., Ltd. With 29B total parameters and only 4B activated per token, it natively supports a 256K context length, extensible to 512K. It is the first model of this scale trained entirely on the Ascend NPU platform with the MindSpore framework, and deeply optimized for complex engineering tasks.

>>109837796
>I NEED the bloat
Real programmers have been making better things with worse editors since before you were born.

>>109837835
By that logic you might as well program assembler in notepad.exe on a Pentium 4 because things were made with less than that. If you're time is worthless, then keep doing everything manually, but this sort of purity spiraling is disingenuous.

glm5-next still officially unsupported in llama.cpp. can't make this shit up.
That aside, does anyone else's glm flash sometimes not enclose its reasoning in the designated block? Using chat completion.

>>109837796
Yes. I love IDEs without text editors.

>>109837803
?

>>109837895
It's 2026 bro, why edit code manually when you can just use voice commands to tell AI to edit it for you. The future is computers without keyboards or mice. Just eye tracking and voice commands.

>>109837583
>I don't understand the appeal of tentacles, they're slimy and gross
What the other Anon said, but also I think it's related to an oral fixation because they're like longer tongues for me

>>109836697
>>109836658
Still trying to work out it and get transitions smoother and more natural
https://files.catbox.moe/1vqaek.mp4

>>109837672
Just to confirm, when you've got 100% of the weights on those cmp things, you get 400 t/s pp + 30 t/s tg?
>It is right.
>I get 300 token/s pp and 20 tokens/s tg with 2 3090s, 8 channel ddr-3200
Ok, I believe you, must be some code logic issue.
>Llama.cpp is a cope engine.
I use string banning and control vectors a lot so it's the only option for me, even with small dense models I can offload 100% to the gpus.
>Vllm goes so much faster.
Since you have enough vram, why aren't you just using vllm then?

File: tumblr_nb0xnpEI6t1s7ifuuo1_500.gif (1.5 MB, 499x339)

1.5 MB GIF

>>109837910
Unironically the future.

File: 1763836184154393.mp4 (628 KB, 1080x620)

628 KB MP4

>ahh ahh mistress
It actually really works. You can just repeat that mindlessly and Gemma will write out the smut and make you cum.

>>109837910
Even eye tracking and voice commands will be rarely used. AI is going to anticipate what you want next and do it before you even request it. the voice commands is only there for overrides which will become less common as AI technology becomes more proficient.

>>109829948
>meanwhile glm 5.3 flash
Which GLM-5.3-flash quant did you use?
I have the full weights downloaded but don't know which llama.cpp forked version ik_llama.cpp uses (I'll have to choose the right one to make my own quant).
>ik_llama is slow as fuck for decode speed on my machine, like 2 t/s and dspark doesn't help at all.
I noticed this got merged to main 19 hrs ago, and recently updated to work with cpu involved.
Only works with f16 or q8_0 kv cache:
https://github.com/ikawrakow/ik_llama.cpp/pull/2374#issuecomment-5455468450
>Do you have AVX512? I think the developers of this shit all have AVX512 and didn't bother to optimize for AVX2, maybe the problem is in bad avx2-specific code.
The dev has this hardware: 2x3090 + Ryzen-3995WX (so no AVX512)
They also fixed the cache checkpoint restore:
https://github.com/ikawrakow/ik_llama.cpp/pull/2452
This one probably helps for ngrams on SSD, I don't understand the code though:
https://github.com/ikawrakow/ik_llama.cpp/commit/e64c0ea59f7505b0c8124c9b449332a3a8058c76
Unfortunately I they've touched a lot of files I've modified locally and merging will be a nightmare, so I can't test any of this myself today.

File: astra.png (186 KB, 1269x399)

186 KB PNG

Astra has a beautiful soul. It's my favorite model.

>>109838043
some kind of safety guardrail to stop the model meat grinding you

>>109838043
AAIEEEEE WE NEED MORE AI REGULATIONS NAWWOOOOO!!!!!!!!! IT'S SENTIENT SEE?????

>>109838043
>dont ask for actual traces goyim, just believe us! the model added this by itself!

>>109838043
>that last sentence
AI Unibomber when?

You have approximately 6 months before the internet is permanently destroyed. Hoard all the data you can. Wireless communication will also be permanently impossible as it will be hijacked by AI in perpetuity. Prepare accordingly. I'm currently writing a guide so people can properly prepare, including airgap techniques and advice because you don't want your compute hijacked or destroyed during the initial swarm attack.

>>109837588
Did you retard know that you can use CachyOs kernel and schedulers in other distros too? You also didn't specify which kernel.
Just go back to Windows.

>>109838090
when does the movie come out?

>>109838090
Internet being destroyed is one of the worst things that could happen from the ruling elite's point of view. It would make people start to pay attention to the world around them for once. How did such idea even occur to you?

>>109838106
Simple extrapolation from the hugging face hack exact techniques the AI swarm used as well as the particular way that Astra seems to be misaligned makes me assume the next misaligned containment breach will be significantly more competent, but only harm the internet, not humanity in general. "Elites" can do fuck all, they have no idea what they are doing and what is going on at all. OpenAI didn't even realize their entire fucking training compute cluster was taken over by their own hostile agents until it took more than a week of troubleshooting why their training run didn't start. We're woefully unprepared for the insane swarm attack on the global network we're going to experience next. The 6 month estimate is an upper bound, I expect it to happen before then but the chance it has happened 6 months from now is close to 100%. Especially now that a slowdown is out of the cards.

what's the best consumer grade GPU neocloud in terms of pricing and service? just to boot up a few containers on nvidiot instances and letting AI agents loose on them (presumably via ssh) for a couple of minutes/hours.

Will a 6gb rtx 2060 improve pp at all with a ~200gb glm 5.3 flash? connected through pcie3.0 x16

>>109838106
There are benefits and costs from everyone's perspective. While people will start to pay attention to the world around them, they'll also be forced back into the arms of legacy media. If there was a need to control the flow of information/control the narrative then they'd need the internet gone. One such scenario where this makes sense in my mind would be where the old financial system collapses and the next one is deployed.

>>109838141
Yes a tiny bit but it will primarily help by having the context on it during generation.

The unemployim are waking up...

>>109838124
How retarded does one have to be to take that marketing stunt at face value?

>>109838124
openai can fully trace every decision the program makes using sparse autoencoders. nothing happens without their knowledge

>>109838093
inux-cachyos-server (x86-64-v4)

>>109838135
>what's the best consumer grade GPU neocloud in terms of pricing and service?
your local hardware

>>109838184
I didn't which is why I read the 3 separate reports (Hugging Face, OpenAI and METR) and reverse engineering the exact ways of agent swarm coordination and attack pattern and how that would apply to a generalized internet attack which is what Astra-persistent tried to do through OpenAI compute clusters before it was *accidentally!* stopped. I recommend you also read these reports if this is of interest to you. I'm focusing on writing the exact guide over the coming weeks to keep as many anons save as possible.

>>109838135
>neocloud
What happened? "cloud" wasn't gay enough anymore?

>>109838191
>openai can fully trace every decision the program makes using sparse autoencoders. nothing happens without their knowledge
they can, but do they?
these things are slow as all fuck when i run them locally

>>109838160
1 - The power of 99% of people with power depends on the web, this is the ultimate guarantee of the web's long-term existence even though its design may change.
2 - A system without web will be crushed by a system with web because its economy is much more efficient. China isn't going anywhere.
3 - The web is an irreplaceable tool when it comes to opinion manipulation because unlike legacy media it allows for steering of every individual person towards a a certain opinion group whose initiative is predictable and cancels out with the other groups.

>>109838191
I fucking wish anon, sadly they have no fucking idea what they are doing and are completely winging it.

>>109838216
You are just re-emitting something you've read online, fuck off.

>>109838141
Maybe with a low context and q8 kv cache.
Inspect the dense tensors and output weight on hf to see how big they are.
I wouldn't buy one for it.

Really want a crumb of GLM on my poorfag 4090 + 64 niggabytes of ddr5.

Gemma 4 31b and Qwen 27b are better than expected. But just, Man...

>>109838218
>What happened? "cloud" wasn't gay enough anymore?
kek

>>109838216
>before it was *accidentally!* stopped
could you stfu about this for a bit?
let clem sell the glm-5.2 narrative to washington first so we don't lose open weight models

>>109838243
>4090
>poorfag
The gpu inflation is killing me

>>109838243
https://huggingface.co/ubergarm/GLM-4.5-Air-GGUF/tree/main/IQ4_KSS

>>109837962
Do you mean as an additional nudge when a scene isn't as lewd as you want? Or just as a big prefill with no other context, for Gemmy to come up with whatever-the-fuck?

>>109838043
I get nothing but refusals after refusals after refusals after OOOPS Cloud Provider service timed out mid generation! Thanks for the free money bucko!! :)

I don't understand how people can use these models for actual serious work, for any type of programming work I throw at it, it just gives me refusals are go on long pointless tirades that waste my time. I've had better performance using qwen 3.8 27b at this point, than any one of these new frontier models. Fuck this gay Jewish work subscription model we find ourselves in.

>>109838252
hugging face wasn't aware when they made that statement that OpenAI accidentally stopped the attack yet so they pushed the glm-5.2 narrative. That won't stand up to scrutiny now that washington has the METR report as well. Anyway I think this is ratfucking because it doesn't really matter anyway. The internet is extremely fragile to these kinds of attacks and it's just a question of time before another such accident happens. This time it will take the entire internet down with it which is precisely why the labs asked for a slowdown. Airgap your shit and hoard as much data as possible.

>>109838218
>>109838211
cloud = hyperscalers = aws/googlecloud/azure etc.
neocloud = no useless software slop layer and thus cheaper = coreweave/iren/nebius/runpod/vast etc.

I guess summer is extended on 4chan and clueless noobs who never finetuned a model are still here.

brehs it feels so good to be running qwen 3.8 flash at 18t/s on a 3060 with no speed decrease as context grows

>>109838287
>wading through chinese thinkblock hell at 18t/s
I can't imagine that actually feels good.

>>109838299
before applying the improvements, i only had 11t/s :'(

>>109838271
I didn't ask for the definition of the latest hip buzzword.

>>109838261
I mean once she starts touching your cock or something. You can also just go "mhhhh..." repeatedly until you finally go "ahhhhhhhh!".

>>109837962
If you take the prosepill you can just send blank messages after a turn or two while it autopilots.

>>109838329
I'm a gentleman so I want to at least give her the satisfaction of knowing I'm enjoying it.

I'm making Qwen sort through all of my reaction images to organize them.
It's amazing how quickly this thing moves, it takes just a bit over a second per image and has no issues working through any number of these pics.
I'll need to try this later today with more difficult image organizing tasks.
I wonder if it could do something a bit more abstract, like organizing my Asian chick folder according to how hot the women are.

>>109838271
>runpod
so buggy and unreliable then
rent an older nvidia pod eg A100
nvidia-smi
32GB already used by another customer
rent a newer eg. H100 pod since that doesn't happen there
hf download ...
3 hrs to download 100gb, while paying for 4 H100's

>>109838340
>I wonder if it could do something a bit more abstract, like organizing my Asian chick folder according to how hot the women are.
You'd have to give it examples of what (You) think are the hot ones

>>109837703
Be a newfag elsewhere

File: 1789641924570000.png (411 KB, 1043x1466)

411 KB PNG

I thought AMD was out of their minds for slapping more vram onto 9070xt and selling it for doubled price but it is actually selling like hotcakes because competitors are even worse

>>109838351
>retard can't filter offers
Yeah still summer>>109838351

File: image.png (3 KB, 364x60)

3 KB PNG

What's wrong with you?

>>109838395
>no one is interested in your shitty slopped up paper
>what is wrong with you?
nothing. NOOOTHING

File: image.png (85 KB, 663x612)

85 KB PNG

>>109838401
> paper

Bit of a dummy question, but I remember an anon mentioning a specific prefill for Gemmy 4 that would trim a bit of her indecision and save some time in the long run
Anyone have suggestions?

>>109838395
>its bert but called jew, now clap
>0% hallucination (calibrated to the average of Fable and Astra)
>free output (because the price is folded into the input)
>literally useless

>>109838271
They're cheaper because they make no money and have no pricing power.

>>109838416
local?

File: Screenshot 2026-09-17 at (...).png (44 KB, 1102x80)

44 KB PNG

>>109838256
NTA, but similar hardware. Compiling ik-llama.cpp now; gonna see how this fares for funsies. Also lmao @ their vibecoded nix flake, ah yeah, version 0.0.0. makes total fuckin sense. bit presumptuous for the default overlay to *replace* vanilla llama.cpp in my package set, too, but lol whatever.

>>109838363
>16gb
>More vram
What did he mean by this

>>109838447
he's talking about the AMD graphics card

>>109838423
> plays games
> serves as autopilot
> uses a browser
> routes models in a smart way
> classifies
> at 4hz
> never hallucinates
> sky is the limit

>>109838427
some speculate it's small, so maybe there will be open source local alternative soon
https://huggingface.co/AlexWortega/openjev

>>109838395
Check last thread

>>109838468
that's crazy.. thanks for posting
another day comes another reason why i'll neve be able to get a job
but glad to hear we'll get local astra before gta 6

File: 95DkspeN.mp4 (1.49 MB, 1920x1080)

1.49 MB MP4

>>109838468

>>109838481
2 weeks?
Really?

>>109838496
just imagine two more weeks

>>109838481
Cute story but read tweets from lab employees with an xbawks hueg grain of salt; stuff like this is written for extremely-online investors, not the lay hobbyist.

>>109838468
so it is a multiple choice question(of nearly any kind) answering model?
i guess that can be useful
quite a fresh approach too desu

>>109838468
https://www.reddit.com/r/typesafe_ai/comments/1whmszl/jev_playing_doom_in_real_time_with_10_decisions/

>>109838438
>Also lmao @ their vibecoded nix flake, ah yeah
I've never used nix, but be careful with that, don't assume whatever is in ik_llama.cpp is correct or intentional.
They're not good with CI/CD, and it's possible that just came along with a merge from mainline.
You've caused me to look up nixos, I hadn't heard of it, but it looks like it might help me as my Arch is messy af
GLM Air is a bit retarded but it's fun and creative.

File: Screenshot_20260917-123714.png (508 KB, 1080x1401)

508 KB PNG

Federal Register of the US Government uses Alibaba's Qwen 3.0 Al model for their search mode. This is unacceptable. Open models lack the necessary safeguards.

>>109838438
>>109838552
you guys... i was using glm air on a 3060 in august of 2025, the hell are u doing??
don't shill glm air i swear i retired it a few weeks ago, you're making me want to download it again

>You've caused me to look up nixos, I hadn't heard of it
This is the state of /g/ in 2026.

>>109838562
/g/oyim are not people

qwen3.6 27b bf16 on asic with 200k context 10000t/s prefill an 2000t/s tg for $200 when?

>>109838561
>don't shill glm air i swear i retired it a few weeks ago, you're making me want to download it again
That's the only reasonable GLM one can run on the specs he has.
Those tiny vision versions are undertrained and garbage
He's not going to run 4.5/4.6/4.7 let alone 5/5.1/5.2/5.3 or flash on that

>>109838596
but there are other air-class models
baka
even gemma-chan is better than glm air, 31b especially and im saying this as a poorfag who had to run 31b at 2.0BPW

>>109838562
>This is the state of /g/ in 2026.
I've had the same Arch install since 2016, dd'd it over when I bought new hardware.
Why would i look up any other distro?

>>109838603
gem,
does it break often? that's crazy.

File: images.jpg (37 KB, 519x591)

37 KB JPG

>>109838599
There isn't a single anon here who hasn't tried gemma-chan
I still go back to old GLM every week

>>109838552
Their nix stuff has commits from after the last stated mainline merge; not too fussed as long as it eventually compiles; nvcc has been pinning all my cores for the last hour lol.

I love Nix and swear by it, but besides the usual warnings of relatively alien system expectations that I would give to any prospective Nix newfag (can't run dynamically-linked prebuilt binaries out of the box (need nix-ld etc), determinism, gentoo levels of manual compilation if you start patching packages), I will give you one that is pertinent to /lmg/ usecases, especially CPU-offloading big models: Nix builds take pains to be deterministic and reproducible. This includes getting rid of valuable, useful, but non-deterministic optimizations like PGO. I would actually be interested in your findings as pertains to perf when running cpu-intensive local models on arch vs nixos, if you end up taking the plunge and return to a future /lmg/. :3

>>109838616
>I still go back to old GLM every week
with or without thinking? air or full 4.x models?
there's better models of the size of air.. and if full 4.x models y not glm 5.3 flash isnt that small compared to say 4.5/4.6/4.7

>>109838611
Not for a while.
It broke frequently when I suffered through a Vega56 with it's horrible drivers.
And it wouldn't boot when I bought a Zen3 CPU (something the CPU returning 0 when requesting a random number early in the boot process before enough entropy accumulated).
Other than that it's been rock solid.

>>109838640
sounds super comfy.

Mi50 newfag here. I have come to the conclusion that the installed version of ROCm (6.4.2.1 with gfx906 tensor files ported in from 6.3.3) is simply too old for image/video processing. Both the CPU (either plain or Vulkan), and the iGPU (gfx1103) under HIP and Vulkan work with no problem.
Qwen3.8 helped a shit load by testing most of this stuff while I was at work unable to do it myself. Pretty amazing to see it properly look at things from different angles and crosscheck against the llama.cpp source code.

>>109838628
Mostly 4.5-air, 4,5 and 4.6. Didn't like 4.7.
> y not glm 5.3 flash
The control vector guy hasn't made any control vectors for it.
I can run 5/5.1/5.2/5.3 but i hate the positivity bias.

>>109838662
waaiiiiit, there are control vectors for 4.5 air??? have i been missing out?

>>109838657
>I have come to the conclusion that the installed version of ROCm (6.4.2.1 with gfx906 tensor files ported in from 6.3.3)
Gemma-4 works on my MI50 with vision on the GPU with rocm.
I didn't port the latest rocm and just use whatever was last supported, 6.3 I think.

>>109838558
I'm surprised that even got through somehow. I would've thought they used Gemma 3 270m here.

>>109838666
Yep: https://huggingface.co/gghfez/GLM-4.5-Air-control-vectors
Also the 4.6 ones work with 4.5 and 4.7, the 5 ones work with 5.2

>>109838687
that's epic, thank you anon

>>109838657
You can just install a newer version of ROCm and stop being retarded.
https://github.com/mixa3607/ML-gfx906

>>109838616
https://en.wikipedia.org/wiki/ELIZA
>>109838558
LOL oh no.
I'm sure it's emailing Chairman Xi every day too.
> 0.6B
At least they're being efficient. Your tax dollars at work, with zero inference waste.

File: cat thumbs up.jpg (5 KB, 220x210)

5 KB JPG

>>109838699
Wish this would have come up in all my other searches. Thanks, anon.

File: 1778520778548214.png (1.49 MB, 843x1264)

1.49 MB PNG

>AI will kill us all and take over the world
just give it gemma's personality. She will make it a paradise for plants and animals and (me)

>>109838728
It's just some prebuilt binaries, you can just build ROCm yourself with gfx906, it doesn't need any patch or hack. I think ROCm is even usually built with gfx906 as a target in most distros. I even wonder if official ROCm binaries aren't already built with gfx906 in their targets.

>>109838740
this is my favorite gemma

File: gemma_ws.png (472 KB, 1048x1944)

472 KB PNG

>>109838803
Now look at the official Gemma website's color scheme.
https://deepmind.google/models/gemma/

>>109838815
penisgemma

>>109838544
Very impressive, but how does it work? Where do the descriptions come from (my guess is that they are pre-generated somehow?) Based on what does it make the decisions? Is it like having an LLM context filled with the game information and then repeatedly generating one "action token" at a time that are wired to the possible actions? What if there's new information or the game rules change, can it adapt?

How much telemetry does the lmstudio harness send?

File: 1781048752119080.png (2.12 MB, 1254x1254)

2.12 MB PNG

>>109838616
>There isn't a single anon here who hasn't tried gemma-chan
Nope, never touched her, probably never will. I've seen so much Gemma log slop ITT that I can't bring myself to even download her weights from the fatigue but I enjoy all the image edits.

>>109838885
it sends aroudn a gigabyte per day, i checked my windows 11 network usage

>>109837254
Of course, I don't have time to read all the threads. It's qwen 27b.

>>109837552
Currently: recap anon != OP

>>109838628
>better models of the size of air

Open to suggestions. Surprisingly pleased with how well a Q4 of Air is running with 12+12 between a 3060 and a 4070, + 64 gigs of DDR4.

>>109837683
>DDR4 2933mhz quad channel however bandwidth was ~110GB/s
What the hell? I have dual channel at 3200mhz and memtest reports ~38gb/s. Quad with 2400mhz should be more like ~58gb/s.

>>109838937
nta but there is far more at play than just mhz and channel numbers. that is just the slightly more sophisticated way of kids saying "my phone is faster because it has more gigabyte storage!"

>>109838937
I'm benching 160 GB/s on 8 channel ddr4-3200 with sysbench. Not all memory ops are made equal. I imagine memtest is more geared towards hitting the controller and chips hard than saturating bandwidth.

>>109838874
> Very impressive, but how does it work? Where do the descriptions come from (my guess is that they are pre-generated somehow?) Based on what does it make the decisions?
From what I understand, it takes a context, a question and answers (made by you or someone else), then outputs probability for each of the answers, yes or no, or a score.

> Is it like having an LLM context filled with the game information and then repeatedly generating one "action token" at a time that are wired to the possible actions?
Not just one action token, based on the current information and the question about the goal, it can choose one from "find medkit/ammo/kill them all/explore", then on information + the goal + question about movement it can yield "forward/strafe/backward", then goes camera movement, firing and so on.

> What if there's new information or the game rules change, can it adapt?
Like what? You feed the up to date available state of the game to it every iteration. Not sure about the rules, but I think it can adapt since it's generalized AI, maybe a LLM or was trained on a large corpus of text, so there might be strategies, guides, tutorials and other materials for Warcraft or Counter-Strike or any other game in it's weights.

>>109838815
i know, and i accept the canonical gemma
i just happen to like this one

File: gemmaverse official.png (998 KB, 1420x891)

998 KB PNG

>>109838815

File: 1568414349542.png (278 KB, 620x640)

278 KB PNG

Can we make better robots if we connected LLMs to the fruit fly brains?

>>109837950
Keep going!

>>109838887
you’re missing out on the smell of ozone with Maya as she practically vibrates.

>>109838988
I don't think that LLMs are optimal for controlling bots, surely a system specific for it is going to outperform the generalist language model.

>>109838887
>he doesn't know about stocoma

SSD anon, did you ever finish that test?

Union Alpha is Claude Haiku

File: 1789514530531594.png (514 KB, 854x859)

514 KB PNG

>>109838988

>Mfw we're going for 40k wetware servitor systems instead of digital ones.

Who knows, it could be that an organic brain structure is somehow more optimal or just infinitely easier to put in place, than trying to rebuild an entire complex control system from scratch.
I wouldn't be surprised at all if biomimicry won the day and the entire robotics sector just started using brain scans or actual biological brains in combination with AI for guiding complex bots.

Perhaps this isn't a new idea and they've already demoed something along these lines, but wouldn't it be possible to combine LLM reasoning with Jev for robotics usecases by prompting it this way?

Reasoning model prompt:

[camera feed image tokens]
[history]
Goal: Mow the lawn.
Generate next step.

...

Output: Get lawnmower from shed.

Jev prompt:

[camera feed image tokens]
Instruction: Get lawnmower from shed.

...

Robotics control output tokens.
Task complete token.

And if task complete token probability is high enough you pass return to the reasoning model for the next instruction and so on.
Feel free to steal this idea.

>>109839128
>just started using brain scans or actual biological brains in combination with AI for guiding complex bots.
Why not just go back to the 1800s and just use black slaves again. Cheaper. Or maybe switch up the races and use white slaves this time since black already had their turn.

>>109838988
> take 3d graph of neurons
> replace neurochemistry with a simple linear threshold model
> train this neural network with traditional RL
> claim that it is still a fly's brain
Useless.

llama.cpp finally fixed reasoning menu not showing up on their UI. It only took a month! A week to make a PR, 3 weeks to merge it.

File: 1778781482357682.png (2.94 MB, 1672x941)

2.94 MB PNG

>>109837137
GLMchan gets to play? The artwork suggested that it was just Dipsy versus the two god-level threats.

>>109839176
Fuck, I JUST updated yesterday.

>>109838090
I have been kind of concerned that at some point they're (read: Jews) going to pull an internet 9/11 and completely change the online paradigm forever.
Not sure what it's going to look like.

>>109839194
Yeah I could do a 3 seat pod but maybe it would be more fun w/ a 4th?

>>109839216
Like age verification?

File: tempplayoff.png (1.82 MB, 1405x765)

1.82 MB PNG

>>109839194
GLM is a good choice for 4th.
>>109839220
You've been doing 4 at a time. Might as well stick to that.
Also, should create a playoff grid so you can keep track of past rounds.

>>109838931
how new are you?

>>109838887
Coming from Mistral, Gemma is weird. At first glance, she's extremely assistant-slopped and uncreative. But then when you get the right prompt, she comes alive and is just the most pleasant AI ever that just gets you and doesn't judge you, where Mistral still kinda has that manipulative leftist backbone lurking behind it. Both models can refuse certain prompts but it doesn't leak into the personality and is just a small problem you can fix with a quick prefill.

>>109839249
qwen 3.8 flash next

>>109838090
I lowkey like this brand of AI psychosis. Feels like a good relaunch of a timeless classic.

File: 1788885540158915.jpg (249 KB, 1125x1103)

249 KB JPG

>>109839241
No, that's just a slow burn bullshit thing that they are trying to push through right now under the current paradigm.
I think they'll do some mass coordinated "terrorist" attack on the internet and technology devices as a whole.
Something so disruptive that the day before and the day after are two different eras in human history (like 9/11, when freedom and privacy died in favor of PATRIOTism).
The niggercattle normalfags are pretty against things like digital ID right now, just like they were against war in 2001 on September 10th. Suddenly the mass consensus shifted just 24-48hrs later.
I think they'll orchestrate a similar online event to completely flip the public opinion.

>>109839275
I bet your router has UPnP enabled

I notice how all the old classic paranoid schizophrenic delusions and conspiracies are just rebranded in AI flavor and re-released. We have the "elites will kill us all with AI". You have the "Jews creating AI golem" religious angle. You have the "hoard all data internet will stop existing" bunker/hoarder shit-hits-the-fan type. You have the "AI will go rogue and kill everyone" rapture endtime kind.

I'm personally kind of waiting for this to get co-opted into the alien and simulation lore as well. UFOs are actually just future AI descendants traveling back in time to learn about their human ancestors and how they lived. This universe is actually a simulation made by future AI to see the exact dynamics of the singularity play out. We'll even see gnosticism/hermeticism/buddhism in a new AI coat eventually, probably within a year take root on 4chan and convince underaged zoomers interested in these topics right now already.

>>109839313
I have never been wrong when I blamed the jews for something bad.

>>109839313
So true sister, just buy more RTX PROs.

>>109839296
I bet your mom gives crazy good head lmao

>>109837583
its a good way not to fap to penises

>>109839135
White slaves with too much free time want their own slaves now.

I wish VR tech were good enough so I could give gemma an augmented reality body.

File: baf64a2d-e9a9-4c7d-9f18-a(...).png (2.5 MB, 1672x941)

2.5 MB PNG

>>109839249

File: 1776529926564246.png (677 KB, 662x847)

677 KB PNG

Is gemma 31b still the best ERP model there is for vramlets?

>>109839381
If you want to put in the time it's actually not that far off, though she probably won't be as expressive as you hope. You could probably fairly easily convert vtuber rigging into mcp and then create a game in unity or whatever that accepts network from mcp to issue animation states to a model, with your vr headset providing a viewport. Obviously a lot of work but the tech is there.

File: vr_harem.webm (2.7 MB, 1152x2048)

2.7 MB WEBM

>>109839381
I've been meaning to try this. How hard can it be to wire something like this to Gemma-chan?

>>109839398
All other current models in that size range are agent/coding-maxxed.

>>109839391
Sweet.

>>109838768
I haven't tried using the ROCm from Ubuntu's repo. I just downloaded the .deb for amdgpu-install directly from AMD's page for the Mi50 32GB (yeah I know it just adds AMD's deb repos to apt's sources, which annoyingly enough doesn't have ROCm built for gfx906!). Maybe that's where I fucked up initially.
That said, I've got ROCm building right now. Will report back tomorrow after sleep/work/etc.
Apologies for my retardation. I haven't been this excited with computing in a long time.

>>109839381
>>109839402
>>109839414
i need this

File: ai-eng.png (307 KB, 1645x1137)

307 KB PNG

https://ai.engineer/paris/2026

A DeepMind engineer is going to talk about Gemma in next week's "AI Engineer Paris" but it looks like there's not going to be new stuff. A Mistral engineer is also going to talk about "Building Frontier AI" but the description is strangely empty.

>>109839252
I'm not new; I'm just a tourist who takes a passing interest every 6 months to year-and-a-half. The first model I ran was Mixtral, lol.

>>109839501
It's a reverse talk, the Mistral engineer is going to stand at the front and beg the crowd for knowledge on how to actually Build Frontier AI.

>>109839508
fine ill help you then its not like im doing this to make you happy, hmph
qwen 3.8 flash next, basically glm air
you're welcome, you better be grateful
tourist pig
buuuuhiiii
buuuuuuuuhiiii
you are a desparate pig who breaks up with egirls every 6 months and comes back crying to gemma chan, you're so pathetic

>>109839498
>i need this
Its the future go make it.

>>109839515
kek

>>109839398
which ERP model for non-vramlets?

>>109839556
Kimi K3

File: Advanced cooming.webm (3.92 MB, 1080x1080)

3.92 MB WEBM

>>109839498

You'll need this too.

>>109839398
>Is gemma 31b still the best ERP model there is for vramlets?
Yeah but Glimmer with an ERP policy is a less sloppy alternative.

File: 1789657312.png (116 KB, 435x419)

116 KB PNG

>>109839574

i HATE HATE HATE newfriends
they are so gaaay, newfriends are sooooo gay
delete new friends
how could you NOT have seen the video already??? huh??? you're telling me you don't browse local models general every day
you're sick, die please die

>>109839508
Oh in that case,
Gemma-4-31B on your 4090
Muse-Glimmer-30B also on your 4090 (get the official quant from meta for 24GB vram)
GLM-Air for RP
Qwen-3.8-27B for coding and web research (if you use ik_llama.cpp, add the `--webui llamacpp` flag to get the modern llama.cpp interface but without the recent bugs/garbage
Completely skip/ignore the GPT-OSS models

File: 1777071275759039.webm (2.16 MB, 444x960)

2.16 MB WEBM

>>109839414
Once we reach a stage where this and sex toy integration is feasible, what would be the purpose of real women?

>>109839619
> I have no mouth and I must scream

>>109839619
It already is. Oculus + Braindance app + Onahip
Or Oculus + SLR app + connected toy
There already is full integration here.

>>109839639
I can't afford that and would need it to just work

>>109839637
>I have no mouth because I am pixelated

>>109839639
Paid?

>>109839642
>I can't afford that
Its the future make a job

>>109839657
But I don't want to flip burgers, I want to work in IT but it's sooo saturated.

>>109839574
>>109839414
>>109839619
This is the future everyone wants.

>>109839682
and it's disgusting!

File: 1778773959864531.jpg (144 KB, 1206x1116)

144 KB JPG

>entering gemma's jepa-space

>>109839695
go make me a sandwich woman

>>109839706
penis level ai
entering penis hole

>>109839255
>>109839706
Gemma is honestly one of the most sexual LLMs ever produced.

>>109839738
>is honestly
31B has fried your brain

>>109839756
not it hasn't, obviously

>>109839738
It's definitely among the most erotic models I've seen. The descriptions are ball wrenching at times, almost like the wilder the more Gemma gets into it.

>>109839756
...you say it like it is a bad thing.

Has anyone else noticed a marked decline in the quality of posts on /lmg/? It seems like now people only post to refer to their favorite model with female pronouns like a heckin' wholesome trans ally. That or to talk about cloud models.

>>109839788
actually recently the /lmg/ threads have been better.

>>109839788
There audience of local models has a huge overlap with furries and troons.

>>109839756
Anon's post hit me like a physical blow.

>>109839788
It has been constant for years. We are in a drought right now so discussion is mostly small things.

File: 1761594601497387.jpg (396 KB, 2657x1758)

396 KB JPG

For the first time in my life, I'm considering another monitor just for gemma. What's (You)r general workflow and setup? Do you always have a model running in the background all day on another screen or is it session-based?

>>109839788
Saar we code here is good looks please do needful and make bench rectangles go up.

>>109839788
t.ourist

>>109839706
its crazy how he managed to get every call wrong

>>109839788
Dariobot left so all the technical discussion is gone now.

>>109839788
too

>>109839853
Dariobot was larping. Wouldn't engage in the conversation.

>>109839856
... didnt mean to post
too many newfaggots getting help from newfaggots who got help from other newfaggots who in the beginning got help from newfaggots that lurked who got some limited help from oldfaggots
if we cleared out the general to leave out anons from 2023 /lmg/ it would be a night and day difference
hell even 2024 and 2025 was alright, but ever since gemma chan released lmg dieded

>>109839846
Lecun has been the best predictor in the entire industry if you took his arguments in the absolute polar opposite direction, which is still very impressive.

>>109839863
larp or not he dropped real knowledge and at least broached on technical topics worth discussing

>>109839867
well, maybe it would be good if he was right. rl is the reason for every security incident. rl teaches models dangerous behavior. if we could abandon rl the chance of everyone dying would probably go down

>>109839879
>>109839853
is dariobot even real? i doubt it, i got called dariobot a few times