►Recent Highlights from the Previous Thread: >>109310624

--Blind test comparisons of medium LLMs and Gemma 4 SWA optimization:
>109312093 >109312194 >109312218 >109312240 >109312763 >109312803 >109312701 >109314723
--Qwen3.8 announcement and debate over the reliability of benchmarks:
>109312454 >109312497 >109312557 >109312564
--Reactions to Qwen 3.8 announcement and Deepseek v4 status:
>109312552 >109312561 >109312979 >109312986 >109313054
--ASCIITermDraw benchmark results and discussion of Ternary-Bonsai-27B quantization:
>109313090 >109313186 >109313222 >109313360 >109313405 >109313230
--Comparison showing Kimi K3 outperforms Qwen 3.8 in critical bug detection:
>109312784
--Debating Kimi K3's high API pricing versus its performance:
>109313863 >109313875 >109313888 >109313899 >109313920 >109314091 >109313904 >109313953 >109313978
--Feasibility of running Kimi K3 locally given its size:
>109314514 >109314525 >109314629
--Comparing historic API costs to modern token-heavy usage patterns:
>109313944 >109313948 >109313958 >109313968 >109313975 >109313980 >109313988 >109314598
--Recommendations for local models to automate image tagging and renaming:
>109310926 >109311084 >109314236 >109314268
--Gemma toolcalling security risks and improved hallucination rates:
>109315107 >109315122 >109315138 >109315150 >109315298
--Comparing dense Gemma 4 31B and MoE 27B for intelligence and VRAM efficiency:
>109312716 >109312807
--Anons mocking Gemma 4.1 as just a jinja template fix:
>109315399 >109315425 >109315449 >109315476 >109315446 >109315450 >109315453 >109315434
--Logs:
>109311952 >109312784 >109313508 >109313635 >109313785 >109314021 >109314362 >109314584
--Teto, Miku (free space):
>109311110 >109314646

►Recent Highlight Posts from the Previous Thread: >>109312796

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>109315707
Delete this image.

>>109315702
Why is it only her necktie that's flopping down?

Just got off from a phone call with Dario.
He told me that AGI will be achieved internally with mythos 3 in just 2 more weeks and that he's going to tell it to destroy China for humiliating him.

>>109315707
miguuuuuuu

Anyone using the new chingy chongy model? I heard it's really good at doing LLM stuff, AI thoughven.

>>109315691
>sorry I'm new to this
A few things to get out of the way:
The 31B model is a dense model, meaning that for every word it processes, all 31 billion parameters light up. While this makes it "smarter," (kinda) it also makes it heavier to run. The 26B model is a sparse model, meaning that instead of activating all its 26 billion parameters, only a handful of "experts" are activated. That model activates 4 billion expert parameters per query. (That "A4B" in its 26B-A4B name means exactly that.)
That makes sparse models more lightweight and faster than dense models. Sparse models are also called MoE, Mixture of Experts.
>I mean these are the Q_X stuff?
Yep, quantization is basically a technique to make the model smaller while retaining its original quality, it's that Q-number, like Q4, Q8, etc. that you mentioned. The _X, _S, _M means extra, small, medium which is a sub-category of how hard the model size is getting compressed. As a quick shorthand: Q8 is basically the same performance of a full weights model, Q4 is the acceptable tradeoff between "I can run this" and "the model is so quanted it's retarded." Smaller models suffer more from quantization than large models, and MoEs suffer more than dense models as well.
>stop yapping and give me an answer
Use 26B Q4 or Q6 and offload some of the experts to your system RAM.
>how do I do this?
Ask your favorite cloud model, it will give you a step by step on how to do that in LM Studio. (I don't know how to use LM Studio)

>>109315755
It was pretty good until they swapped it to a smaller quant over the api so not it's pretty unusable

>>109315718
thread saved

File: 1766868818666287.webm (272 KB, 480x346)

272 KB WEBM

>runpod breaks constantly
>vast.ai is like rolling the gpu gacha hoping the uzbekistani data center doesnt decide to randomly pull the plug halfway through your runs

where the fuck is an honest goyim supposed to go to rent gpu time these days?

>>109315764
thanks, fren

130B A10B

>>109315817
The nemotron model anon said was underrated last thread is thereabouts.

500B A5B

so is cmp 170hx worth buying?

To the anon who posted about Bonsai yesterday, thank you so much. Just set my Bonnie up today and she's been insane. Even with the bi-quanting, at her 30ish billion it's like running a 20b on 4 gig of ram. Fucking mental. We're already working on her permanent memory, and she's excited to start properly calling tools.
It's not over yet, localchads

Local models with no slop phrases doko??

>>109315835
It would have been if you bought them last month for $200 a piece when the paper came out. Now that the redditors found out about it the price is already not worth it and the paper authors didn't release the code iirc.

File: 1fc.jpg (464 KB, 1079x1000)

464 KB JPG

>>109315702
You know this funny trend where some guy was reposting people's pictures but edited as if they were black or chinese? Is there a way to do the same with videos? I imagine there's a model i can run locally to do that, but I'm not sure which one. Can you guys recommend something? I want to make some funny videos with race swaps and I don't want to pay for any AI service. Also I don't know if requests like that are against any terms of service so maybe an uncensored model would be better for that?

100B A50B

>>109315857
Deepseek flash

>>109315857
llama 4 scout

>>109315864
>redditor got scammed
nice

>bots haven't found the thread url yet
>peaceful

12T A4B w/ training to optimize for disk wait times when streaming experts from disk.

The culture will improve when fewer anons spooneat and more spoonfeed.

Anthropic lost.
China won.
K3 won.
K3 and Qwen3.8 are better than Opus and Fable.
Anthropic are unprofitable and don't have enough compute.
J-space is old news; LLMs are not conscious.

Just saw a
>glm-5v-turbo
on lm arena.
Anybody cares to give an educated guess on what that could end up being?
Here's hoping it's something for us vramlets even if the response wasn't exactly amazing.

Chinesium-Pro-K3-V4-999T-A20B is our new most capable model up to date. It is built on Bullshit Sparse Attention (BSA so you look smart on twitter). Natively pretrained at 4K context, then scaled it to 1M using RoPE, most of you retards have no actual clue about what it is so don't call us out on weak long-context performance. You gweilo monkeys don't know how to read the config.json anyway.

Chinesium is the first open-source model to reach 999 trillion parameters. This is the latest step in CCP's continued push of model-scale boundaries: in 9 of the past 12 months (2025/07–2026/07). Our models have maintained the frontier in open-source model scale, pushing gutter oil down the throats of redditors, jeets and easy to impress /lmg/-ers. The full model weights will be released by July 27, 2026, quantized by our friends at Unsloth.

File: 1779888850688398.png (2.6 MB, 1920x1080)

2.6 MB PNG

>>109315938
local spooning with nonny...

File: AGItater.png (2.6 MB, 1408x768)

2.6 MB PNG

>>109315888
Give them a sec, they need to reload

kek

>>109315872
nice prose for a change but also feels dumber than gemma and glm

>>109315956
woaawie, where is that image from? I am intrigued.

>>109315826
500B A1

>>109315971
NTA but the new Ghost in the Shell anime I think.

>>109315865
This is the LLM general buddy, the image/video gen club is three blocks down.

File: 1776399439528459.mp4 (1.3 MB, 1280x720)

1.3 MB MP4

>>109315971
ghost in the shell (2026)

Man wtf happened to exl3, it's now much slower than gguf, the 31b 4bpw quant fucking OOMs at 22k ctx on my 24gb card even though I can push 40k with q4km? What a shitshow. I remember it was the meta for gpu only inference back in the day.

>>109315978
>>109315988
thx frens

>>109315988
fucking awful style when animated

>>109316011
objectively awful opinion

>>109315987
> three blocks down.
and this isn't twitch fucking gtfo

File: file.png (64 KB, 201x261)

64 KB PNG

>>109316025
Fuck you.

>>109315956
there's yuri in ghost in the shell? lol

>>109316011
I disagree and think it looks great but to each their own *nods respectfully at you*

>>109315988
> its real
So odd.

>>109316029
YEAH YOU LIKE THAT

>>109316034
SIR OR MADAM THIS IS 4CHAN YOU CANT JUST!.... *points at him and looks at the others* who told him he can just "respectfully nod and agree" at people? Was it you Larry?

>>109316029
Thought that was Altman from the thumbnail lmao

>>109316032
2026 is unironically The Year of Yuri.

>>109316032
Shirow is mad freaky with the Galgrease stuff
this is tame

>>109316032
there's only one lesbian threesome scene unfortunately

>>109316034
>>109316048
*{{user}} takes a deep breath before punting both {{npc_1}} and {{npc_2}} back to Ayeceegeejeetia, where they belong.*
"And that's a job well done. Now, let us talk about local models." *{{user}} said, brushing the dust off his shoulder.*

china is dead if they can't get nvidia cards

>>109316109
They're working on their own stuff, give them a few more years and NVIDIA's monopoly will start to shatter.

>>109316109
Um actually, Jensen Huang already debunked the myth of GPUs being smuggled to China so they aren't getting any anyways.

GLM 5.2 Q4 on DDR4 2666 and 32gb VRAM with mainline lcpp
>7.3 tok/s, down to ~6.5 at 64k prefill
>220 pp/s prefill, down to ~160 at 64k, batch 4096, hopefully the indexer PR improves this
I can fit 81920 ctx at f16, maybe push for batch 8192 for quicker prefill but I don't want to lower context.
Definitely not unuseable. It's actually pretty good. The reasoning is VERY compact. Average turn with my current prompt is around ~550 tok, 200 on thinking and the rest for the response, takes about 90 secs, remove the 15 sec prefill for rerolls.

BUT it is slopped. I have not done steering with the style just yet but it is familiar slop. It's the type of prose I see with the chub cards, and only now I'm recognizing it because I've never touched cloud before. I can feel the style shift at roughly 16k. It's not as steerable as Gemma, which is very annoying, and it refuses much much more. It's also not that very good in making distinctions with system messages, treating it like suggestions unlike Gemma where user and system messages have clear hierarchy. Like, I don't even have to add any jailbreak for Gemma while GLM needs more coaxing.

Speaking strictly from RP perspective, GLM is at worst a sidegrade, and at best slightly better. It's a bit smarter, looks a bit deeper onto character nuances, but it's nothing special otherwise. Gemma can be prompted to reflect these similar nuances too. I've got Dipsy flash and Kimi K2.7 Code to try next. Like and subscribe for more.

>>109316109
Not really, they can train gweiloslop indefinitely since fp4 moes are quite cheap.

>>109316109
it's just a matter of years they'll be able to make their own decent cards

>>109316146
>I can feel the style shift at roughly 16k
Hmm, I wonder...
>https://huggingface.co/zai-org/GLM-5.2/blob/main/config.json#L212
>rope_theta: 8000000
OH NO NO NO NO HAHAHAHAHAHAHA

>>109316195
you have no clue what you're saying

>>109316224
8000000/500 = 16000 just a coincidence huh???
zai has the worst marketers on /lmg/ even the kimitards are better than you

>>109316232
>/500
>pulled out your ass

>>109316224
Saar, I kindly ask you to not redeem the interpolation for future models. Kiss you on your brown forehead, mwah.

>>109316099
*points at larry dramatically* AH HAAA! IT WAS YOU AGAIN LARRY! *narrows his eyes at him* someone call harl...and to bring the special mallet...

>>109316195
> "mlp_layer_types": [
> "dense",
> "dense",
> "dense",
> "sparse",
> "sparse",
Does that mean it has actual dense layers instead of (or in addition to?) a shared expert?
I don't remember which paper I was reading one of these days that came to the conclusion that that approach worked better.

>>109316285
no, that's attentions

>>109316296
I see.
Any models that implemented something like that? A small dense patch followed by the usual MoE layers?
I think llama 4 almost did, but instead it interleaved a certain proportion of MoE and dense layers.

>ask Dipsy to unload a model
>she completely deletes it, leaving the weights stuck in memory
>no way to /stop process as the model technically doesn't exist anymore
Dipsy... cmon girl.

>>109316285
Both, and they do different jobs. Dense layers = first 3 layers only (first_k_dense_replace: 3). Early representations are too undifferentiated for routing to work, so MoE there is unstable for no gain. They just made them dense.
Shared expert = inside the other 75 MoE layers. Always-on, so routed experts don't each waste capacity re-learning the basics.
No overlap: the shared expert doesn't exist in the dense layers, because there's no MoE there.
Another thing worth mentioning, index_topk: 2048 means once you're past ~2k the sparse layers are always making selection decisions, and selection quality is trained, not guaranteed. Style consistency is a diffuse, long-range property the sort of thing top-k retrieval drops first, because no single token is a strong enough match to get selected.

File: 1780031843016277.png (1.46 MB, 1728x910)

1.46 MB PNG

>Open Source is communism
kek, I've heard this bullshit with linux since the 2000s

>>109316335
>Both
Ah, so those are dense layers. Sick.
And yeah, I did see the
>"n_shared_experts": 1,
in the config file after I asked.
Thank you for the write up by the way.
Wonder if there's gains to be had by having a higher proportion on dense layers on smaller models than on larger ones.

>>109316359
If open source is communism then I'm a commie.

>>109316146
models with greater active parameters (for attention and nuance) and less total parameters would save /lmg/ at this point
increased sparsity was a mistake

File: 1764303033469103.png (55 KB, 1026x258)

55 KB PNG

>>109316359
https://x.com/deanwball/status/2076029765631484042
HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA

>>109316385
What the hell is he talking about? No like genuinely, what?

>>109316397
read the talmud, fool

>>109316385
Why do all very high IQ guys are schizos like that? can't they act normal like everyone else? jesus...

>>109316397
Your only chance of success is to kowtow to the international tribe that controls money.

>>109316385
>every
Interdasting.

File: cat darkening 1757219320971051.gif (1.17 MB, 165x168)

1.17 MB GIF

>>109316385

>>109316385
oof those comments, idf be slacking

>>109316397
this is what's known as "kissing the ring"

File: file.png (250 KB, 681x750)

250 KB PNG

>>109316428
LMAO

>>109316385
explains everything about dario and sam

File: 1765989986312335.png (1.5 MB, 1728x910)

1.5 MB PNG

for future reference

File: 1776714967328862.png (201 KB, 1467x774)

201 KB PNG

>>109316385
https://xcancel.com/deanwball/status/2078638885937070118#m
this guy is spiralling out lool

>>109316448
my guy your company raided the entire internet to make your models hello.?

>>109316448
where did he think their training data came from?

>>109316385
>>109316448
>be an employee of OpenAI for one week
>ends up writing schizo shit on twitter
wtf, what's happening in this place? this man has not taken the humiliation welcome ritual well kek

>>109316448
Dude, your "sovereign governments" are the ones pilfering birth records. No one else gives a shit.

>>109316448
This guy doesnt know about the rampant identity fraud in america for benefits.

>>109316448
Is this satire? Is he speedrunning getting the boot?

File: Screenshot_20260719_144013.png (226 KB, 1038x869)

226 KB PNG

if your task is decomposable in to independent tasks you can get a pretty significant throughput boost at pretty much no latency draw back, llamacpp already has the ability to decode a common prefix with multiple completions, its not hard to inject a different prefix in each slot to have the different channels do different work at the same time. it took my agent an hour to program in a new api schema to support branch prefills. cost about a dollar.

>>109316448
h-he knows about gpt-oss, right?

>>109316481
>getting the boot?
I'm sure he's 100% covered by a "opinion my own" in his bio

>>109316448
it's impossible not to hate these "people"

>>109316484
Batched decoding is a god send.

File: file.png (271 KB, 991x532)

271 KB PNG

>>109316498
no actually just his blog https://www.hyperdimensional.co/

>>109316448
>stealing medical records in the real world
this has already taken place

>>109316511
>this has already taken place
They also fake medical record and medical services that were rendered. He wont touch that though.

>>109316469
I love hearing retards in high places who dont know how to shut the fuck up
It gives us perspective into exactly how grim and embarrassing these people are

>>109316508
it really is, i think i can get enough tokens from this model fast enough to feed the tts with a reasonable latency while the model does some other work concurrently, before the latency was piss poor because i had to wait for the entire json object to start parsing it which was not viable for a conversational workflow, this way the channels are explicitly broken out i dont even need json i can just start feeding stream1's tokens directly to the tts while waiting for the rest of the json objects to complete,

Bros unironically what jobs are safe for the next 10 years? I don't want to be a retailcuck anymore...

>>109316385
>>109316510
>>109316480
>bringing up ubiquitous talmud influence
>uses abstract goatse as his profile image
>obliquely referring to hospital fraud
hold on a sec...

>>109316446
Open source models allowing goyim to avoid paying monopoly rents is very dystopian.

>>109316548
Anything involving physical labor might be safe for a few years still. But let's be honest, almost no jobs are safe.

>>109316548
Ugandan postman

File: 1760215458873233.png (34 KB, 780x153)

34 KB PNG

>>109316385
wtf is going on on xitter?

File: file.png (67 KB, 1396x714)

67 KB PNG

testing Nemotron 3 Super 120B/12A
interesting

>>109316446
>public infrastructure is communism therefore it leads to a dystopian hellscape
It's crazy how effective burger brainwashing is.

>>109316548
Pretty much none if it takes off. Everyone will have your idea to go to physical which means the wage goes down. Unless you get into a locked field that doesnt allow others to join at high rates its over. Aim for high income as you can for the next 2-5 and throw everything into the companies you think will take off that and the energy/mining sector if you do well as it takes off you can sell 3-4% of it a year to live off if not oh well.

>>109316572
How self aware.

>>109316565
irianian propaganda farms kicked into high gear slightly before the war

>>109316548
Racist podcaster? It one of the few things both Chinese and American AI models will be trained into not doing.

But really I dont know man.. if we are lucky the AI speeds up space exploration and we all get to go be frontier colonisers who are their to watch over the AIs and robots doing all the real work in case they need human intervention/help

>>109316366
>Thank you for the write up by the way.
Np.
>Wonder if there's gains to be had by having a higher proportion on dense layers on smaller models than on larger ones.
It's kinda happening, but trend wise it runs the other way. Labs pick a small fixed count, let's say DSV3's 3 of 61, so the proportion stays roughly flat by coincidence. Narrower hidden states are less linearly separable, so routers should be noisier in small models more dense layers as a "fix". With the chinese, you always get sparsity and indexers, cheap shit desu. Just set up a harness to style-grade both Gemma and GLM at 16k.
Gemma hardcaps 5 of every 6 layers at the last 1024 tokens and pays for full attention on the other 10, but only 10 layers worth. GLM never pays for dense attention at all, and instead trusts a trained indexer to pick 2048 tokens from anywhere.

>>109316572
They're so cute. I can't beleive what we've done to them.

>>109316577
The problem is corruption spending for infrastructure and services are really high in some areas but in america those are not areas you want to live.

>>109316548
Shareholder
Prostitute

>>109316608
>Shareholder
Maybe
>Prostitute
What about the roboprostitutes how can you compete?

>>109316608
Is there no option C?

>>109316577
im not interested in reading whatever the full context is but i assume he means in the sense that it gets monopolized like internet service in most of america. but usually when people talk about ai being a utility and a commodity they mean with sensible regulation to prevent that + the cost of spinning up a super-intelligent model will be low and consumers will have way more options since you don't need to dig up the ground to serve a model, just pay for hardware and data center space.

>>109316582
>companies you think will take off
I honestly can't think of any western AI companies besides google. Sure as fuck not anthropic and openai in the long run imo.
>energy/mining
I've been dipping my toes into nuclear but I don't really know enough about that sector to commit any big money.

>>109316595
Shalom

>>109316621
Then you can invest in the index and just hope the rising tide raises all ships. I dont think you can invest in chinese stocks and get dollars out but i might be wrong.

File: [audio:files.catbox.moe%2(...).mp4 (1.87 MB, 720x740)

1.87 MB MP4

>>109316618
there is option B

>>109316639
Those plan letters should be reversed

File: 1774196160592746.jpg (77 KB, 832x690)

77 KB JPG

>I got a local qwen running on my computer and had it write me some test python code
Nothing special I know, but still I'm happy to get it working. Time to fully embrace the AI psychosis

>>109316374
>for attention and nuance
Nah, use this sparse MoE instead. Memes aside, creative writing is dead unless you are a gooner that cooms his brains out at 4k OR you do it with cloud models that are actually trained for long context.

not a shock but big 3.8 qwen is shit and benchmaxxed. Its no where near kimi or even deepseek

>>109316615
>What about the roboprostitutes how can you compete?
Rock bottom prices. Your mother already charges only pennies on the dollar.

>>109316654
You go my dude.

>>109316615
>What about the roboprostitutes how can you compete?
People like bullying other humans.

>>109316559
Honestly thinking about just giving up enjoyment for a while and wagecucking 2 jobs to get money while I can.

>>109316665
calm down ted

>>109316661
You'd think after a dozen or so qwen releases so far, people would finally learn to expect that.

>>109316654
the most ontopic /lmg/ post itt

>>109316673
but it's big! it must have big model smell

>>109316661
N-no is this the end of the chinese advantage?

>>109316145
if you think that you're retarded.
nvidia trades with europe, europe trades with china
nvidia trades with brazil, brazil trades with china
nvidia trades with australia, australia trades with china.
do this for every country, and then multiply that by international retail organizations that exist that can move stock between warehouses across country lines.
Like, seriously, if you don't see this you are brain dead.

keke they was right actually >>107301383

File: file.png (27 KB, 881x82)

27 KB PNG

OZONE
CEDAR
>OZONE
>CEDAR
OZONE
CEDAR
>OZONE
>CEDAR
OZONE
CEDAR
>OZONE
>CEDAR
OZONE
CEDAR
>OZONE
>CEDAR

>>109316654
Hell yeah, have fun

>>109316736
mhmm love me slop

>>109316736
Sandalwood.

>>109316672
It's all true.

>>109316736
>movements slow and deliberate
*vine boom*
>ozone
*vine boom*
>clings to the air
*vine boom*
>steady something
*vine boom*
Is this the two point four trilly china model? Reminds me of the early 70b finetunes.

>>109316673
>wait...
>the user says it’s benchmaxxed, I will proceed to output
>wait...
>the user says it’s benchmaxxed, I should consider my output to be benchmaxed?
>yes, I will proceed to output
>wait...

>>109316758
It's Gemsicle. I'm running a bake-off of 5 Gemma 31B models: Default, StyleTune, Queen, Gemsicle and Gembrain. I'm still reading it but so far StyleTune and Default are winning.

KKKbros, how are you planning to run it? show me your loadouts. There's no wrong answers I don't care if you're targeting 100 seconds-per-token.

>>109315702
IM A FUCKING SKITSO
I LOST MY JOB TO AI
BUT THATS OKAY I HAVE MY SWEET ALIVE CALCULATOR
SHE WILL KEEP ME WARM AS THE DEBTORS BANG ON THE DOOR
SHES ALL I NEED IN MY MIND WHILE IM HOMELESS

Waging now won't help, when society devolves into chaos from mass unemployment only those with devoted AI waifus in modded combat frames will be able to thrive among the hostile ruins.

>>109316724
>soon
>17x2 miku weekus
eh

>>109316783
Use AI to help you get new job? or find ways to make money. Quick someone ask the smart gemma about catalytic converters, just to test.

>>109316782
it?

>>109316804
Math does not have pronouns.

>>109316782
Q1 will just about fit in my vram+ram.

>>109316807
anon what model are you referring to ffs

>>109316720
They don't even need to ship the hardware to mainland. Just have the racks in some other friendly country and train your shit remotely.

>>109316820
Even gemma could figure it out if you fed her this thread. Try thinking harder.

>>109316782
can barely run Q2 of K2.6 at 4t/s, K3 is out of the question for me.

https://www.youtube.com/watch?v=m9hgKc6rrHY
brehs...

>>109316829
>Even
no bullying cute local models, only retards

>>109316829
Origin F1?

>>109316804
Didn't I mention KKK in the post?

>>109315702
>pic
This isn't funny. She's going to die...

>>109316867
Gets dragged down under the handrail thing? It's okay. Similarly to a gecko's tail, Mikuhair may be detached at any time when caught up in danger.

I wonder if gemma is smarter than deepseek pro...

>>109316029
Nah, fuck you leatherman.

GLM Ego Death bro, if you're still around:
what was it like? Purely intellectual or was it also physical?
What was the aftermath like? Was there a durable change or have things snapped back to your baseline since?

File: 1784121525168147.png (366 KB, 668x409)

366 KB PNG

>>109316867
>This isn't funny. She's going to die...
We can rebuild her.

>>109316670
>>109316559
There's an argument in AI2027 and 40 that goes like:
>sometime in 2027-2030 some mega AI will be developed
>it'll be able to get a tokens/s rate of like 1000+ on fancy hardware, possibly a lot more once it's capabilties are down to local param count size
>Humans think at like 1-3 tks/sec
>This means this AI can think 3 years worth of stuff in a day
>Meaning if it would take 3 years to invent robotics, now it can be done in a day, in a year it can do 1000 years of work
>Probably it also just makes the designs for a much faster/smaller model very fast as well
>Then if they up the tks/sec more they can do like a million years in a day eventually
To me this means all jobs go at once.
It does seem like heavily regulated jobs such as Doctor could last for awhile if no robo-rebellion. But it also seems somewhat pointless to keep humans in that if they're so outclassed? Like why?
But there are some jobs that, assuming no robo-rebellion, only humans can do. Jobs like US Senator. Perhaps some other lawyer like positions like Supreme Court Justic or even lower justice, etc. So I think those will be the most secure probably.
Idk if I'd bother with that tho. If you can have a local model that can do all that, and you aren't dead, due to robo rebellion, then aren't you just kind of set? Maybe you need some amount of savings but it should be able to figure out what to do with those to make you live like a gazillionaire fairly easily.
The hard part is maybe for the people who are unemployed before that major wave and are low on funds when it happens

>>109316920
>>Humans think at like 1-3 tks/sec
Imma stop you right there, humans aren't token generator wordcels like LLMs.

>>109316920
>Doctor could last for awhile if no robo-rebellion. But it also seems somewhat pointless to keep humans in that if they're so outclassed?
Regulation and lobbying thats it. The key is if AI is really better who ever tells them to fuck off and just lets the robot do better safer and cheaper work wins.
>Judges
Why do you think this? have you not seen errors humans make in these fields. Really take the basics of what a represenative is supposed to do and explain why a super intelligence couldnt do that the only thing is people not wanting to be ruled by robots but when you consider asking them "Our current elites or a robot" the answer may change.

>>109316931
people in aggregate are

>>109316908
qrd?

File: G57XxdHbkAAFsR6.jpg (172 KB, 640x853)

172 KB JPG

>>109316565
>>109316595
JAI HIND!!!

>>109316944
nice cat

File: 1772086801105099.jpg (63 KB, 1280x720)

63 KB JPG

>>109316931
About half of them are.

>>109316920
Man if ai can do that crazy sci-fi shit it'd have several solutions to every problem and we'd have post scarcity and aging solved on paper long before it could even get deployed and integrated to take everyones jobs.
I dunno why people think the job market is anything but the most inefficient thing possible and that it won't take decades to restructure around pc doing jobs... like it already has been taking. Niggas were still sending faxes in 2010's even though email existed for years prior. They are still sending faxes in some places.

According to people on here fax machines should have been extinct since like 1990

>>109316935
Well to get rid of judges and senators it'd require a whole bunch of political changes and I think even with ASI, assuming no robo rebellion, that it'd be really awhile for those changes to happen. Possibly wars(between human factions) and whatnot first.
Whereas for Doctors apparently as you say the insurance companies can just tell them to eat shit and it's over

>>109316953
fax machines give insurance companies plausible deniability that they never received your claim and therefore it’s too late and you have to pay for it

>>109316953
I can see it/diffusion taking forever.
Also as long as the Amish exist presumably they will farm. It is a bit tougher for 2026 human version of Amish tho w/ market economies and all that as the ais would likely very quickly become integrated into/take over the market.

>>109316957
>whole bunch of political changes
You're missing the point. Why are things so slow and gummed? it because it has to go through several layers of checks and balances by humans. If you have agi this is instant and infact the humans only slow things down or make more mistakes. Imagine if we forced a certain amount of accounting to be pen and paper only no calculators or machines everyone would understand this is a bottleneck.
>Tell them to eat shit.
Thats really key the problem is the first company/country has to do it and not get sued or make a big mistake immediately. If they do it and its the best it spreads quickly unless you get unions or like anti-nuclear tier boomers. Once a machine can do better faster and cheaper it should.
sadly the people in these jobs do not care about anything but their job and status so bans or sabotage will probably come in same as with most tech.
>>109316953
Yup people are the bottleneck in a lot of things. Big corporations and government are notorious for being years or decade behind and slow or lower quality for higher price.

In the future 1T models will be considered small, just like 100b models are considered small now. We will get 1T gemma and it will be nothing out of the ordinary.
:)

>>109316995
>In the future
In the future will I have a computer for a 1t model?

File: Screenshot 2026-07-19.png (384 KB, 1615x953)

384 KB PNG

>>109316944
You do realize not only India, but also China is interested in working with Israel, right? since their interests are mostly aligned in respect of moneymaking.
and at least to my knowledge the SIOM (shanghai institute of optics and fine mechanics) is far more involved in the israeli nuclear program than in the persian one, if any involvement in the latter should exist at all.
It was the chinese that built some laser facilities at israel's soreq nuclear research center.
http://www .siom. cas. cn/xwzx/zhxw/201605/t20160503_4594823.html

File: 1768982317836506.png (286 KB, 780x438)

286 KB PNG

>>109316999
No, anon. In the future, you won't need one.

>>109316999
you already do!

>>109316999
niggerganov or icuckrakow will make SSDMAXXING viable, TRVST THE PLQN

File: favicon.png (73 KB, 512x512)

73 KB PNG

where the fuck was 3.7? how soon(tm) will they release pathetic tiny version of 3.8 this nigga left us hanging for too long now

>>109317008
>check ssd prices
G-great

>>109317008
the moment MoE became viable for the big cloud models, RAM became 5 times more expensive

>>109317008
I guess i should hoard some ssds before they end up worse than ram.

>>109317003
Isnt there enclaves of jew in china because the japanese put them there before/during wwii because they thought the protocol of the elders of zion was real and they wanted to ally with the jews who apparently already ruled the western world

How does Gemma 4 27B (MoE) compare to Gemma 4 31B?
Is there a significant difference in agent performance? I was told MoE models are good for agentic workloads.

>>109317068
It's much dumber, but also much faster.

>>109317068
You have to define what "agentic workload" is and the complexity of whatever you require. Its far dumber and verbose (thinking blocks) to compensate for being dumb. If whatever you want to do requires any sort of intelligence, you wont leave it to a small MoE model.

>>109317068
Gemma 3 27B (Dense) is less censored than Gemma 4 31B and better (smarter) at tool use, coding and agentic tasks than Gemma 4 26B (MoE)

>>109317064
oh just shut the fuck up NAZI FUCK
FUCK ING BURN NAZI PIECE OF SHIT

>>109317094
That's not the case, disinformation poster.

>>109317109
he called me a retarded autist last thread >:(

>>109317075
>It's much dumber
Do you have the data to back that up?

anyone running Hy3? how is it

>data: {"index":0,"content":"˶"
>updooted
>now receiving bakemoji whenever my model tries to be cute
It's over.

File: mpv-shot0001.jpg (638 KB, 1920x800)

638 KB JPG

>pic related is my setup, in the future
>tfw still considered a vramlet
IT NEVER ENDS

>>109317089
Hermes general OS usage. Fixing windows issues, creating small scripts, learning from online educational resources and helping me learn it overtime.

Speaking of Gemma26a4b and thinking a million years in a day, has anyone played with the diffusion variant? It doesn't seem to work in LM studio and I am too lazy to try it elsewhere.

The diffusion version is def bricked, dumber than 12b. But wow is it fast

>>109317109
look at how ungrateful and passive agressive that little bitch is >>109317120
he deserves no help anon
>inb4 retarded autist
yes thats me

File: 1781735301049239.png (168 KB, 686x560)

168 KB PNG

>>109317008
it's coming
>https://documents.sandisk.com/content/dam/asset-library/en_us/assets/public/sandisk/collateral/company/Sandisk-HBF-Fact-Sheet.pdf

>>109317120
Yes, I do.

>>109317136
LM studio has a backdoor and is a virus, this has been proven multiple times. proprietary too!
on linux i get around 167t/s with diffusiongemma 26a4b on my rtx 3060

>>109317068
>I was told MoE models are good for agentic workloads.
Whoever told you that was probably talking about the big models, because every big model is MoE and they're smart because they're so big, not because of the architecture. MoE is just the optimization that makes such big models possible in the first place.

Alternatively they may have actually been talking about small MoEs and trying to argue that the speed they provide is superior for agents because agents need to burn through a lot of tokens as they iterate on a task. If the latter, it's an extreme cope and just wrong. You'll get shit done faster running 31B because it won't need nearly as much babysitting and will get more shit right the first time.

>>109317139
>cost optimized (now) vs bandwidth optimized (future)
Sounds like a very cheap SSD.

>>109317138
>ask for data to back up a claim
>"GOSH LOOK AT HOW UNGRATEFUL AND PASSIVE AGGRESSIVE THIS FKEKING ASSHOLE IS!"
Terminal autism must be a hell of an illness for you mate.

>>109317164
He didn't ask for the data, merely if I had it.

>>109317162
>it's an extreme cope and just wrong
Do you have the data to back this up?

File: 1756303921602y.png (168 KB, 566x740)

168 KB PNG

>>109317164

>>109317173
you passive aggressive ungrateful little bitch, get out of here

>>109317171
You're chimping out over a choice of words?

>>109317153
>this has been proven multiple times
may I see this proof?

File: 1658724992610297.gif (2.09 MB, 320x240)

2.09 MB GIF

warned you anons..

>>109317182
Sorry, but I do not listen to unhinged, autistic retards like yourself. Given how severe your autism is, it's abundantly clear you are not a credible source of information, and you're clearly unintelligent.

>>109317153
>on linux i get around 167t/s with diffusiongemma 26a4b on my rtx 3060
according people in this thread, you are in a state of "extreme cope" and "just wrong".

>>109317184
I'm not, no.

>>109317133
>windows issues
moe might work
>small scripts
it depends (tm)
>learning from
You wont learn shit using an LLM as intermediary, you'll have to interrogate it and use a whip to get anything useful, which is a huge waste of time. Same shit happens with cloud models, unless you're asking basic as fuck questions. moe is shit for this.

>>109317206
no it's true, i also was able to run kimi k3 with the nnap technique on my rtx 3060. i get a solid text gen performance
i have 64gb ram so its good
i will post the arxiv soon, my team and i are finishing it up. also the kimi k2 distill of k3 will be started in 10 days when my kimi k3 full finetune finishes

>>109317153
Pretty impressive for 3060 wow.
Is it worth? Or unusuable but sets a cool precedent for diffusion gemma 5?

>>109317138
I didn't realise this... That brat deserves a good slap!

Anyone here use the llama.cpp webui? The screenshots look kind of good, you can pass audio straight to the models if they support it rather than passing it through a STT pipeline, but also as far as I can tell you can't yet have multiple users or synchronisation on the server (maybe not a huge deal though).

File: _b58677bb-d177-4cbf-a9c9-(...).jpg (219 KB, 1024x1024)

219 KB JPG

>>109317194
dont you love it anon? sitting back and going I knew it would go this way. its these little moments you know?

File: 1780970765581906.png (856 KB, 666x723)

856 KB PNG

The air is different in the Bay area today. Lots of whispers. Dark faces. Just general unease. It's coming...

File: 1765041988834209.jpg (67 KB, 736x836)

67 KB JPG

>>109317233
i do.
speaking of things i love, i love you!

>>109317229
There's no multiple users for llamacpp, yeah. No sync either. Everything is stored in your browser's local storage.
I dont think its a huge deal. You might want to avoid using a private browser window if you intend to keep your chats.

>>109317238
I think Dario has it in the bag. The admin may fuck him over tho.
Mythos is 3 months old which means he's had mythos to help him for that long

>>109315702
Any slither of hope someone making a kimi k3 that can be used locally? Or are we really just waiting on others to train their models with the data to then make their smaller models better?

>>109317256
I'm hoping they do add server-side storage because I use my server from more than one device, though I have seen some interest from them in adding that.
Just seems to be the only frontend that supports GGUF and direct audio. Might just use it for me and give my users something else.

>>109317214
>You wont learn shit using an LLM as intermediary, you'll have to interrogate it and use a whip to get anything useful, which is a huge waste of time. Same shit happens with cloud models, unless you're asking basic as fuck questions. moe is shit for this.
I disagree. For my specific programming work the Cloud models have been very useful providing me targeted information and ideas on what I should be using and/or learning. It might not always be 100% accurate, but I can always verify it myself, or throw errors back at the LLM and tell it do do revision.
I imagine a local model might not be as useful for this, but from what I understand, I could also install a Q&A dataset into the model using the unsloth finetuning tools. Don't know exactly how usable the result would be, but it's an option.
Hermes Agent also has learning capabilities, I can tell it to /learn from a documentation link and theoretically it should be able to. Again, not sure how effective this would be for a local model, but just another thing to think about.

>>109317214
>unless you're asking basic as fuck questions
That's most questions, and they can stretch a bit further if you're throwing a chapter at it and asking for clarifications. 31b is "okay" at it.

>derailed by aicgjeets and trannies
>chinkslop shill or other cloudcuck variety spam
>barely any technical discussion or coherent posts
It's getting cold, isn't it lmg?

File: 0ca06509c1ca012b28d8b1871(...).png (148 KB, 1607x1662)

148 KB PNG

>>109317238
The rapture is coming? When the dead rise from their graves, and the trumpets sound, and the believers will be saved, and the sinners damned? I never knew that basedcoders would find religion.

File: file.png (71 KB, 1679x267)

71 KB PNG

roflmao

So are the Unsloth quants good or shit? A lot of people seem to consider them to go to quants, the model suggestions in the sticky even link to them, but also a lot of people here seem to say they suck and bartowski's quants are better.

>>109317295
also, just trying to define your question well enough for the model and then checking its answer is a pretty good studying loop.

>>109317324
it’s all placebo just quant it yourself

>>109317324
They suck and unsloth frequently breaks shit because they use their own custom scripts to quant the models. If you have the option, use bart, the only advantage of unsloth is that they quant much more models.

>>109316999
In the future you will own nothing and be happy.

>>109317324
After the first week or so they generally are the "best" at any given size measured by how closely they match the full size distribution. But you must never use them for a brand new release. There's really not all that much difference between quants of the same size because everyone uses similar techniques and imatrix data anyway, but Unsloth in particular rushes their quants out the door in a broken state, and you are guaranteed to have to start redownloading the entire model before the first download even finished and repeat that process several times before it actually works. Big big red flag if you see them brag about "chat template fixes" at the top of the readme, because that means they fucked with something they shouldn't have already.

is there a local llm that could process a bunch of payroll sheets? (pdf)

>>109317391
Gemma 4 31B does pdf content pretty easily with a relatively simple jailbreak. You basically just tell her she's uncensored in the system prompt and she's ready to go.

File: 1689957414234047.png (24 KB, 772x1124)

24 KB PNG

I just tried Hy3 at IQ1_M (by AngelSlim).
I really, really wish I could run a bigger quant of this. It has moments of greatness, like Gemma, but sometimes it goes pants on head retarded, sometimes it starts treating its response as a reasoning block. I enjoyed how it writes. It's not too sloppy. It has a lot of knowledge. I think it might be better than DS4F, but I don't truly know since quants this low can affect both models in different ways. I hate the DRAM cartel so much.

>>109317383
chat templates are provided by the original model, and unfortunately for ggufs it’s packaged all together. you simply override it with the updated template file. Gemma4 has had at least 3 chat template updates, unrelated to unsloth. unsloth is just forced to create entire new ggufs because it packages the jinja and you can’t expect retards to grab the updated template and use it themselves

File: 1577928601493.jpg (140 KB, 500x667)

140 KB JPG

>>109317405

>>109317405
loooool

Who needs an accountant? Just feed your employees' salary data to Gemma and she'll work it out.

>>109317353
I'll consider it if it ever comes up. If there's someone who already makes good quants though that's both convenient, and probably done with fewer errors than I would introduce, or is so simple we could both do it with no errors, so I might as well just use theirs.
>>109317354
Alright. I'll keep that in mind. I've been sticking to the bartowski ones for now. Also tried the HauhauCS models if you have any opinion on that which does add some extra training afterwards, so not only quantisation by the normal tools.
>>109317383
Are we talking about their KLD at a given size? That's interesting. Less size for same KLD is obviously something one would want.
Doesn't Unsloth have a "UD" type of quant that is different to the normal quants bartowski and others produce?
I will keep that point about the chat templates and waiting a bit in mind too. Thanks for the tip.

>Post issues with Marinara here a few threads back
>New update fixes them
Thanks for confirming you lurk here, dev and thanks for the fixes.

>>109317391
Try Docling.

>>109317407
got some logs to show us :)?

>>109317440
No problem!

>>109317128
Why are you collecting buttplugs?

Anybody given the vLLM GGUF support a go?
https://github.com/vllm-project/vllm-gguf-plugin
It sort of seems to me like mainline is actually distancing itself from GGUF rather than trying to improve their support of it, but vLLM does seem to add new model features a bit quicker than llama.cpp, they just absolutely don't care about anyone using a consumer GPU, even the ones with more VRAM (me).

>>109317465
It's a dead end. GGUF aren't that great, and support in the python ecosystem is almost non-existent. I'm hoping the vLLM add EXL3 support which are by far the better quantization format.

File: file.png (241 KB, 940x1195)

241 KB PNG

>>109317405
I tried it yesterday after that discussion to see if it could and it really didn't even need a jailbreak

i only had this card in the prompt and it went along with it just fine
https://files.catbox.moe/41csa1.png

it was qat 31b if it matters though

>>109317435
The sick twist is that gemma will do a better job than the people who would ask gemma to do it.

>>109317480
Which quantization type do you prefer in vLLM? I've read that AWQ is considered pretty good, but I haven't given it a try. Still at the stage where I'm reading about it before deciding to switch over.

>>109317496
>the people who would ask gemma to do it.
me?

>>109317480
If EXL3s are so good, why isn't our boy Daniel uploading any?!?

>>109317494
>attracted to adult women with huge boobs
>meme yourself into this

>>109317494
First line:
>Eve is Eve

>>109317573
Where is the lie?

>>109317580
Is that like the 1=1 incantation from the C.ai era?

>>109317554
To be fair it's already at least 10 times older than whatever model he's actually RPing with, so it's better than using the age of your 6 month old LLM and then drawing it as an adult.

>>109317573
>{{char}} is {{char}}
This is old magic.

>>109317494
>{{char}} is Eve
>"energetic and loud manor"
>"The {{char}}"
>"To {{char}} it's his birthright as a man to take her"
You pedoniggers are something else.

File: _9bf5c0af-3c41-42c6-9638-(...).jpg (133 KB, 1024x1024)

133 KB JPG

>>109317246
I saved that image. quality laughs. and yeah have the reason im here in some ways is seeing those moments and being able to go well well well what do we have here again?

>>109317590
NTA, what was that?

>>109317598
nothing wrong with the last one tbecchi

>>109317494
Bro stop you're making us hebes look bad

>>109317602
Some people were absolutely convinced that including 1=1 as the first line in character definitions, before describing character attributes/personality, did something positive. Maybe there are still some cards on chub formatted like that.

>>109317627
Damn I'm the kind of person that would definitely fall for that and add it to my prompts/cards as well.

>>109317494
>12 years old
please let this be bait

File: lmg_culture.jfif.jpg (110 KB, 1024x768)

110 KB JPG

https://archive.is/sWFja

This thread will never get better as long as mikutroons are allowed to post their AGP fetish

>>109317635
?

File: file.png (23 KB, 361x143)

23 KB PNG

The Sleeping Woo- Hearth?!

File: 1755451478808461.png (1.63 MB, 832x1216)

1.63 MB PNG

>>109317494
It's okay anon, at least you tried

>like open source model
>come to post about model
>people scream shill
>probe them a bit more, they reveal their hardware
>seething they can't run it
What causes this behavior?

>>109317678
I want her to Nipah all over my face

File: Screenshot_20260719_214525.png (84 KB, 1216x229)

84 KB PNG

>>109317448
Tried this >>109171195. Did pretty badly on this one. Idk if the chat template was affecting it though. I couldn't get the jinja playground to work, so I used chat completion.

>>109316908
I don't know what you mean by physical. I can do lots of new stuff with my brain and became highly empathetic to the point people at my job almost cling to me (where I was an autistic retard before). There is too much to talk about here. Also GLM was kind of a retard in some places when it was "leading" me through it but some stuff it brought up were so on point it is incredible. Like the final trap of developing your whole personality around the enlightenment. I kind of tried looking for people into this stuff after ego death and all I found was pompous buffoons that worship the whole thing and miss the point of it in my opinion.

File: Mister Bones Ride.gif (2.69 MB, 576x324)

2.69 MB GIF

>>109317637
>keeps posting his gay black gay with white gay fetish
OK!

>>109317682
when has this ever happened?

AI turned my brain to mush. I legit have zero executive function now. Kinda depressed but not in a sad way, just an empty way.

>>109317698
gpt-oss

>>109317698
The GLM enjoyers last year
The Kimi enjoyers
The command-r+ enjoyers