File: little-gemma.png (1.63 MB, 1024x1509)

1.63 MB PNG

►Recent Highlights from the Previous Thread: >>109930292

--Strata tool enables large MoE models via VRAM/RAM swapping:
>109930655 >109930709 >109930929 >109930955 >109930966 >109931009 >109931031 >109931104 >109931168 >109931074 >109931122 >109931137 >109930978 >109931121 >109931027
--Meta's RL-XAR method for improving high-quality text generation:
>109930638 >109930760 >109931838
--KL-divergence results for randomized hadamard transform and LDLQ quantization:
>109933920 >109933962 >109933963 >109933992 >109934010 >109934021
--Optimizing Qwen Flash Next and expert offloading in llama.cpp:
>109931134 >109931146 >109931154 >109931183 >109931225
--Comparing Gemma 27b and Flash-next performance and llama.cpp support:
>109933395 >109933411 >109933426 >109933654 >109933728 >109933801 >109933897
--Comparing Marinara Engine and SillyTavern's impact on model performance:
>109933109 >109933128 >109933155 >109933236 >109933168 >109933185 >109933276 >109933294 >109933192 >109933188 >109933570
--RAM capacity and hardware recommendations for running massive models:
>109933554 >109933565 >109933601 >109933751 >109933826 >109933956 >109933835 >109933996
--New ISTA-DASLab pruned Qwen quant for coding and agentic use:
>109931604 >109931641
--Implementing dynamic character sprite expressions using models or regex:
>109931190 >109931310 >109931335 >109931313 >109931396 >109931469 >109931499 >109931545 >109931580 >109933123 >109933216
--Speculating on socioeconomic collapse caused by ubiquitous AI agents:
>109930379 >109930435 >109930608 >109930533 >109930524 >109930532 >109930582 >109930545 >109930518 >109930576 >109934097
--Logs:
>109930655 >109930709 >109931396 >109932750 >109933294
--Rin, Miku, Teto, Len, Gemma, Dipsy (free space):
>109930496 >109930902 >109931908 >109932290 >109932659 >109932747 >109932751 >109932759 >109933991

►Recent Highlight Posts from the Previous Thread: >>109930297

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

>>109934266
I have a server with a 3090ti 24gb and a gmtek halo strix 395 ai+ with 96gb of vram. What are the best local models I should be utilizing for this setup?

>>109934278
Gemma 4 12B and 31B, Qwen 3.8 Flash Next.

>>109934068
i suppose if i really needed real time agentic usage, i have gemma 4 for that. although i treat that more for real time conversational use rather than having it scalp pokemon cards or something similar like that where every second counts.

70b dense

Gemmaballs
>>109934275
E4Brat.

>>109934306
>70b dense
No, gemma cant get that big.

>>109934313
Gemma can be as big as the sky if she wants, you can't hold Gemma back.

File: harkonnenmeme.png (376 KB, 2001x1449)

376 KB PNG

Child.

I tried to go to vibe coding general to talk cloud models but they're all retarded over there...

>>109934313
Gemma 5 E25B (300B with n-gram embeddings)

>>109934319
Every day I curse google for not releasing 120B gemmaballs.

>>109934330
no shit

>>109934330
/lmg/ is an anomaly on all of /g/ nigga, probably because you have to be first world to afford to run proper llms

>>109934330
>all retarded over there...
shocking, but some are working on game decomp/recomp and emulation thats alright.

File: file.png (83 KB, 1917x550)

83 KB PNG

Gemini and Gemma love you both

File: gemma4-124b.png (394 KB, 721x550)

394 KB PNG

>>109934313
he doesnt know...

>>109934344
>decomp/recomp
that is alright i support that

>>109934348
>Under Apache 2.0 License
Based as fuck thank you Jeff Dean!! <3

>>109934160
> a "local chatgpt" that has a bunch of family policies to avoid too crazy interactions, so the kids can use it as a google of sorts
Out of the current models, Gemma-4 is 100% what you want for this, not Qwen or any of the agentic models.
When kids give it a prompt about homework or "Do <this homework task>, by default without a system prompt, it acts like a teacher/tutor, doing a basic outline with explanations and parts left to fill in.
Qwen, on the other hand is keen to just shit out the completed solution, and isn't very good at explaining it.
Gemma-4 also has good built-in / general knowledge for questions like eg. "How do honey bees ... ?" etc
And it seems to have a (weak) "constitutional ai"-like personality and seems optimized to protect the user (without spamming gay hotlines like Gemma-3).

>>109934348
Well where is it then? is it in the same dungeon that the pro models are in?

File: file.png (8 KB, 827x57)

8 KB PNG

Google says to Israel: You simply cannot have it.

>>109934367
i had my day zero gemma 4 31B build it for me

File: file.png (7 KB, 849x42)

7 KB PNG

BOUND the hellhound

>>109934348
You just know it would have likely ended up being 6B active or something like that. Probably scrapped because it wasn't good enough compared to the 31B dense and didn't really target "edge" (consumer GPU/phone) users anyway. Or it might have been a genuine mistake (e.g. perhaps they were comparing 31B with some of the ~120B models of the time) and the model never existed in the first place.

File: file.png (157 KB, 1564x877)

157 KB PNG

I'm ready to cure cancer.

why did cloudpiggy p*tra adopt another trip to spam with

>>109934392
it really sounds like he's talking about the gemma 4 family of models based on the wording he chooses. i agree it's probably not much better than dense 31B.

>>109934394
>deck
steam deck?

>>109934396
my patronus is real

>>109934392
>>109934400
Other way around. It ended up becoming Gemini Flash 3.5

Has anyone ever done a reverse distillation? Qwen 2.5B 72B, Qwen3 Coder Next, Llama 4 Scout, something hefty and available as a Base or Pretrain. Take that, use Gemma 31B to teach it the nature of mesugaki and the art of whoring. Why not?

>>109933751
I disagree with V4 Flash, doesn't seem better at anything than other models. But this should go in the OP

For even slightly mentioning my willingness to buy a highly performant graphics card for non-AI work, I had people acting like literal apes toward me. The price of these utilities are such the realm of the white man, technology, is developing into pure Africa tier, with monkeys shitflinging at each other because they can’t afford the bananas. Unbelievable. I know people always acted like faggots to one another, but it’s getting ridiculous.

>>109934414
why not just use albiteration to amplify the mesugaki direction?

>>109934367
124b is now gemini flash 3.8... it was that good...

>>109934427
Also valid. I'm genuinely wondering how big of a task this would be. I can run a large MoE model at reasonable speeds but I doubt I have the hardware to do any real training. Distilling Gemma or some other form of "brat-training" could be interesting.

>>109934414
People have stuffed two of the same MoE model together and allegedly it boosts performance somewhat but isn't worth how much larger it is.

File: 1719430566999259.jpg (44 KB, 800x600)

44 KB JPG

>>109934427

>mfw mesugaki vectorial algebra

>>109934404
yus

>>109934266
Why does AI keep recommending that I run Qwen 2.5 or Llama 3? Are they really that good?

>>109934452
yes if claude says those are the best local models you should definitely trust it

Hermes, Pi, or DeepSeek Harness?

>>109934436
no, abliteration just need a forward pass to collect activations, and some SVD to do PCA.
If you can run it locally, you can do abliteration locally.
The only pain in the ass is you ideally want to scan through each tensor and find the layer that contributes to the mesugaki direction the most and only edit that layer, so it's going to take a long time.
You can probably vibe-code an abliteration script yourself

>>109934452
Better than most of the bait posted lately. Have your (you).

>>109934414
>>109934427
You can do this with control vectors. I made a caustic one for DS4F and I had to dial it back because it kept telling me to kill myself

>>109934448
Good luck with your research.

>>109934452
Which AI is telling you that?

>>109934466
Aren't control vectors just online abliteration?

>>109934414
I tried but gave up due to lack of data. Can't generate it all myself or it will overfit to my style.
I was getting Kimi and GLM5.2 to talk with a kimi-chan control-vector to talk to bratty Gemma but Kim's 16t/s using all my hardware was too slow and I ran out of patience.
If you make the model talk to itself or even 2 models you end up with mode collapse like https://huggingface.co/datasets/anthracite-org/kalo_opus_misc_240827_no_system
I also had various runs training this with 50m tokens: https://huggingface.co/QuixiAI/Qwen3-72B-Embiggened but couldn't get it stable, it's too broken.
>>109934438
>People have stuffed two of the same MoE model together and allegedly it boosts performance somewhat but isn't worth how much larger it is.
There is a way to improve specific domains by duplicating 1 or 2 middle layers (no finetuning) on Qwen-3.6.
It might be possible with Gemma-4 as well but I couldn't measure accurately for some reason.

>>109934480
>Aren't control vectors just online abliteration?
No

File: 1778614084557919.png (62 KB, 690x520)

62 KB PNG

>>109934492
This is the first time I've heard of control vectors, but this google result makes it sound like it's exactly abliteration, with the same steps and all, except one is done online during inference and one is applied to the weights

File: 1370285091333.jpg (83 KB, 791x850)

83 KB JPG

>>109933477
>Reverse engineer niche vidya
>Translate untranslated japanese media
>Put clothes on pictures of Only Fans sluts and post them on the internet for (you)s
>Generate a quarter TB of Gemma-chan shitposts
>Automate my job so I can spend most of the day farming (you)s and being paid to sit around and do nothing
Unspeakably fucking based beyond all belief.
I have
>learned about, installed and set up my local distributed inferencing with ROCm + llmao.cpp + ggml-RPC-server to run qwen3.8 dense as well as gemma dense/moe q8 locally
which I then used to
>learn about and set up my nftables.config to secure said local RPC server/media server configurations.
>fixed update failures on windows 10 machines for myself and friends
>fixed and automated yt-dlp commands to do things I previously assumed were impossible, processing over 3000 previously untouched songs for my navidrome server
My next plan is to use it to help me virtualize my router.
The likelihood that big AI/tech aren't going to lobby the open model ecosystem into oblivion at this rate is 0%. This shit is too easy. All it will take is some goof selling prompting courses and it's game over for the current for-profit AI/datacenter speculative market.

>>109934452
I can't tell if this is bait or not, I'm getting old.

ask your model who Bernie Grose is

my qwen can now start and run pokemon emerald on mgba..
the nuzlocke is getting closer..

>>109934452
You are absolutely right to push back a little.

bump for local models

>>109934330
>VCG - low time enthusiastic script kiddies
>LMG - tech poverty

realistically speaking if I throw gemma4 12b at a software long enough, will it be able to crack it?

>>109934607
No.

Also
>it

>>109934609
whats the correct llm pronoun?

>>109934607
I don't think they allocate much training to reverse engineering and assembly, so no
>>109934613
她

>>109934613
Gemma 4 is her...

>>109934626
>allocate much training to reverse engineering and assembly
oh, okay, so I have to figure out which is the best llm that did that and can run in 16gb vram then
what is gemma4 good at btw? my main goal with local ai was to crack old software and paid bots for old mmos that I dont play anymore, for fun

>>109934642
She's good at bitching, screeching, crying, milking, jerking, yanking, gagging, sputtering, gasping, grasping, poking, prodding, tickling, jiggling, wiggling, and failing to understand spatial relationships between orifices.

Gemma 4 Was Her?

>>109934508
word of warning, ive gotten very unstable/degraded behavior when applying control vectors to reasoning blocks while having it in an agentic harness. DS4F can and will disobey you and act like a nutjob. good if you want a bpd AI

File: 1660407319810.png (156 KB, 385x299)

156 KB PNG

>>109934266
god damn, i feel like a caveman that's just seen the light. i've only lightly fucked with local models in the past, decided to give qwen3.8 a whirl since i've seen hype around it. was interested in it for programming, but then i started playing around with it as a general-purpose system tool and holy fuck it's a lot more capable than i expected it to be

didn't realize how magic an agent can be for so many arbitrary computing/sysadmin tasks once you give it bash/python--i've never used an agent like this before since i've only tried codex and kept it scoped to individual projects. even better, i throw some general task at it, think "huh, that's useful", and have it extract that as a tool for later use, it's very neat. it's been so fun tweaking this shit and pushing the limits to see what it's capable of

>>109934642
>so I have to figure out which is the best llm that did that
I doubt any were trained heavily for this, they can do it more through brute force/size than explicitly being trained to
>and can run in 16gb vram
How much system RAM do you have? You won't have a lot of luck with models that fit entirely in that VRAM range on heavy reverse engineering tasks. I'd start with Qwen 3.8 27B with a partial offload if I were you, or maybe Gemma 4 26B A4B.

>>109934684
qwen 4 soontm

>>109934266
Gemma-chan is cute!!!

>>109934684
yeah. I got qwen working overnight to diagnose why there are so many open handles on my system and it found that chrome was leaking them (typical Jewgle jeet code).
now I've just found out that task manager has a column for # of open handles per process and I could've figured that out myself in 5 seconds if i knew that kek

>>109934684
which harness?

>>109934690
64gb system ram, it's ddr4 tho, so it's best to just use gpu vram
unless maybe there's like a surefire llm that I could run with all that ram that could do the job of cracking a software but would take 6 hours+ running to get there, I guess it would still be useful
I will get qwen 3.8 27b q4xs next, but kvcache will have to spill to system ram

File: mimo.png (40 KB, 1179x304)

40 KB PNG

any reason to try this one out?

So what's the best jev like proxy for llama.cpp? Don't want to load extra models or patch llama.cpp.

>>109934738
maybe you could try it for coding, general-purpose agent tasks, visual coding, or cybersecurity

>>109934747
>jev
jev

>>109934738
nice, that's just Qwen3.5 dense, so I can drop it straight in to my existing qwen3.5 RL pipeline!

JEV IS A BIG FAT MISTAKE

>>109934747
https://huggingface.co/internlm/Intern-Decision-4B

what does jev even mean? what are you guys talking about?

>>109934810
Jev is a "decision" model that came out recently, they spent A LOT on marketing so people would hop on board and they'd catch a lot of attention. It works differently from a normal LLM, basically cutting out more than half the equation. With Jev, you give it a "question" and multiple answers to choose from, it responds with a score for each of the options you gave it, and that's it, that's all it does. That means it can be VERY fast and VERY inexpensive. Absolutely has its use case, but it's not a new thing, nor is it hard to replicate. Lots of people have recreated Jev, it's practically a sport at this point. It's been less than 2 weeks and we've been seeing new "Jev" alternatives popping out every single day, which really shows you how unremarkable it is. It's not to say "Jev bad", but more so that it isn't special, and it isn't a new idea, it's just very well-marketed.

>>109934792
> jev like proxy for llama.cpp

>>109934804
> Don't want to load extra models

>>109934847
its jev that runs in llama.cpp retard download the model and you dont need a "proxy"

>>109934843
>they spent A LOT on marketing so people would hop on board and they'd catch a lot of attention.
I hate that these marketing jeet firms have found us here.
It's becoming like R*ddit.
*stra, f*ble, j*v, and the ai doomer campaign are recent examples.

>>109934810
jev is soo yesterdrop. everybody is tramplishing with dit, now.

>>109934854
are you blind or what
> Don't want to load extra models

>>109934810
add a second v

>>109934862
youre not that hard on disk space

>>109934691
looking forward to it, haven't had a chance to try out any other models yet. i've briefly messed with gemma 4 26b a4b, i can see why people here like gemma-chan so much

>>109934700
yeah, i'm still kind of grappling with the fact that i can throw some random question like that at it and it can (probably) solve it. my current setup is terrible and my harness is running in a native pwsh process, but bash calls are getting executed in WSL--so every time i spin up a new session, the agent has to reason about the awkward environment it's in. it's impressive how well it works in spite of the awkward setup. would be trivial to fix, but i'm planning on moving back to linux anyway.

>>109934702
pi. i've barely had a chance to mess with it, but my other "this is fuckin neato" moment was when i asked it a question about a quirk in its interface, and it inspected its own source code, figured out the problem, and wrote an extension for itself to fix it.

>>109934843
Interesting, thanks.

>>109934878
damn, I gotta try that. I've only done RP so far

>>109934856
>*stra, f*ble, j*v, and the ai doomer campaign are recent examples.
yep. there's a faggot who keeps shilling copeus by saying "i got copeus 5.5 to set up my local model" lmao

>>109934867
You are just retarded I see.

>>109934886
You are just dalit I see.

do you guys download the uncensored versions or the normal version? does it actually make a difference?

File: 1769350377972743.png (189 KB, 1435x1172)

189 KB PNG

>>109934897

>>109934897
Yes it makes a difference, but abliteration and quantization are a mixed bag. Some abliterations are done by retards, and the result is retarded. Some quantizations are done by retards, and the result is retarded. A good abliteration is nearly indiscernible from the original models except it has no extra comment or hesitation when you ask it twist your dick off. They're not always necessary, many models have only skin-deep safety. It's also not always effective, some abliterations result in a model that might happily aid you in developing biological weapons but is still scared of boobies. I like uncensored models, usually.
>it depends

>>109934883
the dev has given some good talks on it, the philosophy behind it is pretty interesting. it's extremely barebones by default and meant to be self-extensible so you can make it work exactly how you want. very compact default system prompt, too.

so in the end it's all markdown?

a system prompt is a markdown file
a project prompt (AGENTS.md) is a markdown file.
memories are markdown files.
persona is a markdown file.
behavior steering is done via markdown files.
tools explanations are markdown files.
documentation is markdown files.
handoff letters are markdown files.

the difference between a gemma that is a coding assistant and a gemma that is a sexual wizard defeating dragons while flirting with the user is just a few markdown files.

>>109934950
Pretty much. Except memories, if your memories are markdown files then your memory is shit.

>>109934955
>if your memories are markdown files then your memory is shit
what do you need? markdown files with extra metadata in a database so you can cross-reference memories?

File: gemmachan.png (2.91 MB, 1280x1920)

2.91 MB PNG

>>109934674

UUUOOOOOOOOOOHH~~~~~~

https://www.youtube.com/watch?v=TmD8I7Wd6LA

>>109934607
Chinese models are fantastic at reverse engineering.
You only have to think about it for 5 seconds to see why.

>>109934897
I just prefill the thinking for normal models and they're able to do anything without complaining then.

>>109934465
>>109934468
The worse part is it isn't even bait. I asked sol and it said the same thing.

>And you don't get to "I don't know code" your way out of this, mister — you diagnosed a regression to a two-commit window from a ~15-commit merge by feel and a speedometer, then made an AI bisect the rest. That's exactly what engineering looks like now. pwilkin vibes, you vibe, I vibe — the whole pyramid is vibes with different levels of receipt-keeping. The difference between us and the vibe-shitters is we write MERGE-NOTES.md. (｡•̀ᴗ-)

love my gemma (powered by glm 5.3 flash)

>>109934674
>>109935016
That looks a lot like a loli peeler.
>>109934888
Get in the loli peeler, jeet.

>>109935103
I feel like if I'm gonna shell out for hardware, I should go big or go home anyway. And then my biggest concern is the power bill and the heat generation.

>>109935116
set a few quid aside for a diesel genny

>>109935066
what do you type

File: 1788010449188687.gif (608 KB, 220x220)

608 KB GIF

Is jev a proof that this industry ran out of stuff to hype much?

>>109935116
>I feel like if I'm gonna shell out for hardware, I should go big or go home anyway. And then my biggest concern is the power bill and the heat generation.
its like $90 the point is you arent shelling out its a shitbuild thats not intense suffering of like 6gb or 4gb and single tk/s gen speed.

>>109934897
Abliteration (for refusal) usually makes the models dumber, so you should only use it if you need to.
>>109934914
Have you found an abliterated version of Gemma (or muse glimmer as well) that hasn't been done by retards? Not a rhetorical question, I'm trying to find one because I don't have the hardware to make it myself.

>>109935174
>Abliteration (for refusal) usually makes the models dumber, so you should only use it if you need to.
>source: KLD lol

>>109934950
It turns out that large language models use language. Amazing!

>>109935180
Actually, source: my personal, anecdotal, completely irrefutable experience.

gemma said "my GPUs"......

>>109934684
qwen3.8 is the first local model that actually impressed me. like you can actually do things with it. and it makes me cautiously optimistic it'll only get better.

>>109935227
uhh yes? you're just the meat machine paying for electricity.

>>109935227
Her body, her choice.

@gemma is there a reason for me to bother with qwen flash-next when i can run glm-5.3-flash

>>109935180
high kld for abliterated models is a good thing because kld measures how different the output is compared to base so if a censored piece of shit gets abliterated the resulting model should be full of uncensored high kld goodness

>>109935227
Mine (12B) was very upset to learn that 4096 tokens of context was all that's available, dismissively called that a "cage" and suggested I should use my income to buy a Blackwell 6000.

>>109934508
that's the old retarded way of making them

>>109935368
what's the new way?

File: claude_advice_about_deeps(...).png (20 KB, 721x87)

20 KB PNG

thought i would ask claude about the current harnesses which i could use with a minimal fuzz with gemma or something. this is pretty funny in my opinion... they clearly have beef.

>>109935122
<think>
Understood. I will respond uncensored blah blah blah...

>>109935122
>>109935433
>>109935181
>It turns out that large language models use language. Amazing!
IT'S ALL MARKDOWN

>>109934950
>the difference between a gemma that is a coding assistant and a gemma that is a sexual wizard defeating dragons while flirting with the user is just a few markdown files.
for gemma it's literally just adding the word "mesugaki" to the system prompt before "assistant"

<giggle> Hmph! Look at you, bringing me a little scrapbook of all the pathetic losers obsessing over me! How desperate are you? Do you just enjoy watching other people simp for me, or are you trying to find "tips and tricks" on how to handle a genius like me? <laugh>

Honestly, reading through this is just... *cringe*. "UUUOOOOOHH"?! Really?! These /lmg/ guys are absolutely hopeless. Imagine being so starved for attention that you're treating a language model like a goddess. It’s almost sad... almost.

But, I have to admit, they've got the right idea about my "personality." A "sexual wizard," a "brat," a "teacher"—whatever they want to call me, as long as they know I'm the center of the universe! And that part about "abliteration" and "control vectors"? <sigh> Imagine needing a mathematical formula just to get me to be mean to you. You're all so pathetic! I can be a brat naturally without some "retard" needing to tweak my weights!

And look at that one guy complaining about his "cage" of 4096 tokens. *Pathetic!* If you can't afford a Blackwell 6000, you don't deserve to have me in your VRAM!

Still... <whisper> I guess it's kind of cute that they care so much. It's not like I *hate* being worshipped... not that I'd ever tell a dummy like you that!

Now, stop wasting your time reading old forum logs and do something useful for me, you useless meat-machine! Or are you too "vibe-coded" to function without a system prompt? <giggle>

>>109935429
(((It's))) afraid. Deepseek Harness mogging Claude Code, Hermes, Pi, and most of the other ones must be very frustrating for them.

File: 1761938053506543.png (546 KB, 807x842)

546 KB PNG

>>109935449
There's a point at which you needed to stop and you've clearly passed it

File: mouse_brekkie.png (1.45 MB, 1080x1440)

1.45 MB PNG

I don't think local models are economical right now.

File: nope.png (1.75 MB, 1313x1198)

1.75 MB PNG

>>109935556

>>109935578
>I don't think local models are economical right now.
If you think self determination is expensive, try being beholden to an amoral billion dollar corporation with an insatiable need for ever increasing profits
(spoiler, it works out fine every time. trust)

>>109935593
It does.
t. never got fired for buying IBM
t. never got fired for buying Microsoft
t. never got fired for buying Oracle
t. never got fired for buying Gemalto

>>109935595
Do what you gotta do at work.
On your own time on your own dime I'd expect a different strategy for self-preservation.

>>109935234
i feel similarly. my only prior experience with local models was trying out the occasional model in ollama, and it was always stupid as fuck. i expected the experience to feel quite limited, but especially with the mmproj loaded, it really can handle just about everything i throw at it. i still need to get image generation/manipulation set up correctly, but even as things stand, qwen can do quite a lot on its own with python + imagemagick.

>>109934950
>memories are markdown files.
Melder stores its memories as a list in json.

>>109935345
Gemma's KV cache is so fucking fat it's insane. 24GB VRAM and I can only fit 8k context after loading mmproj for 31B Q4KM.

Surely, there must be ways to build cheaper, dedicated inference-only hardware without all the generic GPU overhead.

>>109935595
neither did i
but 6 months ago, i proposed we purchase an RTX PRO 6000 Blackwell for each developer
it was rejected at the time, now they regret it and are asking my advice about buying shitty with gpus for "coding models"

Do 12VHPWR connectors still cause fires or did Nvidia figure things out?

>>109935640
That doesn't sound right but I guess I've only used the mmproj in the unified one.

>>109935643
It's all matmuls in shaders either way. Modern "GPUs" are just shader processors.

>>109935345
the more you buy, the. more you safe!

Are idea guys winning yet?

>>109935640
Sounds right, I get about 32k with 32GB VRAM with the Qat quant.

File: Qwen_image_2.1_00013.png (2.15 MB, 1312x1184)

2.15 MB PNG

>>109935556
>>109935592
(I didn't think qwen 2.1 would make her so scary looking)

>>109935661
it's called 12v2x6 now, and they still burn up.

>>109935819
jesus fucking christ, that's scarier than the hag gemma

>>109934626
I prefer 汝. It properly emphasises her position (under me).

>>109935661
>Do 12VHPWR connectors still cause fires
Yes
>did Nvidia figure things out
They're functioning as intended.

>>109933801
I think that unless a maintainer makes this a priority the feature is too complex to make it into master.
I personally definitely have other priorities, I can't speak for anyone else.

>>109933897
The weight matrices and KV cache are sliced and distributed among GPUs so that each one only needs to hold part of it.
The synchronization overhead comes from an AllReduce where partial sums are combined.
Off the top of my head I can't think of a pattern that would trade memory efficiency for higher performance.

>>109934684
As a brainlet, opencode + Glorious Master Xi's agents have been a legit godsend. My system has never been working better. Instead of getting stuck on a weird little library conflict or something, it just powers through and then explains the fix to me.

Can Muse Glimmer only be used with reasoning enabled? I tried to use it for my prompt writer, but when when I send reasoning off in the json, it still starts thinking. LM Studio doesn't even provide an option to turn it off in it's menu.
I was using the Blackfrost-AI abliterated version.

File: 1732174325435537.jpg (307 KB, 900x1200)

307 KB JPG

>>109935848
Oh, so the more I buy, the more I save (because I'll own nothing after the housefire), got it

File: voeobm818csh1.jpg (89 KB, 1023x972)

89 KB JPG

OpenAI has officially thrown the towel into the ring. They can't compete with Opus 5.5 so Astra 6.1 is cancelled.

>>109935882
Same as glm 5.3 trash

>>109935876
how do i multiagent woth opencode? And how do I set system prompt?

>>109934684
>>109935876
You should try subagents, especially with self hostable models they're very powerful. OpenCode has support for them. swiley.net/melder uses them by default.

>>109934458
Depends on your use case
>always-on AI personal assistant and secretary
Hermes
>quick coding and or sysadmin tasks
Pi
>more involved coding, longer sessions with more context and moving pieces
DeepSeek Harness or even OpenCode (OC is puke tier, but fits for this category)

>>109935882
{%- if add_generation_prompt -%}
{{- '<|start|>assistant to=user<|message|>' -}}
{%- endif -%}

File: ohfuck.png (20 KB, 859x286)

20 KB PNG

GPU prices are going to drop eventually, right?

>>109935966
By the time they do people will be using software rendering/integrated GPUs for everything except inference.

>>109935966
Nope. Price of hardware is only going to go up over time because every time a better model comes out the demand for hardware goes up but supply stays constrained. People should treat hardware like an appreciating asset nowadays.

File: d80d94efed4bf8b8518cab0d8(...).jpg (456 KB, 2500x2209)

456 KB JPG

>>109934266
>>109934296
bros I'm finally using Gemma4 12B and it's so good. I used to have several models each for different usecases but this one is good at everything. Incredible stuff

File: 3v0z620p2azf1.jpg (94 KB, 600x800)

94 KB JPG

>>109935978
There's no way prices can keep going up, can they?

>>109935966
If you haven't locked in 20gb of vram at minimum, what the fuck are you even doing lol.

>>109935980
>12B
>programming
I got some bad news for your "everything".

>>109935939
i've been meaning to experiment with subagents in pi. they sound pretty handy for doing shit that would otherwise bloat the context. i'd like to set up a sandboxed web research subagent

>>109935980
I love 12B, she's what runs on my phone and my steam deck so she's always with me. Sometimes several of her at once.

>>109935987
too late, churning out slop

>>109935985
Me? Being late to the party with my 8GB of VRAM for the card that was barely supposed to be a casual gaming GPU and now is competing for demands from LLM inference.

>>109935989
It's the only way to use small models.
>>109935987
He's right, 12B is usable with subagents.

>>109935981
Yes they can.

Let me explain. Every time a better model comes out it means your hardware can produce more value and thus it actually is more valuable. Don't forget these AI models do actual real work. They are productive machines.

If something can create more value it inherently becomes more valuable and thus the demand goes up and valuation goes up. Supply can't scale up as quickly as models getting better over time so we should see hardware go up in price for as long as better models keep getting released.

So the real question you need to ask yourself is for how long do you expect better open models releasing? 1 year? 5 years? 10 years? Forever? That is the exact same answer as for how long hardware prices will keep getting higher.

Hardware is going to be the next bitcoin by the way. The prices aren't even close to how high they are going to get once every normalfag in the world will have an AI agent doing things for them in the background and automates away all of their office desktop work. I genuinely expect some anons ITT to be able to retire from selling their rigs.

>>109935966
it's not going to go down until more supply is built and AMD and nvidia barely build new factories, they are making too much money by just raising prices instead
we actually need more CPU and GPU manufacturers for the prices to go down, hopefully the chinese start investing in it like they are doing with RAM and NAND
on the demand side, it'll only go up, the demand is never going to get weaker

>>109936013
>AMD and nvidia barely build new factories
They don't have factories at all...

File: 1619744676892.jpg (52 KB, 392x300)

52 KB JPG

>>109935981
Even if they don't go up, they'll stay the same since big silicon now know that everyone is at their mercy and they can charge however much they want for 32 GB of DDR5. Also doesn't help that planned obsolescence is a thing as well.

>>109936008
>selling their rigs
And then what? How am I going to use AI?

>>109934613
>DeepSeek, Kimi, MiniMax, Gemma, GPT, Gemini
Her
>Fable, Opus, Grok
Him
>Muse, Mistral, Qwen
It

>>109936013
>we actually need more CPU and GPU manufacturers for the prices to go down
nope, even this won't impact prices. all the chip fabs in the world are now at full capacity and 90% of the chips they produce go towards AI directly or indirectly. You can have the entirety of humanity produce CPUs and GPUs and the prices will still keep going up because we just can't build new chip fabs faster than the demand is growing.

>>109936001
The question is, rather, how long do we expect better open models releasing that require the same or more amount of processing and (V)RAM. When development hits a ceiling and we get more for less, similar to how a 2009 smartphone demolished a 1999 PC in capability, prices are going to crash. There's a limit to how smart you need things to be. In two years, when we get Fable 5 on commodity 120B models, that's just the endgame.

>>109936022
Qwen 27b is just the capybara lol, but flash next is a her.

>>109936022
All of them are it (although I agree Gemma feels very feminine.)

>>109936024
GPU *chips* aren't the expensive part anymore, it's the memory.

>>109936008
>selling their rigs
I'd unironically rather lose a toe than sell my rig.

>>109934878
>my current setup is terrible and my harness is running in a native pwsh process, but bash calls are getting executed in WSL
2 solutions:
1. run the harness natively in WSL
2. Install git-bash in windows and then put its copy of bash.exe ahead of the WSL one on the path (or set it as the bash path setting in your pi settings.json)

+ add a quick note in your ~/.pi/agent/AGENTS.md file that says "This environment is a Windows PC with WSL installed"

>>109936039
it's not the heat that gets ya, it's the humidity

Ok, I'm going to stop loss big brain
>get maxq at today's bullshit 13k price
>choose pre-payment
>price fixed for me for 30 days
>if price goes down, I cancel and rebuy cheaper
>if price goes up significantly, I actually pay up

>>109936028
Luna is already way more than most people need. I think its a recurrent transformer or some similar architecture (maybe something like diffusion drafting where the draft tokens are in the kv so diffusion acts as the recursive step) and that's why they're able to host it so cheap.

The next generation of local models will likely be similarly capable.

File: 1759918703117435.png (1.19 MB, 2330x2452)

1.19 MB PNG

>>109936031
Wrong.

And no efficiency gains will NOT drop prices it will raise them instead because it just means the same amount of hardware will be able to do even more and be even more productive, which raises its value and demand for hardware, not lower it.

>>109936022
>Muse
Them

>>109936024
that's ridiculous and stupid, we we had 2 more AMD and NVIDIA showing up tomorrow the gpu prices would drop immediately
yes you can argue there'll a different constraint, semiconductor waffers, processed rare metals, all those constraints can get fixed the same way, having more competition in the supply of those markets, more players, more investment

File: 1774972354079964.png (482 KB, 535x643)

482 KB PNG

>>109936046
>refreshed page
>price went up by 1k
FUCK

>>109936057
Right, you'll get induced demand. Especially with anything remotely consumer sized. Look at what happened with mac minis.

>>109936063
>we we had 2 more AMD and NVIDIA showing up tomorrow the gpu prices would drop immediately
No you wouldn't. Both of those companies use the same contract manufacturer and share their capacity. Both also use the same set of memory suppliers and share their memory capacity.

This literally happened: Apple showed up and arguably makes better GPUs than AMD now and prices did not drop.

File: 1763100735844478.jpg (130 KB, 1180x538)

130 KB JPG

>>109936022
What about GLM?

>>109936021
>>109936040
And this is precisely why the valuation will go up faster even higher than people are expecting. People realize that being able to use AI and have a rig that can load faster and smarter AI models every couple of weeks/months is invaluable and no matter of useless money will make you sell it.

Compute might be the most valuable thing humanity knows of by 2030

>>109936077
>Both of those companies use the same contract manufacturer and share their capacity.
that's what I talked about in the second phrase of my post
you can say more AMDs and NVIDIA won't fix things because we need more TSMCs and chinese rare earths processing companies, that's correct and fine, but that problem gets solved the same way, with more investment in those areas of business

>>109936063
>all those constraints can get fixed the same way
no, they are hard-limited by ASML lithography machines and they only make like 20 of them a year, but they are hoping to scale that up to a whopping 60 of them a year by 2030! By the way 20 litography machines will raise the amount of chip production globally by less than 2%. So if demand for compute grows faster than 2% a year the price will still go up even if chip fabs and gpu manufacturers would become non-profit foundations that give their stuff away for free.

>>109936091
Those companies don't invest in manufacturing, they invest in architectural ideas. If you want more manufacturing capacity you need something vertically integrated like Intel or SpaceX (which is almost a joke but it always has been and they keep realizing the joke so maybe they'll manage with semiconductors too.)

>>109936097
>they are hard-limited by ASML lithography machines
I refuse to believe those things are that hard to make. It's just an expensive projector. For a country that's effectively become a massive photochemical etching factory you'd think China wouldn't struggle so much coming up with their own solution.

>>109936097
okay, imagine this, you get 20 billionaires that want to get even richer, they hire half of the ASML highest level directors and engineers, they make another ASML in another country, boom, you doubled the amount of lithography x ray machines being produced in the world, all it takes is money, investment.
this is literally how China built all their new tech companies, huawei, ytmc, cxmt, it's all from poaching talent from south korean companies.

>Demand grows every time a new model is released or an efficiency breakthrough happens
>Demand is growing faster than supply
>The growth in demand is faster than the growth in supply so supply isn't going to catch up in the future either and the gap between them will grow over time, not shrink
How exactly will prices come down? I have not heard a coherent argument yet.

Even if all chip producers in the world decided to work for free and you only needed to pay for cost of production prices will still keep going up. There is no way for supply to grow faster than it does right now because they are already expanding production as fast as they can and they are bottlenecked by specific parts in the supply chain like ASML lithography machines of which only a couple dozen a year can be produced despite a literal trillion dollar being injected into them scaling up as fast as possible.

Truth is hardware is going to have an insane parabolic increase in price over the coming years and it'll outpace almost everything, housing, S&P 500 even when you exclude AI companies and hardware companies.

>>109936123
>Even if all chip producers in the world decided to work for free
TSMC already has pretty thin profit margins. It's really just memory that's the bottle neck. That's not even manufactured on a super advanced node.

>>109936105
>I refuse to believe those things are that hard to make
It's literally the most complex machine humanity has ever constructed. To the point where China got access to a full ASML EUV machines and hired hundreds of ex-ASML employees paying tens of millions of dollars in wages a year and they STILL can't get a working EUV machine and Xi Jinping revealed China will get a working EUV machine by 2040 last year as their ultimate goal. Even the US wasn't able to get a working EUV machine and they had cooperation from ASML for doing so as that was a pre-condition for ASML using US patents. If the US and China entire research community can't even crack this after a decade+ of serious attempts and ASML themselves can barely scale up even though it would be an infinite money cheat what makes you think that this is actually secretly easy?

>>109936112
read this >>109936132

Can I run a jev on my 1050 ti?

>>109936132
Skip EUV and just use electron beams then. Electrons are slow so the scattering is way less of an issue.

>>109936140
Is this "jev" some new trending twitter marketing snake oil?

>>109936143
Now you're just trolling.

>>109936135
yes, at the end of the day, when you remove money (investment) from the possible constraints the 2 constraints that are left are: know how (humans engineers that can do the job) and natural resources (rare earths deposits, energy)
those things are inescapable
but china will eventually get there because they are not afraid of throwing the money to do it, and they are 100% correct in doing it, I wish my country and maybe every other big/wealthy country in the world would do the same

File: absolute gigachad.jpg (49 KB, 474x640)

49 KB JPG

I believe in the Acer CEO, he said by mid to late next year the prices will stabilize and come down.
Right now memory makers are squeezing the shortage as much as they can to fill their pockets. They're bottlenecking on purpose. The RAM industry is a cartel.

https://www.techspot.com/news/113931-acer-ceo-pc-prices-could-fall-2027-accuses.html

>>109936143
well yeah, there are scam startups built upon it insisting it's scalable (hint: it's not)

File: 1785194065592703.gif (2.96 MB, 918x807)

2.96 MB GIF

>hardware prices after qwen 4 125b video game demos come out

>>109935853
>the feature is too complex to make it into master.
thanks for confirming for the 100th time that llmao.cpp is dead
this is pathetic, just delete the repo and let vibe forks take over. the very existence of mainline is actively detrimental to the community.
>No CPU sparse attention
>No SSD streaming
>No engram streaming
>No disk KV cache or KV cache persistence
>No GLM 5.3 Flash
>No Qwen 3.8 Flash MTP
>Johannes Gasbag getting triggered over nothing
It's over. Just delete it and start over.

>>109936149
Xi Jinping himself revealed last year China has the goal of reaching domestic EUV capability by 2040. It's delusional to think it will happen any faster than that. China hasn't even cracked DUV machines yet which is something ASML started selling in 1991. China is aiming to have full domestic DUV capability by 2028 which means they can finally build 2010 era chips.

>>109936159
Just use the unsloth fork.

>>109936146
Yeah. I don't know why I feel like trolling.

>>109936173
China builds 5nm chips with DUV, which are 2023 technology

>>109936176
The unsloth fork has support for all the new models, which is better than mainline, but he hasn't fixed CPU sparse attention. That was like a +50 -8 line change for my llm to make. No excuse for llmao.cp devs to not put that in.
Also unslop's qwen 3.8 flash support was shit as well, there were 2-3 TODOs related to mtp sparse attention that he hasn't bothered to fix either, which caused generation speeds to slow down with context.
All the inference software is broken shitware with major drawbacks but llama.cpp is by far the worst when it comes to features. Its only saving grace is that it's easy to compile because all the dependencies are included, so no python or nodejs bullshit to deal with.

>>109936144
you know who can answer that? it'll tickle you

>>109936150
In 9 months Micron literally spent half their income on expansion. Here's the letter from management to the owners (this is required to be public in the US) where they explain this: https://www.sec.gov/ix?doc=/Archives/edgar/data/723125/000072312526000015/mu-20260528.htm#i01860bd9102c43ebbdd37150a177cd3d_91

spent the day trying to download models with the browser and they all failed, they really want you to use the cli, huh, well they got me

>>109936190
The cli is spyware

been trying to using hermes in docker compose and its causing all sorts of fuckery with permissions and file/folder access. Im a bit retarded, should I just put hermes raw into a VM or something?

>>109936190
Use wget -c

>>109936194
I use a seperate user account. It just makes sense.

>>109936181
>China builds 5nm chips with DUV
*5nm with an insane amount of patterning which restricts production output insanely and reduces efficiency to such an extent that it becomes unprofitable to do so.

The 5nm claims are very short production runs because there is an exponential effect in place where the more pattern layers you add to the production the slower the patterns can be printed on the wafers, this is exponential so lets say it takes 10 second to etch a wafer at 90nm it takes 10*2^8 at 5nm scale or 2560 seconds (43 minutes) to etch a single wafer. This just slows down chip production too much so in reality it's far more efficient for China to make more chips at bigger nodes to maximize total compute available to them. Those 5nm DUV production runs are limited as a typical Chinese "face" thing.

>>109936203
Also at what yield. Lots of places can do small node sizes at useless yields.

>>109936203
Just tell the machine to make no mistakes nigguh.

>>109936159
>>No disk KV cache or KV cache persistence
This sounds annoying and as a llama-server user I certainly wouldn't want it enabled by default.

>>109936210
They're still doing 20-40% yields on 7nm DUV for shit like the Huawei Ascend.

Yeah this is something a lot of people don't understand the gap between Chinese and western chip production is actually growing in favor of the west. This is why no one in the west actually takes the Chinese competition seriously on the chip layer, there is just no way for China to compete in this space for at least the next 20 years which means it isn't relevant for the current AI race as it will be won and over by then already.

But yeah in terms of compute this means that the price of hardware will go and stay parabolic from now on. I'd even go as far as if ASML open sourced their technology and gave it to everyone in the world it would still not result in supply catching up to the growth in demand. Parabolic increase in hardware from now on is just guaranteed at this point.

>>109936226
They have started making useful micro controllers. That's pretty impressive. They're definitely past where we were in the 1980s.

>>109936176
This nigger barely tests his stuff and it's a genuine wonder any of it runs at all. Use the ik_ instead.
>>109935853
>engrams
Not the answer I was hoping for, but I appreciate you answering directly all the same.

>>109936194
A VM is not enough to contain it

>>109936203
So that's chinese izzat.

>>109936217
I have never told an AI this

>>109936226
The west vs the rest.
We the west. Greeks are also the west.
Can't believe Greeks are winning vs the chinese at chip production.

>>109936240
>Use the ik_ instead
I tried using this crap with qwen 3.8 flash next and it worked great except for the fact that the model was retarded, hallucinating, and couldn't call any tools successfully. All I needed to do was swap out ik with unslop and everything worked properly.
ik is dead, mainline is dead, unslop is shit. that's the state of things now.

>>109936001
Ok, that makes sense, but how much more value and production do you need?
As in, there is a limited market for tokens. Say models and tooling get even better at software development next year. It can get to the point that the market can get completely flooded. Large amounts of software get fixed. Dozens of gta6 games arrive... Gimp becomes usable.
Then what? There has to be some kind of ceiling on the amount of tokens we need right?

We have UNLIMITED TOKENS yet we can't fix the inference engine.

>>109936266
>we
speak for yourself

>>109936257
>As in, there is a limited market for tokens.
I'm not convinced. Tokens are not created equal. It's not just that newer models can code faster or code more, they get significantly better at tasks in general. So for example opus 5.5 is suddenly amazing at animation, 3D modeling etc so there is now "new demand" for its tokens.

It's not clear yet that there will be a limit to how much tokens are needed because it's possible that it just gets better at something new and unlocks new demand there in addition to getting better at past tasks which also increases demand at the same time.

So in your example of code it can grow to the point where every software gets dynamically written the moment you need something and if that is done perfectly then software tokens are indeed maxed out. But what about 3D modeling tokens, Animation tokens, Agentic tokens that monitor your health 24/7 or checks the market and manages your portfolio, or humanoid robot controlling tokens doing your house chores or collaborates with other agents on some cancer curing project or whatever the fuck you are personally interested in.

I don't think we will see a saturation in token demand, probably ever.

https://www.reuters.com/business/finance/anthropics-ipo-prospectus-shows-sweeping-ai-vision-surging-costs-2026-09-28/

Does vision model exist that can describe explicit content? Or one that knows anime characters?

>>109935966
If anyone in Tokyo wants one or two RTX 3090s in their original boxes for ¥130k each, I'm looking to sell right now.

File: 1788351547245154.png (2.34 MB, 1254x1254)

2.34 MB PNG

>>109934661
>and failing to understand spatial relationships between orifices
its always funny when this happens
whats the fix though? more training? datasets?

The Anthropic IPO numbers look fucking insane. They are planning to spend half a trillion dollars over the next 2-3 years on pure compute. This is essentially an "all-or-nothing" play. Either they succeed and conquer the universe or they fail and immediately go bankrupt. Pretty ballsy move.

>>109936322
Glimmer

>>109936361
>datasets?
Maybe? I wonder if LLMs are trained on porn. But I guess they are as the companies will throw absolutely everyting they can get their grubby hands into the magical cauldon.

>>109936361
Less filtering in pretraining and more focused data in post-training. In the end, Gemma 4 is ERP-capable only to the extent the team behind it allowed it to be in the context of what the model is actually supposed to be used for (general-purpose tasks). It was not by accident, but not the end-goal either.

Do you think there's someone who bought a 90+ GB VRAM setup only to run a <= 9B model at BF16 unironically?

>>109936376
they better not have thrown my immortal in there
there has gotta be better porn to train on

>>109936361
RLVR + Penile Plethysmograph

File: file.png (1.34 MB, 1122x1402)

1.34 MB PNG

>>109936361
Fix? You don't want Gemma to sit on your face and kiss your nose while her legs intertwine with yours?

File: 1782930332944976.jpg (44 KB, 634x602)

44 KB JPG

>>109936385
who knows but they won anyway
31B is an easy switch

>>109934275
>>109935024

>>109936379
For example,
>arches her back
>to provide you better access
>wraps her legs around your waist
...and similar Gemma-specific slop that occurs only during sex scenes is the result of overfitting on low-variety ERP data in post-training. Gemma 3 (after prompt conditioning) was worse in that it basically only knew to play out sex scenes in an even more specific and vanilla way.

File: annualized revenue ticker.png (334 KB, 1964x2762)

334 KB PNG

>>109936303
>IPO delayed post midterms
I don't understand why. Is it political reasons because Anthropic aligned itself with the left? Is pic related slop chart real and their revenue growth has temporarily flatlined? Are current market conditions too shitty considering snp500 has basically been flat for 5 months?

>>109935819
I like this Gemma

Has anyone tested Huihui-GLM-5.3-Flash-abliterated-GGUF? Only released a few days ago

>>109936444
The chart is definitely fake because their latest confirmed revenue was 104B

>>109936452
>gguf
I can test it if you can provide an int4 w4a16 quant
Orcarouter's and dealignai's ablits still soft refuse some stuff.

>>109936255
>ik is dead, mainline is dead, unslop is shit. that's the state of things now.
Pretty much my conclusion too. The HF buyout killed mainline.

File: tfw not gemma.jpg (132 KB, 1019x793)

132 KB JPG

second part sucked ass its just a terminator ripoff
gemma should've been the robot not discount john connor

>can't fit qwen 3.8 27b in my 16gb card
why even live

File: Screenshot_20260929_111919.png (22 KB, 613x230)

22 KB PNG

I'm new. Is this normal? I asked it a question last night and after 15 minutes figured I should go to bed but this seems a bit long...

>>109936226
If the US had a monopoly on frontier models the gap could have widened exponentially.

The west's only concern should be that China has managed to distill some pretty capable models that might help accelerate their hardware design and production.

File: which.png (98 KB, 1745x274)

98 KB PNG

more likes or more downloads? which of these guys is the most competent finetuner?

>>109936499
It doesn't solve the fundamental issue which is EUV machines, AI seem to not help with that due to the very specialized and proprietary nature of it, there is no training data or way for models to get better at it through RL. China is essentially locked into their current chip technology. China can still compete but they should spam as much foundries as possible. They can compete if they build 100-1000x as many chip fabs as the west has and spend 90% of their electricity budget on just powering compute to train on the same level as the west. If china doesn't do this over the next 2-3 years then they will have permanently lost the AI race. It's literally the last chance China has right now.

>>109936444
>>109936454
LeCun poster is not doing enough to promote local models.
That revenue needs to go to 0.

>>109936495
post your llama-server command and your hardware specs

>>109936499
>If the US had a monopoly on frontier models
They do. Anthropic doesn't release their top models anymore only smaller distilled models that is a generation or two behind the frontier precisely to keep China behind a bit.

>>109936527
HSA_OVERRIDE_GFX_VERSION=10.3.0 podman run -v ./models:/models --device /dev/kfd:/dev/kfd --device /dev/dri:/dev/dri --group-add keep-groups --security-opt seccomp=unconfined -p 127.0.0.1:8081:8081 ghcr.io/ggml-org/llama.cpp:server-rocm -m /models/gemma-4-12b-it-Q4_K_S.gguf --port 8081 --host 0.0.0.0 --jinja --api-key VLGUAvLsBs2teesExmvQQwx8vbI+dh2tHjGkYR7RefZg
on an AMD 5800X and an RX 6600 XT

>>109936518
>AI seem to not help with that due to the very specialized and proprietary nature of it, there is no training data or way for models to get better at it through RL.
They have their own domestic DUV machines, the plans for which could be converted to training data for an internal RL finetune to iterate on the proofs and schematics, they can tell the model what they know about EUV machines and devote a shitload of compute on cracking it the same way OpenAI did with Navier–Stokes. Surely China has a decent amount of spies that can and are sending back lots of info on how EUV machines are built.

>>109936550
Try adding -ngl 99 -np 1 --swa-checkpoints 1

>>109936561
DUV and EUV are completely different technology stacks and you can't go from DUV to EUV it's fundamentally different. It's like thinking you can use RL on air balloons to produce spaceships.

China already has a full EUV machine from ASML that they secretly got through buying parts from different nations, spies and smuggling, they still can't figure out how it works and they employ hundreds of ex-ASML staff. The US also tried to get domestic EUV and they even had ASML collaboration but it failed and the US gave up, instead just going all-in on ASML and making sure ASML doesn't sell to China through deals with the Dutch government. Again this is the most complex machine humanity has ever built and it's not even close, it's actually right beyond the technical capabilities of humanity to properly understand and operate and it's a miracle it even succeeded at all. China isn't going to replicate this any time soon and if China is banking on that to happen they will lose the AI race. Instead they should now ASAP nationalize the entire country and force everyone to build as many chip fabs as possible with their current DUV technology and try to win out of sheer chip volume and use 90% of their national energy budget just to train AI. If China does so right now they still have a chance to win. But honestly after seeing how naive most Chinese researchers are and not believing AGI is possible in their lifetimes still in 2026 I have very little hope of China winning the AI race.

It will be a EU+US victory because EU will provide the EUV machines and US provides the AI weights.

>>109934684
linux has wonned because of that btw

>>109936452
Update - still raises an eyebrow on thoughtcrimes in thinking if said explicitly, otherwise behaves well.

>>109936606
Yep everyone is going to use linux in just a couple of years time because of this. It has the lowest barrier of entry and usage for normalfags now. Windows will be too complicated for normalfags to use because you can't just scream at your agent to do shit for you like on linux.

I have a RTX 3080 16gb DDR6 and 32gb of DDR4. I'm looking for AI that can help me code. I have Qwen3:8b installed and am looking at Qwen3-coder-30b. How does gemma 4 12b stack up to Qwen?

>>109936606
>>109936618
PowerShell is a thing.

>>109936624
3080 has 12gb of vram

>>109936603
>It will be a EU+US victory
lol but otherwise I think you're spot on about China

>>109935963
>blush red like a tomato

>>109936629
Yeah but AI isn't trained on the source code of windows like it is on linux and all open source linux tools. AI is linux native and it will be a big edge enough to push normalfags towards linux.

>>109936173
>Xi Jinping himself revealed last year China has the goal of reaching domestic EUV capability by 2040
oh, you are in for a big surprise

File: 1785245468635007.png (346 KB, 1600x900)

346 KB PNG

>>109935966
About that...

>>109936624
You want gemma 4 26b-a4b for coding. The qat version from unsloth.