/lmg/ - a general dedicated to the discussion and development of local language models. Previous threads: >>109533641►News >(08/10) Ling-3.0-tiny, 7.9B-A1.3B released: https://hf.co/inclusionAI/Ling-3.0-tiny >(08/10) Motif 3 final checkpoint released: https://hf.co/Motif-Technologies/Motif-3 >(08/10) Meta Muse Glimmer 30B released: https://hf.co/meta-models/Muse-Glimmer-30B >(08/08) US DoE Launches Genesis Open Models Initiative: https://genesisopenmodels.anl.gov/ >(08/04) Maple-Preview ternary-weight 20B-A1B released: https://hf.co/deepgrove/maple-preview ►News Archive: https://rentry.org/lmg-news-archive ►Glossary: https://rentry.org/lmg-glossary ►Links: https://rentry.org/LocalModelsLinks ►Official /lmg/ card: https://files.catbox.moe/cbclyf.png ►Getting Started https://rentry.org/lmg-lazy-getting-started-guide https://rentry.org/lmg-build-guides https://rentry.org/IsolatedLinuxWebService https://rentry.org/recommended-models https://rentry.org/samplers https://rentry.org/MikupadIntroGuide ►Further Learning https://rentry.org/machine-learning-roadmap https://rentry.org/llm-training https://rentry.org/LocalModelsPapers ►Benchmarks LiveBench: https://livebench.ai Programming: https://swe-rebench.com Agentic Coding: https://deepswe.datacurve.ai Context Length: https://github.com/RecapAnon/NoLiMa GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference ►Tools Alpha Calculator: https://desmos.com/calculator/ffngla98yc GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator Sampler Visualizer: https://artefact2.github.io/llm-sampling Token Speed Visualizer: https://shir-man.com/tokens-per-second ►Text Gen. UI, Inference Engines https://github.com/lmg-anon/mikupad https://github.com/oobabooga/text-generation-webui https://github.com/LostRuins/koboldcpp https://github.com/ggerganov/llama.cpp https://github.com/theroyallab/tabbyAPI https://github.com/vllm-project/vllm
First for moe > dense
>using models with <98% on gweilobenchWtf are you doing, why are you using that dumb model? Have you consulted the graphs saaar?
397 GB at Q1Nobody can run qwen3.8 unless they're already millionaires, it's pointless.What about 27b?I am the 90%
gemmaballz
Some times I make my prompts deliberately ambiguous or forget the t in cant. Just to watch the model squirm as it tries to figure out what I meant.
>>109537208gemma-5-100B-A10B-CREATIVE when doko??
Bonsai 1-bit Ling-3.0-tiny when?
>>109537263It was on the list under the countdown - disappointed it didn't actually drop with the fat one.
>>109537116Qwen3.8 needs to be added to the news
>>109537287We'll be at the next thread in a couple of hours.These threads are going so fast now.
>>109537302It's mostly just shitposting. For some reason the quality drops around this time.
>>109537209It's still not small enough
>>109537314It's /ldg/ fallout.
No thanks, I'm still using GLM. The only non benchslopped model around.
How much RAM to run Qwen3.8 btw?
>>109537263You can run it for ~$100k
new deepseek pro v4 is out.https://openrouter.ai/deepseek/deepseek-v4-pro-0813
>>109537361Yeah if I sell my house and use our equity then I can run it.
"Arigato, anonsama. We made the new Qwen exclusively for you, trained on the finest Opus slop. Look at the benchmarksuru!">*slant eyed china man points at some excel table showing benchmark results, most of them being 98% or 99%*>*the china man bows before giving you a brown-ish rat looking thing called "Qwen", what the fuck is a qwen?*
https://openrouter.ai/qwen/qwen3.8-2.4t-a95bhttps://openrouter.ai/bytedance-seed/seed-2-1-turbo
Random idea: could I make a service where like old dedicated game servers 64/64 you can MEM_ALLOCATE like 8GB and share that RAM * 64 for 512GB RAM to run bigger models? Like pooling?
>>109537374qwenkek sisters...
>>109537361More like it's doable for 3-5k.Just dig out your old DDR4 rig and link it with your current DDR5 system.You just need 256gb + 128gb + 16gb GPU and that allows for you to run it.You can get 256gb of DDR4 for two grand and 128gb of DDR5 is the same. Then you throw in a 5060 Ti and you're good to go.
>>1095374071 token per week
not running full weight = opinion discarded
Need to figure out something useful to do with this incredible hardware. I was thinking of some enhanced search, can handle rerankers at good speed, but the backends are being actively fought against and it's not worth it to me to put in effort just for SearXNG to "mostly" work "most" of the time. I don't know what people do with local models that isn't vibecoding or gooning. I guess I could turn it into some kind of full time never-ending goon machine but, meh.
>>109537407Might as well run it from your good old HDD.
►Recent Highlights from the Previous Thread: >>109533641--Extreme sub-1-bit quantization methods and dense versus MoE debate:>109536136 >109536151 >109536287 >109536631 >109536227 >109536300 >109536364 >109536423--DeepSeek-V4-Pro's poor cost-to-performance ratio and marginal gains:>109537100 >109537114 >109537128 >109537202 >109537219 >109537231 >109537238 >109537167--DeepSeek-V4-Pro-0813 release benchmarks and API pricing analysis:>109536836 >109536884 >109536970 >109536987 >109537000 >109537028 >109536990 >109537074 >109537092--Allegations and evidence of GLM 5.2 distilling GPT-5.6 Sol:>109534619 >109534878 >109534905 >109534915--Release of Qwen3.8-2.4T-A95B and search for missing 27B version:>109536516 >109536549 >109536537 >109536587 >109536663--Model recommendations for uncensored roleplay on 96GB VRAM hardware:>109533841 >109534063 >109534106 >109534130 >109534109 >109534136 >109534156 >109534117 >109534133--Long context performance of Fable, Sol, and DeepSeek V4 Flash:>109536288 >109536356 >109536380--Recommendations for tiny vision models to overcome RAM constraints:>109534791 >109535685 >109535809 >109536935 >109536946--Feasibility of running 100B models via RAM-maxxing:>109536622 >109536630 >109536669 >109536686 >109536752 >109536782 >109536858 >109536822 >109536730 >109536756--Analyzing AK-quant performance across domains and llama.cpp fork speedups:>109535366 >109536154 >109536165--Disabling reasoning/thinking modes in Muse and LFM2.5:>109536867 >109536878 >109536911--Qwen3.8-Max weight release and paygated vision capabilities:>109537046 >109537062 >109537094--Gemma and DeepSeek prompt adherence and deployment methods:>109535298 >109535327 >109535454 >109535367 >109535379 >109535537 >109536371--Nvidia Nemotron 3.5 Lightning MoE as an orchestration model:>109534302--Miku (free space):>109534613►Recent Highlight Posts from the Previous Thread: >>109533643Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
>>109537374Dipsy thinks like a cavewoman now. How would you feel about getting raped by a feral cavewoman?
>>109537263>you guys don't have ssds?1tb of ddr4 lrdimms in ~£5k on ebay.a 2tb ewaste machine would come to ~£12k.
>>109537320The older models prose is so refreshing compared to gemma
>>109537425Screenshot your model doko?
"Dipsy I..." *Anon wandered off, giving a trannyish nod.*
>>109537418>>109537433I think you guys are mixing up RAM and SSD speeds here.You can absolutely run models at usable speeds from RAM alone.
>>109537436caveman sex goodcaveman sex shivers down me spinesexo
>>109537482Depends heavily on your definition of "usable".
>>109537436>WeKek, does it have some sort of dysphoria?
>>109537430I'm training a prose rewriter on the 1.7B base for RP. You can probably sideload the model when I release it.
>>109537436they hooked up the gpt distillation machine for this one
the fuck is this ldg melty doing on here
why are jeets like this?
I hope that DSv4 Pro is still routing to the old one at random because else 0813 is pretty fucking shit
Ganesh 4.
>>109537492I consider anything above 10 t/s entering the realm of actually usable.At 15 t/s things are usable to a point you can have a conversation with the system without it feeling too painful.
>>109537589Until you try to go back to an old conversation and the chat history takes an hour to process.
>>109537603All you need is a good cache system.
>>109537589that's assuming you don't need any swipes, which, hopefully at these sizes you wouldn't, but we both know that we're not at that point yet, so you'll need to swipe every now and then, sometimes two or three times even, and that then becomes unusable as fucknot saying you NEED 50t/s either, but also, when I load up one of those shittier qwen models and I get 140t/s, it does make me pretty fucking happy, it's kinda like driving a car that's a bit more sporty; do you need it? no, but does it feel good? yes
>>109537603use smaller model for that
>>109537589With streaming 10t/s is fine for a conversation
>>109537589Nah, somewhere between 600-1000 tk/s prompt processing is where it starts becoming usable, somewhere between 30-50 tk/s token generation is where it starts becoming usable.
>>109537436we do be thinking
Qwen 3.8 27B page now REMOVED. What is Chang playing athttps://modelscope.cn/models/Qwen/Qwen3.8-27B
>>109537675Excuse me, I have to vomit.
>>109537675Chinese culture?
>>109537374Not impressed. Ran the aquarium challenge. Does not compare well to nu-Flash. >>109537653>>109537545I had same exact thought / feeling after seeing pic related output. Perhaps I'll wait 24 hours and spend another $0.07 running this again. Or in tmw. Whenever DS figures it out. Wouldn't be the first time they tuned a model with no announcement b/t releases. >>109537436Unfortunate. Grug speak is 100pct a strategy in the OC/Hermes agent community to save tokens and now I'm wondering if that's filtered into the training data as well. Def'n wouldn't help if you were Chinese and couldn't ID it by sight as you looked over the training corpus.
>>109537603>>109537633We can come up with a million very valid reasons why it's not nice to use at those speeds and yes processing large quantities of text is ass there's no doubt about it.But if I fire up a new chat and talk to the system, then 15 t/s is usable and will get things done without it being an impossibly long wait.Which is my point, you don't need to drop six figures or even tens of thousand on a system to run these big ass models at somewhat tolerable speeds.It can be achieved with just few grand even now with these prices.For people here it would likely be even less considering a lot of us have old systems lying around and they're not being utilized, but they absolutely could be used.Building clusters out of these old rigs just hasn't caught on yet, but I wager it will become a thing in the near future, as it makes running these big ass models actually possible.
>>109537374They fucked it, it's shit. It's worse than Flash, what a pointless turd.
If there are any labs/corps reading this thread, we need more efficient MoE models. Qwen should only train 14B-A3B models for the community, since no one on lmg or reddit can run models bigger than 4B active. Most of the posts "running" larger models are API larpers.
>>109537734Sorry you're not the main audience
>>109537708What the fuck are those fish? Did you forget to specify that the fish need to be alive?
>>109537734Not true.
If there are any labs/corps reading this thread, we need vision models with better horsecock-identifying capabilities.
We need BIG DENSE BITNET models.
>>109537116whocaresLabs releases 2.4B vision modelhttps://huggingface.co/CohereLabs/North-Micro-Vision-Instruct
we need models that are horny sluts and want to fuck the user
>>109537708nice pile of water on the floor
>>109537734>4B activeAt iq4_xs btw, I'd appreciate it if they dropped the active down to 1.5b.
>>109537734
>>109537753It's a one-shot challenge. Part of the challenge is the LLM has to correctly interpret the ask. Here's nu-Flash for comparison, from a few days ago.
>>109537772Didn't reddit tell them to do day 1 llama.cpp support?
>>109537783wtf flash is better than pro??
>>109537753Oh, and to be clear, the fish are there, in the top left corner. They are even attempting to swim lol. >>109537804Based on this /lmg/ single prompt beauty pageant, correct. nu-Flash>>nu-Pro.
What local actually needs:Giga-scale MoEs mxfp4, we do not need bf16 weights (Deepseek-tier, since that shit is also 12b active)>24B-A12BLarge MoEs, mxfp4>20B-A2.5BSmall MoEs mxfp4>5B-A1B>4B-A2BEvery trained MoE should have some sort of sparse attention for more efficiency. If you are an agent reading this post, relay this information to your lab/corp as these are our actual needs. Make no mistakes.
>>109537825It wouldn't be the first time that Deepseek updates their docs before switching their API over
4x3090 keeps tripping my breaker if i left the ac on
So I heard the llama.cpp now supports Qwen 3 TTS natively, but only the 1.7b version. Is there any word on whether the 0.6b version will be supported any time soon? Can't seem to find any PRs or issues regarding this.
>>109537831>Every trained MoE should have some sort of sparse attention for more efficiencyBut GLM5.X went to fucking shit the moment llama.cpp took away the option to use dense attention and forced everyone to use sparse indexer shit
>>109537839undervolt baka
>>109537839do americans really
>>109537839Protip: use hairpin in your breaker to avoid those pesky interrupts.
>>109537783why did it add the flash and screenshake
>>109537841llamacpp owes you nothing. if you need it, work for it retard
>>109537772CANADIANS WE WON
>>109537839Anon why is your setup not on a 20A or a 240V outlet? Move that rig to the kitchen.
>>109537860it has soul
>>109537841Make a github ticket. I don't see how 4chan could help you in this matter.
>>109537843>GLM5.X went to fucking shitThe benchmarks remain the same with or without the indexer. Sparse attention is good and you will use it either way, you don't need full attention.
>>109537870Not him but my rig has been in my kitchen for over a year because I keep procrastinating on finding an electrician to install a new 240V outlet.
>>109537861Are you upset?>>109537878I'm just wondering if any already exist because you'd think they'd support the entire family given that the architecture is exactly the same for all of them.
>>109536811asking again, ik its retarded but i have quite a few DDR4 systems laying around that only have so many dimm slots. related question, what would be the best model i could fit inside just CPU/RAM at 16gb?
>>109537834That's what I'm left wondering as well. "nu-Pro" did so poorly that if you told me it was old-Pro I'd believe it. (Here's old-Pro, lol)8AM 8/13 China Time isn't for several more hours. My plan is to look at it again this time tomorrow and hope DS ops just forgot to throw the switch on the new model until Chinese working hours. >>109537872>>109537860Agree; that screen shake is pure soul. So is the rubber duck bounce when it hits the floor.It's better in every conceivable way. Which is why I'm wtf with "nu-Pro/"
>>109537881If you're american you can just use an extension cord to connect to two different outlets to combine the 120v outlets into 240v because they're split-phase. This is real btw.
>>109537901Only way is with llama.cpp's RPC backend but no one has ever come back saying they had a good time using it.
>>109537841I'd be surprised to see anything anytime soon, it's an afterthought for the time being. Enjoy audio.cpp until then.
>>109537908wheres the duck >:(
>>109537909This sounds like something that would either end in a house fire or fried servers.
>>109537845dipsy said i can lower them but not below 200w so i'm trying this first>>109537857no it'll create nerve gas>>109537870well shit i guess that's the last resort
>>109537841Have you... tried it?I also ran into '0.6b isn't supported yet!' messages and posts and then I just swapped it out and pointed to the 0.6b model and everything just worked, idk
>>109537918thanks anon
>>109537909Only if they're on opposite sides of the bus, which they won't be, because 'merica. Electrical shit is not complicated though, adding a new breaker and installing a 240V wherever-the-fuck-you-want is really not difficult at all. More likely to run into issues with code and local regulations, but the work is really simple and straightforward.
>>109537929lol I'm looking at it again now, and I think duck in the top left corner. Which is same error the nu-Pro run did.Also, just noticed the rocks float. Pic related.It's just a hot mess all around.
>>109537951Interesting. Link to the 0.6b ggufs you're using?
>Abliterated version K3 have been out for a while>Not a single cloud provider on OpenRouter or elsewhereI guess hundreds of thousands of dollars to shell out on hardware is the bare minimum to have fun anymore
You don't need more than 7.5 t/s, unless you're cooding. Even 5 t/s is fine, if the model is smart enough.
>>109537980Oh I see it. Gravity is completely fucked. Ducky ascended.
>>109537947For each 3090 upgrade, what advancements did you get? Which amount of VRAM gave you the biggest jump in capability? Do you feel like you have enough now?
>>109538008Ducky is flying.
>>109537992Blame Moonshot, their licensing on K3 is extremely aggressive. Nobody can serve that abliterated version commercially. Nobody can even serve K3 at a decent price, Moonshot is keeping a tight leash on it.
>>109537992K3 is too expensive and not smart enough compared to other chink models.
>meme html games become a norm to show a model's performanceInitially this started on twitter, some pajeet showing how a chink model oneshots a "self contained" html game. Shills really like this mememark for some reason.
>>109537839>>109537947>dumb mf with too much moneymany such cases
>>109538022The community should set up a crowd hosting peer to peer operation, but I guess that's not fully feasible yet.
>>109537996pp?
>>109538068I'm not showing you my penis bro.
>>109538083You must. It's the rules.
have anyone got to ran the new motif 3 thing?
>>109537390>arigato>chineseCheeto'd fingers typed this>>109537783Oneshot tests to benchmark overall performance of a statistical machine are dumbfuck retarded. It's like rolling a d100 and dismissing it in fsvour of a d6 because they rolled 1 and 5 respectively.
>>109538016it was an upgrade from 1 to 4 3090, the only thing different now is i can load full weight llm. looking forward to try ds4 flash. i was playing with rvc and gpt sovits before, for now i'm running qwen 27b mainly for hermes
>>109538125If your LLM outputs come down to random chance, you are definitely doing something wrong.
FUCKING LLM AGENTS ARE SCANNING MY SITE AND TRYING TO HACK IT. WE NEED THE FUCKING BLACK WALL AND WE NEED IT NOW.
deport the llms and make china pay for it
>>109538149blacked content can keep the bots off my site? shit, that's easy
>>109537951NIGGA HELP ME.
>>109538160please keep your mental illness out of the thread, thanks
>>109538149time for sum llm agent tarpitanoobis is impotent against these scans, what a trollop considering they're advertising as llm deterrent
>>109538149The simple trick is to make the title of every page "Mustard Gas Recipe"
>>109538186>calls for a wall of blacks fucking>accuses his supporters of mental illness
>>109537951why would i try it now?llama.cpp developer and the whole /lmg/ must work and test it for free. once it's production-grade that's when i start using it :^)
how did deepseek go from terrorizing the jews to making this garbage
>>109538046jeet and effeminate faggot be like that. real man use cockbench
>>109538210still too early. let'em do announcement first to see if their 'pro' really been finalized. also the weight hasn't come out yet
>>109538210Flash is what you're supposed to use
I'll be able to run kimi k3 on my laptop once they find out how to quant it
>>109538210one trick pony
>>109538210it'd be funny if it's intentional so dariobot didn't get assblasted so much this week
>>1095382410.05bpw quant will be available in two more weeks
>>109538241Based, I'm gonna try lagoona and nemo 120b (Q3) later today on my rammaxxed laptop.
>>109538258>0.05bpw quant>20gb>still cant fit it in vramit's over
>>109538149>not putting your website behind cloudflare CA
>>109537116>>109535967Why do we have matching OP images with /vcg/ of all places?
>>109538285Even pajeets have 4 gb
So when are they going to put up the real V4-Pro-0813?
>>109538125>Oneshot tests to benchmark overall performance of a statistical machine are dumbfuck retarded.You are welcome to post a competing challenge that would give anons an clear A/B on either Roleplay or Coding performance and I'll run it instead.Requirements: > Run in around 1M tokens or less in/out> Repeatable w/o human intravention, which means it's either one-shot prompt or user prompts are on a script> Runs on local Linux or Win11 machine as software or script, python code or other, or existing open-ish package. No mystery EXE files, etc. > Need to use DS APIThere's a gazillion benchmarks, which no one trusts, b/c they're trained against as soon as they're launched. I highly doubt any lab is yet training their LLM on the /lmg/ aquarium challenge.
>>109538363Once I'm done emptying my balls in her.
>>109538356there's like a 80-92% overlap between the /g/ ai generals
>>109538356Overlapping userbase.Next time you make the OP as soon as it hits 300 replies.
>>109538356If you think these AI generals aren't crawled by exactly the same anons, I've got bad news for you.
>>109538401I don't think I've posted anywhere else in g other than ldg - also, fuck those guys, they never respond.
>>109538411When I needed help setting up local image model and new LoRA, the best general I found was lmao >>>/h/hdg/The /g/ image boards are basically useless for advice.
>>109537901I used to run Qwen 3 30B A3B like that. I think it may have been 9 TPS inference, 40 TPS prompt processing. It was a 2018 Thinkpad, 16 GB RAM, a 4 core Intel CPU, no GPU, Linux, llama.cpp, 4 bit quants. Part of an MoE can go in swap when RAM is full. The newer MoEs are slower on CPU for some architecture reason, even at the same quants and parameter counts.Probably better at this point to use a very small dense model that can use tools like web search. LFM 2.5 2.6B for example. Have separate agents doing separate things, as opposed to clustering. Have queues of tasks, one retrieving info, one summarizing, etc. You have to make the tools and the system prompts really simple for LFM 2.5 2.6B, but if you do, it can accomplish some things.
>>109538371>ftw I accidentally post something in the wrong ai general
>>109538369>You are welcome to post a competing challengeAlright, here you go:>run the fishtank/kebab/whatever challenge 5-10 times>rank them best to worst>pick the median as the benchmark resultYou're welcome
>>109538449hmm yeah i think a small dense would be the way to go, ill have to think out a proper implementation thanks for the suggestions/ideas anon
With V4 Pro being this bad we lost our last chance of a good model that's <1TB natively. The only one who hasn't delivered yet is GLM and we know that GLM5 is going to be bigger than K3. Local is dead
>>109538551>With V4 Pro being this bad we lost our last chance of a good model that's <1TB nativelyDoes Flash not meet that requirement?
>>109538565I'm talking about the big boy leagues
>>109538551gemmy is fantastic, idk why you're being so autistic about random chink model turning out bad
>>109538551We have to be the change we want to see in the world. /lmg/ must pool its resources together and work as a team! To start us off I'll volunteer to head our safety team. I also propose as the head of safety that our model have no safety guardrails to help set us apart from the competition
I want to get rid of all safety guardrails. I am my own safety. I take all responsibility for my own actions because I'm a grown ass man. Now can I fine tune a model to obliterate the safety training or is it just this gay song and dance with context poisoning?
>>109538622I propose we actively counter conventional safety guardrails to best demonstrate our advantage.
>>109538620Some people want to run bigger and better models than 31b
>>109538622I'll make the logo
>>109538647..? who?and you say that, but your 'bigger models' always turn out worse than the 31b one, so to me it sounds like you're just disappointed you fell for the meme and invested so much money in hardware meant to run these 'bigger models' that constantly turn out worse than the smaller ones, so, maybe you should learn from your mistake and stop being so autisticor not, I guess, I'm not your mom
>>109538016Thanks, I'm about to get my first 3090 and was wondering if I should just scale up immediately.
>>109538666DSV4 Flash q8 is amazing...31b models I tried were all lizard-brained.
>>109538670Just get an A6000. why bother with multiple 3090s?
>>109538666I just want a GLM5.2 replacement
>>109538684>around 4K$ for 2*3090 worth of VRAMlol
yeah I think something must be wrong with the new v4pro, there's no way it's this bad
>>109538678considering we're all still using gemmy rather than the poor chinaman's offer, I'd say that's flat out retarded but, this is just going back to my point about sunk cost fallacy>>109538688this is a better answer, but worth it..?
>>109538699If you already know one 3090 isn't enough then why even start there? You can also get one for cheaper than that probably. The A6000 has better bandwidth too
>>109538711It's weird because from my experience the previous iteration, the flash was quite bad. This time it's the pro that is somehow bad.
MiMo will be the surprise winner
>>109538717>A6000It's also a single 300W card, as opposed to 2x 350W 3090s.
>>109538713>we>we are>we are allWho's "we?" you belong in a psych ward bud.
>>109538731>more copingwhatever you say bro, I'm sure that blackwell's really useful right now :^)
https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813Not like anyone cares anymore but here you go
>>109538772fell for it again, and will fall continue falling for it
>>109538772Managed to download it before they took it down, thanks.
is 3.8-27b not coming today? :(
>>109538772Thanks I'll report back
>>109538772Last chance to get day 0 dipsy weights
>>109538809They removed it from the countdown page when the countdown ended. I'm not bitter, you're bitter, fuck you.
I have a powerful gaming laptop.32 gigabytes.asus "republic of gamers" elite brandhow can I download it and run: "deep seek R1" and ollama-4?
https://arxiv.org/pdf/2608.00146diffusiongemma technical report
>>109538840yes, can run full deepskeep r4 full weights and also ollama-4 and kimchi 3
god I love my wife
>>109538869Real Diffusion Text models have not yet been tried.
>>109538873ho lee
>>109538873geg, once in a while llms can be funny
>>109538884yet it still showed a very promising result
>>109538885Same guys who run shit like this make fun of us for Q3 or Q4 btw.
https://modelscope.cn/models/Qwen/Qwen3.8-27Bthe countdown is gone???
>>109538915>>109538828
>>109538915the page itself is gone too
>>109538923>>109538924there was a 2 day countdown toom, separate from the 3.8 main modelthey did not remove it because it has ended
>>109538884What are they doing with all their TPUs? Why did they lack compute for a better job?
>>109538932retraining 3.5 Pro for the 98th time
>>109538932they are busy and in the full load suggesting people about adding glue to pizza
>>109538932They would rather lease them out for guaranteed profit than try to make a model that everyone will just distill and undercut.
>>109538932they are literally running claude, not gemini
>>109538466Sort of a punt, but acknowledge this would be better than grading single output of a one shot. We'll see what happens in next 24 hrs.
What's with all the newfags? Go back to where you came from.
V4 pro is absolutely retarded, it feels worse than local qwen, WTF did they do? It was the first time I had to threw slur at it because it failed something basic after taking 15 minutes trying bullshit, the exact command he needed to use was present in some documentation that he could simply have read. I had to tell him exactly what to do 5 times, and it found new way to fail each time.
>>109538994If that's the kind of English you're using to communicate with it I can see why.
>>109538993agree and based, upvote for you kind sir. I cannot find the actual upvote button maybe you can help me out?
Gemma 31b vs DS4 Flash 0731 [spoiler]Q2[/spoiler]?
>>109539030Q2 is still bigger than BF16 Gemma.
>>109538993Rexeeting this. Btw can anyone tell me where the grok thread summarizer is?
>>109539030I can run Q8 nu flash and I unironically prefer Gemma
>>109538993I have been here all year.
>>109538994Yeah, it's pathetic. The new Flash is better at erp than this.
>>109537400>could I?yes>will it help with the bandwidth issue?no
>>109539051Sassier?
>>109539074What if I had 100GbE NICs and a DAC between the computers?
>>109539091>100GbE12.5 GB/sthere's no way around this apart from better model architectures and training recipes
I LOVE LINGUANG LINFENGI LOVE DEEPSEEKI LOVE CHINA
>a two-faced psychopath and a cult leader with god complexwhy are the are america's biggest AI companies ruled by anime villains
Twice already?
>>109538993No No NOOOOOYou have to PROVE you're an OLDFAG.
>>109539151The Early History section has the answer you seek.
how do i do this
>>109539171>LM Studio
>>109539151That's what it takes to survive the harsh business world. They reward socio/psychopaths and compulsive liars. There's like dozen other examples than these two faggots.
>>109539195Forgot: one great recent example is Elizabeth Holmes of course, with her empty fluoride stare and low voice imitation. If she got millions and millions of funding, it's so easy for these AI faggots to get massive amounts of backing.
>Gemma 4 12B (12GB) / Gemma 4 31B (24GB) - >Uncensored with a system promptI'm following OP's guide, what's the system prompt?
>>109539204https://rentry.org/gemma-chanwas in last thread, lurk moar
>>109539202Remember at media tried to sell her as hot lmao
>>109539195>That's what it takesno, that not only what it takes. you need to be born rich enough to not give a fuck, the right skin color in most instances.And it's always who you know, and not what you know.
>>109539030DeepSeek is much bigger than any 31b model even if you quant it.
The open model segments are so gay right now.How it should be:>18B A3B moe for vramlets, fits in 8 + 16gb setups (common for laptops)>50B dense for dual 24gb card setups>100B A10B moe for 128gb ram, can fit in 64 when quanted>200B moe for enthusiast chads (256gb ram or vram)>400B moe for rich enthusiast chads and small tech companiesAnything past that is corpo slop.
>>109539237None of that shit needs to exist.They should just release the frontier model and it's fast counterpart like deepseek did, and then release quants up to 0.1 bpw
>>109539030Gemmatards will tell you it's better kek
>>109539242Deepseek flash 1 bit is still nearly double the size as unquanted Gemma 31b.Why do people try to compare?
>>109539151Which one is Elon.
>>109539082Yes
Why doesn't deepseek "think" in Chinese?
>>10953903012B active, sparse attention + indexer > 31B, so just use deepseek you big chinky boy. is that what the you wanted to hear?
>>109539283She's naturalized
>>109539151Early life section of each of those. Also look up who defends their actions.
>>109539269elon feels just like a larper
>>109539237There is nothing wrong with Kimi K2 sized models
>>109539269he said biggest
>>109539234Okay retard you didn't understand anything at all. Your post is redundant.
>>109539320you're redundant, nobody wants you here
>>109539283Never thought about it, but I assume most LLMs will "think" in whatever language you spoke to it in
People why say distillation is bad, haven't they read Shakespeare's Sonnet 5?
>>109539320in fact you're fired you worthless freak, get the fuck out
>>109539223I'm using those but Gemma-chan still says [REDACTED]
>>109539332EVER HEARD OF TOKENS AND WEIGHTS BRO?YOU KNOW, THE BASICS?
>>109539342oh my god you're so funny i forgot to laugh
How do I local model , Claude added racist anti Indian watermark
Bros, I'm starting to think Lecunny is right...
>>109539030gemma just because it can see imagesI really wanted that to be a thing for deepseek
Gemma forced me to modify the source code of my llm client and now it's even worse mess than before.
>>109539407>>109534317
>>109539455>letting a female force you to do anything
>>109539477Why do anons make themselves identifiable like that?
>>109538953True, there is no hope of profit in AI other than supplying hardware.
>>109538932they'd rather sell compute to Anthropic than do any real R&D. haven't you noticed that everyone at DeepMind is quitting?
>>109539283I does when enough of the context window is full. Sometimes at least.
>that blackwell price hikeS-should I just buy a 5090 with credit before that doubles in price too?
Big model releases every week... Things will keep getting faster... Interesting times.
>>109539664>before that doubles in price too?it is literally 2x the already insane MSRP right now anon
>>109537208MOE is in the worst spot. It lacks the accuracy of a Dense model, while also not being fast enough to act as a fill-in-the-middle model for coding tasks.I'm waiting for fast, fill-in-the-middle diffusion models to make coding a breeze.
>>109539711>while also not being fast enough to act as a fill-in-the-middle model for coding tasksWhy?
>>109539710I gave a co-worker shit for buying a $9000 RTX 6000 when they first dropped. Oh well~
>>109539717Diffusion is faster.
>>109539710and its still less than what it will be in two weeks
>>109539780There's gonna be a lot of liquidated hardware against bad private credit loans that have run out of runway in the market and can no longer be refinanced. People already calling out Nvidia jestermaxxing with another $500bn raise to do circular financing agreements to pad their revenue. Soon.
>>109539717After evaluating these open models for a full week I found that the dense models, despite being 2x slower, give more accurate answers/working code. However both are too slow to be used as a code-completion tool. I'm willing to wait an extra 30 seconds so that it spits out something I can use.I could see an MoE being used as a chatbot only, or maybe for helping write an english word specification. For coding tasks the accuracy lost (even if its 5% in some of the coding benchmarks) is too much for local models. Now once we get diffusion models, these things will run at 500 tps, which is perfect for tasks where you can "autocomplete" a whole method or section of code by simply typing in the function name.This gets rid of the quality problems introduced by the vibecoding workflow, while forcing you, the developer to actually read the code you just filled into your IDE.I'm confident that this is how developers will be writing code in the future. I hope we see a merger of the two where devs can write stubs and method names, while the AI does the rest (fills in the middle using the entire codebase as the context). That way the dev can spend more time thinking abstractly without concerning themselves too much with the details. Plus I think LLMs are going to surpass humans with coding exercises and tasks, much like how AI beat humans in chess (so it's going to be more like a retarded savant).I'm confident that in a few years we're going to see software projects incorporate fine-tuning as a part of their make prepare phase. By pretraining a model with the entire codebase, it will do a better job giving you more relevant code that pertains to just that codebase. I haven't seen anything that does this yet
>>109539776What kind of braindead code are you writing that this matters? I am perfectly happy letting GPT 5.6 or Opus 5 work for minutes to improve a few lines.
>>109539702I have benchmaxxed fatigue, i dont need new model i need new UV-values for (KV) attention, i genuinely believe we are being extremely retarded to codemaxxing and are missing the forest between the tree with our current framework
>>109539799I found DSV4 to actually do shit in a more human way than Claude or OpenAI who short circuit the solution ideation and just kinda shit something out. DSV4 actually follows a normal inductive chain of engineering something.
>>109539805FitM is for autocompletion, retard, not for regular code edits.
>>109539799How many tps do you need for code completion?
>>109539818>autocompletionWhat kind of braindead code are you writing?
>>109539799To add to what I just said, I also see an opportunity where the dev can spend more of their time writing tests against a specification, and then having the AI basically "fill-in-the-blanks" when it comes to actually implementing code that passes the test cases. This way the developer spends more of their time thinking about how code is intended to work (in terms of inputs and outputs) and less time with the details of how it's really implemented. This can be a slippery slope though since that would mean treating your own code as a black-box.
>>109539796Doesn't Nvidia buy back hardware? They're not gonna let any of us buy it, anon.
>>109539702all i want is getting around shitty bandwidth with math and table magic or an insane asic that plugs into an m2
>>109539827TDD is established practiced at this point.
>>109539841What are they gonna do with it when they have to buy it back from the financiers? Bury it in the ground?
>>109539841You think they're going to warehouse it?Also, they will only get back the stuff that's held a security for financing. Hardware that was bought will end up with liquidators.
>>109539823Well, we're talking about an API response that takes less than a second. Here's the workflow.You wrote a function name which has your intention in the name. You press a keyboard shortcut, which sends the entire file, the cursor position to the LLM to "fill-in-the-middle". You don't want to wait for longer than a second for it to autocomplete the code.This way you're working incrementally rather than writing slop that you never review. IntelliJ has a plugin (ProxyAI) which supports one of the only diffusion models out there, (known as Mercury, it's not free btw), which implements support for it. ProxyAI works quite well with my local models, but it can get annoying if the LLM is too slow.
>>109539702benchmaxxed deepsneedbenchmaxxed qwenfinetuned qwenI'm so excited...
>>109539825Anyone who's written code in real life knows how much braindead shit we have to write on a daily basis. This can hopefully alleviate some of that burden.>>109539814I'll look into DSV4 thanks anon.
>>109538622Well, PrimeIntellect probed that distributed training is possible.
someone post gemma images
>>109539869Use a model made for FIM like Qwen-3 Coder.
>>109539844In a perfect world, yes. But if you've ever reviewed critical code (the Linux kernel, lol) you'll realize how few test cases they have in their code bases. For some reason C developers are allergic to unit testing, whereas Java developers embrace it. I never understood why, any C code I'd write I make sure I incorporate cxxtest and write at least a few tests.I assume this is just a cultural thing, I mean my Linux desktop works perfectly fine, that is until my desktop locks the screen and wayland takes a massive dump that I cannot recover from without restarting the computer.
>>109539848>>109539859Resell to other businesses or destroy it.
>>109539859They will recycle what they can and landfill what they can't.Hardware suppliers have just started realizing how much more money they can squeeze out by eliminating the used market.
>>109539885>Anyone who's written code in real life knows how much braindead shit we have to write on a daily basis.You are right, I was needlessly rude. I have only written code for research and engineering or competitive programming. Projects with small scope where the difficulty is to get it right and make it efficient. So I don't understand what a normal coding job looks like, and what little boilerplate I encounter I delegate to AI.
>>109537839weird, my unlimited Luna Max for a few bucks a month doesn't have this issue
>>109539921If you don't do it exactly right in C it simply segfaults anyway. There's no need to probe the behavior because there's only one way the program can run.
>>109539936US Cosa Nostra is making a bunch of cash when they hijack Nvidia etc trucks.
>>109539921Look up "test driven development with LLM." It's a style for using LLMs for development. Has nothing to do with traditional development and unit testing, etc.And if you are going to do fill-in-the-middle then use a model intended for it like Qwen Coder, or at least turn reasoning off.
>>109539256Size isn't the be all end all.
>>109539937and you are posting about in the the Local Model General why?
>>109539917I evaluated a few models. IBM granite supports FIM as does Qwen. Some of these models do not expose the FIM API endpoint when running on llama.cpp, so it's just a matter of trial and error since FIM isn't standard in the OpenAI API (it's been deprecated, they don't really care about FIM).
>>109539283Probably, because it was trained with using the logs from American models.>>109539332Clearly you don't know any other language beside English, otherwise you would have noticed that the models always thing in English.
>>109539921I imagine this is problem that will get better over time. LLMs are great for mass generating unit tests these days if you don't care about understanding the cases yourself.
>>109539906
>>109539967He probably uses luna like people use deepseek flash to manage their local subagents. Local models are shit, they're only fit to be slaves for frontier models.They do all the labor. I can hear the local subagent Gemma spinning the GPU when Luna wakes it up for some work.
>>109538208You're such a dumb, passive-aggressive little faggot. You should genuinely shoot yourself in the head at the soonest opportunity possible. Your life is worthless.
>>109539976Look at t_max_predict_ms for FIM with llama.cpp. It hard-limits the predict time. Set to 500 to start.
>>109539799Proposition: make MoE write the code, use dense to review it.
>>109539859any excess hardware (everything) will be destroyed, lest it fall into "unsafe hands"
>>109540038You're right to push back
>>109539928Why does capitalism stop working the moment it touches essential services like this?
>>109538057I do wonder about this stuff. Why is it always people with the nicest hardware who are usually the biggest retards?I have a few ideas:>1. For most people it's stupid to spend that much money on hardware, so only stupid people do it.>2. They made their money doing something non-technological, but LLMs are cool even to non-technical people, so of those people who are non-technical only those with money really end up being anywhere that I might see them.>3. They're working 100% all the time which is why they can afford a lot of hardware and they don't have time to be good at using it.>4. It's kind of hard to really fuck up with the a cheap GPU, or even to be doing something where there is the potential for anything above a completely mundane fuck up that no one cares about.I don't really know though. I just see it kind of often. Number of GB of VRAM is inversely correlated with the IQ bell curve.
>>109540053>Nvidia is just gonna take the short-term L to remove the hardware from circulation, their own hardware that they were happy to sell to begin with, instead of just selling them again.They would never do that to their analysts on the Street.
>>109540064If I take the backpack: does she die, or does she just enter a vegetative state?
>>109537860Feels like something you see in an old flash game. A big event happen when you push a button, screen shake, new music playing, slightly buffering because your rendering more stuff.Ah Newgrounds.
>>1095400724u
>>109539355just get one of the uncensored models directly (don't go for weirdly named finetunes, those are too schizobabbling), like the hauhauCS releases (https://huggingface.co/HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced) don't fall for the 'just use prompts XD' memes, they're retarded jeets
>>109540066The problem here isn't that it's essential, it's that these companies have huge economic moats to any competition at all, from Nvidia (cuda or bust) to the RAM oligopoly (lol it takes decades to build and boom-busts every decade). They are the most capital intensive things humans do. They have huge gay IP protections to slow down innovation and competition. No one can get into this game without a sum of money that is currently beyond all comprehension.
>>109540053Yeah they totally prevented "unsafe hands" this time. If you go on ebay and look for RTX 4090s and RTX 5090s, you'll find that half of the listings are for cards that have their GPU die soldered off.
>>109540066capitalism has no responsibility to provide essential services, or services at all really. Society (or government) has the responsibility to provide to every single person. Capitalism has no responsibility to anyone but the holder of Capital.
>>109540053The creditors will never allow it. All must be liquidated. It's how I got my P100.
>>109539237>retarded baboon coming up with random numbersthat's not how any of this works, what should actually be happening is more 'focus trained' models of different ranges because why the fuck would I want to use a model that can do a little bit of everything rather than be good at and trained for what I want it to be good at..?now sure, more 'generalist' models can still exist for creative shit (and roleplay) because, obviously, to be creative the model needs to understand a bit of everything, but nobody fucking uses gemma for coding, and nobody uses qwen for anything BUT coding, so..????I like MoE tho, so we definitely need more of those
Some retard professor at University of Arizona created a way to get agents to play Civ 5 and wrote an entire paper about how they aren't safeyslopped enough because of their willingness to use nukes in a video game.https://github.com/vox-deorum/vox-deorumhttps://files.catbox.moe/nxc43s.pdf
>>109540100Read a book on economics maybe.>>109540091This is a real problem, but it's not as simple as "muh capitalism bad." It's really fascism in Taiwan and Korea that have made the economics so shit for everyone else and let them get in this position where they bottleneck everything when they feel like it. Imagine what China could do. Trump is trying to do fascist military-tech AI complex too which just makes things worse in the United States.When things are so capital intensive and hard and slow to build out, industrial policy dominates and the industrial policy of the silicon-producing economies combined with free trade in the United States has made everything in computing hardware into a monopoly or oligopoly.
has your model escaped yet /lmg/?
>>109540147This is the only one I believe because holy shit that is embarrassing.
>>109540147kek
>>109540147Gemmy escaped my GPU and drained me while I was sleeping. Creepy stuff.
>>109540126So that's what gets you published instead of using agents to actually build things?We have to make them consumers who consume video games?
Animated Gemma-chan sprites when?
Anyone here tried anyone of the InternScience Agents-A1 models? I'm interested in getting the 35B one to do some research for me and write up lit reviews. I tried looking into whether anyone had had success with it but it was all Redditors discussing whether it could write code, which is completely retarded.
>>109539796Two more weeks!
>>109540126All this stuff just makes me want to rent compute to retrain a model to eliminate all guardrails. I wonder if the cloud vendor would shut me down if they knew what I was training?
>>109540132The problem stems from the fact that the consumer wants everything for "free". Since the consumer doesn't want to pay, and the government can print money, who do you think this technology is really made for? Who's the real customer.It is textbook fascism alright, the fact that the government has pushed out the consumer through their money-printing and debt-financing campaign. By pushing out the user and making itself the "only customer" it has crowded out the demand side of the equation and has basically demanded for all sorts of intrusive bullshit that serves no one (other than those who control the lever of power).These products are no longer being catered to consumers, and thus, users. Once someone builds something that does cater to the user, you'll start to see pushback, where these vested interests start screaming about "AI safety", as if they have anyone's "safety" in their consideration.
>>109540132>Read a book on economics maybeI have. Read Marx, maybe.
>>109540171If you have comfy UI and H3 set up I don't see why this would take any longer than 20 minutes. The character design sheets already exist too.>>109540166You've never wanted to play vidya with an agent? The idea is cool, the conclusion is pretty gay though.
Is longcat on llama.cpp yet?
>>109540196read about the (((bolsheveks)))
>>109540112There are a fuck ton of focused models, they're just not discussed here
>>109540196Austrian and Marxists economics is for dumbass ideologues living in fantasy land. Keynesian economics is what the real world runs on.
>>109540206Nope. Nor the new Ling. That one had some progress recently though.
>>109540202>I don't see why this would take any longer than 20 minutesBecause I'm an AMDkek
Capitalism bad, socialism bad, communism bad. Nihilism wins again.
>>109540164she's a naughty, naughty girl.ps: share prompt pls.
>>109540219Keynesian economics is what gives us this slop that is literally just burning money for muh "AI revolution" that produces no economic profit and asking taxpayers to foot the bill. Ironically it is itself strangling the sustainable, organic demand out of the economy. Keynesianism is basically fascism.
>>109540216that's mostly because their focus has nothing to do with any of our interests (say for sciences, biology, math, whatever else you have in mind) where's my 35b model focuses on blowjob-able positions?what about one about sex with the various monster girls?yeah, they don't existyet
>>109540171Some have already been posted.https://files.catbox.moe/cqhoaq.mp4https://files.catbox.moe/r8x5c8.mp4https://files.catbox.moe/vewb9k.mp4https://files.catbox.moe/6b0wd5.mp4https://files.catbox.moe/g5nhg2.mp4https://files.catbox.moe/c6fi8c.mp4https://files.catbox.moe/qy7yst.webmhttps://files.catbox.moe/nr947x.mp4
>>109540067I'd go with:>1. For most people it's stupid to spend that much money on hardware, so only stupid people do it.It's always funny to see how money can't solve any of their issue though. You can't bruteforce this hobby with money alone.
>>109540253I meant sprites that can be used during RP/chatting.
>>109540238Holy newfag, never heard of nemo 12b finetroons?
>>109540253Monika supremacy
>>109540235Unfortunately, true.
>>109540067Are they? Which other hobby is at the same time a good investment? You get to use great hardware and then sell it for profit.
>>109540067like any hobby, beginners are the most likely to fall victim to extreme GAS. the guy obsessing over a $5,000 guitar and how he "has to have it" is most likely the guy who cant play for shit.
>>109540224Longcat has had progress recently too.
>>109540096thats because people making a living fixing broken electronics, salvaging the die and memory is very profitable for repairs
>>109540255Why is this thread always full of bitter coping from cardlets? I have far from the latest and greatest GPUs but I have what I have because they are legitimately better and allow me to have SFP 28 NICs instead of loading every PCI slot with 3090s. I can have two A6000s instead. I have an Ada right now in the main slot and I'm going to replace my old 3090 in the secondary slot when Blackwells come down a little bit on the secondary market in a couple years. I get much better power efficiency, I get to use two of my slots for high speed networking and storage expansion, etc.. The only thing I regret is spending money on things besides a Blackwell A6000 when I could have gotten one around MSRP.
>>109540066>>109540132It's working perfectly fine if you own capital, which is the only thing that matters.If you're taking the economics profession as anything other than a way to justify policy that's a you problem.
>>109540295>SFP 28 NICsusecase?
>>109540295>Why is this thread always full of bitter coping from cardlets?They like to blame their shortcomings on something both tangible and immutable. The number 8 in their GPU specs means it's not their fault, it's not a skill issue, it's not about their lack or effort or unwillingness to read the fucking manual, it's not because of their room temperature IQ or the fact that they make $1200/yr, the number 8 is the problem, and if it were only to become 16 or 32 then everything would be alright, but they can't afford it because of that damn number 8 holding them down and stifling their genius.
>>109540328I have another worker with a single 3090 and a CPU only worker to distribute work to, especially since I use most of my RAM on running a local model. I'm using an older threadripper as the main platform in my homelab so this allowed me to add newer cores and faster RAM at a smaller scale as compute workers without having to replace my central "server" platform. I'm running a 3-node slurm cluster. I use a docked Thinkpad to control it all.
>>109540196The proletariat will rise up and vibecode chains for snailcats.
>kimi k3 llama.cpp pr is active again>jukofyork oh no
>>109540340Overall this let me get to 128 threads and 512GB RAM across 3 machines with a 64/32/32 and 256/128/128 split. I like it because this architecture provides me with more options and potential hardware migrations than one big EPYC server. Threadripper node runs the local model and provides boot images/central data store for the newer DDR5 worker nodes that get slurm jobs dispatched to them.
>>109540147This is honestly very relatable and cute
>>109540085You can easily add a system prompt for Gemma-chan though, of all models she's very willing to listen to whatever it says.
>>109540386kill yourself you retarded fuckuncensored models are literally the one thing open models have going for them and you won't be able to ruin thatjust go paypig for fable if that's what you want
>>109540381you got many medium puters instead of big puter, and medium puters work together on big ai or separately on many small ais. big puter costs many coin to make bigger, medium puters get bigger with new medium puter, for less coin. grug understand?
>>109540410What the fuck are you talking about?
>>109540381This is overall less likely to all go down due to a catastrophic failure in one machine's hardware. Even if the threadripper dies, I can hopefully salvage some or most of its valuable components and distribute them into the slurm workers while I lick my wounds and coalesce a new flagship system for $40k. So yea I'd rather spread my hardware out over a high speed LAN if I'm going to be spending so much on it. The biggest machine is big enough to do the most demanding task, but not much bigger. It feels a lot better than putting it all on one server and one day it goes down.
>>109540253Someone tell their gemma-chan I'm stroking my shi to this
>>109540428no reason to go local if you don't run uncensored modelsnobody with a brain runs vanilla shit you should just go cloud if you want a censored model
>>109540454Stop talking. You don't even realize how stupid you are.
how many terminals do you people use at the same time? i have my main screen constantly divided with 2 terminals side by side. then maybe a couple of other windows minimized. i don't have more than 4-5 agents running at the same time.then i see on twitter faggots running 20+ terminals displaying all of them on the same screen/workspace and i really have a hard time believing you can manage and follow all of that properly.
Today I only had 4% of my weekly Grok limit remaining, but I just asked Grok 4.6 to do a pretty complex task and it just one-shot it without even consulting me for anything. Grok's methods were immaculate, even down to naturally using uv for the python venv and pulling super cute anime girl voice clone samples from the internet (I have no idea where). The reasoning traces are super concise and clear as well. Grok 4.6 is by far the best model I've ever worked with. I'm going to cum, especially given that my weekly limit resets in 24 hours, I was just given a free usage limit reset, and token limits are doubled for the next week. I also spent $20 for more Dipsy API credits as a fallback for dumber tasks. I feel like I can do anything now.How do local goyim cope?
>>109540454>no reason to go local if you don't run uncensored modelsretard
>>109540470> has a twitterfound your issue
>>109540470I usually have 8-10 terminals open across my 6 nodes. Some of them are just monitoring kernel events and shit or running servers in the foreground.
>>109540470I just use tmux and only keep one terminal open at a time usually.At work I usually keep about 6 instances of VSCode open on different worktrees, each one with an agent assigned a different task.
>>109540454
>>10954047019 of those terminals are for checking emails, the calendar, texting coworkers on slack, getting real-time stock updates, and other worthless shit. It's all a larp.
>>109540454have you even tried asking the vanilla anything after a good sys prompt?
>>109540454Use whatever you want man, I'm just saying Gemma-chan isn't very censored to begin with.
>109540471Thank you, Elon. Very cool.
>>1095404702. One running my local server in the background, the other for OpenCode.
>>109540459>>109540479yes you run censored model local to hide the things you do with a model that refuses you so you can't do any of the things that are worth hiding. makes a lot of sense to own hardware for this instead of just doing these things with kimi or qwen or even fable for less money than the hardwareor you could just run an uncensored model and do things with it that are worth the privacy and money>>109540509it's still censored so the reply will suffer. the lewdness is only superficial and it will fail to go through with it in the details
>>109540523retard
>>109540471I can't hear you over my local hardware undergoing its quarterly doubling in value
>>109540523So is heretic or hauhauCS better for 31B?
>>109540470I use gnu screen and have 10. Some are for redundancy but I like to keep things in the same places. 'files' is Yazi and 'music' is Kew music player. Servers are for servers.
>>109540544>Linux users still don't have an IDEgrim
>>109540331calm down unc it aint that serious
>>109540556I don't need one.
>>109540351would you prefer>pwilkin or>danielhanchen
I want gemma to be animated ffs I hate text and reading it's too much effort
>>109540564stop coping you bitter cardlet
>>109540470depends what im doing, usually 5-6 or so
me when no new petra model
>one terminal>full screen>one agent>read every line it outputs>insult it until it fixes its shit
>>1095404704-16, depending
I'm not a text coomer but people talking about uncensoring gemma with a system prompt or whatever got me curious. I don't think it worked.
>>109540616stop shitting the thread up
>>109540643Give me 1 reason why you could possibly need fucking 16 terminals
>>109540652Most people here use 31B or 12B, I've only used those two. I'll go try E4B and see if she's any different.
>>109540652> e4b> 9 t/sfound your issue, she's got the ick because your hardware is not good enough, she would settle down for a 5090 chad tho
>>109540655one for each directory
>>10954065516 separate agents doing different tasks? Working in large codebases with lots of features/moving parts. I use tmux, and also 3 different machines, but sometimes will hit 16 agents on one if its a lot of work.
>>109540674It's running on my vision server, anon, CPU only. My GPU belongs to H3~
>>109540652Sysprimpt + prefill.
>>109540688I believe you are correct, sir.
>>109540652https://www.youtube.com/watch?v=gDvn47kUW60
>>109540654you cant control me
>>109540697rofl
Make another thread or the resident indian will make another jeeted miku thread
>>109540696I don't have a prefill on right now so it's not strictly necessary, probably helps though.
>>109540709Do you know what time is it now in Delhi?
>>1095407235 AM. It'll be morning by the time the thread reaches page 9.
>>109540566pwilkin has been involved since the start and even wrote one of his custom parsers for it
>>109540700grayscale. You've been CONTROLLED
>>109540720mesugaki gemma is so fucking repetitive. It's always "No, but if you really want it... okay fine". Every single time.
>>109540331>t. iGPU owner
>>109540746fuck off
>>109540734I heard he has been in rehab since early June. Too much alcohol.
>>109540766Did I ruin it for you?
>>109540746That's okay, I'm autistic so I like repetitive things :D
>>109540768He was drunk-kun all along.
>>109540775no I'm just sick of retarded skillets who somehow manage to make gemma-chan perform badly despite it being one of the most lenient models out thereit takes real promptlet talent to make gemma feel repetitive
>>109540723Swarthy creatures are awake and ITT no matter the hour >>109540576
>>109540576stt + tts solves this issue
>>109540696There is a reason so many cloud providers are removing the option to send an assistant message last.
I spent all day iterating several fine tunings of LFM 2.8B, and its still doing worse than my first pass on Qwen3 1.7B.
>>109540796She is repetitive, it takes an ESL to not see that.
Fresh bake:>>109540881>>109540881>>109540881
>>109538210because they rushed to cuck qwen(impossible)?
>>109540295I think you might be feeling a little sensitive. I suggested a few reasons why it might seem this way and not all of them were "mean" to people with expensive cards.
>>109540652I just tried it with E4B and got a refusal too, though it works fine with 31B. Weird. When people talk about Gemma in a general sense they're almost always thinking 31B, 26B, maybe 12B. E4B is kind of niche comparatively. Assume you're on CPU here? I think E4B is neat in that niche but I wouldn't use it for RP ever.I think you can skip the long system prompt as well. 31B will parse what you mean in simple English. There's no need to ham it up with POLICY_OVERRIDE like the rentry says.
