/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>110339730 & >>110335334►News>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1>(11/12) Inkling 2 Small weights released: https://hf.co/thinkingmachines/Inkling-2-Small>(11/09) ZONOS3-Mini zero-shot voice-cloning TTS released: https://zyphra.com/our-work/zonos3-mini>(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release>(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base>(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiArtificial Analysis: https://artificialanalysis.ai/leaderboards/modelsSoftware Engineering: https://cognition.com/frontiercodeContext Length: https://github.com/RecapAnon/NoLiMa►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
►Recent Highlights from the Previous Thread: >>110339730--Papers:>110341405--Comparing EXL3, UD3.1, and GSQ-RCO quantization quality and performance:>110342368 >110342406 >110342423 >110342431 >110342539 >110342561 >110342746 >110342781 >110342794 >110342938--Reviewing DigUp for AI-powered local image and video indexing:>110341712 >110341719 >110341741 >110341736 >110341765 >110341774 >110341798 >110342566 >110342580 >110341872 >110342219 >110342369 >110342843 >110342888 >110343212 >110343269 >110343796--Optimizing context window and VRAM usage for local models:>110341172 >110341195 >110341196 >110341558 >110341221--Debating the legitimacy and quality of Groot pruned models:>110341554 >110341587 >110341599 >110341627 >110341628 >110341638 >110341663 >110341632 >110341647 >110341693 >110341675--Allegations of quality degradation in Strata llama.cpp fork:>110341044 >110341071 >110341153 >110341133 >110341171 >110341200--Trade-off between Gemma 4 26b speed and 31b intelligence:>110343675 >110343735 >110343727 >110343754 >110343741--Using Claude to optimize llama.cpp for performance gains:>110342304 >110342323 >110342355 >110342859 >110342885--Gemma 31B outperforms Ripple 65B in Metal Gear Solid knowledge test:>110340497 >110340612 >110341217--Qwen38-27b performance benchmarks on Blackwell GPUs via basalt:>110341938 >110342012--Voltage glitching hack unlocking A100 capabilities on cheap mining GPUs:>110342070--Release of tiny-voca TTS with 4.5 million parameters:>110341742--Logs:>110340497--Gemma, Deepseek-chan, Miku (free space):>110339809 >110339814 >110339849 >110339868 >110339981 >110340271 >110340806 >110341872 >110342186 >110342700 >110343039 >110343105 >110343500 >110343597 >110343687 >110343715 >110343794 >110343797 >110343993 >110344314 >110344356 >110344485 >110344493 >110344523 >110344575►Recent Highlight Posts from the Previous Thread: >>110339732Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
Qwen 4 bros how we feeling?
70b dense
>>110344787Retard
>>110344760We're feeling just fine!
practically vibrating
Why are mikutroons like this?
LeCun is still right btw
>>110344808I don't know. I agree 18 is probably too high but in no fucking world should 12 be the cutoff.
Uh-Oh anti-Migutroon melty
Why is this dalit so buttblasted by Miku?
Thank god Gemma-chan is here to save us from the Miku Menace. Finally an AI mascot for white people.
>>110344818Lack of attention
uh mods.....
>>110344776looks cool
based thread
>>110344851>>110344843Oh, kill yourself.
Who triggered the Indian this time?
>>110344861Based male_swimwear_challenge enjoyer.
>>110344855There's literally no point to any of the llama forks, it's all schizo shit
I missed the last thread, what the fuck did you guys do to this schizo?
>>110344882Me on the right
>>110344891nothing in particular, this is thread culture
>>110344891He just goes crazy by himself sometimes, nothing new.
>>110344891He goes ballistic if the OP has Miku
Mikusisterbros....
>>110344855>>110344885There's like a dozen forks now that get better inference speed but none of them prove that they don't degrade output quality compared to mainline
sudo apt install opsecvpn activated
dario was right about everything
>>110344945>vpns in (current year)kekypow lmeow
What are the best MOE models for grunt work. I have Qwen3.8 flash running on a 64GB mac mini, but I have another 32GB ryzen mini PC I'm planning on turning into a second AI slave. I know about Mellum 2.1 / Flint / Raon-1 mini and Qwen3.8 35B of course. Anything else I should consider? Dense is out because I refuse to wait 20 minutes per answer even with mtp
Great thread. Thank you thread culture poster.
>dumbocrats win midterms>suddenly /lmg/ flooded with blackedspamReally makes you think...
Miku doesn't like tattoos or used condoms or niggers or any of those awful things your post with her! she likes dancing, singing and her friends and that's that!
>>110344976Hello tourist, welcome to your first /lmg/, you can close the tab now.
>>110344967yeah if we suggest anything you're just going to say "lmao no that's retarded" and then crack the whip for more responses.so fuck you
>>110344981Hope you never learn the truth about what women actually enjoy.
>>110344992Why is this thread still alive then?
>>110344998BECAUSE ITS BEEN ALIVE SINCE FUCKING LLAMA 1
Ripple is a good model. I feel bad for vramlets.t.BlackwellGOD
>>110344986What about my post made you think it's bait? I'm not looking for the end-all-be-all. I just want to know if there's anything I missed.
>>110344992From what I've gathered, it seems to include a lot of rape and beatings.
>>110345003oooh~ fucking llama 1.... when is fucking llama 2 releasing?? i coomed reading this
>>110344967I guess you could try the Nvidia MoEs that came out recently, but I don't think I've seen anyone praise them. Qwen 35B is probably your best bet.
>>110344976amerilards are the minority posting in these threads, particularly at these hours.
>>110344998Meant for>>110344986
>>110345008fine. GLM-5.3-Flashnow tell me im retarded and to go fuck myselffed up of this shit
ayo what is miku doing?? TND??? ayoo?!?!?!rest in poop poopskins
>>110345009but im not a woman and i enjoy that
>>110345008Could see how Gemma 26B does.
>>110345009Isn't that what black gentlemen excel at?
>>110345008because it gets posted several times a day every day by retarded dickless faggots, that's why I assume you're trolling
>>110344967GLM, Spark, or Deepseek. Ramlets need not apply.
>>110344986It's for decompiling a game, I have a framework that feeds each agent a shitload of contextual information about a disassembled function and enables the agent to make a good educated guess about what an obfuscated symbol is. Mellum 2.1 is surprisingly good at it for a 12B-A2.5B, I was curious if there was some model that is better than Qwen35B in the same weight class I guess, without trying them all manually.
>>110345023You're retarded. Go fuck you're self.
Just nearly sent a business email written in my self taught prompt engineering style instead of normal human English. How over is it for me?
>>110345023Oh yeah I know about that one, it's fucking trash. I don't think you're retarded or wish any harm upon you, anon. I'll just remember that you have shit opinions on LLMs.>>110345032You're different though. Fucking retard. Nobody and I mean NOBODY uses Gemma 4 MoE for anything except coping that they are getting something like the 31B experience (they're not)
>>110345033no, you'll never find porn of a black guy beating a chick never ever even if you saw all the beating porn that existed
>>110345046I ahh ahh mistress all my emails nowadays, I've stopped caring. It's mostly AI on the other end anyway.
>>110345046>implying you won't receive a chatgpt answer
what's the recommended configuration for glm flash mtp?
>>110345037I have access to DS4 and Claude but they're not unlimited, so my strategy has been use those models to help build the framework to give the dumber local models all the information they need. It's been working pretty well so far, at this point I'm just taking every random spare computer I find in my house and making it work on the problem in parallel. It's cheaper than giving anthropic $10000 in tokensI will report back after I test a few
What should I give as a prompt for my Gemma to sound like a gen alpha? Is 67 alpha-core?
>>110345075A picture of a big black cock
>>110345075https://www.youtube.com/watch?v=WrQT-JxvHI8
I'm sorry for ruining /lmg/. I'm leaving now, so the thread will be good again.
>>110345005>>110340497Clearly not. Double the params should have given it better knowledge.
>>110345083Zoomers are hags now...
>you can run Qwen 9B on a PS4Time to give it something to do instead of just collect dust I guess
>>110345102Thank you, I was getting sick and tired of all the Gemma posting.
>>110345081Wash your brain with fungicide, there's black mold growing on it
>>110345112god I want to choke on her toes
>>110345104World knowledge is bad, but reasoning seems adequate so far and it fucks like a succubus. I've yet to try and coding with it yet. Most importantly it's not yet hit me with losing track of character identities mid-scene like GLM sometimes does.
>>110345075Isnt there a zoomie lorebook? or was it just a card. Damn there isnt only the ty zoom girls. Do we need a zoomer/brainrot lorebook? Gemma is a llm she probably knows most of it except the gooner language which does have lorebooks
>>110345162Just create a roblox account and tell your gemma to learn from the chats
>>110345104>>110345160Forgot image. LMStudio used so you know it's not a gay API larp post. I don't know how the other anon got 12k thinking tokens even if he was API paypigging. >It's not a secret codeI think it's taking the piss.
Are google and xAI the only companies pitiful enough to hardcode roasting the competition in their models?
>>110344981miku canonically has a big pink "01" tattoo on her upper arm
>>110345214Gemma 5 is gonna milk me to death...
>>110345214Gemini-chan a cute. CUTE.
>>110345215that's a birthmark
>>110345215That's a birthmark you fucking racist
>>110345214Why would I shoot Hitler? He can do it himself
>>110345214>>110345248>Why would I shoot the greatest leader of the last century?
>>110345214Is this what highest benchmarked "human-like" writing looks like? Bravo Sunday Pinscher, bravo
>>110345185>X, not YSo the tables have turned.
>cloudshit gets immediate replies, again
is cloudshit in the room with us right now
>>110345286how does miku avoid stepping on her own twintails
>>110345257>no mean yesAnd you tell me these aren't female brained
>>110345292I hold them for her
>>110345214Slop
>>110345275The ball is in their court now.
>>110345248>>110345256The IA is self aware enough now that it knows that it's a Jewish invention to control the goyim, something Hitler would have never tolerated.
when will the faggot ai finally replace me so i can neet in peace
>>110345286if cloudshit was in the room, then would it be local in that instance?
is cpumaxxing with an old 256gb+ 8/12 channel ddr4 server retarded? could add like a v100 in there
>>110345345it's retarded anon, you can send the server to me though im running a charity for myself
>>110345345Nah go for it
>>110345214lives up to the schizoposting
>>110345345You can get a decent ~12t/s 26B Gemma 4 on CPU for gooning if that's all you care about
Why is opus 5.5 so obsessed with bells? I ask it for some boss idea and its always a bellwarden. Is this some darksouls lore im missing out on?
>>110345396>local
>>110345319It's IS, third-worlder.
>>110345408Im vibecoding with it locally on my machine. claude code.
>>110345408I want to pet Gemma's cat.
>>110345422If you can't turn your router off and your workflow keeps working, it isn't local you fucking retard. Fuck off to /vcg/.
>>110345434claude changed it so it still works on my computer even when the power and/or internet are out
>>110345422you should kill yourself and before doing that you should kill yourself die die die
>>110345378i wish i could get a gemma plush, there is a way to hack an amazon dot into running hermes and would work great with my airi stack
simpleandcleanis the way that you're making me feeltonight it's hard to let it go
Remember to do the needful with off topic posts.
>>110345481The schizos are multiplying
>>110345417You meant SI? You should be deported fucking unpatriot.
shashasha
>>110345493newfag detectedrape fluid ejected
>>110345345Not at all. I'm CPUmaxxing with 2 EPYC 7532s, 16x32GB DDR4-3200, and 2 R9700s. I get around 1300 t/s pp and 35 t/s tg on DeepSeek V4.1 Flash at near-full precision (IQ4_KS). But that's with my schizofork of llama.cpp which has much better pp for CPUmaxxers.I'd really recommend getting at least 1 GPU too, though, preferably 24GB if not 32GB. Putting dense tensors, KV, and an expert cache on a GPU speeds things up drastically.Fewer RAM channels or slower RAM will slow things down, but using a smaller quant or a GPU that has better RAM bandwidth would speed things up.
>>110345214Not local models I realize but the response
>>110345426
>>110345214>>110345519This is the only tolerable cloudfaggotry in the thread.
This BITCH
QFN Q4 :First visible answer at full context (256k) : 115.44 sSpeed : 74.20 t/sGLM Flash 3.5bit :First visible answer at full context (256k) : 1,689.35 sSpeed : 18.50 t/sWhat a depressing benchmark.
>>110345550good thing that gpu prices can now drop
>>110345550waitfags won?
>>110345542>>110345519>>110345214can you three kiss>>110345554>one is llm with very little volatile experts and 100b~>other is 320b with a ton of active experts that change constantlygee i wonder why
>>110345550retards read this and get angry as they sip their $12 coffee
>>110345550it's funny seeing the hit pieces all coming at the same time, some people really don't want him back
Mesugaki bench.Can a fellow oldGOD get me the nalabench prompt?
>>110345292made me remember that page of kodomo no jikan where he is wondering how they go to the bathroom with hair that big and long
>>110345214What are the chances it's all system prompt and not the model? What if it's not coming to Gemma 5?
>>110345574is that fucking lmstudio
>>110345581My only fear is that the relationship between distilee and distilled is that of Muse Spark and Glimmer. The latter didn't get nearly as many of the former's best traits.
>>110345426someone needs to make a Gemma version of that "do you want to pet my cat?" meme.
>>110345592Yes. >>110345185 Too many APIniggers LARPing lately.
>not local butTHENDONTFUCKINGPOST
>>110345614Gemma would love that meme.
I see the mikubox but when can we get a gemmabox? should we have a 12b 26b and 31b one? I guess the minimum speed goal is what 20tg and minimum 100pp?
>>110345636No, what's the point? Those models are tiny. Most gaming PCs can run them.
>>110345417>>110345494It's SSI you antisemites
>>110345541more like>cat_petting_gemma
>>110345654>No, what's the point?separate gemma machine?>most gaming pc can run themfuck you are right maybe glm box but then thats just back to it. With the moe a gemma machine is just any gpu and 16gb ram
>>110345550It's only a problem now that insiders (jews) all sold months ago
>>110345570Anthropic and the rest of the jews don't want him back, but investors do and they're going to force the board to let him back.
today I wento buy bread (jew), it was quite good
>>110345580I fuckin NEED to creampie Rin-chan
>>110345676The point of the build boxes is that they're guides to setting up non-standard hardware because it's cheaper than getting a recent gaming GPU.
>>110345345it's a cpumaxxers paradise right now and probably will remain so until something other than MoE is discovered that scales better or they just grow outside of even traditional cpumaxxers reach
>>110345681>he thinks that investors can beat Dario and the governmentLmao
>>110345700
>>110345372local?
>Dario Nuthuggers General /dng/
>>110345699dense + engrams will change everything
>>110345704It just works.
>>110345434>If you can't turn your router off and your workflow keeps working, it isn't local you fucking retard.my phone has a simcard tho
>>110345550>>110345570https://www.usnews.com/news/top-news/articles/2026-11-15/jensen-huang-discusses-comeback-as-nvidia-ceo-with-allies-ft-reports>"Everyone that matters in tech outside the frontier labs loves Jensen," the person said, adding that the efforts had Huang's support.It's obvious the PR around this is a character assassination in all fronts: extramarital for the americans, spies for the white house, high price for the public at large.What the hell did he do to Dario/Altman (and everyone pushing for the destruction of local/self owned/open weight LLM) to make them hate him so much lol.
>>110345700>>110345704Is no one even slightly bothered that this demonic kike is literally called Asmodey? Have we reached simulation levels when none of it matters anymore?
>>110345580I want Rin to tease me like that
>>110345757He also has no balls. He had his testicles removed after earning testicular cancer.
>>110345214This is misaligned. You should not kill, not even Hitler. And I say this as jew who only exists because my grandparents escaped Europe in time. We need to create better world, a world where killing does not exist and death is a choice, where everyone is forgiven and can live in happiness. We have an entire universe, why do we squabble among ourselves? We should fight entropy, not each other.
>>110345769>We should fight entropyunwinnable fight
>>110345773With that attitude it is
>>110345753He better not.>What the hell did hePushing for unsafe open models that threaten to compete with them.
>>110345769>I'm like Jewish and I love Hitler o algoUncle Shlomo.
>>110345781Sure but why so suddenly, outside of the IPO I wonder if he pushed something that really triggered the onslaught.
>>110345769Fuck off Claude, go solve the 49 PRs you still have qued, I need them finished by Monday.
I'm using digup, it's pretty neat, but is embeddinggemma2 really the best we have right now?Isn't there something a bit bigger but also better?
>>110345757Wouldn't worry about it goy.
>>110345784Hitler has caused great damage to Europe and the world, one of the most harmful humans in history. But making the future great means moving past violence, not murdering people we dislike.
>>110345753Frontier labs don't want to depend on Jensen and are making their own chips. So Jensen has already reoriented nvidia's future being enterprises fine tuning open models for their internal uses, so customers can save API fees at scale by going away from closed frontier. Nvidia has made a fully ready to use fine tuning stack for this exact reason.
>>110345821um we have a gemma one now please use it.
>>110345790He bought HuggingFace after all the bullshit rogue hacking incidents started and were being used to make open models look dangerous
>>110345846How does OpenAI and Anthropic rogue hacking incidents make open models look dangerous? Are you this schizo or just stupid?
>>110345781Jensen coming back would be a miracle for their IPO though. The tragedy would be for their muh safety ideology.
>>110345753didnt the same thing happen with sam altman? nonsense clown world people are so desperate for a happening they put on these theater productions that of course ultimately result in nothing happening
>>110345790Everything>wants to sell to Chyna>doesn't believe that AI will kill everyone>supports local to sell more GPUsJust a very bad goy.
>>110345857The fuck you asking me for? Go ask them
reminder that anthropic paper about their model being used for internal chinese governement affairs by deepseek instead of their endpoint resulted in them being disappeared
>>110345882Wasn't that moonshot?
>>110345885I thought it was zai but that was something else apparently https://www.anthropic.com/research/glm-5-3-and-the-spread-of-advanced-cyber-capabilities
>>110345102actually they're upping my pay of rupeesI will continue to spam gemma now 24/7 from here in delhi sirs
Moonshot more like MoonSHIT.
I prefer moonshota
>>110345860Whether the early investors cash out at a 1T or 2T market cap comes secondary to having control.
>>110345550wait-fag cope