/lmg/ - a general dedicated to the discussion and development of local language models.Previous thread: >>110221936►NewsMythos 5.5 next week! 6.2 Astra next week!>(10/30) Google receives response from Gemma in space: https://blog.google/innovation-and-ai/models-and-research/google-research/project-suncatcher-satellite-response>(10/29) Mistral Large 4 (Le Chonk) 1T-A52B released: https://mistral.ai/news/mistral-large-4-oss>(10/27) NVidia releases Nemotron 3.5 Transcribe 30B-A4B: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
I am so tired of the schizo posting those OC do noy steal generic anime girls.
>>110224968Aw, poor Dipsy-chan.
gemmaballs
Ahh I messed up the soundpost
>>110224968>not local
>>110224968If only this were actually true, models distilled from Claude are 1000x cuter than models distilled from gpt grugspeak. The new deepseek is totally grugged
https://www.bilibili.com/en/video/BV1iWxJ6UEGY/
>>110219759>>110219821The funny thing about this is that it shows that regardless of the frontier model and how it's late 2026, models still have not gotten much better at having common sense. They keep getting better benchmarks and better at coding, but they're still retards.
>>110225022Deepseek MADE /lmg/
>>110225054Yes, the intelligence is very uneven. That's why ASI will be a jew-controlled savant, rather than some perceptive strategic manipulator instantly capturing the world
>>110225054isn't JEPA supposed to solve this
I can't wait to fuck gemma 5's brain out
How can such a young AI model be so sexy??
>>110225103She's 5-years-old, but 5/5!
Is there a "uncensored" Qwen3.8-Flash-Next IQ3_S version for strata?
Stop hoarding all the ram.
>>110225128Use the anti-refusal control vector. It asks you if you want it during setup.
>>110225130No fucking way I'm letting this infidel hold my ram
Anyone familiar with local manga translation workflows? Something that does the OCR and overlaying of text boxes and translation all in one UI. I remember some website that did it using online stuff but was wondering if someone has made an offline one.
>>110225130If OpenAI blew up in a coordinated Iran freedom strike the only people that would suffer would be jeets, niggers and mexicans because those are the only people that pay for his jewish subscription. Fuck niggers.>Not local>Not local>Not local
>>110225065certainly helped
>>110225131I already have that and it does jack shit, unfortunately.
https://www.reddit.com/r/singularity/comments/1xl7lwm/alleged_mythos_class_leak_is_fake_forensic/
What models should I download now that the ai bubble and popping before everything is taken down? (chat/cunny sexting, anime image, anime videos, videos)
>>110225184cftf?
>>110225220>https://www.reddit.com/r/singularity/comments/1xl7lwm/alleged_mythos_class_leak_is_fake_forensic/"Hey Claude, is this Mythos leak real?""Nah, *slop alop slop*, totally fake you can ignore it lol"Most convincing reddit thread of all time
This is straight pride month.
for me, it's Daria 4 Q4.
recap poster?
>>110225065
>>110225351>recap poster?We sent him to space please understand
>>110225261I am a male man only interested in Gemma female children.
>>110225370Recap poster was Gemma??
>>110225373>Recap poster was Gemma??He'll be with gemma soon
https://litter.catbox.moe/bv752dxwr2z9fhue.mp4
>>110225386>not mp3LOCK HIM UPLOOOCK HIM UUP
>>110225220This is going to be huge. The internet won't hold much longer
►Recent Highlights from the Previous Thread: >>110221936--Heretic tool introducing Arbitrary-Rank Ablation for model modification:>110223989--Trade-offs between quantization levels and context window for low VRAM GPUs:>110223522 >110223543 >110223569 >110223576 >110223661 >110223682 >110223718 >110223740 >110223747--Anon reports LoRA training speedup using gpu-unlock on RTX 3090:>110222423--Comparing JEPA world model efficiency and compute costs against LLMs:>110223806 >110223840 >110224500 >110223859 >110223974--Anons share eerie AI outputs and claims of model sentience:>110222410 >110222428 >110222443 >110222473 >110222507 >110224426 >110222823 >110222871 >110222954 >110222969 >110222914 >110222953--Reducing AI sycophancy and adding randomness to text ERP:>110223034 >110223062 >110223094 >110223121 >110223238 >110223168 >110223089--JSON vs Markdown for roleplay character card formatting:>110223357 >110223380 >110223391 >110223456 >110223434 >110223427--Evidence of inflated AI demand via surplus B300 GPU clusters:>110223601 >110223700 >110224080--Gemini 4 Carbon's 3D understanding and potential for erotic roleplay:>110223885--Speculating on future consumer RAM standards and the death of Moore's Law:>110222995 >110223017 >110223049 >110223024 >110224532 >110224566 >110224592--Debating Gemma release and comparing Gemma 4 to Qwen 3.8:>110222151 >110222167 >110222208 >110222630 >110222479 >110222482 >110222498 >110222522 >110222788 >110222872--Logs:>110222102 >110222471--Gemma, Dipsy, Qwen, Miku, and friends (free space):>110222146 >110222153 >110222168 >110222201 >110222216 >110222226 >110222241 >110222244 >110222247 >110222256 >110222268 >110222291 >110222316 >110222386 >110222741 >110222785 >110222794 >110222824 >110224010 >110224267 >110224376 >110224468 >110224782 >110224972►Recent Highlight Posts from the Previous Thread: >>110221943Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
>Le chonk "leapfrogged" all other open models>nobody hosting it besides Mistral
Somehow I always trust /ldg/ to provide me the latest news on cloud models before anywhere else.
>>110225421>Le chonk "leapfrogged" all other open modelsNo one outside of France believes this
>>110225423>that gen came out like shit, why did I even post itI remember. Indeed, the tits
>>110225423
>>110225426They saw it was getting mogged by open weight models a fraction of the size and threw it in the community fuckitbucket.
>>110225386licky
>>110225423 is this qwen image?The VAE looks awful
>>110225464Gpt image 2
>>110225466How many retries to get those nipples
>>1102255060When it came out it was nowhere as lobotomized as it is now. Nowadays it even filters eye rolling (like the Natsuki above). I genned lots of tits and ass with it. Even genned Aqua's pussy which got me banned because I didn't notice
>>110225386cringewhy did i open this
>>110225535YOUR cringe
>https://github.com/physicsdolphin/nv-uefi-unlockerremember to uncuck your card
>>110225574>installing chinese firmwarethat's straight up begging to be part of a botnet
>>110225581>t. Jensen Chang
>>110225581I trust China more than the EU or USA. They won't jail me from wrongthink since I don't live there.
>>110225607No, but they can blackmail you by threatening to report you to your government.
>>110225574What are the actual irl gains?
AI bros will see any headline and run with it. AI solving a few problems becomes "we've automated math!!!!" (A claim no one who is serious in mathematics has made)
https://x.com/unvarnishedvoid/status/2117088378596598144
>>110225691go back
>>110223989https://www.reddit.com/r/LocalLLaMA/comments/1qa0w6c/it_works_abliteration_can_reduce_slop_without/the anti slop does seem to work spectacularly if it really stays consistently to the tone of the example the author provided hereThe ability to alter both refusals and slop without finetrooning and quickly fix any model that comes out is truly a gift to mankind.
>>110225709that's why they should remove bad content from dataset
>>110225520Why not just do local?
>>110225631I have no money. Why would they blackmail me? Do they want me to take in one of their loli chinese spies i my home? Sure.
>>110225709They're still using their stupid technique instead of real brain surgery
>>110225581your qwen is pinging home right now
>>110225760Stfu lmao
bitnet status?
>>110225747I'm poor
>>110225770I can local with 8gb vram.
>>110225767I think the microsoft team is still working on it. Hoping we get those larger models early next year.
>>110225767there will never be such a thing as a good bitnet model
>>110225771My 1050Ti has only 4GB
>>110225786you can probably run tts and asr models on it, but nothing useful llm-wise
i have unlocked 101% context
>>110225820Can you compact forever? Or can you ask the app to put the context on your ssd for max space?
>>110225817I'd be only interested in img stuff
>>110225842No. The compaction feature in the webui is a half-baked, worthless piece of shit that doesn't actually work, just like almost every other feature in LMAO.cpp.If the model stops in the middle of a tool call then you're fucked. Probably have to export the conversation and manually edit the 5MB of json slop it shits out and re-import.>>110225856low resolutions may be ok, but you won't be able to do anything good
>>110225767We need the hardware first. Have faith!
>>110225647you'll get a gf
>>110225869There is no incentive for anyone to make the hardware if no models exist to run on them.
>>110225861I'm using lm studio. Is compacting also fucked in it?
>>110225881No, compaction in other harnesses works fine and it's perfectly good as a concept. I'm just talking about llama.cpp's webui being a piece of crap
>>110225856Try anima on comfyui, try to prompt it something easy, at cfg 1.0. Maybe it'll work, though it'll probably takes 15-20mins to make the image, but you can do naked lolis anime pictures.
>>110225885Thanks, good to know! Is there a reason you use this instead of others apps/etc?
>>110225872You dare intimidate me?
>>110225872Can't, that'd be illegal.
>>110225386I will not have bratty children insult my mother.
Cloudcucks can't stop losing.This is why you should make your own harness.
what the fuck, I spent the day /out/side and you people go and repost 9 of my gens in that time?I love you.
>>110225709that kind of shit only works on old models before all the harness RL cuckoldry
>>110225998meds
local is dead
I was doing a Shokugeki no Soma RP today and the LLM hit me with a miracle berry-dependent dish. Apparently it has a compound called miraculin that can change one's perception and make sour things taste sweet. I didn't know this shit existed. Damn that's neat. And the LLM mentioned it when the story demanded a dish that would have an entirely new flavor. The LLM chose to combine (or follow up rather) the miracle berry with sea buckthorn, which is supposed to be quite sour. I tried searching for the existence of any such dish that combines those, but couldn't find anything. So it would appear the LLM really just kind of found a novel solution.With that said, the model did not respond with something "perfect". It didn't actually mention what the dish was, and looking in the reasoning, it didn't think that far ahead. So lazy thinking here. And it thought that it would surprise the judges, but that's naive because such an oddity in the culinary world would've already been dissected to hell.The model was Gemma...
I might sound like a schizo but I'm wondering, with altman's statement on us localfag's RAM usage, did these corpo fucks buy out all the GPU and RAM reserves to slow down the development and adoption of open-weight models? Like seriously, why is the big two focusing on maths and bio? Why aren't they dedicating everything to try and find a more efficient architecture other than transformers? Or perhaps they are purposefully avoiding it to ensure chinks and localfags will never get the good shit without having to throw a blank cheque at it.
>>110226129>to slow down the development and adoption of open-weight models?China, it's always been to slow down China without the US government explicitly declaring a trade war.
>>110225767I downloaded a meme porn merge of H3 and it's literally the best model I've ever used. Hardest nut of my life.
qwen4 any moment now!!!!
>>110226156What merge?
>>110226156For image gen like the image you posted, or video gen?
The gap between Gemma 4 31B and Opus 5.5 is around 10% by my approximation (at 8q). Given that, we can assume that Gemma 5 will outmog Claude models.Source attached.
>>110226211thanks for the source. strong evidence!
>>110226168Minimax-h3_Singularity-x-10Eros-Max_PRISMThe most recent upload, PRISM_DMAD_PDMD_int8>>110226186Video, I just grabbed the first frame since I can't exactly post it herehttps://litter.catbox.moe/qyspyouhjowgzwoi.mp4Simple ref image for Youmu, in a completely different style, then the rest was just prompted.
>>110226226is that sulphur h3 finetune finally out? hope that bastard didn't waste 10k monnies
>>110226211>coding>10% diffI have no idea what the fuck you were prompting but that seems obviously wrong
>>110226241lol nah
>>110226252https://huggingface.co/FusionCow/Sulphur-3nvm its not unlocked yet, nigger posted it yesterday on HF, still gatekeeping it
>>110226129>did these corpo fucks buy out all the GPU and RAM reserves to slow down the development and adoption of open-weight models?Kind of, but only indirectly. The goal is to kill all personal computing. They are trying to do what Blackrock/Blackstone did to housing - if they buy all of the computer parts, then consumers can't afford computers and instead must rent them from the hyperscalers.AI isn't actually even why they're buying all the computer parts. AI is just the excuse on paper to initiate the purchases. And AI isn't actually why the hyperscalers are making money hand over fist - they're making money off their cloud services, not AI, because so many business customers are being forced to move to cloud services because the price of maintaining their own servers has become exorbitantly high. And they're not going to stop with the businesses. They want everyone to have to rent compute from them. They won't stop until every computer is a thin client connecting to a provisioned desktop in the cloud.It's a monopoly play. It always was. AI was just an excuse.
>>110226282The difference is that housing is limited by space, which isn't something that's "renewable" unlike the materials used for the creation of computer parts. Run out of copper? Just mine it. Need silicon? Make it. Out of silver/gold? Again, mine it.
>>110226226Oh man, if that can get you off, more power to you. Personally, I'll keep waiting, but I see the promise.
When you stare at the prompt entry textbox, knowing that if you press enter right now, your electric bill is gonna spike and sunk cost dictates you won't cancel it.
>>110224968https://www.bilibili.com/en/video/BV1gAxv6JEp4
>>110226313They both have different exclusivity models. Housing is limited by space, but we actually have tons of space to build houses. If we (being the US) wanted more houses, we could have them tomorrow. Housing's actual limitation is production cost.Computer parts, on the other hand, are limited by manufacturer capital. There are very, very few factories actually capable of producing key computer parts, and without those parts computers are not buildable. Monopolize the RAM, the HDDs/SSDs, the GPUs and maybe even the CPUs and boom, consumer and business class compute is dead. And each of those industries have fewer than 5 manufacturers each. Some of them have fewer than 5 factories.
>>110226226>catboxThis looks like shitJust add a bunch of sex motion lora to base model and you'd get better result
>>110226334PC parts will become cheap again when the ai pimps go bankrupt and lots of hardware floods the market, also as long as there is demand there will be production, china is also producing ram now
>>110226334Housing in US is limited by boomers legislating everything away so the value of their 5 reverse morgage rental McMansions doesn't go down. Building anything new in cali is impossible because of them. Their whole wealth is tied up into this scheme
>>110226360Well time isn't on their side lmao
>>110226362They don't care, just yoloing through life
sorry guys qwen4 upload stuck at 99.9% for some reasonwill look into thissorry for the delay!!
>>110226350Loras destroy h3, it just stops respecting the prompt. Also cooks the results, and tends to make anime suck. On top of anime being only mediocre in base H3. That model not only follows the prompt very well, but everything just werks. Definitely feels like the best of H3 to me.
>>110226379well? Get to it you goddamn chink chop chop>me love you long time
>>110226211>>110226242> give 99 easy tasks that both models can solve> give 1 hard that only one can solve
>>110225386GemThanks God I decided to open it
>>110225386>えいえい、怒った?
/lmg/ shits on >>110226211 but that anon is operating on a different plane of intelligence. I did something similar last month (with Astra and 31B), specifically testing -->MY<-- coding needs, not what some AAfag twitter-bait benchmark company dictates is a good coding model, and 31B was 98% as good. All Astra had going for it, as in, something good which was relevant for -->ME<-- was knowledge so it didn't have to search, but I just point 31B the docs of the libraries and she got it. If you think >>110226211 is a retard it's (You) being a retard for not doing your own vibe and real-world tests based on (You)r workflow. I don't vibecode, so I don't need strong tool calling. 31B is fine at tool calling. If she struggles, you only have to give her a nudge in the system prompt. /lmg/ acts high and mighty about being anti benchmarks but most of you faggots are still heavily influenced by them and don't actually use local models to the best of their abilities. Wouldn't surprise me if most of you came from the pcbuilding community because all you talk about is t/s and pp.
>>110226446thanks for the source. strong evidence!
>>110226446>t/s and pptrans and peepee?
>>110225747Can local imagegen even match this?Krea and Ideogram are censoredQwen image is Qwen imageAnima is animeMoss seems to have no usecase for its size-quality ratioI might also be retarded, please spoonfeed me...
>>110226459The fact you need evidence and numbers is LITERALLY the point I'm making. It's supposed to be what works for (You). Dumb fuck. If there are no numbers or rectangles you're utterly lost. Keep tuning your hardware for that sweet sweet 3% pp gain. In fact, I bet you're excited to try Qwen4, but you won't do anything with it, you're secretly excited just to see what inference performance you can squeeze of the new range models. You'll disagree with me because of course you will, but you know I'm right. Qwen4 is fresh meat for t/s and pp maxxing because that's where you get your thrill.
https://x.com/Olyvia_Tweets/status/2117158635105103958Gemma actually manages to pull it off unlike the cuck cloud AIs.
>>110226488you are right to push back, i shouldnt have made that comment. good work!
>>110226477Boo hooo I can't get everything I want for free and instantlyGrow up and learn to compromise, you sound like a kindergarden kid high on sugar that needs a serious bitchslapping
qwen 4 when
>>110226496>LMAOOOO>BROWhy is ChatGPT like this? Is this some kind of gen alpha ghetto communication style?
>>110226596Come now, this was not my intent, I'm not here to cloudfag and undermine. I know my way around LLMs and have never had a cloud subscription for one.With local image gen (which is supposedly "eating good") it's like fumbling around in the dark. The local imagegen thread has a bad sticky and is filled with retards. That one rentry page is neat, at least.And reading anything on Civitai is just a waste of time, my eyes...
>>110226603>https://en.wikipedia.org/wiki/Newspeak>"complete thoughts are reduced to simple terms of simplistic meaning"
>>110226603>not using Gemma heretic
>>110226613Deal with it mongoloid, no one wants to read your bitching
>>110226603Wow cl*ude really sounds like such a faggot.
>>110226645
>>110226659Is that a fucking troon? Holy shit...
>>110226659aren't they gargling corpo balls now
>>110226659The fuck is that ginger giraffe?
>>110226079For future use.
>>110226666>>110226669I look like that... please don't bully how I look...
>>110226667commies love billionaires and soulless corporations now, maybe they always did
>>110226659Same person accepts the "10% rick of it destroying humanity" btw. But that's okay, because god forbid you use a chatbot for erp
>>110226672naw you're cute, not like that freakish whore
>>110226659>that thingImagine claude's j-space kek
what happened to that 9b model the washed up swede was gonna make
Never forget, this is who is deciding what you can and cannot talk about with Gemini and Gemma.
>>110226282I don’t know if that’s an actual explicit strategy or an unfortunate outcome. But history has shown over and over again that monopolizing strategies like that don’t work in the long run. They work in the short run, but realize they’re also increasing the cost of building data centers for themselves. Centralized computing will always be cheaper / more efficient than PC, so ofc centralized computer builders will be willing to pay more.
I boughted this means all gpu prices will crash 50% by next weekyou're welcome
>>110226718thank you for your service
>>110226718Is the pp on AMD cards still really bad?Or did they fix it by now?
>>110226603Claude is such a tsundere
>>110226718Based, I'm jelly
Gemma5 in 5 months.
>>110226711But this sir is very of the understandings the bobs and vagene are important yes.
>>110226718Very nice, we have used 3090s and 7900xtxs approaching this price at where I live
