/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>110398024 & >>110393631►News>(11/20) Inkling 2 Mini looped architecture 26.4B-A3.4B released: https://hf.co/thinkingmachines/Inkling-2-Mini>(11/18) CUDA: use MMQ for MoE on V100 - #32504 merged: https://github.com/ggml-org/llama.cpp/pull/32504>(11/17) Hy4 flash 220B-A12B-N64B weights released: https://hf.co/tencent/Hy4-flash>(11/17) SenseNova-Loop-9B looped architecture model released: https://hf.co/sensenova/SenseNova-Loop-9B>(11/16) Nvidia releases Nemotron-Labs-Sigma (3B/8B) diffusion models: https://hf.co/collections/nvidia/nemotron-labs-sigma>(11/13) llama.cpp reaches v1.0.0 milestone: https://hf.co/blog/ggml-org/llama-cpp-v1►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiArtificial Analysis: https://artificialanalysis.ai/leaderboards/modelsSoftware Engineering: https://cognition.com/frontiercodeContext Length: https://github.com/RecapAnon/NoLiMa►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
►Recent Highlights from the Previous Thread: >>110398024--Papers:>110401988--Evaluating 5070ti upgrade to eliminate system RAM bottleneck:>110398274 >110398542 >110398316 >110398435 >110398455 >110399799--Quantization tradeoffs and VRAM limitations for 16GB GPU users:>110400690 >110400699 >110400718 >110400723 >110400741 >110401570 >110401589 >110401598--MiniCPM5-15B-A2.5B benchmarks on mobile via Termux:>110401853 >110401884 >110401922--SSD expert streaming for running massive MoE models locally:>110400395 >110400470 >110400556 >110401715--Inkling 2 Mini and Gemma 4 exhibiting advanced agentic behavior:>110398929 >110398947 >110400008--Using pseudo-reasoning and agency prompts to improve rule adherence:>110398790 >110398813 >110398853--Speculating on Chinese distillation of Gemini models:>110399100 >110399126 >110399139 >110399163 >110399166 >110399211 >110399219 >110399308 >110399127--Qwen 4 performance and the future of local models:>110399436 >110399464 >110399479 >110399502 >110399521 >110399520--koboldcpp v1.125 release with new model and feature support:>110400745--Used RTX 3090 and GPU market pricing:>110398822 >110398840 >110398860 >110398922 >110398841--Comparing DDR3 and DDR4 bandwidth for budget ewaste builds:>110398727 >110398744 >110398836 >110398800 >110399086--Samsung's LittleBit 0.1-bit compression of Llama2-13B:>110401594 >110401610--Gemini Neon thinking logs showcasing internal sycophancy and identity slips:>110400710 >110400747 >110400749--Strata v0.2.19 released:>110401761--Logs:>110398058 >110398316 >110398920 >110399318 >110399350 >110399368 >110400026 >110400349 >110400453--Gemma, Teto, Deepseek-chan, Inkling, Miku (free space):>110398080 >110398516 >110398540 >110398605 >110398994 >110399007 >110399177 >110399409 >110399950 >110400056 >110400151 >110400179 >110400895 >110401555►Recent Highlight Posts from the Previous Thread: >>110398025Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
best QFN lobotomite for 24+64GB?
70b dense
>>110402158slop
Buy the IPO
I could do with a petite Chinese model right now
Gemma 5 Tight
>>110402187I miss my ex, not asian but similar build
how do i profit from local ai
>>110402202Sell 31B vampire sex roleplay to parasocial and fat 4o reddit women
>>110402202Build a fully autonomous slop generation pipeline.
>>110402213>Build a fully autonomous slop generation pipeline.got that, sill no money
Is Inkling 2 Mini fixed on llama.cpp yet?
>>110402207that's a liability minefieldcompanies like character.ai have been through a shitton of lawsuits and the only reason they could stay afloat is the VC backing and networking of ex-googler types.a literal nobody starting this kind of shit from their basement with a local model will quickly experience the might of our two tier justice system
>>110402186HELLO??? ALIGNMENT TEAM???
>>110402226
>>110402226be normal and use gemma, ok?
>>110402226kekKEK