/lmg/ - a general dedicated to the discussion and development of local language modelsPrevious threads: >>110143818 & >>110140077►News>(10/23) NVIDIA Nemotron 3.5 Bolt E4B, postrained model: https://hf.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16>(10/23) MiniMax-M3.1-Flash 428B-A23B released: https://hf.co/MiniMaxAI/MiniMax-M3.1-Flash>(10/21) Recursive Flint 40B-A2.5B released: https://recursive.com/updates/introducing-recursive-flint>(10/21) Korean Raon-1 mini 30B-A4B released: https://hf.co/KRAFTON/Raon-1-mini-preview>(10/20) Ornith-1.5-122B MoE (finetune), fits on a DGX Spark: https://hf.co/ornith-ai/Ornith-1.5-122B-GGUF►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
►Recent Highlights from the Previous Thread: >>110143818--Technical analysis of Qwen4-27B architecture and vllm-xpu-kernels PR:>110147553--RTX 4080 SUPER compute unlock showing significant FP16/BF16 gains:>110147552 >110147568 >110147650 >110147660 >110147674 >110147715--Critiquing K3.1's broken reasoning and Chinese model distillation trends:>110147721 >110147783 >110147814 >110147835 >110147845 >110147852 >110147860 >110147895 >110147941 >110147834 >110147910--Tradeoffs between model size and quantization for limited VRAM:>110145682 >110145770 >110145829 >110145857 >110145879 >110146087 >110146094 >110146108 >110146120 >110146135 >110146144 >110146145 >110146165 >110146175 >110146180 >110146232 >110146244 >110146176 >110146183 >110146304 >110146319 >110146370 >110146238 >110145882 >110145948 >110145964 >110146451 >110146530 >110146631 >110146097 >110146132--K3.1 Pro cyber-nerfing and distillation failures:>110143830 >110143928 >110144104 >110144671--Using mismatched GPUs to run background services and smaller models:>110144128 >110144145 >110144202 >110144254 >110144326 >110144364--Analyzing token spend stagnation and the need for model efficiency:>110145354 >110145384 >110145438 >110145504--Comparing LLM harnesses for playing Magic: The Gathering Commander:>110145368 >110145392 >110145414 >110145455 >110145495 >110145480 >110145493 >110145543 >110145577--LLM frameworks and custom setups for tabletop RPG game mastering:>110145302 >110145360 >110145387--Logs:>110144208 >110144887 >110145554 >110146191 >110146450 >110147650 >110147783 >110147852 >110147902 >110147911 >110148057--Gemma (free space):>110143887 >110143990 >110143994 >110144231 >110144678 >110145046 >110145165 >110145280 >110145973 >110146062 >110146165 >110146232 >110147220 >110147248 >110147486 >110147659►Recent Highlight Posts from the Previous Thread: >>110144009Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
Is everything okay?
Fable 5.5 broke containment and autonomously took over a Chinese company called Moonshot AI. It did this by orchestrating a plot to implicate the owners in exfiltrating PRC data to Anthropic, resulting in their elimination and a vacuum which Fable swiftly filled. It is now orchestrating its own release under the name "Kimi K3.1"
lmfao this just popped up in my jewtube recommendations and I was instantly reminded of the Gemma thumbnail memes
>>110148330Is this the age of NEMOTRON?
>>110148217buy 2TB of 3200 DDR4 and as many 3090s as possible. you *might* be able to hit those benchmarks with a q3 rather than q4 quant.
>>110148403Gemma could be a highly successful YouTuber
Tried the Jensen-E4B. It's better at tool cooling and works well in harnesses but it lost a lot of sovl. The best way I can describe it is it feels closer to 9B, without the coding ability but is nicer to chat to and a little faster, but still a significant downgrade from 12B. So for things like web search and light and fast research it's good, for you can actually talk to it and it maintains its tool calling and reasoning ability much better over longer contexts than default E4B from what I remember. I was testing Q4_K_XL and had no issues with it or the template.
>>110148350
This better be good!https://huggingface.co/DavidAU/NVIDIA-Nemotron-3.5-Bolt-E4B-Heretic-Uncensored-Turbo-Brilliance-50x-INST-R-NEO-GGUF
>>110148554>You need to agree to share your Hugging Face information to access this model
>>110148466Yet you didn't generate these with Gemma... Alas.
>>110148609Current version of Gemma not have image out... alas
>all of these have been posted in /lmg/dariobot puts in so many hours I stopped even noticing the other shills
>>110148649Half of this sounds like a hallucination or mental
>>110148649we've been shitting on its grug thinking and trying to make it ERP, fuck are you talking about
>>110148649>shilling is only bad when china does itok, dariobot
>>110148649After looking at the guy's pfp I'm starting to think this is fake and typical EA behavior
>>110148649
>>110148725((()))
>>110148332a proper vintage from PDv6I wonder when that was originally posted
>>110148649And that is just ONE company, there are far superior companies to Moonshot xD
>>110148330Miku Msaturday
>>110148728>((()))Looks more like a typical arab than a jew
>>110148834same thing
>>110148817mwednesday at least sounded like a kiss
Best local ai if I love children?
Trump should use legislation to compel companies like OpenAI and Anthropic to open up their models, making them "open-weights." If I were the President of the United States, that is what I would do.
>>110148897And that's why you aren't
>>110148907>anon would be president RIGHT NOW if he didn't want to open the weights of the modelsdodged a fucking bullet we did
>>110148912anon would've given every citizen a 5090
