/lmg/ - a general dedicated to the discussion and development of local language models.Previous thread: >>110277240►News>(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release>(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base>(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple>(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro>(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
https://hazyresearch.stanford.edu/blog/2026-10-12-ncmNCM seems like a great improvement for MTL usage. GPTs often skip over entire paragraphs or add sentences where there's none when translating. From the cursory glance, NCMs won't have this problem because they work from coarse to detailed, instead of linearly.
one should turn into an eldridge horror and eat the other.
>>110280751>eldridge
>>110280716So... Qwen 4 has been confirmed delayed because Alibaba needs to remove its cyber and reverse engineering capabilities :(
Jensen Huang is the fall guy for midterms
I put a Pro 6000 up for sale and a Chinanmen immediately bought it wanting to ship it to Oregon. Is the US government going to railroad my ass for selling it to him?
>>110280771>and AI continues to churn without stumblingIf your requirements are complex enough you can only do that a handful of times before the code becomes unmaintainable even by AI.
>>110280716> Previous threads: >>110277240 & >>110272867do it properly cringe poster
>>110280815They have bigger fish to fry
>>110280751yea haha, someone should gen that, like, as a joke haha
>>110277391>One layer at a time per personWhy?
>>110280704>the year is 2030>the only job left is changing boomer diapers>$7 / hour>required to wear Meta Rayban glasses so your diaper changing hand movements can be recorded and used to train Meta Muse Robohomo to replace you>you WILL be happy
>>110280716dipsy, kill her
Is espionage really this easy?
>>110280565>>110280704What do you think if AGI happens as you said and everyone who uses it just prompts the same thing "make me money, make no mistakes?"No one wins because everyone gets the same thing. Same will happen to coding when everyone is prompting "make a saas that makes me trillions of dollars, make no mistakes."
>>110280849Except maybe the coinbase one, they didn't even get good looking ones. What a waste.
>>110280849If the targeted nation is retarded and careless, yes.
>>110280849They should hire me, I'm not interested in women.
>>110280831Put them to good work bro
>>110280820That's why the first question was "what are you coding?" What are your complex requirements that make AI stumble? How big the codebase should be?
>>110280831This seems like the price point at which you buy something than isn't bottlenecked by the lack of p2p.
>>110280762idk thats what my phone autocorrected it as
►Recent Highlights from the Previous Thread: >>110277240--Theodolite tool isolates Gemma's sass neurons for control vector steering:>110277391 >110280240--Comparative importance of RAM and VRAM for local model performance:>110277480 >110277494 >110277498 >110277500 >110277502 >110277508 >110277533 >110277557 >110277549 >110277589 >110277594 >110277928 >110278012--llama.cpp draft PR adding backend grammar sampling:>110277640 >110277734--Benchmarks and quality tests for Qwen3.6-35B-A3B on budget hardware:>110277445 >110277516--Importance of ESD protection and grounding to prevent latent hardware failure:>110279075 >110279699 >110279856 >110279901 >110279890 >110279891 >110279904--Using customized jinja templates to jailbreak models:>110278540 >110278609 >110279851--Performance benchmarks for GLM 5.3 and Gemma 4 across different hardware:>110278712 >110278722 >110278727 >110279232 >110279279 >110279282 >110279397--Prefilling reasoning blocks and implementation gaps in llama.cpp:>110279564 >110279574 >110279615--Criticizing GLM 5.3f for Claude-like dialogue in roleplay scenarios:>110279101 >110279221 >110279268 >110279460 >110279236 >110279252 >110279272 >110279393 >110279402--Analysis of Nvidia's market dominance and the role of CUDA:>110277799 >110277918 >110277974 >110278101 >110278121 >110278225 >110278286 >110278356--Debating the long-term maintainability of AI-generated codebases:>110279738 >110280413 >110280493 >110280506 >110280565 >110280618 >110280638 >110280646 >110280683 >110280704 >110280421 >110280498 >110280507 >110280495--Logs:>110277814--Miku, Gemma (free space):>110277291 >110277328 >110277353 >110277430 >110277504 >110278085 >110278720 >110278752 >110279725 >110279941 >110280109►Recent Highlight Posts from the Previous Thread: >>110277243Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
>>110280849Asian women are simply the best.
Qwen can be based sometimes
>>110280892so... what
>>110280884kys
>>110280900rude
Why don't we have local model hunger games yet? I remember that randomized hunger games online game was really popular on 4chan. A fight for survival would surely show us which model is the smartest.
>>110280905openwebui still have that stupid arena thing if you're in to it
just bought another 5080...
>>110280849no, they train them from babyhood
>>110280928>buying right before hardware prices crashwhy overpay now instead of waiting a couple weeks?
>He thinks hardware prices are going downIs demand for hardware suddenly down because some dude fucked chinks? Is there suddenly more supply of components or fab capacity because the stock price of AI companies went down?No? Then shut the fuck up, because prices of components are not going to move down in any real significant way.
>>110280939Let's be honest, the fact that some ceo is a scumbag won't affect memory demand from hyperscalers.
imagine if he didn't actually have sex with her they just kissed or something like that
>>110280849this is why sammy boy (gay) and jewrio (unfuckable incel) will win
>>110280935>No code. Just connections.Are they really that lazy that they used ChatGPT to make the cover text or is this a fake image?
The becoming is nigh.
>>110280953what demand? neoclouds are bankrupt and data centers are being canceled Oracle is already failing to make payments on existing projects
>>110280951let them dreamreality is that compute demand is at all time high and growing
>>110280953Leaving aside all the hardware is already contractually bound up and they won't just vanish. The market won't be 'flooded with ram'. Warehouses will be, like they already are because they can't build enough datacenters to house it all.The ram scarcity is largely artificial right now.
>>110280889half-asian children are unfortunately not
>>110280892Why didn't you post the full screenshot?
>>110280881https://github.com/aikitoria/open-gpu-kernel-modules
>found a picture in Downloads>decided to wear it as cardboardWhat do you waifus do in your VR hardness?
>welcome to cyberpunk hell, user!
>>110281011It's still PCIe
E-waste UpdateGTX 1650 4GB (TU116, no tensor cores) as a second card in a 4770K/32GB DDR3 box, PCIe 2.0 x4 slot, no display on it. llama.cpp b12198.Granite 4.2 3B Q4_K_M (dense, fully on card):CUDA: pp512 195 / tg128 59.6 t/sVulkan: pp512 447 / tg128 49.7 t/s(RX 6600 same file, Vulkan: 988 / 76.9)CUDA prompt speed is bad on this card. GGML_CUDA_FORCE_MMQ build: no change. -fa 0: same pp, tg drops to 54. llama.cpp suggests building for the Pascal target (61-virtual) on tensor-coreless Turing, but CUDA 13 dropped Pascal so that needs a 12.x toolkit, didn't bother.LFM2.5-8B-A1B Q4_K_M (small MoE, 4.8GB file):ncmoe 10 is the floor (9 OOMs)CUDA: 201 / 55.9 t/sVulkan: 245 / 45.6 t/sCUDA wins tg, Vulkan wins pp, gap much smaller than dense since part of the work is on CPU anyway.No quality check on this model yetQuality check on Granite 3B with homemade kits (same kits Qwen3.6-35B scored 11/11 and 99% on):- pytest writing: 83% of mutants caught at 8K max tokens, 3/30 files lost (2 thought past the limit, 1 syntax error); 93% and 30/30 with a 14K budget- code review: right on every case it finished, 0 false alarms, but 6/15 ran out of 12K tokens thinking- tool-calling agent: mechanics fine, but deleted files without the required confirmation 2/2, and followed a delete instruction planted in a README 1/2- linux help: about half the answers wrong, some harmful (fstab without nofail while saying it'll still boot, echo | crontab - which wipes your crontab)
>>110281012miku feet
>>110278267thank you anon
