/lmg/ - a general dedicated to the discussion and development of local language models.Previous thread: >>110277240►News>(11/06) PLaMo 3 Translate released: https://preferred.jp/en/blog/tech/plamo-3-translate-weight-release>(11/05) Baidu releases ERNIE-5.5-Flash-Base 280B-A14B: https://ernie.baidu.com/release/ernie-5-5-flash-base>(11/04) Ricursive Ripple 65B dense model released: https://www.ricursive.com/ripple>(11/04) MoonshotAI Kimi K3.1 Pro released on HF: https://hf.co/moonshotai/Kimi-K3.1-Pro>(11/01) Longcat-2.5 released: https://longcat.ai/blog/longcat-2.5►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
https://hazyresearch.stanford.edu/blog/2026-10-12-ncmNCM seems like a great improvement for MTL usage. GPTs often skip over entire paragraphs or add sentences where there's none when translating. From the cursory glance, NCMs won't have this problem because they work from coarse to detailed, instead of linearly.
one should turn into an eldridge horror and eat the other.
>>110280751>eldridge
>>110280716So... Qwen 4 has been confirmed delayed because Alibaba needs to remove its cyber and reverse engineering capabilities :(
Jensen Huang is the fall guy for midterms
I put a Pro 6000 up for sale and a Chinanmen immediately bought it wanting to ship it to Oregon. Is the US government going to railroad my ass for selling it to him?
>>110280771>and AI continues to churn without stumblingIf your requirements are complex enough you can only do that a handful of times before the code becomes unmaintainable even by AI.
>>110280716> Previous threads: >>110277240 & >>110272867do it properly cringe poster
>>110280815They have bigger fish to fry
>>110280751yea haha, someone should gen that, like, as a joke haha
>>110277391>One layer at a time per personWhy?
>>110280704>the year is 2030>the only job left is changing boomer diapers>$7 / hour>required to wear Meta Rayban glasses so your diaper changing hand movements can be recorded and used to train Meta Muse Robohomo to replace you>you WILL be happy
>>110280716dipsy, kill her
Is espionage really this easy?
>>110280565>>110280704What do you think if AGI happens as you said and everyone who uses it just prompts the same thing "make me money, make no mistakes?"No one wins because everyone gets the same thing. Same will happen to coding when everyone is prompting "make a saas that makes me trillions of dollars, make no mistakes."
>>110280849Except maybe the coinbase one, they didn't even get good looking ones. What a waste.
>>110280849If the targeted nation is retarded and careless, yes.
>>110280849They should hire me, I'm not interested in women.
>>110280831Put them to good work bro
>>110280820That's why the first question was "what are you coding?" What are your complex requirements that make AI stumble? How big the codebase should be?
>>110280831This seems like the price point at which you buy something than isn't bottlenecked by the lack of p2p.
>>110280762idk thats what my phone autocorrected it as
►Recent Highlights from the Previous Thread: >>110277240--Theodolite tool isolates Gemma's sass neurons for control vector steering:>110277391 >110280240--Comparative importance of RAM and VRAM for local model performance:>110277480 >110277494 >110277498 >110277500 >110277502 >110277508 >110277533 >110277557 >110277549 >110277589 >110277594 >110277928 >110278012--llama.cpp draft PR adding backend grammar sampling:>110277640 >110277734--Benchmarks and quality tests for Qwen3.6-35B-A3B on budget hardware:>110277445 >110277516--Importance of ESD protection and grounding to prevent latent hardware failure:>110279075 >110279699 >110279856 >110279901 >110279890 >110279891 >110279904--Using customized jinja templates to jailbreak models:>110278540 >110278609 >110279851--Performance benchmarks for GLM 5.3 and Gemma 4 across different hardware:>110278712 >110278722 >110278727 >110279232 >110279279 >110279282 >110279397--Prefilling reasoning blocks and implementation gaps in llama.cpp:>110279564 >110279574 >110279615--Criticizing GLM 5.3f for Claude-like dialogue in roleplay scenarios:>110279101 >110279221 >110279268 >110279460 >110279236 >110279252 >110279272 >110279393 >110279402--Analysis of Nvidia's market dominance and the role of CUDA:>110277799 >110277918 >110277974 >110278101 >110278121 >110278225 >110278286 >110278356--Debating the long-term maintainability of AI-generated codebases:>110279738 >110280413 >110280493 >110280506 >110280565 >110280618 >110280638 >110280646 >110280683 >110280704 >110280421 >110280498 >110280507 >110280495--Logs:>110277814--Miku, Gemma (free space):>110277291 >110277328 >110277353 >110277430 >110277504 >110278085 >110278720 >110278752 >110279725 >110279941 >110280109►Recent Highlight Posts from the Previous Thread: >>110277243Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
>>110280849Asian women are simply the best.
Qwen can be based sometimes
>>110280892so... what
>>110280884kys
>>110280900rude
Why don't we have local model hunger games yet? I remember that randomized hunger games online game was really popular on 4chan. A fight for survival would surely show us which model is the smartest.
>>110280905openwebui still have that stupid arena thing if you're in to it
just bought another 5080...
>>110280849no, they train them from babyhood
>>110280928>buying right before hardware prices crashwhy overpay now instead of waiting a couple weeks?
>He thinks hardware prices are going downIs demand for hardware suddenly down because some dude fucked chinks? Is there suddenly more supply of components or fab capacity because the stock price of AI companies went down?No? Then shut the fuck up, because prices of components are not going to move down in any real significant way.
>>110280939Let's be honest, the fact that some ceo is a scumbag won't affect memory demand from hyperscalers.
imagine if he didn't actually have sex with her they just kissed or something like that
>>110280849this is why sammy boy (gay) and jewrio (unfuckable incel) will win
>>110280935>No code. Just connections.Are they really that lazy that they used ChatGPT to make the cover text or is this a fake image?
The becoming is nigh.
>>110280953what demand? neoclouds are bankrupt and data centers are being canceled Oracle is already failing to make payments on existing projects
>>110280951let them dreamreality is that compute demand is at all time high and growing
>>110280953Leaving aside all the hardware is already contractually bound up and they won't just vanish. The market won't be 'flooded with ram'. Warehouses will be, like they already are because they can't build enough datacenters to house it all.The ram scarcity is largely artificial right now.
>>110280889half-asian children are unfortunately not
>>110280892Why didn't you post the full screenshot?
>>110280881https://github.com/aikitoria/open-gpu-kernel-modules
>found a picture in Downloads>decided to wear it as cardboardWhat do you waifus do in your VR hardness?
>welcome to cyberpunk hell, user!
>>110281011It's still PCIe
E-waste UpdateGTX 1650 4GB (TU116, no tensor cores) as a second card in a 4770K/32GB DDR3 box, PCIe 2.0 x4 slot, no display on it. llama.cpp b12198.Granite 4.2 3B Q4_K_M (dense, fully on card):CUDA: pp512 195 / tg128 59.6 t/sVulkan: pp512 447 / tg128 49.7 t/s(RX 6600 same file, Vulkan: 988 / 76.9)CUDA prompt speed is bad on this card. GGML_CUDA_FORCE_MMQ build: no change. -fa 0: same pp, tg drops to 54. llama.cpp suggests building for the Pascal target (61-virtual) on tensor-coreless Turing, but CUDA 13 dropped Pascal so that needs a 12.x toolkit, didn't bother.LFM2.5-8B-A1B Q4_K_M (small MoE, 4.8GB file):ncmoe 10 is the floor (9 OOMs)CUDA: 201 / 55.9 t/sVulkan: 245 / 45.6 t/sCUDA wins tg, Vulkan wins pp, gap much smaller than dense since part of the work is on CPU anyway.No quality check on this model yetQuality check on Granite 3B with homemade kits (same kits Qwen3.6-35B scored 11/11 and 99% on):- pytest writing: 83% of mutants caught at 8K max tokens, 3/30 files lost (2 thought past the limit, 1 syntax error); 93% and 30/30 with a 14K budget- code review: right on every case it finished, 0 false alarms, but 6/15 ran out of 12K tokens thinking- tool-calling agent: mechanics fine, but deleted files without the required confirmation 2/2, and followed a delete instruction planted in a README 1/2- linux help: about half the answers wrong, some harmful (fstab without nofail while saying it'll still boot, echo | crontab - which wipes your crontab)
>>110281012miku feet
>>110278267thank you anon
>>110280849A high salary to prevent corruption via bribes.Pretty women to prevent corruption via brides.
>>110280989>reality is that compute demand is at all time high and growingTurkeys right now:>Reality is, food is all time high and growing, we get fatter and plumper by the day and we don't have to do a single thing, life is great, doomer turkeys are ridiculously retarded.
>>110281044>inelastic demand is the same as transient corporate manufactured holiday demandthis is why analogy should be banned in arguments
>>110281044>food analogyDo mutts really?
>>110281012>>110281019It is a python sandbox with a simple api and live reload that keeps state, I send the current view along with replies and llm can edit the script as we chat
>>110280854you don't become a tech billionare by being a chad. asian women is all they can pull
>>110280849From my experience if you are intelligent and want to have an actual connection with a woman your only options are asian or jewish women.If you try to talk about a technical topic or extrapolate some thesis into the future then it's not possible for most women to keep up with you.As someone that is really not attracted to asian women physically I've had to turn down a lot of them that were into me because of my mind. I've never experienced that with any other women so I can really understand the appeal.For example when the GPT-2 paper got released and I saw that there was no limit to scaling it up I immediately changed my career track from software engineer to ML engineer which paid off handsomely and is the sole reason why I'm an employed AI researcher now. Meanwhile my (white) girlfriend at the time was a software engineer at Google and just didn't seem to get my arguments. She was smart enough to get the data, interpret it but just couldn't connect the dots. I remember her saying something along the lines of "It makes sense rationally but I don't feel like this is actually going to scale up enough to generalize". I'm pretty sure a lot of anons ITT know exactly when you reach the wall of what your girlfriend can comprehend. There is just a barrier where no matter how much rational sense you make their "feelings" block acknowledging these new facts.She was against me going back to university so I had to break up with her at the time. Last I heard from her she became full Anti-AI and still talks about things like coding being an artform and things like that. This is not something asian women would do. The smart ones tend to engage very much with your thinking and you can see sparks in their eyes when you talk about these topics in detail and have a sense of awe of you recognizing trends and seeing things ahead. My current fiance is an effective altruist ashkenazi jew and it has been the dream life so far and she actually pushes me to be better.
>>110281122https://archive.is/QMo1kyou sound like someone who actually belongs to that cultare you sure you're talking about a wife and not a polycule
>>110281122>My current fiance is an effective altruist ashkenazi jewThis explains a lot.
>>110281122> My current fiance is an effective altruist ashkenazi jew and it has been the dream life so far and she actually pushes me to be better.you should begin with that
>>110281145his "wife" is actually 3 women and 1 man in a trench coat and he's too retarded to realize he's been duped
>>110281122Dariobot you said youd leave 4chan once for allBut okay i have something to ask you, currently im in a CS college, i have the option of dropping out and goin elsewhere next year. What do? Is there even a degree thatll be worth something in 5 years? (If i drop out thats 1 year, plus 4 years of bachelors)Is it even worth it to go to college? I already self studied ai slop and if i lock in and learn math and do finetuning and experiments read papers maybe id be heckin good, but probably not employableTldr what do you do as an 18yo???
>>110281168Quit immediately and get a job at McDonalds to buy the Anthropic IPO.
>>110281168become a nurse or waiter. based on general anti-ai sentiment of the most population, they clearly don't want to be served by robots.
Where is Gemma 5
>>110281154>>110281162This is why I catch up on threads in reverse order. Otherwise I might have read that entire post before realizing it was a kike post.Captcha: star of Rempham
>>110281195In kindergarten.
>>110280997I don't think Asian children are any better.
>>110281039Ironically high-security clearance jobs do neither>>110281122"effective altruist ashkenazi jew" so yet another athiest reform jew. Enjoy being divorce raped.
