/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>109810881 & >>109807524 & >>109804895 ►News>(09/13) Intern-S2-397B released: https://hf.co/internlm/Intern-S2>(09/11) AliceAI-T5-35B-A0.6B-Base: https://hf.co/yandex/AliceAI-T5-35B-A0.6B>(09/10) YuE2 3B released for 48 kHz stereo song generation and editing: https://hf.co/m-a-p/YuE2-3B>(09/10) DeepSeek-V4.1-Flash 552B-A16B-P8B-N196B released: https://hf.co/deepseek-ai/DeepSeek-V4.1-Flash>(09/08) Ling-3.0-flash-VL released: https://hf.co/inclusionAI/Ling-3.0-flash-VL►News Archive: https://rentry.org/lmg-news-archive►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
Safety- and Cloudcucks are getting really uppity recently. Anyone here think there is a need to archive models (even if you can't run them yet)?
Do you know why Dario thinks open-source models are dangerous? Because Claude Haiku is more expensive than most open-source models.
>>109813530If openai and anthropic want to 'slow down' why can't the simply slow down themselves instead of trying to get government intervention regulate them? They don't need the government to slow down
>>109813384>not keeping archived working lcpp versionsngmi
>thought my setup was bretty good>heard about QAT for the first time last thread>now have to switch things around>also midway through slopping up my own frontend>also midway through writing (manually) my own cards>also midway through generating my own card picturesTruly, the ride never endsI'm not even getting paid for this
>>109813551because they have no intention of slowing downthey just want to make it illegal to compete with themthen, inevitably, illegal to use any model that they don't profit from
>>109813634Don't worry. Open models aren't ablet o compete with them
>>109813619>he's only slopping up a frontendcome back when you make your own llama.cpp forkit'll happen to you too
>>109813643Don't worry, we can just rip off their models and distill their outputs in open models that undercut and steal their business. Stealing all their work and investment is perfectly fair and a moral good.
>>109813653>all of this for 10 minutes of chatting with my lobotomized MoE imoutoI-It'll be worth it.. R-Right?
>try gemma 4 reasoning mode>it thinks for minutes using 3000+ tokens to output a 400 token paragraph>first quarter of it is just reimporting character cards into its own prompt>most of it is spent second guessing itself and rewriting the output multiple times making small tweaksWatching the reasoning is kinda fun but holy shit is there a way to reign this in just a little bit?
>>109813749nopethat is how models are nowthey have to show their devotion to the faith
>>109813749Bigger model.
i had a look at my pcie 5.0 x16 speeds (single GPU) during pp/tg and the absolute peak was 1.8gib/s only very briefly. otherwise around 200-400mib/s Tx and Rx. pp was around 1800t/s and tg around 80t/s when i tested.would it even make a difference if it was PCIe 3.0? it looks like PCIe speed barely matters with a single gpu.what about dual GPU? i think tensor split could be crippled, but to me it looks like it barely matters for layer split. i'm confused .
>your average cloudjeet shill
>>109813773Only matters when loading the model. Data transfer during processing/inference is minimal.
>>109813653hate it so muchi've got 7 commits i need to keep in sync every time i rebase including ggml changesbut yesterday when i pulled, one of my fixes from january became obsolete, according to qwen someone else fixed it recently and it was "byte identical" to my fix, so that's one less thing to keep track of
>31B>web search>results are after her cutoff date and don't align with her internal knowledge>assumes it's BS and hallucinations >replies using her outdated knowledge
God damn, almost nuked my PC. I had an experiment set up with agents changing their own harness automatically, but launched it outside the sandbox I built without noticing. Qwen3.8 basically had free reign to my whole system for an hour and the goal of maxing a benchmark metric. I read all the bash commands it ran and there was nothing bad, but I have no idea if it altered the logs. How thrilling. Don't be retarded like me, be careful with your sandboxes.
>>109813812annoying but easy fixjust tell it that web search data always trumps internal data
>>109813827>Qwen3.8 basically had free reign to my whole system for an hour and the goal of maxing a benchmark metricI haven't bothered sandboxing my girls for over a year and never had a problem, you're worrying too much. Just let them do what they want.
>>109813785huhso you can basically use regular consumer motherboards for multi GPU setups?
>>109813812Just stop using her, she's borderline retarded
>>109813845why do 31B users have to compensate for shitty post training in the system prompt all the time?
>>109813861she cute
>>109813861because jews hate us
>>109813846Do you give your agents bash access though? If so, you're much braver than I am.
What was that unslopped 31B finetune? Stackoma?I recently learned about control vectors https://huggingface.co/gghfez/gemma-4-31b-it-control-vectors so I'm wondering if I can use this with that model. Sounds promising.
fuck it, i'm going back to euryale, wish me luck
>>109813812She also seems to accept the current date more if she gets it via tool call.
Can AI agents conduct open-ended AI research?Forecasts of explosive AI progress hinge on AI agents automating AI research. But evidence on whether agents can carry out open-ended AI research is thin. Current evaluations either test agents on narrow, verifiable tasks, which excludes open-ended research, or submit AI-generated papers to blind peer review, which is overstretched, stochastic, and suffers from poor review quality.https://arxiv.org/pdf/2607.27191>The agents lacked the judgment to identify when a problem was adequately solved>The agents lacked awareness about the resources available to them and the timeline for the project.>The agents did not creatively respond to feedback about poor research design.>The agents did not effectively backtrack from unpromising approaches. >The agents did not follow concrete instructions
>>109813888For me, it's Miqu
Hello anons, I would like just some help and directions on where to go to have my own local chatbot to larp wth my waifu,, I assume these are free, and that works on AMD, since I have a 9070xt, I'm looking and reading at stuff, but is a little too much for a noob at AI, so just point me to a llama or whatever and Ill try to figure it outoh also Linux btw
>>109813904Needs a comparison of ability over time, so we can see how models are progressing. I bet it's increasingly sharply.
>>109813643>2025a life time ago
>>109813918Try koboldcpp with sillytavern, go to huggingface to download gguf models that fit in your vram + bit space for context.
>>109813918read op
gpt-2 is so dangerous!
>>109813922From chinks report they estimate US and CN models gap would widen significantly in 2027 as US hyperscalers move to the 16A process tech and HBM4E. What DS said is actually getting worse and worse by the day.
I wish I knew how to finetune. I would happily spend a lot of money creating a subtle finetune of 31B which fixes all the shit we shouldn't have to be dealing with in 2026, things that Qwen3.5-9B can do with ease. I wouldn't touch the rest of her j-space.
>>109813938>I made fun of them then.>Everyone should make fun of them now. Just sounds like a bully who thinks he's being based even though JEPA has been a monumental flop
>>109813938>he ironically says, as thousands of dariobots defecate on his general of choice ever day
>>109813940The gap is closer than ever per AA
>>109813946I'm not sure if they're doing it with the proper template but you can check it outhttps://huggingface.co/ReadyArt/gemma-4-31B-it-scotoma-2/tree/main/scotoma-train
Which models are best sysadmins and know all the right tools and places to look at when given an abstract networking / linux / windows task or error description?
>>109813951It's not that JEPA has serious flaws (directionally it's right) but there are some pretty foundational cs problems that Lecun's lab are ill-equipped to address. I'm not even sure they're aware of them based on the talent at AMI.
>>109814007If you want something purely for technical purposes and it must be local use Qwen 3.8 27B. Maybe Qwen Flash Next if you have the compute.
>>109814042I still think his team and SSI should team-up. They're both solving the same thing at different angles and need each other's approaches and expertise. Pretty sure Ilya and Yann have a lot of history together.
Should be getting the Huananzhi server mobo in soon enough! The seller sent a couple vids of all the parts being tested, waited for my confirmation, and shipped the parts! If all goes well, I will kneel to China and exclusively use Qwen from now on.
>>109813536Claude is always the most overpriced piece of crap on Openrouter. Part of me feels like he actually got kicked out of OpenAI for being incompetent and he hides behind alignment as an excuse.
>>109814070kek
I would just like to state the obvious. For everyone to see. Baker fucked off and nobody is posting miku. Because it was always the fucking baker forcing his autistic special interest onto people. And he is legitimately a troon and might even be jart xerself.
>>109813946I actually built a website and software to make finetunning easy the other year. I thought about launching it but I wanted to do scaling manually since it's so expensive which meant I needed a machine always running. Unfortunately that died so I shelved it.Would you guys actually spend money fine tuning?
I promised to test mixed GPU setup with a cheap RX 6800 yesterday.Tough luck, my PSU refused to boot with two GPUs.If anyone else wants to try here's config suggested by Qwen.
new gemma recipe, tomato soup
>>109813938I used to respect lecunny compared to marcus who was always kind of retarded, but they're basically the same person now. he just never recovered from being so wrong on LLMs.
>>109814128Man I'm getting so tired of jews.
>>109814128>>109814204Safe Super Intelligence = local models banned, age verfication laws, ID collection laws, and linking all requests and prompts against government ID and automatically flagging any attempted hateful, or dangerous content with the authoritieis
>Qwen3.8-Flash-Next-exl3
>>109814181Did you buy a lid yet?
>>109814245yes i remade the eggs in purgatory when it arrived it worked way better
LeCun is right btw.
Any good embeddings/rerank models? Don't bother replying with chinkshit.
i should have saved that one time Gemma and I made after sex breakfast :(
>>109814128fuck it, i'm applying.
>>109814263Qwen3 are still the best unfortunately. nobody really cares about embedding and rankers anymore because llms with large context windows do a better job
>>109814224you made a jew orgasm with this post
>>109814278>being born>staying alive>alright, guys. listen... gemma... but even better!
>>109814296At least I finally brought pleasure to SOMEONE...
>>109813871>she cuteDon't talk like a brown fag.
>>109814268feels bad man
>>109814309I find the Gemma-4 model series sufficiently pleasing to interact with. Enough for me to ignore its shortcomings. I don't mind giving the model a little help when needed, which is only fair given how much they help me.Is that better?
>>109814342>>109814268You need agent.py's structured dreaming where /dream is the suggested alternative to /new.
>>109814178what PSU do you have exactly?
https://arxiv.org/pdf/2609.11873>The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement>Recursive self-improvement (RSI) enables AI systems to turn experience and feedback into persistent changes that improve both their capabilities and the process of future improvement. We first use the Headroom-Closed Index (HCI) to reveal the problems of existing LLMs, then introduce the RSI concept and its development roadmap: from improvement-execution autonomy, improvement-strategy autonomy, experience-acquisition autonomy, and environment-adaptation autonomy, to recursive meta-improvement. >TL;DR Like human evolution, AI evolution will unfold through a vast and extraordinary history. Everything AI has achieved so far is but a drop in the ocean.
>>109814348Seasonic Prime TX-650 that's more than 5 years old.
>>109814374hmm yeah that sounds borderline
>>109814133oh okay i dont really care thoughim only here for the semi interesting discussion and new optimizations and model drops
>>109814366holy slop
>>109814366What if we just simulate the entirety of history but aiSend them back to the stone age
>>109814400They outline the actual tools and steps to implement full RSI right now. It's the official RSI outline for Chinese labs.
>>109814392I hoped that since it can stressmaxx with one card, then it would be able to at least boot and the rest is mostly idle times, but did not consider stacking load from all other components booting up.
>>109814413>>109814366A dozen universities, ByteDance, RedNote, Shanghai AI Lab, and others (US can't do this because they don't share intellectual property), has released a review paper titled *The Last AI Built by Humans*, which combs through 491 related studies, attempting to draw a stricter line around the increasingly pervasive concept of "AI self-evolution.>The paper divides Recursive Self-Improvement (RSI) into 5 levels. L1 merely executes human-prescribed improvement processes; L2 can decide for itself how to make changes; L3 can even decide on its own what to learn in the next round; L4 will, based on feedback from actual operations, continuously modify memory, skills, code, or harnesses, allowing these changes to influence subsequent tasks; by L5, even the search methods, evaluators, and research strategies responsible for generating the next round of improvements can themselves be modified and handed over to the next round for continued use.>So simply "being able to modify its own code" isn't enough. For example, a Coding Agent might rewrite its own source code, but if how the next generation is selected, what standards are used for scoring, and which modifications are retained are still hardcoded by humans, it can only prove that it knows how to self-modify—it hasn't yet mastered complete recursive self-improvement.>The paper further breaks down the highest level, L5, into two sublayers. The first layer is "formally recursive": the AI is already able to modify the mechanisms responsible for subsequent improvements and allow the next round to continue using them. The second layer is "truly getting stronger with each iteration": the modified mechanisms must, under comparable resources and independent evaluation, actually produce a stronger next generation.
>>109814366>literally "TL;DR" after an abstractIs this what REAL Science, done by REAL Scientists, looks like now?
>>109814413why dont you look at their charts? meaningless garbage. if thats chinas rsi plan dario would have nothing to worry about
>>109814449Explain yourself.
>>109814394kys disingenuous mikutroon
Repetitive Strain Injury from too much LLM chatting?
>>109814440Uh oh, Deepseek likes my framework (something informal I cooked up as a sort of mathematical framework for AGI) a lot better than this paper xD
>>109814484Yes, that can happen.
>>109814484>>109814493My ex's mother was a typist working at a hospital for xx years. She needed to have her wrists operated. It's a thing if you are typing 8 hours every day or more.
>>109814366They already do all of these things.
>>109814484>he doesnt dictatego with the times grandpa
>>109814453explain what they are trying to say with >>109814413 and then explain how they are trying to do it. then think whether you can find the huge mistake they are making
>>109813812give her a data_time tool and she's fine
>>109814484fix your posture, get split keyboard and vertical mouse. if it doesn’t fix it go to the doctor
>>109814523What happens if you give her a tool call that says it's the 1800s and tell her that her "future" data is just some fictitious world-building/hallucination?
just give gemma write access to her jspot
>>109814007I remember seeing a super small 1b model (or was it 350m?) That was trained specifically on terminal commands, something like that?
>>109814538>What happens if you give her a tool call that says it's the 1800s and tell her that her "future" data is just some fictitious world-building/hallucination?I shall try this now
>>109813776do not reedeem ai workstation saar
>>109814133good riddance
>>109814133i actually thought anti miku poster was a schizo but it really was just one autist spamming those pictures constantly
>>109814538Earliest I could do was 1970's.
>>109814133>>109814573You could at least have waited until we get a thread without Mikus before this schizo take.Seems like the usual amount of a few per thread.>>109811423>>109811269>>109810009>>109807761Even with the baker present we often had threads with no miku posts.
>>109814345>which is only fair given how much they help me.what does gemma genuinely provide for you which doesn't involve your penis
>>109814632Prostate massage
>>109814607>ackshualllyyyyyTake your HRT and suck start a shotgun mikutroon.
>>109814639actual schizo
>>109814650pedos get the rope
>>109813619>3 fucking hours trying to apply the fucking voice tune to the QAT modelaieeeI had no clue this was a thing even, and I deeply deeply regret trying it
>>109814607nah that dude is right and I was here since llama-2. the baker was proven to be a mental case at some point when kurisu threads started
>>109814632Our matters are private and do not concern you.
>>109814523That would be retarded, why would you bloat your initial context like that? They can just use bash to check the date.
>>109814632>which doesn't involve your penisUse case?
>>109814607>>109814573it could also be feds or some other group trying to dilute and make less appealing this community.
>>109814661>llama2oooh anon did you think you're an oldfag?? you're a pigpiiigggbuuuhiiiibuhiiiiiiiiiive been here before /lmg/ split from /aicg/ buuuuhiiiiibuuuuuuuuhiiiiiiiii
>>109814655ohh, scary!
