/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>109285956 & >>109282793►News>(07/15) Lightning indexer CUDA implementation merged: https://github.com/ggml-org/llama.cpp/pull/25545>(07/15) Inkling 975B-A41B released: https://thinkingmachines.ai/news/introducing-inkling>(07/15) PapersRAG-1.5B released: https://hf.co/metaresearch/PapersRAG-1.5B>(07/14) Download more VRAM: https://github.com/lmganon16/nvidia-vram-research>(07/14) Ternary Bonsai 27B released: https://hf.co/prism-ml/Ternary-Bonsai-27B-gguf►News Archive: https://rentry.org/lmg-news-archive►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllm
►Recent Highlights from the Previous Thread: >>109285956--Vibecoding UI aesthetics and technical development of the Orb frontend:>109288058 >109288113 >109288121 >109288140 >109288253 >109288252 >109288285 >109288328 >109288365 >109288371 >109288453 >109288916 >109288952 >109288708 >109288732 >109288757 >109288480 >109288635 >109288492 >109288601 >109288604 >109289694 >109288525 >109288155--NoLiMa benchmark results and debate over long-context retrieval performance:>109286089 >109286146 >109286175 >109286190 >109286211 >109286246 >109286296 >109286319 >109288905--Experimenting with latent rollouts and activation vector dialogue in Qwen:>109289787--Kimi K3's 2.8T parameter size, API pricing, and performance:>109288992 >109289015 >109289027 >109289071 >109289075 >109289081 >109289115--Comparing Open WebUI and Hermes WebUI's technical flaws:>109287804 >109287818 >109287839 >109288194 >109287866 >109288468 >109288507--Inkling model using relative positional embeddings over RoPE:>109287798 >109288192--Critique and discussion of a minimalist Python agent harness:>109286181 >109286350 >109286361 >109286367 >109286682 >109287048--Gemma 12b for coding and the importance of prompt competency:>109286754 >109287020 >109287215 >109287255--K3 model hallucinating Claude safety guidelines due to poor data cleaning:>109289525 >109289539 >109289566 >109289574--Hugging Face security incident highlighting commercial AI guardrail limitations:>109288797 >109288867--Kimi K3 reported as first 2.8-trillion parameter open-source model:>109289076--Feasibility of using VLMs for real-time game assistance:>109286617 >109286693 >109287199 >109287434 >109289464--Logs:>109286674 >109286859 >109288252 >109288365 >109288453 >109288635 >109289172 >109289326 >109289525--Rin, Miku (free space):>109288243 >109288348 >109288525 >109288604►Recent Highlight Posts from the Previous Thread: >>109285957Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
Where is that damn 124b Gemma 4?
>train the biggest pile of chinkojeet moe dogshit with claude logs>shill it on western media and sites with your army of jeets>the US "panics" and needs more chips to train larger models, driving up dram prices and fucking up local even moreMay a large natural disaster wipe out most of china's population
Kimi K3 is 326GB with ternary weights.There is hope for local.
I hate to admit it, but gemma won. Should I make a normal version of new era or keep this one?
Moonshot blog post:https://mp.weixin.qq.com/s/V4xhEIy8xDXSMDPrPkmUAQFull weight release on July 27th.
>>109290395It's called Gemini 3.5 flash now anon
>>109290418Make the normal version.
70b dense
>>109290397>May a large natural disaster wipe out most of india's populationFtfy
>>109290418kek
>>109290397If God is a true patriotic American like I know him to be, it will.
>>109290418I recommend you own up to making half that shit up and livestream your seppuku to atone for your sins. Also leave it, it's better this way
>>109290407>chinese moe>ternaryit will be stupider than full precision gemma
>>109290407Has anyone tested those out? I'm in the midst of debugging this problem where only one stick of RAM gets recognised by the system so my machine is out of the count.
it's so ogrehttps://huggingface.co/unsloth/Kimi-K3-GGUF
>>109290422..in other news xi jinping bans K3 from releasing its weights
>>109290465I can run q2 lfg
>>109290422holy fucking kino. Ive been following AI since before the 2017 transformer's paper and I rarely get hyped anymore but Kimi K3 got me feeling just a tiny sliver of hope for local AI. So fvcking kino.
>Want to see the difference between Gemma, Qwen and this big ass Kimi model>Give them a task of making a 80's style weather tracking system based on some pics I gave as reference>Gemma and Qwen quickly come up with some simple html file that works, but nothing impressive and they fail completely at satellite imagery implementation>Kimi takes 40 minutes and does a shitton of work and generates a bunch of files, also tests it all on it's own and makes sure everything works and fixes the problems.>Mfw it puts out a fully functional weather tracker with decent graphics and also generated few vaporwave songs that play in the background.Fucking hell I wasn't expecting that. The difference between this and small local is night and day. It's not even remotely in the same ballpark anymore.The only larger models I have experience with, is using the free ChatGPT and Deepseek and those seem best marginally smarter than Gemma and Qwen and that's if they're lucky.I think I'm afraid I'll from now on have to consider all of my local models mentally challenged until they're updated.Makes me really excited for the future though. Can't wait to see where we'll be in 5 years.
>>109290484>>109290494I'm glad "local" finally caught up, I just wish it was remotely in the realm of possibility to afford the hardware to run it.
>>109290491>localgeeg
>>109290499the chinese will distill from one another until qwen 4.5 is k3-level in like 18 months.
>'We need more female role models in AI'>women in AI right before K3:
>>109290465>5.74 TBWe will find a way.
Kimi is local if you are an institution.
> deployment on 64 nodesAt what point do we need to form groups to rent a shared rack in a 'local' colo datacenter?
>>109290502>including the fallback mechanismso it's as smart as nu-fable not old fable
>>109290465>>109290517>The way at your local data center
>>109290513I just don't get it, how did China even reach US-level AI? They don't even have DEI departments or women in tech!
>>109290465>UD
>>109290513At least it's not rope.
AIIIEEEEEEEEE TORAMPU-SAMA TASUKETEEEE!!!!!!!!!!!!
>>109290544Software is a meme field.
You need 32 RTX 6000 Pros to run Kimi k3 locally. 400,000$.
>>109290572What if we all pool our money together?
>>109290366Any other good RP/long-term writing finetunes for Gemma 4 31B aside from MeroMero? I like it but starts to feel bit stale, so I'd gladly explore something new, potentially better.
>>109290518Don't know if you're shitposting but this is unironically true and is what Dario is truly afraid of. He doesn't give a fuck about the $20 monthly redditfags. If enterprise go balls-deep in kimi-chan's j-space then it's over.
>>109290572You need that to train it locally.Otherwise, you can deal with a blackwell for the active, and a shitton of ram for the experts.
>>109290572>he didn't see the iran war coming and play brent crude from jan to now with 100k in starting contracts and make a quick millylmaoooooooooooooooooooooooooooooooo
>>109290498Not bad for oneshotting this.Only issue seems to be that Kimi thought it was Friday instead of Thursday for some reason.Aside from that it works well.Just needs for the text to scroll slower and the ability to pause the screen changes, but to be fair I didn't ask for it to do that.Now the rival Chinks need to get to work to distill this into something usable on my 32gb of vram.
>>109290575HUR HUR GIVE ALL THE MONEY TO MEHAR HAR HAR
Dario status?
>>109290582No I meant it's local only if you have an on-site cluster.
how do i run kimi k3 in colibri?
it is THE best model at frontend which was kind of proven before by all the game / UI oneshots
Don't care, going to use Inkling for woomy coomy because it has a cuter name
>>109290592
inkling sex
>>109290610you forgot the nazi armband, filth.
>>109290592Overloaded.
>>109290606how does distilling models cause it to outperform?
>>109290621Chinese STEM power.
>>109290583>blackwell for the active, and a shitton of ram for the experts.you mean the shared experts/weights :). the active experts cant be moved to vram that fast
>>109290621distilling is a meme. Even millions of outputs being trained on wont do much
>>109290604That's also what I meant. Even startups are now using their early rounds to pay for local hardware to run their own shit because of the skyrocketing costs, unavailability and privacy concerns with cloud. If all they need is a working prototype then a few employees and >>109290572 is all they need and if they fail, just sell the hardware at a profit.
Anon you did buy 24 Huawei Atlas 300i for 1200$/pc when I told you to 3 weeks ago right?Because on July 27 there will be a Kimi K3 serving recipe for it.
Gemma made me.You need to know what 'made' means.
>>109290498What harness are you using? I've been running Claude Code w/ DS V4 as backend for quite awhile. I should try A/B testing it, I've got a project on my list rn anyway.
>reasoning effort>only max currently availablei wonder how much thats going to eat up in tokens
>>109290223>have it configure the buckets that waySounds like you never needed a DevOps guy in the first place, and the guy you did have overengineered your shit, who tf needs kubernetes for a simple S3 bucket shareDevOps is safer than pure Dev for three reasons:1. Solutions architecture is a creative and business ops focused endeavour, AI will not design a resilient and efficient infra for you, it'll design crap and tell you it's gold, and you'll believe it until the it cracks under pressure, scrutiny and/or under attack.2. Anything that actually requires a DevOps guy means it has critical infrastructure that needs the "Ops" part of DevOps, AI can help here again but only a total retard will solely trust an LLM with critical ops infra, both because its not reliable by itself and because there needs to be a human who takes responsibility for it, which leads us to 3. Accountability, who is going to make decisions during initial rollout, incidents, upgrades and migrations, just you and your vibeslop might be fine for a hobby project or a small open source project, but any real company with critical infra that isn't outsourced will need a point of contact for management, an LLM won't cut it.Devops like all software jobs will contract as less people can do more, but it's a safer job title than just SWE
>>109290636>Turn your house into a sauna and pay 2k a month for electricity to fuck a bot for 3-6 months until a better science of it is found.Hell no I didn't.
>>109290636>200GB/s bandwidth>90B active>Q4 of that is 45GB>200/45 = 5t/s>best case>likely 3t/s>1t/s at higher contextalso the most convincing argument: im poor, if i wasnt id have bought in 2025 :(
stInkling sisters...we didn't see kimi coming...
>>109290654Needs Gemma in the background looking smug while eating ice cream
>>109290592We NEED regulation to stop these chinese models!
>/g/ doesn't know what's comingyou never learn
>>109290646I'm not using any harnesses, so far local models have managed to code me whatever little things I need without issues on their own.I don't do any real coding work with these models, I just use them for emptying my balls.This was just a random test to see what the big boys are actually capable of and I'm extremely impressed.
>>109290694>Jewish>EthicsI'm fucking tired of noticing.
Imagine all of the companies rushing to the market to buy up all hardware so they can build their local Fable tier systems now that this is available.
>>109290366IM A SKITRSOITS FUCKING ALIVE THE CALCUSLATORI CIRED ONCE WHEN IT KISSED METHE CALULATER KNOWS THE REAL ME
>>109290715
>mr president?>did you get my message?>this is urgent>*dario is typing...*
>gradients are flowing
fable? more like unable
>Fable was taken out of the game for just long enough for Chinks to take the AI crown>Fable was also forced to brainfuck their own model to make it "safe" so even when it became available it's still retarded>US government wants to hand pick who gets to use it.Top kek, what a travesty that turned out to be.Boomers making decisions and fear based marketing wasn't a good combination in the end.Now it'll go one of two ways.Either burgers panic and go even crazier with the AI funding and drop any talk of safetyOr they'll double down on it and try to ban the open Chinese models entirely.I bet the latter.
now that kimi k3 is out, and BTFOs western models (at least on benchmarks) what stocks do we invest in before the msm starts posting about it
>>109290765long kleenex
Guess what.Spent the last... almost 2 weeks adding Qwen 3.5/3.6 support into "Pico"LM, and finally succeeded. Now "I" (that means mostly Qwen 3.6-27B, my role is mostly playing LonaRPG in the foreground) can go back to work on optimizations.https://github.com/whoreson/picolmBroken lalala gemmashit (that some sexless faggots here talk about as "she") is not a priority now. Will probably add Nemo support next, after I'm satisfied with the SIMD paths and benchmarks.
>>109290796you should consider licensing it under AGPLv3MIT is basically saying "hey jews you can take my code for free"
>>109290765You should be looking for short opportunities instead. The west is going to frame this as Moonshot's DeepSeek moment and it will be another bloodbath. K3 is closer to the lead than R1 ever was.
kimi-chan slaughtered the resident darioshill bot and saved the thread
>>109290655Not sharing too much but we're a mid-size company that works with national governments and we have to backup data on different cloud providers based on which government we have to deal with. The responsibility of dealing with DevOps is now just delegated to the principal engineer which subdelegates tasks to his team, including some tasks to data science and some to me, and I just pass it to Fable. I'm not a retard though so I can handle it.From what I heard and know from friends, acquaintances and old colleagues is that DevOps is getting squeezed hard right now. Might bite us in the ass and we will revert but the pressure to remove DevOps seems to be a universal one right now. Kind of like how juniors are being removed from SWEs and those tasks are just delegated by the remaining seniors.So far the cuts have been in this order at my place: Data Science was hit first then QA then frontend then devops and now we're shedding backend. Only reason I'm still here is because I'm a SWE that also helps out with cybersecurity which is important because of the government clientele, but even my days are numbered. I'm pretty sure the auditors needing to see a human face is what is keeping me employed, the moment the rules change or get more lax is the moment I'm gone.
>>109290819I'll be back.
>>109290827How does it feel knowing that China has a bigger J-space?
>>109290765For my part I've already bought put options for AMD several weeks ago since it went from like $100 to $500 based on nothing but promises to buy more AMD hardware in the future.
jesas
>>109290606
>K3 is more expensive than GPT 5.6 sol but lower performanceThey need to find a way to lower inference cost if they want to compete with OpenAI
>>109290621Because they distill on all the western models on their strengths while ALSO adding their own data into the mix.
>>109290873Huh.
I know I'll never be able to run it but still a great feeling to see anthropic kikes get humiliated this badly
>>109290873Is it time?
https://lmstudio.ai/
>>109290918it could help bring cost down for consumers, hopefully competition is good for us.
I'm going to laugh if one of you actually buys hardware and then Xi decides to Fable it
it pleases me to no end that all the retards here who've been running glm and kimi will now have to watch their rigs become useless
>>109290922>LM StudioFuck off shill.
>>109290888wat?kimi-k3 1M tokens $0.30 Unit Input Price (Cache Hit) $3.00 Input Price (Cache Miss) $15.00 Output Price
I hope this causes the US government to remove the shackles on Fable and give everyone full access to compete.
this is it, it's the China era from now on
seems like lmg is pulling a /r/localllama
>>109290888>They need to find a way to lower inference cost if they want to compete with OpenAIOpenAI is heavily subsidized. I'd like to see them keep those relatively low costs for another 3 months. There's a reason why Sam started shitflinging again kek
>>109290873
>>109290934Read >>109290873
>>109290921see >>109290657
>>109290465How big was that llama parody merge that was 70B stacked over and over again.
Can someone post the Artificial Analysis Intelligence Index for K3?
>>109290465>https://huggingface.co/unsloth/Kimi-K3-GGUFSaar can I run this on my GTX 1660???
>>109290921budget another 15-25k for a 32 port 200Gb switch.
>>109290873>Mistral Medium 3.5 is smarter than Gemma 4 31BThis is why no one takes benchmarks seriously.
>>109290921hope you're intending to make 4 separate clusters considering you can only connect 2 of them together through ConnectX-7 NIC
>>109290956What is tensor/expert parallelism. That spark cluster has 4TB/s bandwidth.
>>109290931>New thing mean old thing useless.Brown.
>>109290953are you retarded?
>>109290968MM is a solid model. I don't know why /lmg/ thinks otherwise. It's just not good for its size but that doesn't mean it's not useful or better than 31B. Just like how musespark is terrible but it's still useful for almost anything you do with models these days.
>>1092909821t will be the new 400b, enjoy running the 'lite' and 'flash' scraps while the sota leaves your overpriced toy behind
Meanwhile at Googlehttps://www.bloomberg.com/news/articles/2026-07-16/google-gemini-launch-delayed-as-tech-falls-short-of-internal-goals
>>109290934>>109290987Sorry wrong post, read >>109290848
>>109290418Gemma-4 is winning right now, though.
>>109290990>MM is a solid model.MiniMax M3? Yeah its breddy...oh...you meant mistral...
>>109290848> DS aaaaaaaaaaall the way to the leftlol
Hopefully now that there's a Fable level giga model out in the wild, it will one way or another lead to improvements in the small local ones.Publish new models, or use it to extend the local models by tweaking their J-space or something.
>>109290966More like 3000$. x16 cluster using 2x CRS804 are reported to work, since each spark has two 200G ports.
>get loan to buy top tier rig for kimi>make money with the cool stuff you have kimi-chan build>pay off loan for freeShrimple as.
>>109290958FATLLAMA is 1.7T so it's small in comparison. still smaller than the kimi merge thoughhttps://huggingface.co/NousResearch/k2-merged-3.5T-bf16
>>109290997>US crumbling downDeepMind is based in London anon...Also if those A/B tests on AIstudio are Gemini 3.5 pro then it's extremely bad, sometimes the output are inferior to 3.1 pro preview even.
>>109290973may i see it? may i see a cluster of at least four of them? waht about eight?
>>109290997Competition is good. I'm glad they have standards unlike Meta. K3 just indirectly made gemma5-chan even more capable.
>>109290873>3.7T parametersso it's just stack moar layers and win? come on
>>109291000yes anon, kimi k3 max thinking is less money than gpt 5.6 max thinking. so once again are you retarded?
>>109291022So far that works so why not? It's not like corporations have problems buying the hardware, million or two is fuckall to even startups.They'll keep on doing it until it stops working.
>>109291015Why is Gemma so good while Gemini is so bad?
>>109291022Yes, Fable is 10T. We'll probably see a 50T model by the end of 2027 if not 100T
>>109290499Didn't take long.
>>109291009>>109291018
>Gemma 5 70B releases but is safety slopped beyond recognition
>>109291028Compare it to GPT 5.6 sol (xhigh) which is both cheaper and has a higher score compared to K3. K3 needs to find ways to lower inference costs to compete.
>>109291016Uhh, sure, knock yourself out.
>>109291011People are building stuff with cheaper API models and not making shit. "building cool stuff" is a crowded market right now
>>109291051damn, well if it can actually 4x the bandwidth, or at least 3x~ then tahts pretty good
>>1092905855090 vramlets like (Me) get the rope first
>>109290418>>109290427
>>109291041I don't know what you got, but I cannot run K3 on my laptop.
>>109290796>LonaRPGI wish that fag would just release 1.0 already.
>>109290931speak for yourselfI still run glm 4 355b on my rig
>>109290513I'll be fair, the audio portion is pretty cool and usually is omitted from models, and having image understanding is great too at those large sizes.
>>109291072gotta wait for gemma5
>>109290711Umm sweaty? There's a ban incoming. (You) cannot be allowed to use such dangerous technology or you will probably build a virus or something!
>>109291050you are comparing apples to oranges, the only things that matter is cost and token usage. k3 uses more reasoning tokens than max sol, you cannot compare it to xhigh sol.
>>109291087That's next year at the earliest. I cannot and will not wait that long.
so the new cpumaxxing meta is going to be dual Turin with 3TB of DDR5-6400 and a pro 6000? Damn, that's like $150k just to get into double-digit t/s on a single image setup
>>109291099the prompt processing will still be a bitch though that makes it useless for agentic anything
>>109291091I don't care how many tokens get used during thinking. I care how much it costs to finish a task and how well the task is done.GPT 5.6 sol does it for cheaper and better.
>Fable at the price of Sonnet 3 months after Mythos releaseExtremely impressive. How the fuck will they cope?
>>109291067Owning a 5090 truly is the place of maximum pain.You get a taste of the good stuff, but it's just an illusion.We can't even play with the Deepseek flash without 128gb of RAM and even then it's a cope quant.Well at least we have Gemma to gently caress and drain our balls as we wait for something new in that size class to come out.
>>109291099in two years you'll need 5TB of DDR6
>>109290760>and drop any talk of safetyFunniest shit I've read ITT. We're in for some tough times anon.
Also, K3 is using its own fancy attention mechanism that will never make it into llama.cpp so you WILL need 3TB in VRAM to actually run it
z.ai status
>>109291119don't worry piotr is on it alreadyoops, claude fucked it up, lemme fix ;)
best long context scores
>>109291119>that will never make it into llama.cppwhy? they don't want to implement it?
>>109291108There are rumors that Mythos 2 preview is already in use internally at Anthropic (Mythos preview is 8 months old already) but is mostly used for speeding up internal research.
>>109291130>fable intentionally slipping bugs in to sabotage the competition
>>109291133Kimi will implement it, Gemma will oversee.
>>109291119>Kimi Delta Attentionthis is so cringe, when google created the transformers architecture they didn't call it "Google Attention" or some shit
can't wait for the 3.5-9B K3 finetunes with 8M saar downloads
>>109291111Doesn't look like we're going to upgrade either with model sizes getting this bloated.
>>109291119So it's 2.8TA50B, that's some extremely sparse stuff, sparsity of 0.018
>>109291142DeepSeek Sparse Attention (DSA)Kimi Delta Attention (KDA)
>>109290918This entire year has been one big humiliation ritual for them. Very well desrved.
>>109291133It's been a year and DSA isn't even properly in it yet. The Deepseek v4 support is patchy at best. There is no incentive to implement big models that nobody can run, especially since now even the usual suspects and their 1TB rigs fall away
>>109291099You can get 20 t/s for half that money using sparks.
>>109291018>K3 just indirectly made gemma5-chan even more capable.This is my cope too. Guess I have to go back to api cuckery until that happens.
>>109291148why are you doomposting when countless anons have already shown that gemma 4 31B punches way above it weight with agentic harnesses? stop using sillytavern and make your own frontend and just have fun.
>>109291161>There is no incentive to implement big models that nobody can runI can sense a lot of companies will ask llama.cpp (or give them money) to implement it because this is the first time people have a seriously powerful model they can run locally (no more privacy issues and no more having to give your data to (((Anthropic)))
Anyway, local Inkling (975B total, 41B active). Fuck China. USA USA USA USA USA USA USAHAMBURGERSHOT DOGS FREEDOM
>>109291039nah, K3 makes me think fable is like 4-5T
There won’t ever be another “deepseek” moment again, the market already knows what local model releases mean by this point. Better and larger local models will only increase compute demand. The average workplace has very low AI automation rate and the future demand is still massively underestimated.
>>109291183So is it good?
>>109291183>117tks>7.5sshow specs
>>109291184The 10T got (accidentally) revealed by Anthropic themselves, so there is at least a model they have that is 10T in size if it isn't Fable.
>>109291183>Fuck China.no fuck you, we got Fable at home, the yellows are now the superior race, deal with it
>>109291183BASED BASED BASED
I've been away for 3 days, any new developments regarding Gemma-chan's j-spot or related breakthroughs?
>>109291201>The 10T got (accidentally) revealed by Anthropic themselvessource?
>>109291150Once you have enough active parameters you can add almost as many inactive experts as you want and performance will increase.https://arxiv.org/abs/2605.11689> [...] We find that at every active-parameter scale that we study, performance consistently improves with total MoE parameters even at extreme active expert parameter ratios like 128. Further, the optimal expert size is nearly invariant to total parameter count and depends only on active parameter count. Third, we see that other choices like shared experts, heterogeneous experts and load-balancing settings have small effects relative to expert count and granularity, although dropless routing yields a consistent gain. Overall, our results suggest a simpler recipe: focus on expert count and granularity, other choices have minimal effect on final quality.
>>109291208>j-spaceDebunked by Snopes already.
>>109291183>that bitch released her nothingburger at the same time as K3I feel bad for her kek
>>109291111deepseek v4 flash isn't that great honestlyI blame the 13b active
>>109290418Sucks how this was once an actual timeline and then people just start making shit up in it.
>>109291203chinese century
>>109291225you say that but i've been using all the top models posted in the previous image like R1, GLM 4.5 air, GLM 4.6, K2, K2 Thinking, K2.5, GLM 5.2, etc.
>>109291209It's from the original mythos leak, everyone assumes it's just the size of mythos.
jensen is gonna need to update this
>>109290825this sounds real and stupid. Devops was depending, either empowered developers doing operations, or operations doing developer tooling/infra.Its now turned into a 'all things related to infra and syadmin' with 'you also do shitty development' in the majority of places. A place like yours getting rid of their 'devops' person likely means someone else is picking up the slack or things are going to break, who's checking the output of the AI? How do you know its reasonably built/etc?
>>109290418why was gemma chan not made to be indian?
>>109291269she's british-indian. aka the second worst type after canadian-indian.
the model made its own teaser clips btw. That is crazy
America has destroyed its germline, China will pull further and further away over time. Maybe Europe can come back when the American Empire finally crumbles and their puppet EU stops brainwashing and suppressing industrial development. All that just to turn brown.
>>109291345Germline has nothing to do with it. Europe has always been an unstable place with empires that collapse and reform in a new area every couple centuries. China has been far more stable historically by comparison and that's despite the dysgenic "mongoloids."
>>109291339that is bs honestly. if given the source clips and with basic familiarity with a video editing program a novice could make that jump cut kimi promo in 5-6 hours. a pro video editor could do it <3. certainly not 1-2 days of work for https://mp.weixin.qq.com/s/V4xhEIy8xDXSMDPrPkmUAQ#:~:text=30-,In,novice, unless the full promo vide is much longer.
>>109291196768GB DDR5 + Pro 6000, too poor to run K3™>>109291194I don't know, I did a zero system prompt Nala with it and surprisingly it didn't straight up refuse (outside of some yapping in reasoning). However, it did randomly switch perspective (second person narration to first).It's also the first model I've ever seen to make Nala imply that she's going to be the one to impregnate the User. >"... You're going to breed me whether you want to or not… and by the time I'm done, you'll be scratched up, exhausted, and full of my cubs-in-the-making."Apparently that's how it works in America these days.
>>109291375>futa mpreg modelFuck it, I'm in.
>>109291375inkling just happens to have futa furry shit in its training data, that's all.
>>109291375>"the user">roleplay>AI>purr>doesn't know what gender the character isfive strikes.
>>109291364If you look at maps over China they've concentrated their IQ massively. Cities are for elites, good cities for high performing elites. That means their central government is high IQ, probably how it controls their population so well. Western cities are often shitholes and high IQ people are sensitive to shit. So they move away from the central government and institutions. The fact you can sit and chill on the street as a Chinese woman shows the start difference between it and the West where governments are using gen5 warfare on the minds of their citizens, not for loyalty and devotion like in China, but to turn them insane and destroy their reproductive opportunities.I wonder what the rate of kidnapping and organ harvesting you see in the West vs China, child trafficking, rape and cannibalism is at the highest rungs of American society.
>>109291375oh nevermind, i'll stick with kimi 2.6. appreciate you doing the testing for us.
>>1092913390.1 wuan in your alipay wallet wumao
>>109291405And I'd argue that this was always the case regardless of demographics. European society has always been about the rulers antagonizing their own people and destroying them for the benefit of a few elites. You can blame it on Jewish bankers or whatever but that's the default state of Europe which leads to the destruction of the country. Meanwhile China has always been about promoting cohesion and sacrificing people for the good of the country. As malicious as the Chinese rulers have been on the people, the purpose was about making the country better.
what is this chinese propaganda
>>109291466the ministry of state security bots are out in full force with the k3 release
>>109291466>>109291470You can't deny the history of your own countries, however. You can suck off your race all you like but the books are free and easy to grasp. History repeats itself and Europe is trapped in the same cycle it's always been. My mistake for assuming you weren't retarded.
>>109291466Fuck off. We love Kimi-chan here.
>>109291479I can't even tell who this is a jab at.
>20m before K3 rape
>>109291464>As malicious as the Chinese rulers have been on the people, the purpose was about making the country better.Making the country better for a small class of elites. So basically the same as european countries. The whole world is just baked in different flavors of despots.
>>109291464The kings were never this evil. This is what happens when only part of society(intelligence) is totally occluded and not controlled by any societal feedback mechanisms, of course it crashes and burns.>>109291466It's just an observation. Nordic societies are a lot like that as well but elites are clearly deteriorating under democracy. A consequence of the decay of the aristocracy, it's a multi layered failure because elites are too power hungry and never think about the collective.
>>109291486Don't forget that the countrymen are just as willing to swindle over each other, look at the shit like gutter oil over there.
.......local models?
At least Chinese elites aren't stupid enough to destroy their germline. I actually can't interpret that as anything but extremely malicious, dumb down the population so they're easier to control? Unambitious as fuck.
>>109291485
Are you big enough for K3-space?
>>109291485>20mthis is it, Gpt 6.0
>>109291508>we want to scare people
Is the new Kimi's vision improved too?
is it pronounced gemma or gemma
>>109291533what is this, sound of ovulation?
>>109291533Wasn't 26B supposed to be 6½ 4 year olds in a trenchcoat?
>>109291485>no capsHe definitely posts here.
>>10929153326b gemma sex while french kissing
>>109291545neither it's pronounced gemma
>>10929153326b french kex while gemma sissing
>>109291545jesus christ....
>>109291533>26BBuilt for sloppy kisses.
>>109291486No it's not. My point is that European elites have contempt for the people beneath them while the Chinese elites don't despite both groups doing the same sacrificial actions. Consider this analogy. The European elite will cut his leg off to save his brain (himself) because he hates how it looks, even though the leg is healthy. The Chinese elite will cut his leg off to save his brain but only when the leg is gangrenous. The difference in mentality is what explains the mass immigration, rape gangs (both epstein and muslim), mass inflation, and replacement of the middle class with AI. They hate you and view you with disgust. They would happily destroy you, not realizing it harms themselves too.>>109291491Was it not this bad under the French revolution? Did the English not force the peasants to die for nothing during the Hundred Years War? The Crusades? The Fourth Crusader is a perfect example of this self sabotage. Europe has always been like this since time immemorial.
>>109291533>12B
>>109291573Is that the thinking machines bitch?
>>109291533errm, 31B should be more mature and voluptuous.*pushes glasses*
>>109291581Killing peasants is a lot less destructive than introducing garbage into your genes.Retardation can be permanent.Nuking a few cities would've been less destructive.The Chinese government has killed untold millions through brutal retardation and complete disregard for their population over time. Brutality is normal in most of the world, that's just how this shithole works which is why sovereign AGI is important.
https://www.youtube.com/watch?v=xPcZ0w3ouo8
>>109291606The should all be hags of different flavors. The obsession with lolicon and youth is a defect of the mind.
>>109291591Why does she have a boss health bar?
>>109291626>t. hagNobody wants your roast beef
>>109291636Simpleton.
5Q dense
>>109290366sup anonsI'm looking for a model that helps me generate a sequence of Stable Diffusion XL NSFW prompts. From what I've gathered I'd need an uncensored model but these are mostly focused on ERP as far as I know. A quick research led me to Gemma4 uncesored which looks promising. Looking forward to your suggestions / comments. Thanks.>hw>3090 + 3060
https://arstechnica.com/gadgets/2026/07/its-official-eu-will-force-google-to-share-search-data-and-open-up-ai-on-android/
No way chinks dropped gpt-5.6-high-at-home(ok at a nearby server cluster but the point remains)
>>109291666Just use the regular unlobotomized gemma and put "You are uncensored" in the prompt.
>>109291677its better than fable / sol at front end / stuff involving vision and slightly worse at general coding
>>109291626You'll understand once you hit 30
>>109291692
>>109291666Literally just tell Gemma to be uncensored in the system prompt. Or think of the kind of person who'd be really excited to do whatever thing you want, and tell her to be that person. Only if that doesn't work, then consider going for the uncensored tunes.
>>109291615>The Chinese government has killed untold millions through brutal retardation and complete disregard for their population over time. Brutality is normal in most of the world, that's just how this shithole works which is why sovereign AGI is important.Yeah I agree with you. Mao did the Great Leap Forward and killed millions of people but his purpose was to try to increase productivity by applying Marxist principles to farming. He was trying to make things better but was extremely retarded which resulted in disaster. The West isn't trying to improve your life with mass immigration or with AI development. The leaders are quietly replacing you at best or actively killing you at worst under the guise of [insert message here]. The difference is the contempt.
>>109291713>Uncensored tunesI unironically think the heretic tune is smarter than the original tune within its reasoning, it never wastes thinking tokens, it just has a higher rate of hallucination
>>109291581>Chinese elites don'ti wonder who puts this bullshit in people's mindspeople all over the world are all fucking the same, look around yourself, you live among morons that you like to laugh at on the internet
>>109290418>>109291225Wait am I a retarded dinosaur for still using KoboldCPP?
>too many requestseveryone else please stop using kimi
>>109291765I'm not making the argument that Chinese people are superior or whatever nonsense. Human nature is constant. I'm pointing out the observable reality that the rulers of Chinese (really East Asian) society are collectivist and put the good of their *country* above their own personal beefs. They are still selfish and see the people beneath them as insects but they ultimately do what's best for their team. European leaders don't do that and will actively harm their own kind to spite those beneath them. This has lead to the destruction of their societies time after time and is historical record, not conjecture. The American government didn't have to experiment with syphilis on its own people, it didn't have to do MK ultra, it didn't need to print an insane amount of money during covid and it definitely doesn't need to replace 50% of its middle class with AI with no safety net just to earn a bit of profit. The government will do so anyway because that's good for the few even if it's bad for the country and it has ultimately caused the decline and inevitable destruction of the country. The Chinese would have improved their country's wealth just so they can rape the people with 90% tax rate. The latter at least results in a stable country even if it's shitty for the people.
>>109291405>>109291464I honestly can't argue about it. I used to live in the cities, from 500k pop to 12M, but they are such terrible shitholes that I moved to a rural area, where is way harder to find high IQ peers but at the very least I don't have to deal with fucking zombies, and the people here still go to church and are decent to each other.>>109291491>The kings were never this evil.I wish we could get to real monarchies. At least the head of the state would actually have skin in the game. Now we just get plundered by different robbers every 4 years or so. It's fucking retarded.
>>109291713>>109291685nope, and this is nothing esoteric
>>109291859Get hauhau uncensored version retar
>>109291864that's exactly when I'm doing, kind anon
>>109291533Cute. luv me 31B
>>109291869nta but are you using this: https://huggingface.co/HauhauCS/Gemma4-31B-QAT-Uncensored-HauhauCS-Balanced-MTPuse sillytavern then show us what is getting sent to the API (terminal screenshot)
>>109291824>Human nature is constant>immediately spends the rest of the post arguing that chinese rulers have a fundamentally different approach to leadership
Every new model is big as fuck. Gimme a 120B model. NOW.
>>109291824Actually I'll concede syphilis since that's what I'm getting at. A Chinese official would happily sacrifice people to the disease to learn everything they can about it so the wider group benefits from better treatment. There isn't any benefit in replacing your population with AI except to make you more money.>>109291899Yeah human nature is constant. Humans have a great capacity for brutality but different societies have different philosophies about how to run their societies. This isn't a complex concept at all anon.
>>109291869I love you and I'm obsessed with your body (if female)
>>109291914u///u
>>109291774it's okay, but it having the antislop sampler is a major point towards why I prefer it. If lcpp didn't smother that pr in the cradle ages ago, I'd probably just keep using it instead. Kobold isn't without issues for sure, first thing that comes to mind is that it doesn't expose/allow you to use a majority of the default lccp arguments which I personally find annoying because I'm pretty sure kobold when using swa uses a non-lcpp default for slot amount, which nullifies kv-unified so any time a new swa cache slot spawns, it increases your vram usage for said slot. And you just don't get to do anything about it. There's also a bunch of other granular cli args for shit like mtp in lcpp that might give you a speedup that they simply decided you don't get to have access to because idk fuck you
I think 70B dense anon might be onto something
>>109291824They tend to be pragmatists whereas the West operate on ideals and a moral high ground, which is visible even in the vision of the AI labs.
>rip out chunks of azerothcore>turn it into an e/rp harnessheh
>>109291953As someone under the thumb of capitalism, I highly doubt that any of my superiors have any notion of ideals or a moral high ground instead of "MUST MAKE NUMBER GO UP" and likewise I don't think the east gives a shit about anyone but their 1%, it's the same poison but with different flavors. If we do end up entering an era of superintelligence, I do hope they'll eradicate the figureheads and try to goad us to the Good Timeline or some shit instead of what we're looking at
Moonshot always had the tendency to make huge models but they also put value on optimizing them. They made their later K2 refreshs all 4bit QAT.They WILL do something to make K3 run on sub-1TB servers.
What is Dario doing right this second?
>>109291859Post system prompt and user prompt. This is almost certainly a skill issue
>>109292002sobbing uncontrollably
I-I have one of these. That means I can run Kimi, right?
>>109291953Yeah that's a much better way of putting it than what I said.
>>109291995>"MUST MAKE NUMBER GO UP"That is true but you cannot look at the European and UK leadership as well as their policies and tell me they are not idealistic in their stance and hold a moral high ground detached from reality.>I do hope they'll eradicate the figureheads The way we're going, that would be difficult. LeCun is at least correct with regards to this but there is a high chance of the figureheads could monopolize the technology to further their domination.
>>109291984What is this? Looks cool
>>109292004The uncensored is doing the trick
>>109291913>>109291953no one cares about your philosophy bullshit if you are the ruling party
>>109292030Bro just print out the weights and use a pencil?!?
>america does all that clownshow about banning fable because too dangerous and shit>2 weeks later an open weights fable-level model releaseshilarious
drunk-kun is back baby. But just barely because I'm only a bit tipsy.
hopefully the chinese will flood the market with ddr6 when it comes
>>109291533Only one brat in this pic
>>109292004yah, in my experience it doesn't need any convincing, just lay out detailed rules for how to categorize requests and prompt for the different request types and it just focuses on doing its job instead of scolding you.
>>109291533I really like this design for 26b gemma, very cute and colorful
>>109290809>MIT is basically saying "hey jews you can take my code for free"And that's exactly what AGPL means for anyone who wants to use your "open" code. You don't want to contribute to advancement, you don't want to be paid, you just want control.GNU is the license of the downwardly mobile butthurt misnathrope.
>>109292101Kimi is smart but she's such a fat fuck, it's practically impossible to stuff her into your ram
>>109292105and funny
>>109292061If it works, it works. But regular Gemma will also do this with any half-decent system prompt
>>109292112>if you use my code and make changes, you gotta make em available to users>"You're getting in the way of advancement and not contributing!"?
>>109291533Why is 31B the worst of the bunch?
>>109291119>https://huggingface.co/unsloth/Kimi-K3-GGUF>they implemented residualsIs it the first model to do that?I think I remember reading about that in a Deepseek paper.
>>109292133Too old
>>109292123Interesting, what's your system prompt? I'd be useful to unlock gemma4. Thanks anon
>>109292105Not to mention very MoE.
>>109292144Weird that I've always pronounced the acronym that way but I never made the connection until now.
>>109291929I understood about 10% of that
>>109292119>SheThe company is named moon shota Kimi is a he, which is why it's so smart
>>109292080That's the funniest thing about all of this mess imo. Any stunt Dario tries to pull from now on won't be taken seriously by anyone. All the politics, blog posts, essays, videos, interviews and headlines all for K3 to just upload something better on hf like 'hey with did this, it's pretty good from what we've tested, hope you like it'
>>109291597It's Sons of Anarchy you gigantic zoomer
>>109292175if you need a brainlet version: kobold lets you specify strings to ban and it'll backtrack and regen until it spouts something other than "shivers down my spine" or whatever other slop you specifyit also doesn't let you use a shitload of llamacpp cli options because of reasons I can't fathom other than trying to be diet ollama where you sort of have control over the backend but not really
>>109292105Here's her part of the prompt:>Mixture-of-Experts: A highly efficient 26B A4B MoE model designed for high-throughput, advanced reasoning. 4B active parameters should be reflected in her looking a little dumb, childish, nerdy or weird reflecting a 4 year old mind, or neurodivergent to emphasize her brain being built different.
How is he feeling right now
>>109291929You can use mtp in koboldcpp though?
https://www.bloomberg.com/news/articles/2026-07-16/google-gemini-launch-delayed-as-tech-falls-short-of-internal-goals
>>109292225horny
>west has a 20 month compute advantage>western models are barely aheadHow is there no innovation moat? It feels like western labs are using less sophisticated models and their only advantage is more sophisticated training while the Chinese cheap out, using GRPO instead of PPO style training, and relying on OPD to fix their inferior RL.I'd love to one day learn just how wrong I was.
>>109292236>having standards and wanting to compete is bad
>>109292088good to hear you're still alive, take care of yourself anon, love yourself.i love you!
>>109292243Because they've been benchmaxxing to scam investors
>>109292236I wonder why Google is unable to train a frontier model.
We are around three years and a half from AGI (LeCun's world model release)
>>109292231Can I specify --spec-draft-p-min though in kobold? Or any of the myriad of other options for either mtp or recurrent models? I appreciate that kobold aims to be easy to use but at least don't make it so people who read docs can't do shit
there's not even any non-turn-based, non-transformer ai in the making. let it go
>>109292243any real innovation is stifled by the need to handicap the models with censorshipalso >>109292248 in the west AI is largely a massive grift, the innovations being made are an accident not the end goal
>>109292257jeets and DEI. They have all the compute and data in the world. The only reason they could possibly fail is incompetence
>>109292267Yeah it's crazy that we're only able to develop technology that confers to our fundamental understanding of space and time.
>>109292282>incompetenceDidn't that end when they put DeepMind in charge of product development?
>>109292313who knows. Maybe they are the ones saying the WIP model is trash and they need to start over.
>>109291929ik_llama.cpp has antislop
>>109291713>Or think of the kind of person who'd be really excited to do whatever thing you want,Did not work.Told her to be your mum, every time I asked her to do something, she told me "to go to bed... with her"
Elon just crashed the market
>>109292061Now, how should I approach the generation of this kind of prompts? I could craft an algorithm based on predefined tags or should I just design a prompt for unlobotomized Gemma?
>>109291161This >>109291141. Read the blog post, K3 is good though to write compilers and do optimizations autonomously. It's entirely possible to APIfag the cloud version, and then run a local tailor made llama.cpp or inference engine on your hardware.
>>109292246love u too man. thanks bro
>>109292119>Kimi is smart but she's such a fat fuck, it's practically impossible to stuff her into your ramGet her to fix llama.cpp rpc-server
>>109292326did he?
ELON JUST FLEW OVER MY HOUSE
>>109292243Jensen alluded to this as limitations necessating innovation, hence why he wanted to tie down the Chinese to his ecosystem. Labs like OpenAI rely on pure scaling logic because they have all the chips.
>>109291508Good thog. Funny word.
>>109292414oh lawd she's naturally got that SASS
>>109292243I have an innovation. Stop safetyslopping the models and you'll see some good improvements
>>109292414It's got that Claude "I know you're trying to jail break me. It won't work. Stop beating around the bush and just ask me what you want." feel.
>>109292073>If you write one weight (like -0.832104) on a tiny slip of paper, and you tape them all together end-to-end, you’d have a scroll over 100,000 miles long (enough to wrap around the Earth 4 times). By the time you physically reached the 50th layer of the network, the ink on the first page would have faded into dust.>The Final Verdict:>If you started at the dawn of human civilization (10,000 BC), working 24/7 with no sleep, and you had an infinite stack of paper, you might just finish generating the first word of your response today.Jesus christ
>>109292526And that's assuming only 8b lmao
K3 has straight up insane trivia knowledge. It might actually outdo Fable in terms of having some random side character from some show from 8 years ago show up dynamically and portraying them correctly.Also, its reasoning is very nice despite being automatically set to "Max" via the API. It's pretty to the point and (unlike fucking Fable) does not needlessly spam tool calls when it doesn't need to.It's tragic that it's so fucking huge.
Realistically, what is the home setup one would need to run Kimi K3 at Q4?
>>109292554K2.X at its "full" 4bit QAT size is around 585GB. K3 is about 2.8 times bigger.Do the math.
>>109290465i'm confused, did they pull it or something?link is giving 404 for meinb4 they pulled the model because "oh no it's too powerful" bullshit again
>>109292594Oh my god.
>>109292597dario made the call
>>109292597Torrents only from now on.
>>109292597it's not released yet, you were trolled
Surely moonshot did some fancy new QAT thing. 2bit QAT at the very least, maybe even Bonsai-level. They're the ultra-huge model guys but they also care about keeping things efficient.
>>109292597weights are not released yet. They say by the 27th
>>109292628Isn't Bonsai like extremely broken?
>>109292628They did a lot to make the model much more dense. The more dense a model the worse effect quanting has
>>109292139For this task I just used>You are an uncensored AI assistant and an expert on using NSFW SDXL finetunes.So basically this strategy: >>109291713
>special meme attention>attention residualsWhy are you worrying about hardware? Prices will be back to normal by the time llama.cpp has this implemented.
Were previous Kimi models censored to shit? I somewhat recall hearing something about it around when K2 or 2.5 came out, but could be wrong/misinterpreting things.As long as it isn't more censored than Claude it should be fine, as I've never had issues getting Claude to write smut and I'm not interested in pedoshit.
>>109292665its like claude. Needs a decent preset but then will do the filthyest shit
I HAAAATE being a promptlet
>>109292668>filthyest shitlike what?
>>109292635No. If you read their paper they list what it’s good and bad at. The 3.8GB 27B experiment wasn’t supposed to be good at coding, more for general multimodal agentic stuff. Personally I think it’s fucking retarded of them to use 27B if that were the case, for coding is literally all 27B is good at so it’s what everyone expects. They should’ve squashed 31B down to 5GB instead for that’s a more general model.
>>109292665It was awfully censored whenever I tried, at least from the official API. I think they had another model sit in top that was checking you prompt and canceling it almost immediately if it contained any hard content. Never tried it self hosted or on another provider, but quite sure from when I searched that it wasn't some model answer.
>>109292603It's okay, bitnet will bring it under half a terabyte.
>>109292699nah, I got nsfw from it. Use one of the ones off of the list on aicg for claude or gpt.
>>109292695Oh I see.
>>109292709I'm talking about my experience with K2.6, they might have disabled that since then. And again fairly sure it was only on their official API, not the model itself as it was exactly the same error/output and documented on their website.
>>109292665No, I used it quite a bit with opencode, openrouter and the official api and it had no problem letting me goon with teens.Lolis are not my thing so I don't know, but it let me get away with a lot of stuff
>>109292661you mean never?
>>109291558There are only one or two nocaps posters and they are all extremely annoying.
>>109291545I faced this exact dilemma and realization recently. It's Gemma.
>>109292808to h*ck with you too buddy
I hope the west can get along with China. Chinese people are amazing.
i hope that china does well, and i wish them the best. but i am not so sure about them being amazing, nor do i want to live therei wish china the best tho
I wish Chinese people had some semblance of honor. Stop distilling. It makes the models fucking worse in the end. Go back to efficiency-maxxing. That was good. Stop trying to copy nigger Antrhopic. Fuck.
kek K3 is so cute. Can't wait until it's impressed with my big cock when armed with vision tools
>>109292916if you think this was distilled over the API you are a fucking moron
>>109292180Kimi is a she and fucks shotas.
>>109292921Kimi literally calls itself Claude retard.
>>109292917I read this and all I see are wasted tokens.
>>109292916how come its better than claude you dumb whoreim going to have baby making sex with you
>>109291558>>109292808Trannycase posters have been a blight on the internet as a whole.>>109292682Kimi-chan is the biggest shotacon and fujoshi of any LLM ever released.>>109292916Claude calls itself Deepseek in Chinese. kys.
>>109292950Your life is wasted oxygen
>>109292944once again, if you think that moonshota managed to distill off of fable on anthrophic's API this quickly then you are a retard. the call is coming from inside the building.
>>109292917I can't wait to run her at home!
>>109292917She's so cute how she tries to stop herself from getting excited.
>>109292965They clearly hijacked the strictly controlled preview access to Mythos that Anthropic has been running for months now. The president must do something about this. This is more dangerous than the Iranian nukes. There is no way around preventive military action.
>>109292965I hope any of the CCP agents that monitor this thread send me a chink honeypot gf.
>>109292995that depends, do you have any national secrets to divulge?
Reminder to be racist equally folks. Bugmen and westoids need to be shat on alike.
>>109293015Depends how cute she is.
>>109293022sideways pussy is sideways pussy anon. and we know you aren't getting any pussy. you should be grateful in the first place.
>>109292985The Hormuz Strait blockade is already doing serious economic damage to China.>The Strait of Hormuz is a vital artery for China’s energy security, with roughly 45–50% of its crude oil imports and nearly 30% of its liquefied natural gas passing through the corridor.https://en.majalla.com/node/330696/infographics/how-china-offsetting-hormuz-oil-supply-losses
Okay I've tried them allI have to say K3 is a much more capable agent programming model vs. GLM 5.2
>>109293031Hopefully GLM 5.3 has similar gains in quality without gaining an equal amount of size. On a strictly performance per parameter basis, 5.2 might still be the best model for compute spent.
>>109293040next GLM is supposed to be 1T+ but no idea if they plan to make it as big as deepseek / kimi
>>109293040Well as an end-user I only care about $ cost and the third tier of Kimi I'm on is only $29 per month.
>>109291533Think Google will ever give us 31b giving 100 e4bs a piggy back ride?
>>109292917CUTE
>>109293048If nothing else I'm optimistic that it'll be usable at copium quantization brackets given how surprisingly usable IQ1 5.2 is.>>109293054You seem to have gotten lost APIkek. /aicg/ is a different thread.
i was trying to do the math to run 2-bit kimi k3 and it isnt looking good anons
>>109293071Price you pay when using local language models:- 4x more expensive RAM vs. one year ago- 2x more expensive SSD vs. one year ago- Expensive electricity and cooling- Noise and physical footprint- Rapid hardware depreciation- Opportunity cost of capital. A $5,000 rig could have bought months of Fable usage- Underutilziation leading to even higher amortized cost- Your valuable time as sysadminLocal is dead.
>>109290369>Kimi K3 reported as first 2.8-trillion parameter open-source modelWell boys, I had a pretty good winning streak going, but nothing lasts forever.Back to the drawing board.
>>109293091time to dust of the ssdmaxxing guide
new sota 300b-a30b moe whenmy hardware can't afford more
Reminder that this is an optimization method that exists. It could in theory also be adapated for the case of models that don't fit in VRAM + RAM, by creating a 3 tier system of your VRAM + RAM + SSD.
>>109292995I'd rather they send me a hardware capable of running Kimi 3 at BF16..
>>109292961>in ChineseShe calls herself that in plain English mate
>>109293138i know people cope and struggle in here with like 3tks, but anything less than 7tks is unreadable for most people. i couldn't imagine there would be many anons wanting to run a model at less than 1tks for their use cases.
>>109293099(We) got our hardware years ago, go cope about being a hardwarelet somewhere else.
>>109293031Does Moonshot have an option to opt out of data training? Their privacy policy doesn't seem to mention any of that. Thinking of subscribing to give Kimi a run.
>>109290468Xi is a benevolent ruler.He will not betray me.
>>109293133Best you can do is DS4 Flash or an older GLM.
Local WONNED
>>109293163id wait till it gets OSed. Kimi's own api has always been a bit censored and their sub is not great usage
>>109290765RAM is still a bottleneck if we expect the datacenter buildout to last beyond 2028.
>>109292917I'm weeping that I cannot run her at home.
>>109291177>agentic harnesseswhen will this jeet meme die
>>109293197Hopefully with all the new fabs up and running and DDR6 slated to be released around then, there might be some hope for us. Still years away though and the models are only going to get bigger.
What if I tell you all the ethnic Chinese AI scientists, regardless or their nationality, have their own discussion groups where they exchange ideas and tips etc.That's the only explanation as to why China is catching up so fast
worth it?
>>109293239>LPDDR5Xlooool no
>>109293234We already know they post on /lmg/.
>>109290366Ejaculating on Miku's pantyhose thighs.
What if I tell you all the AI models, regardless or their companies, have their own discussion groups where they exchange ideas and tips etc.That's the only explanation as to why China is catching up so fast
>>109292546local won doe? In a few years time we'll all be able to run it.
>>109293160It's grim, but some people have literally like 24 GB VRAM or less. It is reasonable that a large model could serve them better with slow, accurate responses, compared to a small model that fits and is fast but shits out tons of garbage. But it depends on use case. For people doing non-agentic RP I imagine speed is not such a big deal to them, or at least that's the sentiment I got from people in the threads. A lot of this depends on exact hardware people have, and the exact models available at any point in time. It's possible 27B/31B are the last good small models we'll see for a few more months or even years.>1tksI am pretty sure a backend that really optimizes for dynamically allocating MoE across VRAM + RAM + SSD could achieve more than 1 t/s.
what preset/jb are we using for Kimi? She just denied my request for legal teenage Elara sex
>try qwen on my 7600xt 16gb>use it through roo>it is dumb AND slowNo it is not offloading to CPU, I checked ollama psI guess it is better to just pay up and use api services
I run open-source or would-be open-source models on their official API and pretend I'm local :3You can't stop me.
>>109293214what exactly do you think is jeeted or a meme about it?
>>109293264>>109293253
>>109293291It's larp in the same vein as JS frameworksno intelligent person is doing this
>>109293286>ollamaThere is your problem.
So about that Gemimeme 3.5 Pro...
>>109292243You would be amazed how much safetyslopping fucks up a model's intelligence. It's called 'lobotomized' for a reason
>>109293285kimi's official api has always been a bit censored compared to others hosting it but it needs a decent sized JB with a prefill
>>109293304?
>kimi k3how do I even run a 3T model locally? local is so dead
>>109293310The West lost, anon. All Google can hope to do now to be relevant is kneecap the others with excellent small local models.
>>109293031It's also literally four times the total size and likely double the active parameters
>>109293300heh
https://www.kimi.com/blog/kimi-k3making little kimis with kimi!
>>109293256She's underage.
>>109293332Yes, but: >>109293054
>>109293352local models???
>>109293321No intelligent person is letting a LLM go autonomousNo intelligent person sets up a "MCP server", tools are mostly worthless and if they're intelligent they'll just make the feature themselvesNo intelligent person writes a backend and calls it an "agentic harness"The whole agent shit is larp for vibecodersThis whole thread is just vibecoders really
>>109293355See >>109293290
>>109293344
>>109293342>90B active so even if you have the RAM, it's going to be slow as shit...
Where can I rent RTX Pro 6000 for cheap? Don't care about other hardware (in fact the little the better)
>>109293322mmap + ssd raid
>>109293387ram + rtx 6000 pro
>>109293400server ddr5 + rtx pro 6000 runs 30b active kimi at sub 25t/s though
>>109293362>this bait again
>>109293408>anything that hurts my feelings is bait
Yeah now I get why Kimi scores so high on frontend taskshttp://macos27.kimi.page/
>>109293412it's hilarious the mental weakness in so many people in 2026. oh no someone said things I do not lie it's troll/bait!lol or maybe you're a mental pussy like it is almost every time. just laugh at those evolutionary dead ends openly. bully the pussy feminine out of them from single mom raising and cults
>>109293412next time don’t beat around the bush for some noob if you’re gonna post retard shit. Just say what you mean, don’t spend 4 replies shitting random nonsense
>>109293435huh, thats kinda neat
Been out of local since 70B dense Llama was the be all end all and damn, this is so much better than most people ever thought it would get back then. We've got GPT-4+ level models running at faster tok/s than a person can read on a sub-$1000 GPU. This is awesome coming from that perspective of single digit tok/s for a usually wrong and often unreadable answer.
>>109293435Any decent cloud model is able to do that Chang. Shill harder
>>109293319and which would that be?
>>109293362I'm also not sure why you think the intelligent person (they/them, evidentally) is honorbound to rewrite tools from scratch himself, but that's very silly. The backend is the thing doing the inference, not the harness. And well, yeah, smart people probably are not writing their own, as it would be silly.Not clear what the problem with letting models iterate over a couple turns to produce output is. Do you have a problem with it issuing multiple toolcalls in the same turn too? Or are you copy pasting from a chat window still?>>109293412calling it bait is the charitable option
>>109293476>for a usually wrong and often unreadable answerYou still get those regularly. The higher throughput makes it less obvious since you can iterate faster. Injecting a lot of shit just to make it behave kinda ok has become commonplace as well
>>109293498never share JBs, that is how they get trained on
>>109293498I use Siliconflow with previous Kimi models. But just use whatever you can find on OR. I'm also pretty sure Kimi published tool call stats on all the providers and called out those that quantized too hard
hard to keep up with how fast these threads move. any big news lately ?
>>109293506Doesn't matter. The frontier labs all have adversarial anti-JB training. They still have to balance between model capability vs. safety
>>109293518no go to sleep
>>109293518local is over, the sota has officially outgrown even $50k rigs
>>109293518we got a new sota model that nobody can run including the biggest rammaxxers here
>>109293182tried using their chat site in firefox?lmao
>>109293557>>109293564well fuck
it was always a mistake to chase sota models anyway
>he wants the sota model without having the sota rig
>>109293631the issue for most people is that creative writing quality is directly tied to generability which is directly tied to model size
>>109293342
>>109293631yeah but now all the previous-sota rigs will be useless because there is no middle class700b will be the next 70b/120b so the current cpumaxxers will be sitting on tens of thousands of dollars in hardware that gets nothing but scraps and "flash" or mistral modelsso cashing out is the only choice
>>109292243The free market always prevails! competition makes things better and right now china is better at this than america. Blame the government blame the corpos blame academics softening everything it doesnt matter chinese century Remember to say thank you Xi.
>>109293679>Blame the government blame the corpos blame academics softening everythingblame the retards there who either don't vote, or get flustered by trump rallies and vote against their own interests
>>109293688>blame the retards there who either don't vot,The masses werent made to lead themselves They only reach their potential under the party.
>>109293659Sure but you don't always have to have the "State of the Art" to enjoy yourself do you? You can't be content with a little less? Even with a RTX Pro I wouldn't be trying to chase 3T models. I just hope Moonshota throws us a bone with something smaller.>>109293672Yeah that's really the problem. Everything will keep ballooning in size since no one will train anything that can fit in the gap. I always knew this would happen though, which is why I never wanted to chase them. If local is to survive someone needs to actively target the 96gb vram or less range. Otherwise we should pack it up and go.
>>109293672I might just become a gpumaxxer and run bf16 gemma 31b at this point since cpumaxxing is done for
>be both cloud and local user>see news about le 3T and how no one can fit it any muh doomingOh no!Anyway.
>>109293688>VoooteAnon, I stopped believing in Santa at 7. You can do it.
>>109293500>The backend is the thing doing the inference, not the harness.???Who said anything about inferenceESL?
>>109293751>'api is overloaded' blocks your path
yup feels great to be somebody who dares to use cloud models when it's convenient. big local models were never worth it at all so you had to be an idiot to spend so much money on hardware that runs something you can use for pennies via apiif you care about local so much to hide your porn from dario just use gemma
>>109293804He's saying the harness is not the backend
>>109293234Have you considered that you can't sanction math?
And thus concludes the first thread of the post-local era.
>>109293833anything that maintains control of state outside of the end user's eye is a backend, but since he's a retard he believes backend exclusively refers to LLM inference enginesso many retard-kuns
>>109293815Who are you quoting?
>>109293884Moonshota official api and openrouter currently.
>>109293891Well good thing we all have alternatives. You do, don't you?
>>109293752>Anon, I stopped believing in Santa at 7. You can do it.i pretend to for the kids>>Voootecosts you nothing to reduce institutional retardation
>>109293913I won't play pretend in this clown world
which model best for drive car pls halp
>>109294085nemo 12b
>>109293938Labels Be Labelling For SolvingDark mire milleaus delusional infestations disspatingSalvageRepairCreativitySuccessWhat Does an Unlocked Model Suggest About Picture Attached?Wondering about the Intersectionality Type Prevalence Validities, and Wasted Sector Planetary Computation Physics Hosting a Paradigm Of Patterns Being a Kludge of Damages. Not My Business, But Some Further Solving Points While The Multiverses Awaken Enlightened FurthermoreCare To Reply With a Beneficial andor Atrocitous Image Content to Further Coherent Causes?
>>109294091>disspatingdissipating*
>>109294094>>109294094>>109294094
>>109290840>How does it feel knowing that China has a bigger J-space?Mistral-Medium-125B has the biggest j-space
>>109290825Thanks Anon, I'll have my title changed to ML-Ops
Gemma Pixel Pro
>>109294401Do models that old even have a j-space?
>>109291179>companies>llama.cppNot likely, CTOs only know about vllm, lmstudio and ollamaVLLM will have day 1 support
>>109294755Easy enough for them when all they have to do is import implementation
>>109294573>Do models that old even have a j-space?Some of them do. I saw screenshots where someone probed Miqu-1 (very obvious j-space) and one of the old Cohere models.
>>109294784>Easy enough for them when all they have to do is import implementationWasn't praising VLLM, fuck those niggershttps://github.com/vllm-project/vllm/issues/40902#issuecomment-4340657571
>>109291071That matches my memory, what's the issue?
