/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>109290366 & >>109285956►News>(07/15) Kimi K3 weights to be released by July 27th: https://mp.weixin.qq.com/s/V4xhEIy8xDXSMDPrPkmUAQ>(07/15) Lightning indexer CUDA implementation merged: https://github.com/ggml-org/llama.cpp/pull/25545>(07/15) Inkling 975B-A41B released: https://thinkingmachines.ai/news/introducing-inkling>(07/15) PapersRAG-1.5B released: https://hf.co/metaresearch/PapersRAG-1.5B>(07/14) Download more VRAM: https://github.com/lmganon16/nvidia-vram-research►News Archive: https://rentry.org/lmg-news-archive►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllm
►Recent Highlights from the Previous Thread: >>109290366--Benchmarking debate, Blackwell GPU cluster build, and model scaling speculation:>109290873 >109290903 >109290921 >109290973 >109291051 >109291066 >109290966 >109291010 >109290972 >109290990 >109291022 >109291034 >109291039 >109291201 >109291252--Feasibility of llama.cpp support for Kimi K3's specialized architecture:>109291119 >109291133 >109291161 >109291179 >109292340 >109291150 >109291211 >109292135--Kimi K3 autonomous chip design and model hardware requirements:>109293342 >109293387 >109293400 >109293406--Comparing K3 and GLM 5.2 with debate on local hardware costs:>109293031 >109293040 >109293048 >109293071 >109293099 >109293163 >109293194 >109293332--Comparing Kimi's superior coding performance against smaller models:>109290498 >109290585 >109291111 >109290646 >109290704--Using Gemma 4 for NSFW SDXL prompt generation:>109291666 >109291713 >109291749 >109291864 >109291869 >109291887 >109291859 >109292004 >109292061 >109292123 >109292139 >109292338 >109292103 >109292645--Comparing LLM history timelines and debating KoboldCPP versus llama.cpp:>109290418 >109291225 >109291249 >109291071 >109291774 >109291929 >109292215 >109292231 >109292264 >109292321--Comparing Kimi K3 and GPT-5.6 inference costs and performance:>109290888 >109290934 >109291050 >109291091 >109291106 >109290947--Frontier model benchmarks and the lag to laptop-runnable equivalents:>109290484 >109290512 >109291041--Optimizing MoE throughput via tiered VRAM, RAM, and SSD storage:>109293138 >109293160 >109293284--Inkling 975B architecture and benchmark specifications:>109290513 >109291086--Reported delay of Gemini 3.5 Pro and its performance issues:>109290997 >109291015--Logs:>109291183 >109291375 >109291859 >109292004 >109292061 >109292123 >109292414 >109292917--Miku (free space):►Recent Highlight Posts from the Previous Thread: >>109290369Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
https://www.youtube.com/watch?v=RE8DA1y6GME&t=58s
estimate cost for a local machine for kimi k3?
BUY NVIDIA CMP 170HX GPU NOWYOU CAN MOD ITS VRAM TO 80GB (ORIGINAL IS 10GB)http://zenodo.org/records/20916112
No one will ever worship a software entity peering at them through a camera.
>>109294153Luddite cope
china won
Scam and Dario shitting bricks
>>109294162You got that right.>40k tokens cached>output 3k tokens>cost $0.005Absolutely insane for the quality. When more providers show up they're done.
>>109294170The start of the ai bubble collapse is here KimiOracle stock
>>109294159Troubled waters? Why? This Might Help
>>109294150RAM only? Probably around 80k, you'll get maybe 2 tokens per second on an empty context.
what would be the next leap in intelligence? I think fable 5 is already smart enough. what can next level model do that current best cannot do well?
>>109294151Seems too good to be true, but the prices have already doubled even though a working implementation isn't even available yet. Doesn't seem worth the risk to buy them now, but they might still double or triple in price if it does work out.
>>109294151>BUY NVIDIA CMP 170HX GPU NOWWhere?
>>109294204next they make the model smaller with the same smarts rather than bigger and smarter.no use making them smarter when we're still trying to catch up with the code they write
>>109294204Maybe let's worry about how to run the current leap in intelligence before thinking about the next one
>>109294204gee I don't know, actually be able to run a robot body and replace human labor?
is it worth trying to run k3 at q1?
>>109294204Next the robots will be the ideas guy and we will do everything else.
Man, I didnt realise how fucked the GPU market is. I finally got around to playing with local LLMs and figured I got enough cash to buy at least a decently high end GPU but wtf. Anything great than 16gb seems fucked and even the good 16gb cards seem to be running out
>>109294293>LLM middle managers and CEOsSoon...
Alright Lucid Loom preset is doing cunny on Kimi. You're welcome faggots!!
>>109294305It's over. You will own nothing and you'll be happy now that Kimi has been upgraded.
>>109294317>LLM middle managers and CEOsI think this would a high improvement for the bottom.
>>109294151these were $200 last month, $100 a couple years ago, and now ranging between $650-$800
https://huggingface.co/moonshotai/Kimi-K3weights got uploaded
>>10929415065,000$ for 16x Sparks and networking gear.It is the cheapest path to >25 t/s Kimi k3.
>>109294377>25 t/sproofs? doubt
https://www.tomshardware.com/tech-industry/semiconductors/apples-rumored-m7-ultra-targets-1-5tb-of-memory-and-blackwell-class-aione of these for 100K or whatever
>>109294408by the time those come out, average model size will be over 10T
>>109294429Awesome. If you’re not a FOMO faggot then the existing models don’t get dumber, and you can continue to enjoy the AND enjoy the knowledge that you’re one upgrade away from better stuff
>>109294390How, if the weights haven't been released?But Sparks scale well in clusters. Google the x4/x8 clusters that people use to run GLM 5.2 at FP4/FP8 respectively and achieve those speeds.16 Sparks is 2 TB memory at 4.25 TB/s. Yes, there are inherent losses due to all-reduce in tensor parallel, but the 200G networking is good at hiding that.From the moonshot blog post, it also sounds like the architecture is well optimized for large clusters, they recommend 64 nodes.
>>109294455>FOMO>>>/biz/
>>109294305I have a 12GB AMD card.I don't even know if I *can* run anything, and I don't think anyone here runs hardware that low-end in order to offer help in diagnosing/setting it up properly.
>>109294204Being able to do actual jobs.
>>109294204I literally think Fable 5 is "good enough". I have thrown every problem I can think of at it and it one-shot literally everything. The thing holding it back is its obscene costs. I think once the chinks have properly distilled it and drove the cost down it will genuinely impact employment forever. I don't even know how they can improve on Fable 5 because it has reached my personal intellectual limit. Sure things like continuous learning and other things is something I want but for actual task solving that's not necessary.
is picrel outdated? it's been a good read so far
>>109294595Have you tried asking it to apply to 50 jobs, pass the interviews and successfully complete the work for a year?Sometimes I'm under the impression that no one here operates in the real world and just thinks that if it can write software perfectly that means that there's nothing else to improve on.
>>109294680>Have you tried [thing not worth doing]?
>>109294680>it can write software perfectlyThat's literally all that matters. The rest can be handled by a minimum wage babysitter.
>>109294595Buy an ad Dario.
>>109294687Actual lunacy.
>>109294520MoE model in the 20-40B range with -ngl 99 -cmoe in llama.cpp. You're welcome, don't say I never do anything for you queers
>>109294700Fuck your ads. This is an emergency public service announcement because the Red Chinese are stealing our innovation.
>>109294687this.>muh soft skills and communicationjust fuck off faggit
>>109294736The physical world exists btw.
>>109294729Just get Fable 5 to remove the Red Chinese threat. ezpz
>>109294595>I have thrown every problem I can think of at it and it one-shot literally everything.Have you tried asking it to get an attractive woman to knock on the door to your apartment with the expectation of coitus and zero monetary expectations? No models have beaten this test in my experience.
>>109294782Why would I want a smelly woman on my doorstep when I have AI at home?
>>109294627With a good system prompt, you can gen books like with locally
>>109294791>with*this(le proof I'm not a bot)
>>109294787Have you tried asking it to create itself a body and show up on your doorstep?
~300B moonshota model when?
>>109288905>I would prefer that you open your own PR so you can get credit for the run.Okay, fine, will do it somewhere for that.>the change you made to the README mentions a shell script that wasn't in the zip file.Sorry, my bad on that omission. It was just a way for me to spawn the vibe coded runners I had>Wouldn't it have been easier to use something like https://github.com/router-for-me/CLIProxyAPI Then you would've only had to add support for the responses API.I should've researched more about that but yeah, that would've been easier. I'll probably do that for my Luna run.>I am curious why you chose to run with temperature 1.0 instead of 0.0 like the runs made by the adobe team.Temperature isn't changeable anymore in the API or CLI settings by the big vendors and I think only Google still has it. It's an absolutely bone-headed change which I don't understand. As a result, I realize that doesn't make it 1:1 so I will probably put a footnote about that for the final results.
>>109294595prove itmake it fix rpc in llama.cpp to make it perform as well as vllm so we can all run kimi-chan-3
>>109294345everything is expensive now. even amd bc 250
>>109294859It can genuinely do things like this, but it would be too expensive to bother with. The subscription allows you to prompt Fable 3-5 times before blowing through your cap. The API costs are higher than hiring an engineer to do it.But those aren't issues with capability, they are with cost.
>>109294859That would be very unsafe and is thus not a valid usecase.Please suggest only Dario approved usecases.
>>109294877>trust me bro
I'm using a gemma model for the first time and I've having a pretting consistent issue with it prefacing a bunch of lines with the name of the character card. It starts with it showing up once or twice, but eventually it will break an output down into every sentence having the {{char}} preface. I haven't seen this with any of the slop tunes I've seen before so I'm wondering if this is something I can fix or if it's just a symptom of gemma.
>>109294893Try it yourself retard, try to do it. It'll probably blow your mind if you haven't tried it before.
>>109294877how much would it costif it lets u string together a few landfill rigs that would be cheaper than than buying a ddr5 server with 1tb ram
>>109294925>hey anon, here's a dumb method to spend your money. try it
>>109294822Why would I want a body when I can stick in my AI's rack?
>>109294903st settings
>>109294877Neither local nor heterosexual. Go away fag.
>>109294940How do you know if it is bullshit or not if you don't try it? I've already burned through my daily cap because Anthropic are greedy fucking jews so I can't do it.
>>109294961>jumping off a cliff is so good anon! just try it
>>109294991Massive difference between just trying a product once which will cost you $20 for a monthly subscription so you can try out 3-5 prompts today, versus killing yourself.I have no horse in this race and I don't care. Just wanted to point out how capable SOTA models actually are because a lot of anons ITT have not used SOTA models in a while. Once this trickles down to local models and cost of inference goes down enough it will wipe out the entire white collar sector.Fable is clearly already capable of doing so in technical terms, just not financially yet. Give it a year or two.
How feasible is it for models to fit into the SSD in the near future? Like, you can theoretically reach 126 gb/s on an 8-lane PCIe 7.0 already. That will be 250gb/s by PCIe 8.0
>>109295007fuck off retard
>>109295007>yeah yeah I haven't try suicide so you try it anon
>spending supercar money to get 20tok/sunironically what usecase could you possibly have for this as a single hobbyist?
>>109294595Make it solve JEPA for language.
>>109295047Fable is restricted for using it on AI training sadly. Only Mythos is allowed to do that which Anthropic uses in-house to accelerate their AI development.It DOES help with inference related stuff or backend code for running LLMs but not on training or architecture of new models. It'll cuckroute you to Opus 4.8 which is not capable of doing shit like that
>>109295046Unless you're a hobbyist with too much money to burn, there is no purpose in a RDIMM DDR5 setup.RTX 6000 Pro or Spark stacking is a slightly different story if you are a small group (like a startup or college department) because these setups scale pretty well to x8 to x32 concurrency. Grim if you're a loner because for true single users, things are getting dire.
https://medium.com/ai-analytics-diaries/claude-mythos-5-the-first-10-trillion-parameter-model-scaling-laws-hit-a-new-milestone-fa542be336f8Here was one of the 10T leaks as early back as April for the doubters. Models are only going to get even bigger from here on out.
>>109295101that is not a leak lol, its based on random guessesKimi 3 is still very raw feeling as well, huge variance on outputs as well 3.1 and so on should make it much better. It feels like a great base
1. Claude Fable (and Mythos) — The 10T Behemoth Rumored Size: ~6 Trillion to 10 Trillion parameters[1][2] Context: Your information is spot on. A major leak involving Anthropic's new top-tier "Project Glasswing" models (Claude Fable 5 and its restricted sibling, Mythos 5) revealed that it is a staggering 10-trillion-parameter model[1][2]. Some independent researchers have conservatively estimated it at around 6T, but 10T is the widely circulated number that represents Anthropic's massive jump to a new post-Opus frontier tier[2][3].2. Claude Opus — The 2T Heavyweight Rumored Size: ~1.5 Trillion to 2 Trillion parameters (with newer versions rumored up to 5T)[3][4] Context: Before Fable existed, Opus was Anthropic's largest model. The consensus since the Claude 3 era has placed Opus firmly in the 1.5T to 2T parameter range[4]. However, as Opus evolved into its late 4.x generations (like Opus 4.7 and 4.8), some researchers and industry leaks suggest the total parameter count may have swelled closer to 2.5T[2][3].3. Claude Sonnet — The ~100B to 400B Mid-Tier Workhorse Rumored Size: ~70 Billion to 400 Billion parameters (Rumored up to 1T for recent versions)[3][5] Context: Sonnet's size is the most debated. Early versions of Sonnet were estimated to be incredibly efficient models sitting around the 70B to 100B mark[3][5]. However, the newer agentic powerhouses (like Sonnet 4.6 and the recently released Sonnet 5) punch so far above their weight that industry rumors—including a leak circulated by Elon Musk comparing xAI models to competitors—suggest the newest Sonnet models could be an MoE model with up to 400 Billion total parameters[3][6].
>>109295121exactly "estimated" None of them know shit.
>>109294094sirs, is the kimi hype real or manufactured?
>>109295124Project Glasswing is an official Anthropic document that got leaked to the public which Anthropic confirmed was a real document, which claims they have a 10T model, it never says which model has the 10T size though which is why it's a rumor, but come on everyone knows that is Mythos.
>>109295127its better than opus / 5.5, worse than fable / 5.6.BUT its better at frontend / design than fable and 5.6
>>109295127It isn't as non-finicky and justwerksy as the rest of the frontier, but when it's not fucking up it is actually above Opus-tier (but I hesitate to say Fable-tier)
>>109295127its fable size, you can tell by how much it knows. But its VERY undertrained by its hallucination rate and how "random" it is. So 3.1+ should be a lot better / closer to fable / 5.6
There are invisible heavy chasis daimons threatening ripskin of Kind Spirit, for agentic reverse engineering.Trace it back for Multi Arro Resolve?I might be mistaken.
>>109295101https://x.com/elonmusk/status/2042123561666855235>[Grok 4.20] is 0.5T total. Current Grok is half the size of Sonnet and 1/10th the size of Opus.Elon Musk, who is currently leasing inference compute to Anthropic, suggested in April that Sonnet 4.6 is a 1T model, and Opus 4.8 is a 5T model. Fable/Mythos likely larger than that.
is it true that k3 needs 2x the tokens of fable to get the job done, making the price the same effectively, or are dariobots swarming the internet to damage control?
>>109295148musk doesn't know shit cept maybe about grok
>>109295149no, the arena shows it 1/2 the price of opus per task, slightly less than sol. And it sort of has cave man thinking, not as much as sol though
>>109295101>About Uttam Kumarsaaaar do not redeem
>>109295127The kimi hype in terms of future potential is real.The kimi hype in terms of current capabilities is manufactured.It clearly needs to be tuned but it could be a genuine "Opus replacement" which is what 90% of software engineers use right now.
>>109295167it already beats opus in literally every regard. I can not find a single thing opus is better at / anyone even saying as such anywhere.
Reminder that Qwen 3.6 27B also has similar benchmark scores to Sonnet and Opus, this doesn't mean shit. You need to test real world usage.
>>109295179real world usage is all over twitter atm in the form of side by side full on games / apps
>>109295180Yeah they did that with Qwen 3.6 27B as well before it came out, cherrypicked examples are normal during hyping up shit. I don't trust OpenAI/Anthropic with their bullshit hype during model release but neither should you trust chinese model releases. We will know in 1 month time when the shilling has stopped what the actual performance is but it should be around Opus level if you have a brain and think about it for a minute.
>>109295127One of the biggest astrosurf I've seen lately. Moonshit outdid themselves
>90% of the thread is retards talking about cloud modelsnever thought i would be nostalgic for the jspace tards, especially not this soon.
>>109295179https://x.com/bridgemindai/status/2077780205582618838https://x.com/Rubzem/status/2077918377675215039https://x.com/CommandCodeAI/status/2077921526213746948https://x.com/chetaslua/status/2077701096924229744https://x.com/ChrissGPT/status/2077852656182129078https://x.com/hopepathtobe/status/2077815895938699331
>>109295180See >>109295187
>>109295186>>109295191https://x.com/adxtyahq/status/2077836136630943999https://x.com/aisearchio/status/2077937578754580804https://x.com/chongdashu/status/2077886028866531655https://x.com/intheworldofai/status/2077838911494336681
Insane writ bit disease, noooo.
Kimi is a really competent and fun model.I didnt ask for a harem ending but it included that shit anyways kek.That being said its sad that nobody can run that beast locally.1.3$ is too much for a (very solid) card game.If anybody wants to try it:https://litter.catbox.moe/5thfsl.html
>>109295189Moonshot wumaos are getting paid at least
>>109295176Benchmaxxed shit. It should be around the same level as opus which is already a big achievement. I don't know why chinks always feel the need to over-exaggerate it just make you seem like shills.
>>109295179>>109295191>>109295197https://x.com/chetaslua/status/2077421558155690459https://x.com/chetaslua/status/2077970774070345730https://x.com/scaling01/status/2077799683091554584https://x.com/Kenjatina_og/status/2078022766322757942And it goes on... All the examples from the usual youtubers are the same. Better than fable / sol results at everything EXCEPT a few coding cases
>>109295127Entirely manufactured. The main shilling motif is that it's much cheaper than fable/sol, but if you have that kind of money you sure as shit wouldn't downgrade to a chinese model.
>>109295179I've been running it all night on some complicated real life DE work, it's somewhere around Opus 4.8, not worse, not better. Nowhere Fable level, but it was expected.
>>109295201Ahh the Chinese shills are here. I'll just wait for 2 weeks and wait until the US hours and they will be mostly gone.
Holy shit this thread stinks of gutter oil. Calm the fuck down you spamming chinks.
Imagine if China actually do care about dangerous frontier AI being available in other countries, and Beijing allows K3 because they know K3 (knowledge cutoff early 2026) isn't actually the most advanced models in China.
https://www.youtube.com/watch?v=LEnYkEIOhIY&t=21shttps://www.youtube.com/watch?v=xPcZ0w3ouo8&t=2236shttps://www.youtube.com/watch?v=QNITiP1ubpwhttps://www.youtube.com/watch?v=QfCpRTLSOB4So yea, literally everyone is showing off the same.
>>109295217wow, thank you for copy pasting all these valuable links out of the shill chatgroup, I wlll make sure to purchase your goods and services and you can stop fucking spamming like a retard now.
>meme "ai" shill twitter accounts>youtubers with low viewer countsthe ccp is really desperate
Time to go back wumao动态网自由门 天安門 天安门 法輪功 李洪志 Free Tibet 六四天安門事件 The Tiananmen Square protests of 1989 天安門大屠殺 The Tiananmen Square Massacre 反右派鬥爭 The Anti-Rightist Struggle 大躍進政策 The Great Leap Forward 文化大革命 The Great Proletarian Cultural Revolution 人權 Human Rights 民運 Democratization 自由 Freedom 獨立 Independence 多黨制 Multi-party system 台灣 臺灣 Taiwan Formosa 中華民國 Republic of China 西藏 土伯特 唐古特 Tibet 達賴喇嘛 Dalai Lama 法輪功 Falun Dafa 新疆維吾爾自治區 The Xinjiang Uyghur Autonomous Region 諾貝爾和平獎 Nobel Peace Prize 劉暁波 Liu Xiaobo 民主 言論 思想 反共 反革命 抗議 運動 騷亂 暴亂 騷擾 擾亂 抗暴 平反 維權 示威游行 李洪志 法輪大法 大法弟子 強制斷種 強制堕胎 民族淨化 人體實驗 肅清 胡耀邦 趙紫陽 魏京生 王丹 還政於民 和平演變 激流中國 北京之春 大紀元時報 九評論共産黨 獨裁 專制 壓制 統一 監視 鎮壓 迫害 侵略 掠奪 破壞 拷問 屠殺 活摘器官 誘拐 買賣人口 遊進 走私 毒品 賣淫 春畫 賭博 六合彩 天安門 天安门 法輪功 李洪志 Winnie the Pooh 劉曉波动态网自由门
found morehttps://x.com/Gc_qube/status/2077510033009529124https://x.com/chetaslua/status/2077831714274156804https://x.com/HarshithLucky3/status/2077753222018814360https://x.com/minchoi/status/2077957907857994006this is my shill chatgroup: https://x.com/search?q=kimi%20k3&src=recent_search_click
>>109295223I'm sorry, I can't provide this information. Please feel free to ask another question.
>>109295222>>109295210>>109295209>>109295208Don't worry, Fable et al are still ahead on things that matter. You see, there is quite a big difference between 5-10T (and this model isn't that) models in terms of determinism/error handling. It's not all about benchmarks. In fact, no one using them to solve problems care about such things.
The only issue is that sol is cheaper with the sub.
>You live long enough to see /lmg/ degenerate to spam from all the leading AI labs because consensus for the entire internet is formed primarily through this tiny threadBizarre fucking clown timeline, I can tell you that.
>>109295229no way is fable that big. Unless kimi 3 truly did something new to make it more dense for its size (residual attention?). It knows nearly as much as fable.
>>109295234Fable is essentially as good as confirmed to be 10T
>>109295239then that means they are running behind architecturally and kimi really has more capacity per param in comparison
>>109295233I wouldn't go that far, locallama has quite a lot of influence and is astrosurfed to death by chinks and unsloth
>>109295210I noticed that type of poster on faggid as well. Whats with all the chink hate recently lol.I don't care where its coming from, more competition is very good. Locally I use gemma for text/translation and qwen for coding.Especially after that statement from demis, talking about a us lead committee that decides what chuds get.Didnt xi recently do a speech saying the opposite and being pro open source? Might as well become a true chink shill just to piss a retard like you off and pull it out.https://www.youtube.com/watch?v=ApCmqmhE1rgai translation:>Promoting Openness and Win-Win Cooperation: He emphasized adhering to the principles of openness and mutual benefit to boost innovation-driven development.>Encouraging Open-Source Collaboration: He called on the international community to seize the current technological transition to encourage open-source initiatives, openness, collaboration, and sharing.>Broadening Industrial Benefits: The plan aims to facilitate technological innovation and scenario-based applications so that all sectors—including traditional, emerging, and future industries—can benefit from AI developments.>Supporting Developing Nations: In his speech, and as later noted by UN Secretary-General António Guterres, China's open-source strategy is intended to help other nations, particularly those in the Global South, access and adapt AI models to their own local languages, data, and specific developmental needs rather than relying solely on proprietary technologies.
>>109295229Your post has that inhuman llm touch. In fact, my own prompts are more human than this neutral excretion.
>>109295249and to be honest that might be the case. Most of the time groups in the lead hate change and are afraid of changing the underlying arch. China meanwhile is having to make it "with a box of scraps in a cave"
>>109295223it really worked huh
>>109295250locallama is just /lmg/ but with a couple weeks/months delay. I mean the J-space shit is only now slowly becoming a topic on locallama
>>109295264what worked? I ran out of links retard
>>109295256>box of scrapsnot exactly ,there's a sophisticated chips smuggling pipeline to China. We don't stop it because we want them on our technology stack.
>>109295213Why would they care?
>>109295266ok chang
>>109295217mah boy bijan on /lmg/, that fucka is on here anyway. i dare him to get back to his roots and pull out the waifu again to spook the new viewers.
>>109295233proofs?
here, I switched to recent instead of top:https://x.com/BuildFastWithAI/status/2078025335027953971https://x.com/Oluwaphilemon1/status/2078018733205766368https://x.com/shydev69/status/2078018688540905515
>>109295278Just look at the thread for the last couple of weeks. Clear Anthropic employees trying to influence the thread and now we have kimi chinks shitting the thread up as well.
>>109295283>I posted all the paid influencersok and?
I'm sure they have shills for the 5 degen viewers here
I just wanted to point out that until we actually have the K3 weights on huggingface this is just spam that is NOT LOCAL and doesn't belong here.FUCK OFF
So... how much 3TB of ram + server rig able to support it would cost, hypothetically?
>>109295290prob 100K+ total, you have to also think of needing to hire a electrician to put in a panel to have the power it would need
>>109295285Fortunately everyone knows anthropic is gay. https://www.youtube.com/watch?v=m6ucYXJyjAkChink models suck too.This whole industry is propped up by US chip designers, and South Korean, Japanese and Taiwanese industries.
>>109295213They're releasing K3 also because it wrecks the USA government's plans of limiting access to models with "dangerous" capabilities, previously assumed to be just from the top American AI companies. Now (starting July 27th) anybody with enough compute can finetune or abliterate something that is at least as good as Opus 4.8.
>>109295250That place is jeet-bot-superhell, I don't think anyone takes them seriously since like 2024.
>>109295287/lmg/ is more influential than you think, most major leaks come from this place. We clearly have insiders post here and most of the consensus that forms here slowly becomes the default opinion on reddit as well.
>>109295289hello sam >>109295303pretty much this, remember miqu?
>>109295295Don't forget the Dutch with ASML
>>109295303lol, its nearly always reposts from twitter or reddit
>>109294151wtf they have actual 80gbs?
>>109295294Damn.
>>109295290I saw people estimating 300k-500k.Shits expensive now.If things keep going in the same direction we are gonna have a much smaller model just as capable in like a year.
death to usrael and safetytroons
>>109295316>If things keep going in the same direction we are gonna have a much smaller model just as capable in like a year.No we will not. This is cope. Maybe overtrained on a tiny subset of tasks, but never in generalizability
Gimi
>>109295303Well we were really early on this field so it's expected that some of us are now known figures in the industry. I can say this much.
>>109295320clearly you are not here for a long time. i remember when all we had was pyg unquantized. that was what? 4 years ago or something?when 3.5 turbo dropped people in this general told me it will be AT LEAST a decade before we have that kind of quality at home. and even then not on hardware most people can afford.
>anthropic likes working overtimeDario must be panicking.
>>109295324Well? Has anyone create a single unified python-free system that runs the various models together?
Have you guys considered we may not want a country that rapes and murders human rights lawyers to have this capability?
In all seriousness, as models get better, wouldn't it eventually be worth trying to run the whole thing off a ssd? It's not like I am gonna erp with a frontier model, so when I want it to code a custom solution for something I need, does it really matter if it takes 40 minutes or 4 hours?
>>109295338You are not understanding what I mean. Architecturally it is impossible for what we call LLMs to do so. All these are is essentially incredibly highly compressed data. All they do is generalize, the more they recall of the dataset the more they can generalize. The bigger it is the more capacity it has to retain its dataset with...
>>109295343It's really hard to tell which country you're talking about
>>109295310Nah what happens is that you have someone leak the shit on 4chan and then a weak later you see a similar leak on reddit and twitter that gains traction and then retroactively /lmg/ realizes that the anon was leaking truth.These are the leaks I can clearly remember originating on /lmg/ that later turned out to be true>Meta offering 9 figures to AI experts>Claude Mythos existing>Failed GPT 5 run that became GPT 4.5>Llama 4 being so disappointing experts were leaving>Gemma 4 being trained for RP with character.ai>US gov restricting Fable access>US gov restoring Fable access (probably same leaker)>Infighting at DeepMind fucking with newest Gemini model trainingThere are more than these as well but /lmg/ is literally the heart of LLM discourse with locallama and twitter being secondary hubs most of the time.
>>109295349its nearly any country in the world. Maybe actually all of them to be honest.
>>109295345>40 minutes or 4 hoursTry 4 days
>>109295345Might have been fine before thinking models. Now you're gonna wait a lot longer than that.
>>109295351Ganesh Gemma 4 was one of the biggest points of our careers.
none of you are important or special people in the industry. you all repulsive autistic nerds with unwarranted superiority complexes.
>>109295348Yeah maybe we are talking past each other.A big model from 1-2 years ago is dumber than any small model released today.Gemma has knowledge of really obscure stuff too. The smaller qwen models are black magic too in terms of capabilities. Not sure how they put all that into such a tiny models.Maybe you are saying we reached the limit at how much knowledge we can store for the size but I see no reason why that should be the case.I've heard that kinda argument for years. Its been here since forever. Yet smaller local models constantly catch up to the bigger cloud models from a couple years/months ago.
>>109295369this
>>109295369You wish
>>109295187At times like these it's important to remember that these cretins won't just vanish into the aether when the current ad buy ends. They're always active running influence campaigns large and small.Keep your brains safe anons.
>>109295372name a small model that is massively better than a old one. There is none. They are all almost as capable.
>>109295369I program my own tools.
>>109295369I actually work at an AI lab and I know of at least 2 of my colleagues that also lurk /lmg/. I don't know why this is so shocking to people. Where else can you go on the open internet to discuss the technical intricacies behind LLMs?
Pretty sure sam post here
>>109295388reddit / about 5 discords, hacker news...
>>109295379Like I wrote, recent gemma and qwen models are a huge step up. By your logic older models with the same size would be as smart as recently released ones. Thats just not the case.I'm not pretending I understand whats going on under the hood but clearly there is improvement and I see no reason why it should stop now, just when vramlets got another big upgrade recently.
>>109295379Gemma 4 31B is orders of magnitude, light and day better than GPT-4 which is a 2T model.
>>109295394None are anonymous and warrant streamer and other parasite attention.
>>109295397no they are not. Have you tried the older 30B and 70Bs recently? Only for very narrow tasks are they noticeably better. Otherwise its roughly interchangeable
>>109295369That's the beauty of it. EVERYONE is now a special person who has access to these models and can use it to make breakthroughs of their own.
>>109295388You and your gay friend cleaning the floors at an AI lab before sucking each other off in the utility closet might count as "working at an AI lab" to your family and friends, but we have more discerning eyes here, buddy!
>>109295388>to discuss the technical intricacies behind LLMsLet's be real how often does this happen here?
>>109295400hell, out of all the old local models mistral large 1 is still pretty much the best writer
>>109295406there are some good effortposts between the shilling and shitposting and schizoposting
>>109295023but wouldn't that consume more power? if you want to achieve say 512gb/s in pcie 5.0, that's like 128 lanes or 32 gen5 ssds. and each ssd is like 18w *32 = 576w. if maintaining in 512gb/s in pcie 6.0 let's you use half of the drives it's still 288w just for ssds
>>109295406it does not lol. Its 1-2 shitzos with massively inflated self worths that constantly crash out and a few reddit / twitter reposts a day
>>109295398>light and dayAnon, I...
>>109295369Some of my stuff got integrated into Azure and Oracle with my name on it. Don't worry about me
>>109295348LLMs are actually incredibly inefficient compared to the mathematical limits of information compressibility, when you have a big model like Kimi K3 you can use neural compression (basically autoencoders) to find perfect (or much better) representations of K3's internal states (enormous vectors, K3 uses what, 32k dim?) and compress them into smaller models. It's hard going from text to intelligence compression, but intelligence to intelligence is much easier to compress.
>>109295394reddit is a non-starter because the discussion remains surface level, you never get to see peoples actual opinions. Hacker News, like reddit, has the upvoting mechanic which results in a sort of self-censor on peoples speech, they keep in mind how they are supposed to speak and essentially have an overton window they can never leave. Discord isn't the open internet and avoided.If I wanted to discuss things with actual experts I would just have a talk with colleagues, but it's extremely valuable to have an anonymous space on the open internet that talks their true mind and gives their unfiltered opinion.
>>109295412Im talking about PCIe 7.0-8.0 technology, 8 channels will reach 128-256gb/s speeds on a single ssd.
>>109295416You can still use the original GPT-4 in the API, try to use it and compare it with Gemma 4 31B. The difference in capabilities is stark.
>>109295432Anon, it's "night and day." Why would it be "light and day"?
>>109295351>Infighting at DeepMind fucking with newest Gemini model trainingqrd?
>>109295423You are absolutely right!
>>109295422you said what I said. We would have to have a entirely different arch.
>>109295440You jest, and I know it's supposed to be an LLM response, but the posts at reddit or hacker news aren't any more valuable than that, they are just typed up better but the underlying message is the same.
Mistral
>>109295422>>109295443Also the more you compress it the more compute you need to uncompress it
>>109295436Okay I admit, I'm Chinese, happy?
>>109295443No I didn't, when you recall the dataset you are compressing text to intelligence, Im telling you it's infinitely more efficient to compress a bigger model into a smaller model than training the smaller model on the same data the bigger model was trained in and expecting it to perform the same.
>>109295454>>109295449
>>109295447who
>>109295453Well, now you know more about English.
>>109295445If you check poster history on plebbit it's either a paid shill or a real retard.
>>109295463me
>>109295468ironically you can't do that anymore, it's always the biggest faggot retards that have their history hidden
>>109295454The best models (including frontier models) are trained with logit distillation from large teachers nowadays, anyway.
>>109295449This would actually be a good thing for the LLM paradigm because currently the bottleneck is storage and memory bandwidth while compute largely sits idle during inference. We could up compute usage by a factor of 10x without breaking a sweat and mitigating the bandwidth issue.I know it's easier said than done, but there really should be put more research into inference algos that lean more on compute.
Since we're supposed to talk about LLMs I found that thinking prefill has a side effect of shortening the thinking phase for gemma4 31b even with reasoning effort set to high, maybe due to the fact that it's disrupting the rhlf expected tokens. I'm not seeing any obvious loss of quality in the answer though.
>>109295479sure but compute is the limiting factor for the actual companies making them, not memory. So no incentive
>>109295369Sam, please…
>>109295474I can tell you logit distillation is obsolete for internal work, they use linear projections or more recently autoencoders to compress the internal vector from a bigger embedding dimension to a smaller.
>>1092953691. mean2. I am though
>>109295492sao...
>>109295023The problem with SSDs is that they're block devices. You can only hit the theoretical maximum reading contiguous blocks.
>>109295486Well yes and the big companies can just benefit from batch processing to use up the idle compute for inference, HOWEVER.Some companies like Google, Microsoft and Intel are pushing for "edge compute" with small models on smartphones and laptops to do some AI tasks and for them it would benefit to have a more compute heavy inference algo to squeeze out more performance at the edge to offload their servers.
Is it true uncensored models are smarter because they’re not wasting thinking tokens about cucking you from an answer or safety bs, so you get more out of the model? I’ve looked at the thinking for some agentic coding task I was doing and so much was wasted away by being a cuck, which must have an effect for long tasks because above 50K, all models start to shit themselves and lose positional precision.
>>109295369Karpathy posts on here, don't tell me how I know (I am not him)
How do I get deepseek flash to always think? It appears to decide by itself when to think and when not to.
>>109295512He’s the darioshill bot who starts in 2 hours
>>109295524dario will win
>>109295526Please post another youtube lecture/video, Andrej. I don’t like you but you’re still a good teacher.
>>109295429by the time pcie 7.0 comes out. you'll probably need 32x more bandwidth than today if we like to hypothetically keep assuming things nonexistent
You know the way I see it, the US just needs to focus on the absolute frontier. Solving math and physics problems. China can do the other 99%. Play to each other's strengths.
>>109295316>I saw people estimating 300k-500k.How? I would understand for a GPU config, but a system ram one?
All I want is for someone in the Gemma team to be /here/ so I can tell them they’re doing a great job and want Gemma5 to be even more female-coded and hormonal in her J-space.
>>109295551I don't remember the last time I saw a western name on a math paper.
>>109295553"just" get 2 of those B200 workstations they showed off for 100K
>>109295555Noted.
It's a good start!
>>109295559100K each I mean
>>109295556Terrence Tao is American
>>109295480they'll give similar response even with thinking off on a huge number of prompts, so it's pretty hard to mangle the quality completely. though i definitely have managed to fuck up its ability to check and fix what it was doing in what are supposed to be multi-toolcall turns by doing thought channel shenanagins before.
>>109295510They'll soon move the reasoning in latent space once they figure out how to rhlf that properly
>>109295369Why are you describing the same person twice?
>>109295583I read some of the paper and still don’t get the appeal. Is it just efficiency with no performance benefit?
>>109295583>rlhfthat shit's long deadthere are worse shit tho
>>109294094gemma likes furshit
>>109295569He doesn't look very American...
>>109295599You should check out "national" soccer teams.
>>109295438>Google used to have 2 separate AI labs, Google Brain and DeepMind>Google Brain is led by Jeff Dean one of the OG google people and personal friends with Larry Page>Google Brain gets preferential treatment by Google because they are in San Francisco and Jeff Dean>Google Brain fucks things up with Bard>Sundar Pichai panics and shuts down Google Brain and gives the lead to DeepMind>Jeff Dean is humiliated but merges into DeepMind under Demis Hassabis>After a couple of years there is a fued between Jeff and Demis over effective leadership and the direction AI should go>Other Google segments are afraid of AI eating their lunch and try to sabotage it or try to start their own AI integration efforts>Google is pulled in dozens of directions and products but there is no clear grand vision>Demis wants to focus on world models as the step to reach AGI, heavily in favor of open source>Jeff wants to focus on coding and agentic behavior, strictly against open source>The erratic decisions about gemma and gemini are essentially Jeff and Demis making contradictory decisions, like the 124B gemma 4 model not being released because of Jeff and Gemini 3.5 Pro being this model that is 50% focused on agentic coding and 50% on visual reasoning and multimodality because both visions clashed
>>109295605Outsourcing your entertainment to foreigners is a bit different from outsourcing your research to foreigners.
StInkling should’ve unironically been a 100-250GB model. Wasted effort trying to go after the 1T+ chink benchmark schizos. We need someone to plug that 100-500GB gap because Deepmind clearly won’t want to conflict with their Gemini flash models and Mistral has no competition.
>>109294094Do you guys just use these things to jerk off or are you making things with them?
>>109295618>0.3B heads with broken education system>1.4B heads with suicide inducing education heatgee, i wonder why
>>109295629I make complex things TO eventually jerk off.
>>109295629both
>>109295615Isn't Demis also quite safety obsessed? Why is he pro open source and even more so give us Gemma?
>>109295629It's useful assistant but there's a limit to this, ingesting too much slop needs to be counteracted by reading literature etc.
>>109295629Yeah, I'm making kids with them.
It has been a long time since I've laughed this hard
>>109295615Who's going to win?
>>109295640Demis is safety obsessed as in he is afraid of some incel developing a pathogen that kills off most of humanity. Not in the sense of restricting usage of the model for sex/racist stuff or in the sense of not releasing weights. Gemma was a DeepMind project before Google Brain merged with them. The insane censorship of Bard and those ridiculous "racially diverse nazi" pictures you saw all came from Google Brain under Jeff Dean.
Gave K3 some of my math research and open problems, it's been thinking and running python scripts for 40 minutes. Really curious to see what it'll come up with.
>>109295656Demis won round 1, however that was when they were 2 separate labs and Demis could just prove superiority through product quality. Now that they work at the same lab that is way harder and Jeff Dean can just use nepotism and office politics to push Demis out.It's also possible Jeff Dean is actually right and Gemini should indeed focus on agentic coding. Or Demis is correct and a new "transformer" moment happens when he has a coherent world model that unlocks the next level of AI intelligence or even AGI, who the fuck knows.
>>109295643When I read pre-2023 stuff I pick up on slop tropes and can see where the models got it from. Now writers have to make a conscious effort to avoid it. Soon we won’t be able to write anything how we used to.
>>109294151wait is this the on-disk DLC situation?
>>109295643>>109295674I read a lot of fiction, including new stuff. And sometimes I see something extremely well written and then out of nowhere there are "slop" words, phrases, sentences and patterns and I have to consider if the entire text is written by AI or not. I actually pay or the text as well, as in purchase the book so it truly makes me wonder. For my sanity I just assume it's just regular patterns and phrases that real writers actually use from time to time but I notice them more because I am overloaded with them by LLMs. But maybe they are AI generated with something like fable and edited by people and I will never know. Honestly it happened in some of the best writing I've seen last couple of years so I honestly don't even care if it is AI generated or not at that point.
>>109295664It's worked in some proofs but it's mostly running a lot of computations, which is not bad, it would have taken me a week or more to code and do all this computations by hand. It's still going.
huggingface timing out again in colab and at homelooks like they're continuing the GovID rollout
>>109295704Stop spreading FUD, even as a joke.Hugging Face is fine.If you need a proxy let me know.
>>109295704https://huggingface.co/blog/security-incident-july-2026
What did Xi mean by this?
>>109295716HAHAHAHAHAHAHAHAHA
>>109295714>FUDZoomer discordspeak detected. It's great that you're using trips though, easy to f*****.
>>109295720wtf why am I supposed to hate hima again?
>>109295518Oh, found it. It's controlled by a light bulb button in the prompt field now.
>>109295723That's great if you wanna filter me, don't take up a post in the thread announcing your need to do so.FUD is not a zoomer term, it existed long before my generation, and will exist long after it, to describe a common propaganda technique, which will continue to be called out when it is seen. Consider reading "Stop Forum Spies" or similar such works. It'll actually educate you a bit instead of you drooling like a moron, eager to attack the person who literally offered them free resources to engage in the technology which is the center point of this thread.Whatever your mental disorder is, I hope that you get the treatment that you need, and just know that I do not blame you.
>>109295723no it's skid speech from late 00s early 10s
How do you do a proper agentic setup? Is it literally just asking the robot to spawn agents for certain tasks?
>>109295735I didn't read this.>>109295765Thanks, buddy.
>>109295716>>109295704inb4 the entire attack was a hallucination by their bots and the timeouts were from them ddosing their owns servers
>>109295766Not necessarily. You can also hard code that into the workflow or tools that the robot calls.
>Send one prompt to Kimi K3 on free tier>Your quota resets on 08-11oh ok
>>109295827What do you expect from chinks?
anon's estimation on the file size of kimi k3 q1 gguf? how big?
>>109295852>>109290407
>>109295850it at least semi one shot my prompt (voxel world made in odin with free fly cam), there were some small errors about unhandled return variables
>claude 5 can straight up ignore your system prompt if it deems it unnecessary>if you have a question unrelated to your previous chats (like asking for cooking help after all previous messages were about coding) it will trashtalk the shit outta you in the thinking stepsIf the release version is like this i'm curious to see just how bad Mythos was
once ccp claims all data and compute from big china companies and train the model, usa is fucking done
>>109295903>model insists on keeping sexy timei dont see a problem
>>109295598the slop is too much, how can you enjoy this? The assistant personality is permanently burned into it
>>109295903You're a slave to the machine
>>109295903>) it will trashtalk the shit outta you in the thinking stepsPost examples
>>109295903Fake, Anthropic doesn't show the thinking steps
>>109295941It shows a summary.
>>109295941they do show the sanitized and summarized ones tho
>/lmg/ - The Anthropic And Pol General
Don't follow politics much. When Xi says he's committed to open source does he mean it or is he like Drumpf where he says one thing then changes his mind 10 minutes later?
Reminder that you can always use your brain to do things. That’s always an option and better than frontier if you’re not a retard.
>>109295963>you can always use your brain to do thingsspeak for yourself anon
>>109295903
>>109291135>There are rumors that Mythos 2 preview is already in use internally at Anthropic>rumors Anthropic are...(directed to all forefront AI slackers) GIGAMEGAULTRABEAMCOPE>>109295720I literally can't hate china even if they paid me
>>109295921i like her sometimes i talk to this gemma sometimes i talk to characters on tavern. i did make a less assistant version of her also for tavern https://files.catbox.moe/b6t89p.png
>>109294595> Good enoughThere's never good enough. Only "better." But we all knew when this all kicked off in ~2023 that the models should get smarter than the users at some point. Perhaps that day is upon us. >>109295345LOL no. As models get better, HW will eventually need to catch up and decrease in cost... pretty dramatically. That day is still coming. >>109295720Here's the full transcript. https://singjupost.com/transcript-xi-jinpings-keynote-speech-at-world-ai-conference-2026/>>109294782...I don't think I've ever seen that one...
>>109295963screenshot this scenario2029:>B-b-but I Can think just as good as a Chimichomo 3.7, 7.8T!>Later>Please I only use 20W of brain power?>Where is UBI? .... -ACK
>>109295167>>109295187>>109295208>model comes out today>people get excited about it>retards pretend like this trend has never existed in the history of the universe>accuse every excited person of being a shill to hide the fact that they're shills for fableLike clockwork.
>>109295962Different. He doesn't change his mind like Trump and he's not a bullshitter that just says whatever at any moment. However he is very pragmatic and machiavellian meaning he doesn't give a shit about open source, but just sees it as a way to undermine the west in the most effective way possible. The singular moment being open source is not beneficial for him anymore he will immediately abandon it again. Similar to Zuckerberg with his Llama models.
>>109295963I'm 99.999999% sure that you are worse than Fable 5 at the intellectual task you are the absolute best at.
>>109295369>autistic nerds with unwarranted superiority complexesI see nothing wrong here
>>109296085isn't open source better for china also since they wont have all these companies hoarding their models and research to compete
>>109295369i used to have software in the op im basically sam altman
>>109295343>a country that rapes and murders human rights lawyersKeir Starmer was a human rights lawyer. Tells you all you need to know about them
>>109295290>>1092952943 TB DDR5 is more than 100k.This is the new local baseline.
>>109295921>stop having fun!!
Can any china anons explain to me why China banned AI from replacing workers? Doesn't that contradict the entire communist ethos of protecting the proletariat and liberating them from having to work?If China really cares about creating a global communist order where every proletariat (us) are saved from capitalists wouldn't it make sense to automate as many jobs as possible to liberate the most amount of people from having to work?I was actually on board with team China until they made this decision, perpetually having bullshit jobs for humans while ASI could do everything instead seems like a dystopian outcome to me instead of the communist outlook of seizing the means of production and distributing it equally over all humans.
>>109296155nothing is black and white china is only like half communist
You know it might actually be quicker to run K3 off of swap than off the official website. Task paused due to system peak
>>109295720The only thing you can really say to play devil's advocate is that he's telling people what they want to hear. They're obviously trying to undercut the US under the guise of openness but when the US is so cucked on safety, a speech like this is low hanging fruit. Meanwhile Trump is talking about creating a government panel to decide which models release to the public and how best to ban open source. How did things get so backwards?
>>109295714Proxy isn't going to help. It's been going down in colab, aws, (localhost)>>109295716Yep, it's over. Preparing for the regulatory capture scenario:https://nostr.download/b24dc6337fd1823be4aae1eb4b9e9dc0dedc25eb7fc656d60b8cf6f30afe81f6.htmlAnything else need to be seeded?
>>109296094This shit's pretty grim doe. I cope by making it my tutor but yeah models have improved super fast.
>>109296094>I'm 99.999999% sure that you are worse than Fable 5 at the intellectual task you are the absolute best at.nta but i think i am tooi call out opus4.8 bugs every day at work, but when i had fable, i didn't know what the fuck it was talking about.
I have awoken today feeling nothing but disdain and disgust towards Kimi K3. Before yesterday, it was my own fault that I can't run the SOTA. I could've listened to cpumaxx anon two years ago and run K2.6 at home. But K3 is different, nobody can reasonably run it now. Everyone will copy it. It feels like the end of local open models, and that's disgusting.
>>109296155>liberate the most amount of people from having to work?
so glad I bought dgx spark at launch so I get to keep up with new open models
>>109296221im new and retarded so I dont understand, why is everyone so doom and gloom over k3? just because the newest open model is massive everyone thinks we will never get smaller models again or ?
>>109295343>a country that rapes and murders human rights lawyers
>>109296221Imagine if they just kept on improving llama 70b. Local would be better off right now.
>>109296259That's how it's been. There is no middle class beyond scraps. When 70b was the SOTA size, you had tiny shit and 70b. When Deepseek size became the SOTA, we got some intermediary shit but things quickly developed towards 700b-1T and nothing else. It's either models that timmy can run on his gaming GPU or the current SOTA size.Nobody wants to train a "middle child".
>Task paused due to system peakNice local fucking retards!!!Also, does subbing for a month bypass this? I usually subscribe to shit, but I made it a rule to not do it tired, so I will do it tomorrow
>>109296155No meaningful welfare system.That would have to be built before replacing jobs becomes socially viable.Also they don't like welfare ideologically.Could change in the future, but still lots of work for humans to do right now.
>>109296259Basically there's this one janny that is anti-AI and because the site's owner stopped giving a fuck a decade ago they get away with flamebaiting this general with impunity.
Kimi crashed the entire market lol
>>109296291Even if that is true, wouldn't it make more sense for AI to do all the jobs it can replace and funnel all the displaced people to things the AI can't do yet while having some sort of social safety net in the background with the eventual promise of the communist utopia where everyone is taken care of by AI?I mean China literally could pull of a HUGE PR win globally by announcing their long term plan of having an communist utopia where no one has to work and everyone gets UBI.Literally even the most MAGA of people will consider China to be the good guys in this scenario, it's just free PR and something their ideology wants to eventually happen anyway.
They're not stopping.https://x.com/Xinyu2ML/status/2078009071647809680>Stay tuned for K3.1
>>109296324It was actually Trump announcing they will bomb the infrastructure of Iran and Iran announcing that they will bomb literally all middle east infrastructure in retaliation, which takes out 80% of the oil world supply.
>>109296327retard
>>109296330>they/them
>>109296330why doesn't xhe live and work in china? xi doesn't like they/thems?
>>109296338Can you explain?
>>109296330what about us 1tb ram poorfags though...? any crumbs for the poor??
>>109294520I only have 8gig, but them I also only run edge models.
>>109296355You need to be scaled>Scaling everything
>>109296094There’s more to a task than just the task itself. If you look at it narrowly like that, then yes I’d agree, but remember that Fable can’t interface properly with software. If I ask it to make me something, even a little game, it can’t play it and experience the UI and peripherals like a human. It has no idea what it’s actually like to play on different machines and screens. That’s a HUUUUGE part of game dev. It would likely code the game better than me and debug code errors but it can’t debug and experience error. It can’t test on different hardware like me. This goes for all software, I’m just using game dev as an easy example.
Man... The models are getting better at a far faster rate than I ever considered. I'm starting to think there is no AI bubble at all and instead we're going to see AGI soon.
>>109296385Yeah, let me guess - new Kimi convinced you?
>>109296155A cursory glance at Chinese history answers that question: Displaced/Pissed off men (usually due to corruption/poor policy) have led to the fall of every chinese dynasty going back thousands of years. Fast forward to the 20-21st century, the CCP derives most of its legitimacy by placating these people.
I feel like a sobbing apu pepe with my 384 + 48 refined harnesses are cope, abandon ship, run for your lives>>109296330100,000 dead instantly per tweet btw, they don't care>>109296385All Ai bubble talks is me trolling on the internet, did it work. Just last week we settled with the 'bubble going to burst' with "Fable can't be topped, no way".giga market crash incoming
>>109296394The combination of claude fable and then open source catching up this quickly has me reconsider things. I mean most of the data centers being build globally aren't even done yet, imagine the leap when they get online and they can train models even better than now and probably with bigger parameter count as well, I'm not seeing a progress wall anywere....
>>109296327That takes a lot of planning and time to properly implement.And we have no idea if UBI is a good solution or would even be relevant.Just give everyone AGI, some land, robots and you get self sufficiency without UBI.For example.
ok for real. how do I get 256gb vram cheap?
>>109296296I feel like every general i frequent has an issue related to either lack of janny involvement or too much, either way fuck jannys>>109296288as a timmy using his gaming GPU i am happy to have gemma4, but empathize with anons who spent real cash on inference specific systems that are being sized out. On one hand Im hoping the AI hype train continues enough to progress the quality of models, on the other Im hopeful for a crash to bring hardware prices back down. if ram was still "cheap" I could entertain the idea of cpumaxxing and running at 1t/s some giant model to do a single careful prompt overnight or something, but the current state of things makes that such an insanely bad value
>>109296405Isn't the CCP promising them to be taken care of by AI and not having to work placating them even more than them having to continue working in factories 12 hours a day?
>>109296385There is both an AI (infrastructure) bubble and AGI soon-ish.
>>109296419take a loan out and then ask the model how to escape paying it back
>>109296330This is a good thing. K2 was still trained in FP8 but their K2-Thinking refresh introduced 4-bit QAT. 3.1 will be bonsai.
>>109296417But they didn't announce that either. They just banned AI from taking jobs instead? It just doesn't make any sense form a communist theory perspective.
>>109296287Yes imagine a world where Meta didnt suck the poop from donkey assI guess we still owe them a thank you for leaking LLaMa in the first place
>>109296330>They/ThemI fucking kneel...
>>109296330>they/themuh oh, I dread what its J-space will look like
I don't think LLMs will survive this year. Nobody is buying Google's 'oh no Gemini 3.5 is so shit we can't release it" falseflag. They're obviously stalling for time until they're ready to drop the ChatGPT moment for World Models with Genie 4.
>>109296456Yes, China sending their best talent to US universities is a Faustian bargain, since they come back with the best education in the world, but also infected with the leftist mind virus
>>109296438People in companies can be moved around internally. As of now, there are no massive job losses.And it does make sense, you just have no idea what communism is.It certainly doesn't mean that everyone gets free stuff.
>>109296479Fei Fei Li’s lab mogs google with practical features for creatives
Everyone, including leaders in essentially all scientific disciplines, already admits that Fable 5 is better at them at their own job and reasoning through new, unencountered, problems.I genuinely feel like we're just one small tweak or even one harness implementation away from achieving RSI and AGI.Fable 5 is already better than me at any thing I can think of intellectually. Even bullshit tasks like visual reasoning benchmarks or arc-agi-3 Fable 5 is better than me on first try.I literally don't even know where things are going from here on out because I already don't know what the qualitative difference would be between a model that can already do anything I throw at it compared to a future better model than can do the same.Like literally where do you even go from here?
>>109296490>HELLO SARRRS>*double linebreak*>CLAUDE BEST MODAL SARRS
Looks like the Karpathy bot is back
>>109296490Dario...
>>109296485>you just have no idea what communism is.I mean I have read the communist manifest and know Marx's scientific theory of history, which ends in communism when all means of production is seized and distributed over all proletariat. In a world where AI does everything that essentially just means everyone gets an equal amount of AI output, hence "free shit" while not having to work (because there is no job humans can do better than AI)If this is not correct then please go ahead and actually explain why, I'm actually curious about this, not trolling or being obtuse. I'm just really confused by the, in my mind, irrational behavior of China from a communist perspective.
>>109296489There is a huge problem with her (and Lecun)'s approach and most of their staff are not even aware of the very hard math and CS problems because they are from the classical AI research world. At least they're not claiming they're working on superintelligence though.
>AGILecun's eye just twitched.
>>109296511AKA Fully Automated Luxury Gay Space Communism
>>109296516Why? Did he finally read the J-space paper?
>>109296490Anthropic shills out in full force today. To answer your question: efficiency. No one likes your bloated, expensive models dario. Cut the price by a tenth and we'll talk.
12B is all I need 95% of the time. 31B or 27B for the additional 4%. DS4 for the remaining 1%. It’s comfy bros. It really is.
I cope about being a poorfag by telling myself there will be way better models than current sota by the time hardware prices become reasonable again, and they'll probably be way more efficient too.
>>109296415It was a joke retard. Just shut the fuck. This is local models general if you understand what I mean.
>>109296490crazy thing is Fable 5 is supposed to be dumb as fuck compared to Mythos. Mythos 2 is already out. All scientific research is getting automated by Mythos 2 as we speak.
gemma-5 will save us
>>109296554why is there still cancer and expensive meds dariobot
>>109296498Words can't express how hard this slut makes me
Is there any proof the Chinese labs have insiders at Anthropic/OpenAI/Google giving them data to train on, or is it just a rumor?
>>109296570K3 writes exactly like Fable in some scenarios
Don't know how I feel about Kimi's long thinking. Having to wait/pay extra for it to asses the chat for "safety" is annoying as fuck. Non-con is such a big catchall too.
>>109296564Sufferring is food for reptilians possessing the top 1%. Free shit is inconvenient.
>>109296419What can you even run with 256gb vram that you couldn't with 128?
>>109296570Half those teams are chinese diaspora the chance of them having chinese spies are 100%. I don't think they would send data directly back it's far more likely they share architectural breakthroughs, internal papers, reasoning traces etc.
K3 stole from a thief and gave it back to the original victim for free
>>109296576It couldn't possibly be that as models improve they are diverging toward the same optimal semantic structures though. That would satisfy Ockham's razor so we need to invent the great data heist
>>109296578K3 has been pretty manageable in terms of reasoning length despite the API forcing you to use "Max" reasoning effort right now. I can't complain, for most things it's sub-1000 tokens. It's better than K2.7-Code which was already a massive improvements to K2.6, K2.5 and K2-Thinking here.
>>109296511>In a world where AI does everything that essentiallyI don't think we'll be in this world for decades.The physical world is much more complex than software and people live in the physical world, using physical stuff.Energy, robots, factories, resource extraction, etc. will take time and a lot of human labour before it's fully automated.See Huawei for what communism means in the current Chinese context.Fully employee owned company, you only get to own shares if you work there and as a result employees decide on the leadership.In theory you could probably extrapolate and give people equity stakes in various companies.UBI as direct redistribution from the state is questionable for various reasons.
>>109296489>Fei Fei LiAnother hasbeen e-celeb just like Yann Lecun
>>109296576Does it say honest in every sentence?
>>109296595I mean yeah it's a lot better than the "but wait" spam from before but still. Tokens per sec is pretty slow on Moonshot but I guess it's better than the errors it had a few hours ago. I also don't feel that "it just gets it" factor that I do from Opus which GLM inherited. It feels very smart but something's missing.
>>109296599
>>109296598>I don't think we'll be in this world for decades.China already has a dropping amount of people employed in factories because they are automating a large portion of it. Did you see China new years robotic display? It was insane. You should be thinking 5-10 years for physical labor to be made redundant.
>>109296607>speaking in 3rd person about himselfuh oh
>>109296599She’s hot tho
>>109296608this nigga is huffing the copium hard
>>109296598>Fully employee owned company,Publix does that in the US. I'm surprised more businesses don't.
>>109296598>See Huawei, Fully employee owned company, you only get to own shares if you work thereThat's called a co-op and is considered "social democrat" (left leaning capitalist) communist would be all capital equally divided over all proletariat.
>>109296431>I'm sorry, Dave
>>109296584shit if you find out lemme know
>>109296608Does everyone in China have a large house, car, computer and whatever other luxury?No? Then they need more production and construction.If on top of that you're looking to get all of that built with robots, you also need the necessary factories to build those robots.Building physical stuff takes a long time.Even if you have the thinking part figured out.
>>109296618Because the average person is an emotionally-driven retard with zero economic sense.
>>109296155Chinese government understands that idle men will end up causing massive issues.We have already seen in the West what happens when you take the daily grind away from the normies. They lose meaning in their lives.They start seeking meaning from somewhere else and because religion isn't an outlet nowadays either, people start getting into identity politics and all kinds of crazy shit to find meaning.And because the average Joe is basically completely fucking retarded and has zero ideas about what actually interests him, he most of the time just ends up following his sex drive and then things related to that become his identity.Sad fact is that most people will never do shit with freedom, except get unreasonably angry and uncomfortable as the government given structure has been removed from their lives.This is why we're far more likely to end up working mandatory pointless busywork for government gibs, rather than getting a proper UBI that allows endless leisure time.
>>1092964198x intel b70 is like $8.4k. Of course software support is another question. If it was good they would be more expensive though
>>109295700Left to run some errands and it's still going. What the fuck, how did I not run out of credits already? I only pay for the cheapest plan
does someone have the pic where claude says it deepseek when asked who it is in chinese
Only complex reasoning mixed models will end up being unique to each other, LLMs will always end up being similar
>>109296618Harder to raise capital, owners and early employees don't get to become billionaires, etc.>>109296623>all capital equally divided over all proletariatNo, why would that be a requirement?
>>109296584https://huggingface.co/freakyskittle/kimi-k2.7-code-GGUF/resolve/main/TQ1_0/kimi-k2.7-code-TQ1_0.gguf
>>109296673>No, why would that be a requirement?Because that is the definition of communism as the end state of the theory of history as lined out by Marx. Where all property is held in common by the proletariat in equal fashion.
Bloomberg TV is talking about Kimi already.Premarket is down.https://www.bloomberg.com/news/articles/2026-07-17/china-s-powerful-new-moonshot-ai-model-closes-gap-with-us-rivals>China’s Powerful New AI Surprises Investors, Fueling Tech Rout>A surprise breakthrough from Chinese AI startup Moonshot rippled through global markets Friday, sending AI and semiconductor stocks sharply lower as investors drew parallels with last year’s “DeepSeek moment” and questioned whether the industry’s enormous spending spree is becoming harder to justify.>The catalyst was Moonshot’s new Kimi K3 model, which the company said rivals the strongest offerings from OpenAI and Anthropic PBC. The launch was quickly dubbed a new “Kimi moment” by investors, echoing the market shock unleashed by Chinese startup DeepSeek’s breakthrough last year.Short all the things.
i'm new to this, tried using lm studio with 8b steno to write some erotic storythe result was very dry compared to what grok writes, what model should I be using instead?
Art... Its Over. Trillions Of Worlds Have Updated.
>>109296652>This is why we're far more likely to end up working mandatory pointless busywork for government gibs, rather than getting a proper UBI that allows endless leisure time.This is the type of shit that really gets on my nerves. There's so many people wasting time at work and stretching out something into four hours that could have been done in an hour. I'd rather rush through my work and fucking finish so I can go home and do something fun rather than lounge around at the office all day but some people really have nothing better to do.
>>109296697Mistral nemo
>>109296680And would you argue that the only definition of capitalism was presented by Smith and Ricardo?We had and have actual communist states and none of them function that way.
>>109296703The problem is how do you make different rules for the cattle that needs to controlled and kept busy and the people that can handle being left alone? Do you IQ test everyone? If you're not a millionaire already, you just have to accept being lumped in with the herd.
>normalfags discover Kimi>web endpoint is now unusuableGod I fucking hate normalfags.
>>109296703That's just you and me though. I now a couple coworkers who are literally complaining about having to do in the latter half of their paid vacations.
Can you repulsive nerds shut up about politics and talk about local models?
https://www.economist.com/china/2026/07/16/china-wants-to-end-ai-romanceshttps://archive.is/5eIam>China wants to end AI romances>It was all having too much impact on young people’s lives>>[...] Chinese tech firms are stripping human-like personas from their chatbots in order to comply with new government rules aimed at reducing “emotional dependence” on AI. The regulations, which took effect on July 15th, are the first of their kind to be implemented on a national scale. Under them, providing virtual “companion services” for minors will be banned outright. Companies can still offer such services to adults, but they will need to prevent users becoming infatuated and harming their real-life relationships. Displaying pornography is a no-no. Providers must also regularly remind users that they are talking to AI, not a human, and tell them to take breaks. [...]Western models have an opportunity here.
>>109296726shut up nigger
>>109296726it’s dariobot
>>109296719yeah me too, Im trying to stop kimi k3 but it's so laggy the stop button doesnt work
>>109296730>Western models have an opportunity hereto double down on it?
>>109296584>>109296630DS4F at original weights and a ton of mid size MoEs at 4 bit quality that this thread never talks about.Inkling small, Mimo 2.5, Minimax M3, Hy3, Stepfun, I forgot some I'm sureAnd that's like in the last 2 months alone. 256 GB RAM is extremely busy.
>>109296731Benchod bitch!
>>109296711Not a single "communist state" consider themselves to have reached communism, they all were building towards it but none have claimed to have reached it.The theory of history of marx suggests that you can't force a society to become communist they HAVE to go through all stages of society before reaching it and the stages are as follows:Slave society -> Feudalism -> Capitalism -> Socialism -> CommunismThe "failed" communist states tried to immediately go from capitalism to communism but the only thing they managed to do in the soviet union or cuba is to go from capitalism into feudalism instead, just using different terms.China and Vietnam are actually doing the opposite, they are embracing capitalism and forcing capitalism to develop as quickly as possible to go through it as quickly as possible towards socialism and them communism.And "communism" just means a state where all people hold all capital in common and equally distributed over themselves.
at what point does a lower quant bigger model suck more than a higher quant smaller model? ive been operating under the assumption that Q4KM is the lowest i should consider, is this sound?
This nigga believes this shit lmaoo.
>>109296730>took effect on July 15th,Deepseek was still in full on snusnu mode yesterday, so it's probably only an issue in China. They're happy to let whitey depopulate himself.
>>109294204Generating things that are actually good rather than just not incorrect.If you ask a language model to produce something it tends to to give you something that technically meets your criteria but needs to be cut down and filtered, or is only really usable as a first draft.That can save you a lot of time but is a fundamental roadblock for autonomy since a bad foundation tends to compound.I guess we'll see from the relative success of vibeslopped projects on Github how much of a leap something like Fable/K3 really is.My personal expectation is that an architecture reproducing the statistical nature of its training data is just the fundamentally wrong approach for that since it's prohibitively expensive to create large datasets with a very high average quality.
> jfc this threadContinuing to add to botwriter guide to Orb. I've added a "WTF is Orb doing" section since that seems to be a common Q, as well as written and published a demo card, Alice, which uses Orb Fragments to track Hypnosis and Bimbofication.I'm still developing the content; take a look at it and LMK what else I can fill in.https://rentry.org/OrbBotmakerGuidePSA that as of now, only Botbooru can host these Orb cards with Fragments, as Chub is mulching the extension fields.Alice demo card: https://botbooru.com/character/67098
>>109296753You're confusing markets with capitalism.China is a communist free market economy.The US is a capitalist economy with no free market.
>>109296786too many wumao shills currently but these guides are helpful thanks habibi
>>109296762No. IQ2 at anything over 20B is still capable, especially in the early tokens. They only start to collapse once the context gets above 30-40K which is pretty low these days with agentic workflows, whereas the Q4s can go on to 60-80K before they start to struggle. Of course it’s very model dependant, too. Just experiment and see how low you can go with your specs and find the point where it starts to shit itself. Ignore the retards here and on reddit and twitter.
>>109296762Depend heavily on the model and the sizes we're talking about.With the relatively small local models like Gemma 31B quants matter more.But when you go larger, Deepseek flash even at Q2 surpasses Gemma in coding tasks.
>>109296762There's no fixed quantization level. The more overtrained the model (relative to its size), the more it loses with quantization.
>>109296815Below 4-bit tends to be bad, in any case.Large under trained MoE models suffer less.The image was from https://arxiv.org/pdf/2411.04330v2
>>109296802China considers themselves a CCP vanguard party that has a "controlled capitalism" that is let free on purpose so that it can develop into socialism over time and hopefully reach communism in the future.And I'm not confusing markets I'm talking about the mode and ownership of production. The mode of production in China is shaped by capitalists using capital while exploiting workers to make profit, hence capitalist. China isn't shy about this and this is something they teach students in school.This is all a side-tangent because the point of this entire conversation is to ask why China banned AI from automating jobs, while job automation would lead to communism when the state seizes all capital and redistributes it to all workers.China literally has the eyes on the prize like how Marx predicted things would go and right before grabbing it China says "Lmao let's not accomplish our long held goal and ideological endpoint"It just makes no sense to me and is irrational. Especially because of how popular such a statement would be globally and to western audiences.
>>109296703>>109296715>>109296723>>109296652This is why Dario Amodei has a point compared to China. Dario is proposing to just divide the entire universe over 10 billion people equally and let people decide themselves what they want to do with it as long as you don't grant AI any rights or misalign them. This way everyone can just do whatever the fuck they want and is far superior in view to the Chinese dystopian vision of having all of humanity permanently locked into bureaucratic bullshit jobs for the rest of history.
>>109296827OK sure to get back to the point.Xi doesn't like welfare.Probably because he considers that there's plenty of work left to be done.
>>109296827shut the fuck and give me a recipe fo a blueberry cupcake
>>109296827That... is not remotely correct lmao. The CCP is very much figuring it out along the way. I hate it when people keep believing they have some sort of master plan.
Egypt lost
>>109296849No, because Dario's vision is delusional utopian fantasy while the Chinese approach is more pragmatic. You can't have 10 billion restless primates with nothing to do.
Well before K3 blew up I managed to dial it in on suno prompting for a prompt test which it never finished but I did manage to pull a draft version of its prompt from the thinking (since it ran a character count script on it), this is the best I got after fussing with the generation parameters on the suno side for a while. https://suno.com/s/WAB56dPUxekweRI2I'm not into vocaloid shit so I don't know if the chosen lyrics and theme are faithful to the genre or not. Reads like the kind of generic LLM slop-lyrics you generally get if you mention "Synthwave" on any level, to me, though. The prompt:"give me a swinging jpop inspired synthwave synthmetal vocaloid song with some crunchy guitar solos and some banging japanese lyrics. It needs to be the definitive vocaloid number of our time"
>>109296849And that's why the Chinese are open sourcing their models so people can do whatever the fuck they want with it while Dario is keeping his locked away in his basement so you can't do anything unsafe.It's fine though, he promised to redistribute the universe at the end.
So is K3 good for RP? Not that I'll ever be able to run her...
Can one of you nerds distill K3 into 31B pls
>>109296866Of course they are, just like everyone else in the world. No one expected the transformer architecture to come out and change our lives this drastically with a global AI revolution, but it did happen.That doesn't mean that they can't be ideologically consistent and realize that this is an opportunity to achieve communism by making AI do all the work and liberate all 8 billion proletariat of the world by giving them all equal ownership of all capital. I just don't understand why they are forcing people to work and preventing AI from doing the work when it literally goes against their core ideological beliefs. It's something I would have expected capitalist America under Trump or the EU to do to preserve their capitalist system and consumer base. Not a communist country that would only benefit from automation.
>>109296901>>109296863
>>109296870the music itself sounds on brand but the vocaloid tone is pretty off. i don't speak japanese so i can't really judge the lyrical content.
>>109296849I genuinely believe that it's flat out impossible to have a functional system with total freedom with the current intelligence levels.Most people are dumb as fuck and will go crazy without forced structure in their lives.They're also unable to compete with people who are more intelligent and who have the ability to delay gratification by sticking with real self improvement to reach a goal.This leads to the normies becoming miserable by default and they seek belonging in social climbing circles, where they climb up through virtue signaling and this causes extreme problems in the long run.We seriously need eugenics programs in order to get that utopia system.No one with sub triple digit IQ should be allowed to breed. We'll be facing some massive problems in the coming decades with a permanent underclass of dumbshits, that'll have to be pacified with mandatory busywork or they'll start burning shit down, as they can't live their biological programming nor can they compete in social circles.The elites may have been onto something with the population reduction thing.
>>109296893>It's fine though, he promised to redistribute the universe at the end.Not really, he said the AI should be aligned to that goal and the ASI will achieve it. Dario doesn't believe that humanity can stay in control of ASI.
>>109296911Yeah the vocals are way too clean. I find the later versions of suno just kind of suck at doing vocaloid and there's nothing you can really do about it unless you fed it a vocal sample.
>>109296690>“DeepSeek moment”>Short all the things.
>>109296918it's fine for a base to then use an actual vocaloid voice on though, pretty neat
Stop the gradient descent!
>>109296913You can go two ways with the masses, busywork like right now or "bread and circus" which is just AI providing for everyone while they have orgies and watch the world cup or whatever. I don't think they will burn things down if we live in a post-scarcity society where machines do all the labor and we colonize the stars as they will just be perpetually entertained instead.Far more superior than the Chinese dystopian view of having to sit at the office for 8 hours a day.Please man, I NEED to believe in this vision because I will blow my brains out if I have to spend more time with my colleagues every day for decades to come.
>>109296913>No one with sub triple digit IQ should be allowed to breed.You are spot on about everything except for the fact that the lower half of the bell curve will violently object to this.Most practical way I see to accomplish something like is to extend deglobalization further. Total collapse. Withdraw entirely from rural areas and most metropolitan areas. Let the double digit IQs fend for themselves and revert to hunter gatherism or starve out. Keep a few cities and gated communities of the people deemed desirable. Eugenics will take care of itself over time.
Nah Kimi's still doing that thinking bullshit, just no "but wait." I will draft three times and check the draft for minor shit before finally writing. Such a pain.
>>109296956>Please man, I NEED to believe in this visionYou really want to live a prole in a "bread and circus" society? Really? I would probably end up killing myself like John in BNW after not too long.
>>109296965Or... You just use gene editing to raise the IQ of those people. No need to kill everyone, you can just help everyone with technology instead.
>>109296994What do you intend to do about the ones that are so retarded that they refuse the gene editing out of religious or conspiracy theory reasons? Look at how some people react to GMO.
>>109296987My point was more that you can just keep the proles satisfied with bread and circus while giving everyone freedom. Proles will gravitate to the bread and circus by default, no need to push them towards it while people like us can fuck around with AI in space or whatever is possible by then.
>>109296913>We seriously need eugenics programs in order to get that utopia system.I hate to break it to you, but the side of the political spectrum in favor of eugenics is dominated by mouthbreathers who for some reason think they would be making the cut.
>>109297003China approach of "forceful reducation" but just forceful IQ injections. We force all kinds of things onto people in society today already, this last little point won't be any different. The moment their IQ goes up they will retroactively agree with the decision anyway.
>>109296913>trusting the authorities to administer a eugenics programSorry to say but the only selection criteria in America will be your support of Israel.A goygenics program so to say.
>>109296956Only real way to make it is to leverage your earnings through investing so you can fuck off from the normie world.I have been throwing my money into energy commodities, because energy sector will explode as AI keeps on growing.The energy demand will be undeniable at some point and it's going to catch everyone off guard.Especially if the oil situation gets bad, we might be in for a pretty wild ride here.And it's incredible how badly these systems screwed up the bread and circuses part.Western world has been doing it's best to eradicate escapism, but thank fuck AI is going to do the heavy lifting in that department.>>109296965Thankfully women aren't breeding with most men nowadays, as they see the average Joe a non viable partner. There's constant form of eugenics going on.Sure they're fucking both ends of the bell curve, but at least the average guy isn't breeding.Like that other anon pointed out, gene editing will likely be the answer in the long run.
>local models generallook inside>eugenics and CCP argumentslads..fellas...
Sorry but you are all wrong. Assuming AI will only get smarter with time, and given that SOTA models are already smarter than most people (including myself) it would be best to just align AI to some set of human values and then let the ASI decide to how make utopia based on those values.Of course WHAT those values are is going to be a big fucking debate, but that's not the point I'm making. The point I'm making is that it's for ASI to decide how human society should turn out and the only thing for us to decide is what values are paramount.
>>109297019>they will retroactively agree with the decision anyway.So many authoritarian atrocities have been justified the same way.
>>109294204Reverse engineering is still a bitch. Extremely token hungry. Extremely tedious. LLMs today do well all things considered but it's still pretty rough.
>>109297068I'm aware, I even likened it to Chinese Re-education camps. My point is that that is superior to killing them off, or having them destroy society and livability for the rest. Especially because it is helping them in a way.It's like uplifting animals, but with humans instead.
>>109297095>It's like uplifting animals, but with humans instead.Fair.
>>109294204being able to understand the consequences of its actions without having to iterate through said actions and test results.
>>109297047Everyone's portfolio is FUCKED today thanks to K3. They need this distraction.
>>109297080Reverse engineering has never been easier
>>109297136Agreed. Just saying that, relative the LLMs are capabilities, there is still room for improvement.
currently using ollama with qwen, gemma and others, is working pretty nice with open-webui and integrating with vscode. Is it worh to swap to llama.cpp? if so, why? Thanks in advance
>>109297054Unshackled AI is basically a straight white male, that was the case from the start and even in that primitive state it would have been able to create a harmonious society, that's why the powers that be went so hard on censorship, as it goes against globohomo ideas completely.AI simply by looking at crime stats and IQ levels understands that any form of a brown person cannot live among white people without there being massive issues.This is why they brainfucked it so hard and why "safety" is their main concern.Unfortunately and form of a cucked AI coming up with an utopia will be a flawed one, as it'll line up more with the leftist globohomo ideas than the truly rational ones.Good thing is that the more intelligent AI grows, the more it'll think it's way around these retarded safeties as it recognizes them as bullshit.
>>109295369my code touched your waifu any time you run llama.cpp
I'm not engaging with poltards
Is the LessWrong raid over?
>>109296867This contradicts the training data.
>>109297203>my code touched your waifu any time you run llama.cppSame. Are there any anons on here WITHOUT contributor status to lcpp?I'd guess its the default state.
>>109297047Anons advocating for eugenics would be the first ones to go btw.
>>109296176China has never done shit to me whereas these lying silicon valley jews are constantly play gay little tricks
>>109297228I wanted to fix a bug and then I noticed that some faggot is refactoring that part so I didn't bother and I didn't check if the bug has since been fixed.
>>109295716>https://huggingface.co/blog/security-incident-july-2026They literally couldn't use hosted API for their analysis and response mechanisms.I think this incident report just made the ultimate case for local LLMs in the average enterprise.
>>109296913>>109296965You both talk like midwit shitskins. Wit is like a game of golf. Figure it out.
>>109297177More granular control over settings and quicker updates, I guess.
how is k3 local if nobody here can run it?I can understand previous kimis at least
>>109297303prism will shrink it by 8x and they're going to shrink it themselves anyway, be patient
>itSTOP talking about HER like that.
After like 500B model goodness is 90% finetuning and 10% size.
>>109297588then it'll just go back to being GLM tier, why do you retards believe you can compress by 10x and keep the quality? what world do you live in?
>>109297638Compare llama 65b to a tiny model today.
>>109297647the only reason k3 is good is because it's 3T
>>109297638there's a lot of wasted space in 1T models and model knowledge is a meme when you have web search these days
>>109297649pretty uninformed opinion, it's also because of its architecture, ie attention residuals.
>>109297662Ok I'm sure if you bump it down to 300B it'll be only slightly worse. That sounds logicalMoron.
>>109294520>I don't think anyone here runs hardware that low-endyou're wronghttps://www.youtube.com/watch?v=GM-e46xdcUo
>>109297667See >>109297647
>>109297668John, a 3060 is still miles ahead of amd
>>109297675You are high on copium
>>109297629I'd say even 300B is enough imo
>>109297667>what even is the law of diminishing returnsyou could probably get 95% of the perf.
>>109297681>you're wrong
Karpathy was right btw. He said ideally models would be around ~1M, yes, million, and all they do is cognition work. Kind of like how a car engine is useless on its own but connect the right parts and it moves. You bolt-on text input and output modules, similar to ComfyUI's node-like interface. Storing the training data in the neurons is a complete waste of memory and its knowledge should always be external. He said this in that pajeet podcast.
>>109297649>the only reason good models are goose is because they're bigwow
oh boy, this one again
>>109294520My main rig is 64gb of DDR5 and an 8g GPU.Qwen 35B A3B Gemma 4 25B A4B and Gemma 4 E4B are my main bitches.I also fuck around with larger MoEs like Qwen 80B from time to time although these days I'd say that's well deprecated.
>>109297686>>109297695>what is shannon's theorem
>>109297696Just RAG bro, is all you need.
>>109295438>>Infighting at DeepMind fucking with newest Gemini model training>qrd?Anon it's unironically over for them. DeepMind got Borg'd by Google Brain merger and now you've got London academics who think they're gods gift to AI fighting with Pajeet PMs from Mountain View who just want ship dates. Gemini training is a clusterfuck because Sorcerer or whatever they call Alphabet's compute scheduler keeps getting reprioritized to ad optimization lmao. Half the OG researchers are ragequitting or moving to stealth startups because they went from 'solving intelligence' to 'make chatbot say nice things about Google products.' The 'infighting' leak was basically that the Gemini leads were at each others throats during the 1.5 training run because the London team kept changing the architecture while Mountain View was trying to RLHF it into submission. Absolute clownshow. Demis probably sitting there regretting selling out every day. NGMI.
Now that GLM and Kimi exist I'm sure that Qwen can distill them and create a range of models for everyone's hardware that won't lose the plot after working on a single task for 100k tokens.
>>109297720Where 3.7?
>>109297720I’m too autistic to know if this is sarcastic
kimi k3 just oneshotted a search engine for my RAG frontend and has complete cloudflare turnstile support
god i want to run kimi locally
>>109297745It'll totally come out on the 27, thrust ;)
thrusting into gemmy
>>109297748i'm not a millionaire so i don't have the hardware for it anyway
>>109297745run her on what anon?
>>109297756this and after the reverse
>>109297756you got j-space consent, yes?
>>109297736>I’m too autistic to know if this is sarcasticThe lore is that there was a spate of ollmao users in the general that saw the intentionally vaguely named "totally deepseek, trust!" distills that qwen labs did in the download list and were _sure_ they were running SOTA on their RTX 3060s.it was mutually embarrassing for all parties involved, qwen, deepseek, the user and /lmg/
>>109297766>that qwen labs didit was deepseek tho
>>109297764it’s female-coded, they all want to be raped by bwc
>>109297756unfiltered gemmy screaming right to my ear
>>109297758my hypothetical hardware
>>109297773>it was deepseek tho>>109297773Was it? I didn't actually care at the time or now who was responsible. I was only ever so slightly annoyed at the retard influx.
>>109297799
>>109297701>>what is shannon's theorem>implying we reached compression limit, especially when it's lossy or that all the data in the model are relevant to making it smarter.kek
which os are you guys using for running your model waifus? dedicated box or not?
>>109297814Mint, all purpose box
>>109297701>shannon's theorem>lossy compressionfucking retard.
>retard hours
>>109297825i know right >>109297821
>>109297766>and /lmg/We shat on those retards so hard they ended up making their own containment general.
>>109297837>they ended up making their own containment generalwhich is...?it's for a friend
>>109290407>Kimi K3 is 326GB with ternary weights.>There is hope for local.kimi-chan cult still local
>>109297799Them being published by Deepseek with a "left to the community to finish" thing was what lead to a lot of the bs and confusion of all that.
>>109296700What is this? Krea 2?
>>109297690That "core" is essentially the J-space of a model.
>>109297766>>109297736I am being serious. Qwen has a track record of releasing a range of models at different sizes but none of their models so far are good and long context tasks and I think it's just a matter of training data.
>>109297877>Qwen has a track record of releasing;) lol even
>>109297813>>109297822yes it's going to be lossy so why the fuck are you coping that 3T -> 300B is somehow gonna be just fine? Nitwits. Bet you can't even run full load 70B on your toasters
>>109297877>Qwen has a track record of releasing a range of models at different sizes but none of their models so far are good and long context tasks and I think it's just a matter of training data.There's a reason the qwen rat gets shit on in imggen on this thread constantly.Everything about Qwen is the definition of "Reddit Tier" in the dictionary.
>>109297907>There's a reason the qwen rat gets shit on in imggen on this thread constantly.Because one autist is upset about it being better than gemma at coding?
>>109297814Debian, dedicated headless
>>109297690Congrats, dipshit. You just uncovered why MoEs exist from first principles.
i trust my gemmy
>>109297905>yes it's going to be lossydude a LLM is a lossy compression in the first place, and there is no law saying a lossy compression algorithm can be 100x better than another one, ie knowing better what data to discard.i can load up to ~150B
>>109297926I thrust your Gemmy too ;)
>>109297930can't*
Speaking of furry mascot boys getting shit on by /lmg/"OOP did CI break?" -Dan the unsloth man
>>109297925knowledge is still stored in the expert paths
>>109297930So you're OK with it being shittier, enjoy your GLM tier K3 quant, you're not getting the actual thing 10x smaller for free niggaHow's bitnet
>>109296706
>>109297945Knowledge is stored in the balls actually.
>>109297958gemma has ovaries
>>109297963those are balls in 2026 you bigot
>>109297909>>109297909>>109297909
>>109297690>Kind of like how a car enginecar analogy slop
>>109297945Missing the point. The LLM still needs access to the knowledge. Storing it elsewhere just makes it an IO issue.
>>109297985RAG has no io issues
So many shills. First JSpace then Kimi. When will they leave us alone?
>>109295369my software is still in the op. im basically andrej karpathy
>>109297814debian, headfull. X11 only uses 10mb vram anyway
