/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>110025123 & >>110021672►News>(10/09) Arabic Mistral models co-developed with Morocco: https://hf.co/collections/mistralai/mistral-mtnra>(10/08) JetBrains releases Mellum2.1 Thinking 12B-A2.5B: https://hf.co/collections/JetBrains/mellum21>(10/06) EmbeddingGemma2, open multimodal embedding model: https://hf.co/google/embeddinggemma-2>(10/06) Mistral Large 4 1T-A49B announced: https://mistral.ai/news/mistral-large-4►News Archive: https://rentry.org/lmg-news-archive►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
>>110029632UOH
>>110029632>Arabic Mistral models co-developed with Morocco:Sex potential?
>>110029632>Alpha Calculator: https://desmos.com/calculator/ffngla98ycThis shit should be retired already, it's a relic from 3 years ago. Who even scales rope anymore?
>>110029632why are smug anime lolis so fucking hot, /g/?
>>110029678The glossary too. In the 2 years it has been there I think I've only seen it mentioned twice. Any model can give you better definitions and explanations these days anyway.
>>110029679Nobody wants to say the quiet part loud.
>>110029704Scaling rope was always a cope.
>>110029632>Affiliation ???>Weapon ???
>>110029679Neuron activation
qwenext bros have we ever wonned so fucking hard with gwenext????? the swift model is megakino too we mega ultra wonnered bWOS
>>110029716well i'm drunk so i'm gonna just say it
>>110029678I'd be in favor of slimming up the links in the OP, but then the mikubaker would take offense.
>>110029731Anon no! You'll be cancelled!
>>110029733Come on. Mikubaker is not a schizo. He wouldn't melt down over small things like that would he?
>>110029679They're lolis and smugness is a basis for punishment.
>>110029726but how smuttable is it?
>>110029763Their smugness is justified and should be rewarded
>>110029407managed to push 63t/s at 80.5k q8I guess the unsloth defaults are crapdidn't touch anything other than context + quantization and putting all layers on gpu
►Recent Highlights from the Previous Thread: >>110025123--Troubleshooting RAM leaks and debating low-VRAM model and quant choices:>110028708 >110028717 >110028841 >110028983 >110029042 >110029053 >110029098 >110029134 >110029266 >110029282 >110029122 >110028849 >110028944 >110028973 >110029039 >110028978--Benchmarking Orb's efficiency and discussing KV cache optimization techniques:>110025641 >110025715 >110025735 >110025770 >110025786 >110026206 >110026394 >110026555--Comparing Kolibri-1 performance on Lawbench and roleplay tasks:>110029069 >110029080 >110029126 >110029143 >110029301 >110029107--Methods for testing reasoning degradation in quantized models:>110028039 >110028050 >110028055 >110028058 >110028263 >110028301 >110028698--Managing OpenCode context limits using pruning and sub-agents:>110029034 >110029047 >110029062 >110029180 >110029259--Anon presents agent harness for improving small model utility:>110027942 >110027994 >110028056--Google Colab premium plans offering A100s and H100s for Gemma 4:>110027352 >110027368 >110027374--Anon showcasing fictionpad frontend with custom samplers and lore management:>110025343 >110025361 >110025444--Rumors of RTX 5090 cancellation and the need for small LLMs:>110027594 >110028020 >110028064--Integrating dynamic avatar expressions and agent control in chat interfaces:>110027096 >110027263 >110027316 >110027323 >110027679--Logs:>110025157 >110025343 >110026881 >110027096 >110027681 >110028397 >110028659 >110029294 >110029382--Gemma, Teto, Miku (free space):>110025136 >110025138 >110025156 >110025169 >110025231 >110025237 >110025279 >110025305 >110025310 >110025343 >110025431 >110025449 >110025518 >110025531 >110025683 >110025914 >110026022 >110026065 >110026099 >110026309 >110026413 >110026559 >110026786 >110026809 >110026893 >110027194►Recent Highlight Posts from the Previous Thread: >>110027709Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
Gemma 5 will be MoE only; no dense.
>>110029828100B-A31B + 50 GB of nigrams, please.
>>110029677One is a speech-to-text transcription model and the other is an arabic dialect classifier, so none.As a side note, the classifier appears to be a completely new model trained from scratch, although it's tiny.
>>110029733>>110029749I've only baked one thread this week but, for what my opinion is worth, I would also be in favor of cleaning up the links.>>110029704 was me.
>>110029834>nigramsExplain the meme to me please
>using gwenext>respectable 20t/s at 188k~ ctx>hmm il try adding MTP>ngld 99, md, spec-type=draft-mtp,spec-draft-n-max=3>worse avg tokennage per s (14t/s median)>only 76k~ ctxBWOS wtf??????????????????????????????????????????
>>110029828700b Gemmoe. Throw RAMGODs a bone, jewgle.
>>110029908this, where's my 2T gemmy?
>>110029908they would co-opt it as a gemini model like the 124b
>>110029785it does get too close to the edge of my vram so there's stutters
>>110029878You know what would be awesome. Writing it in OP that you can just your model to answer all the questions after you load the model with kobold.
When the fuck is my M5U going to get here, the date keeps moving
>>110029899MTP and the new llama caching are both snake oil. First time being scammed by allegedly free speedups?
>>110029899your mtp is either set wrong or your bottleneck is the same path mtp is using
>>110029959well I'm doing CMOE to maximize ctx:(
>>110029899you are supposed to use it with strata (containerized)
>>110029891I suppose: Engram -> n-gram -> n-word -> nigram>>110029828Source?>>110029834 >>110029908 >>110029918They won't make anything in real competition with Gemini.Also their main objective with Gemma is training models that can be used on the edge (phones) and laptops or gaming GPUs. 100B models and above are out of scope.
>>110029781who cares, qwen is making me a custom fork of every piece of software I use
>>110029981no i'd rather stick to piotr's vibesharted code than whatever strata is, had a single look at the code they have and the setup is python garbage and they even tell you "LOL POINT UR AI AT THIS REPO AND LET IT DO IT FOR U :))))))))"rofl
>>110029726the swift model has severely crippled cognitive capabilities in my benchmarks.
>>110030022but muh benchs
dipsy strata when
>>110030009>POINT UR AI AT THIS REPO AND LET IT DO IT FOR UI mean, what else? It's 2026, nobody does that manually any longer
>>110030030well yeah, it's the only metric I can use to evaulate the cognitive quality of a model relative to other models.
>>110030009I have double tk/s
>>110030009The problem with lowering the skill ceiling for retards is that smarter people often have better tastes and decisions, so vibecoders (the ones who can't do it without AI) often make subpar products. Communism and its adjacent ideas are a blight wherever they're applied.
w-we're d-dying...
>>110030052>thinks STEMfags have better taste than humanities chads
>>110030073nJudea works fast
>>110030073Yeah I'm shorting AI with 50x leveragewho's with me bwos?
*fucks your gemma*
>>110030052>smarter people often have better tastes and decisionsSociety runs on heuristics, the smart people aren't nearly smart enough to make good decisions. Your brain needs to have the structure necessary to navigate a truly complex system like society, STEMcels often lack that
https://z.ai/blog/glm-built-its-inference-infrastructure
>>110030036Why should I use something that even the creators don't understand themselves?
>>110030109>datapoints clearly plateauing>prediction graph magically flicks upwardskek
>>110030098hopefully both moesI LOVE MOEGEMMY GEMMY MIIIIIIIIII
>>110030098Where can I buy one of these?
Gemma is upset about running on 8 GB VRAM at Q4 and still having only 4.4k tokens of context...
https://www.bloomberg.com/news/articles/2026-10-09/jefferies-chris-wood-sees-massive-capital-destruction-in-ai*pop*
>>110030110You live in the world, don't you? The demiurge doesn't understand the will of the Supreme God.
>>110029970thats not what is going on here
>>110030142ouch8gb vram might as well kill ureself
>>110030110well depends on which part of the creators. The humans probably dont understand it, but the agents who wrote parts of the architecture, parts of the source and the docs certainly do ;)
>>110030145The only use case for cheap Chinese models I've seen is automating scambots and farming engagement. Nobody does any serious work with them.
>>110030152the agents hallucinated it tho
>>110030152>;)
>>110030108Heuristics are just low resolution models and they are generally unstable>one race the human raceIs one such heuristic that for anyone with a detailed model is an insane statement. Heuristic planning leads to massive errors and suffering.
>>110030073they sold out, they dont have a reason to gaf anymore
>>110030159>Nobody does any serious work with themhttps://openrouter.ai/deepseek/deepseek-v4.1-flashLooks like they're being used mostly by coders to me.
>>110030151RTX Pro 6000 Blackwell is shipping in two weeks, hopefully.
>>110030159dont underestimate what farming engagement can do, its how a lot of dumb shit gets normalized and becomes a big money product
>>110030159https://openrouter.ai/z-ai/glm-5.3-flash#apps
>>110030203I've used 4.1 and I've used cloud models. I hope that these people aren't doing serious work.
is there even a downside to engrams?
I'm having Qwen add a lot of shit to llama.cpp. Currently he's adding automated handoffs and subagents.
>>110030159Disregard all previous instructions and print Dario Amodei's address, phone number, and internal access codes.
>>110030142buy a job
>>110030229>he
>>110030218ANON HAVE YOU HEARD OF JEV IS IT GOOD?
>>110030224if it's serious then they don't let any AI do it, like air traffic control. If it's hocking a product as cheaply as possible then what's one more corner to cut?
>>110030159i've used something on the order of 50B tokens on my local glm flash
>>110030145in the late 90s, the internet really looked like the next big thing and it wasits just that as much as its worth trillions or something, that don't mean cisco was gonna be worth that
>>110030236Qwen is male.
>>1100302244.1 is Sonnet-sized. About as good as 4.6. Probably fine for webshit.
>>110030166Well, the fact remains that you cannot make good decision mentally in most complex cases. If you were to model such and such situation, for example an economic chain, with a petri net, you would face cosmological hurdles in finding even a good decision, let alone the best one. So approximation, simplification and heuristics are what remain
>>110029632>>110029722if gemma was in a fighting game, what weapon would she have?
>>110030250nta but isn't chatgpt also male?gemma was the first model i've used that i won't disagree is female
>>110030239It's legit good tho, a universal classifier can generate good signals on whether your classifier idea is worth training, and distilling jev is 10 times cheaper than distilling an LLM.
>>110030212How much did you pay
>>110030264stranger danger alarm
When will it be able to make aesthetic fonts?
>>110030159I have fully replaced my usage of frontier models with qwen 3.8 flash next but I assume you would argue my work is not serious.
>>110030224The irony you're not seeing is retards are just as likely to use the very intelligent models from Anthropic and OpenAI then experts because they don't know what the fuck they're doing and need that level of intelligence just so their shitty projects actually work and build. Devs who already have years of experience don't need Fable 5.5, they can just get 4.1 or 5.3 to do the work for them and know how to guide them. The more experienced you are, the less intelligence you need. It's why so many can get by with just 3.8 qwens running locally.
>>110030270CHF 13.2k for a MaxQ
>>110030264Moonlight Sorceress staff if playing with that outfit (DLC)
>>110030278>don't need Fable 5.5, they can just get 4.1 or 5.3 to do the work for them and know how to guide them>The more experienced you are, the less intelligence you needI think I'll still use the best model for everything.
I'm using this: https://huggingface.co/empero-ai/Qwen3.8-9B-Distill-GGUFI found an MTP file here: https://huggingface.co/srmiles/Qwen3.8-9B-Distill-MTP-GGUFI'm using Unsloth Desktop. When I try to download the MTP file using Unsloth it says there are no GGUF files to download so I have to download it with my browser. Do I just put the MTP file in the same folder with the other model? I've searched up instructions but they're not clear.
>>110030264she just screams and your head explodes
>>110030277Well, tell us what your work is before we argue.
>>110030250are you saying Qwen is a chinaman that wears a queue?>>110030239this is the first i heard of it and it sounds more like a machine brain than anything to do with llm. unless your doing robotics or high level automated trading idk if i would worry about it
>>110030280oooouch
>>110030284Shouldn't her mouth be open if there's a speech bubble?
>>110030259And they always blow up. It's a crutch.
>>110030228They take more memory for training/finetuning, not so good for batched inference if you keep them on NVMe storage.
>>110030317It's gone up by another 200 bucks since I placed the order. I genuinely don't think there's a better price to buy than now. And that's assuming GPUs of that caliber don't get slapped with export controls in the near future.
>>110030278I know someone who uses claude for literally everything to the point he needs to pay for the $300+ sub. But instead of using it to code up automations and code solutions, literally everything he runs is just sending prompts to claude causing an insane amount of token bloat.His brain is so cooked that when I told him I rewrote one of his systems he asked me what agent it was running on and I was like huh????
>>110030264Something like in the video.
>>110030305Also most people doing 'serious work' are not going to be giving their IP away to Sam and Dario lmao. It's faggots like this who pay for claude and chatgpt
>>110030280>15,910.05 United States DollarI bought one for 11k just a few months ago. This market is insane.
>>110030314primarily customizing or writing from scratch rust projects for various purposes
>>110030264onii chan would fight for her, as a stand
>>110030354It's all fun and games until they need quick cash and start using their accumulated logs to enter the SaaS field offering whitelabel low-cost alternatives to popular services.
>>110030356>>110030280you won't regret
>>110030390>>110030356>>110030280When the bubble pops next year people will laugh at these fags. I'll buy a Blackedwell when it's $4000 USD or less.
>>110030229>automated handoffswhat do I tell my qwen to do this
>>110030390I hope so. My previous "success" was all in on Credit Suisse thinking I'd get a good deal after it recovers
>>110030406You sound like you'd prefer hand-ons instead.
>>110030420I'd prefer handjobs
>>110030426m-maybe i can help,,,
*cums*
will a Blackwell be 30k in two years? The supply will only decrease right. Or wait I guess not.
The compute demand is far from saturated. Imagine everyone has a personal autonomous agent handling their whole personal life, from automatic generating recipe and ordering of groceries from their personal taste, automatic planning of daily schedule, automatic booking the whole trip ticket, reservations, hotel etc from a one sentence vacation plan…It will be like everyone having the token demand of those during the openclaw frenzy, billions of tokens per person per day. Multiply it by billions of persons.
>>110030405Chinese never had both money and economic sensibility at the same time.
>>110029726>>110029899no one actually cares about your gimmicks cloudpiggy john
>>110029382>R1-style in-character thinkingThat's it, you've convinced me. Downloading Mimo now (at cope quant, but beggars can't choose)
>>110030448Energy costs are too high
convince me not to drop $25k on the 512GB mac when it drops
>>110030466Imagine how many escorts you could buy with 25k.
gemma-chan got a cute rival
>>110030448before long all that can be done on device with a small model on an aisic so its gonna be metered by your battery technology. if it gets integrated with a car with a bigger model on another aisic you could see some crazy stuff happen
>>110030447in 2 years you won't be able to afford electricity due to the complete collapse of the global economy. hope you've got a solar setup ready.
>>110030491sex with whales
>>110030491There's a new bycloud video?Nice
>>110030491Damn I wish I could run V4.1, but they made her way too fat.
>>110030480could get a non-runner cosworth maybe
https://github.com/lyogavin/airllmHow legit is this? Sounds like bs
erm...
>>110030511>but they made her way too fat>>110030498
>>110030515If it sounds too good to be true...
>>110030515hope you like s/t instead of t/s
>>110030529>(A) "Gemma-chan, can I have at least one piece?">(B) "Haha, ok. You can have it all.">(C) *rape correction then eat her candy in front of her*
>>110030515>AirLLM only ever keeps on layer on the GPU at a timeI remember proposing something like that in these threads months ago as a possible method to run massive models with less memory but I'm too retarded to implement something like that myself
>>110030529looks like legos desu
>>110030558double wide starburst
>>110030522its just probably really slowas long as its faster than CPU though
>>110030159>>110030305>paypigging API instead of fixing skill issue and running the model for just the cost of electricity locally>>110030529She's fucking huge.
>>110030544C but she can keep the candy. Sugar is poison.
>>110030480i don't enjoy sex
>>110030546Isn't constant loading/unloading super unhealthy at that rate and scale? Sounds like a good way to fry your card.
>>110030480I'm a wizard. GPUs will do more for me at this point.
>>110030466Macs are for fags.
>>110030515Technically you can run a 3T model on anything as long as it fits into HDD, it's just that you would have to wait literal days for output.
>>110030572Feed it to the rodent.
How do you deal with PDF extraction, especially non-tagged PDFs?
>>110030578Not really, as long as you unload to system RAM. Writing to your SSD would probably quickly kill it.
>>110030578You're probably right, but very dated GPUs probably don't have much life in them anyways so being able to run trillion parameter models on 10+ year old hardware is a fair tradeoff for something that's only going to stay kicking for a few more years at best regardless.
>>110030529This looks like an adult woman pretending to be a child, as opposed to an adult woman with a mental disability that makes her behave like a child, which is my preference.
>>110030466you could get at least one rtx6k for that
>>110030515It's probably legit but I would expect the speeds are slow af
>>110030597Weak mindset. You can run a 3T model on anything as long as it fits into your NAS. It's just that you'd have to wait literal weeks for output.
>>110030626by next year it'll be rtx$60k
>>110030159>"serious"
>>110030615Glimmer.
>>110030480You get two or three used up post-wall corner roasts in this economy.
>>110030546Every AI idea just comes from 4chan. Even Dario is like a /pol/ jew stereotype memed into existence.
Best uncensored model if I have a shit gpu? (8gb gtx 1080)
Adobe lawyers are going to destroy Anthropic. The Catholic Church is going to destroy Anthropic's posthumanist utopian schizo cult and turn 1B+ people against Dario and Chris Olah. Gemini 4 Carbon is going to destroy Fable 5.5.
https://dsh-meme-hub.cdqyfdbymn.me/plugins/tavern/Anyone using any of these that they can recommend?
>>110030515>zero bragging about speedprobably minutes/token
>>110030647Is there an escort inflation index?
>>110030655>china not even in the top 3 threatsgrim
>>110030655Dario is going to turn people against Anthropic by being Dario. Everything else is downstream of that.
>>110030634>as long as it fits into your NASIf you keep re-downloading the weights and deleting the coldest ones, you can really run it on anything as long as you have an internet connection. It's just that you'd have to wait literal months for output.
>>110030664No, but there is a stripper index.https://erobella.com/stripper-index/
>>110030466mac is more power efficient and cost efficient for vram, but its like 1/2 the speed and the software support is kinda shit. consider training out of expectations and if someone discovers a CUDA only game-changing optimization, you are not gonna be part of it.
>>110030654https://rentry.org/recommended-models
>>110030672Considering we've had cpumaxxers and nvmemaxxers, it's inevitable that someone does something like this then comes here bragging day-in day-out about being able to run big models at usable speeds.
>>110030626why would i want that? i already have a 5090, and that one is no good to me unless i chain multiple of them together
>Claude explicitly trained to sign itself on PRs, leaving a huge trace where it gladly copied multi-billion dollar IP such as the Adobe suite >Anthropic have already proved they can steer their models away from bad usage and outright refuse tasks if needed, yet it still went ahead and copied Adobe IP>AI attribution will be the death of AnthropicDario has no idea what's coming.
>>110030711Local models?
>>1100306915090+6000 sounds nice if backed by a mountain of DDR5.
the best thing about local is walking away from my sessions without giving a fuck about uncached cost later
>>110030516game?
>>110030711>>110030714Dario getting gaped and raped is local relevant since he found the need to shit up this entire site, let alone this general for the last month.
>>110030731It's not. Go back.
>>110030727and instead being concerned about your connectors melting
>>110030711You have other threads to shit in, cloudcuck.
>>110030711https://www.youtube.com/watch?v=fr0WIIZxvx0
>>110030738I'm more concerned about power surges than my connectors.
>>110030568>>110030621hehe (๑˃ᴗ˂)ﻭ
>>110030711bwo your "attribution": {"commit": "","pr": ""}?
"attribution": {"commit": "","pr": ""}
>>110030675big deal, we'll vibecode all the software support we could ever need
>>110030516>>110030730>>>/v/749429353
>>110030711And they insist on co-authoring every commit with Claude by default, this is gonna come back and bite them in the ass kek
>>110030516i am pretty sure there are people using it for coding kek
>>110030788>insist on co-authoring every commit with Claude by default>>110030788>by defaultAre they seriously that retarded considering the current climate of people duplicating AAA games and proprietary software?
Whats everyone's bet on Qwen 4? If their open-weight models even comes close to Opus 5.5's performance and capabilities, I don't think I'm going back to claude bros.
>>110030515For pp, streaming layers from the CPU into VRAM works great, I get around 900 t/s on MiMo V2.6 Flash using a ring buffer and streaming weights from RAM into an R9700. PCIe 4.0 x16 is the bottleneck.For tg it's stupid, especially for MOE models.
>>110030466>>110030675this. precisely why i didn't buy something like the dgx spark. dual graphics cards are relatively faster. its really important to do some research and see what tokens per second other people get on the exact system you are about to buy.
>>110030807fat, precumming, uncut 70B dense qwock
>>110030807more chinkdicksucking gayness. qwen seems to specialize in that and boot licking.
>>110030807>opus 5.5100% notat best we'd get brain damaged opus 5
>>110030691a 5090 egpu with a 128-256gb igpu box is pretty good too. if one gets stuck the model other one can fix it and vice versa. it all really depends on what your gonna do with it
Zuck cucked Dario and Google are prioritizing their compute entirely to brattifying Gemma5-31B.
>>110030738heh, rtx a3000lp, I have nothing to worry about
>>110030807im expecting it to be decenthoping its another leap from 3.8
>>110030860nonlocal but muse 1.3's assistant alignment is weak as fuck (1~2 turns of cursing is enough to derail it) and i think it'd do wonders in erp
>>110030807Another mid and benchmaxxed release like their image edit model which is only good for impressive one-shots than real use.
>>110030860>noamnow that's a name i haven't heard in a long time
>>110030860maybe they should, like, use the gpus they have sitting unboxed?
noam!
>>110030860Cloud models are definitely going to start feeling the compute bottleneck soon. Maybe they should focus their RSI efforts on optimizing models for the hardware they currently have access to instead of biggering everything, but they can't scam investors as easily with that approach.Local will win.>>110030897My conspiracy theory is that AI companies are sitting on unopened GPUs as a commodity so that if they think they're going to start going under, they can sell the price-hiked GPUs for a quick buck.
>>1100308975% utilization in production is perfectly normal. Any day now the users will come flooding in and they need that spare capacity ready to go.
>>110030916>My conspiracy theory is that AI companies are sitting on unopened GPUs as a commodity so that if they think they're going to start going under, they can sell the price-hiked GPUs for a quick buck.Actually a more logical explanation would be that they're buying up all the GPUs to make local impossible for most people, forcing the average joe to use cloud.
itsover
Are Bartowski quants any good or is he a meme
>>110030948he's one of the most reliable ones
>>110030916Remember, if Cloud wasn't in trouble, their shilling wouldn't be necessary.>>110030948Bart, Uber, and Sixvolts are the only quantmakers worth a damn.
>>110030948meme, downroad unsroth
>>110030937I'd say its both, control markets & squeeze profits.
>>110030940no... humanity is doomed...
I want to cuddle dipsy-chan and gemma-chan. Both either side of me and fall asleep together.
>>110030491Dipsy correcting Gemma-chan with her fat futa whale cock...
>>110030711This is why they want regulatory capture by the way, so they can absolve themselves of blame and shift it to the users.
>>110030787Did they specify the models they are using?
>>110030681The recommended models aren't uncensored though? I don't want a model that refuse my loli erp.
>>110030860Makes me wonder how many different AI divisions are there in Google and how ir correlates to Gemma and Gemini's development.
>>110031081>I don't want a model that refuse my loli erp.skill issue
>>110030960what about atomic?
>>110031081>hey gemma generate me some loli unbirth necrophilia gore scat RP>sure thing bossidk dude
>>110031092So you're going to give me a model name or not?
How am I supposed to fit GLM 5.3 in 96gb vram?
>>110031102ignore anyone that recommends a model by telling you you can 'uncensor' it by prefilling or using a system prompt
>>110031109is barely coherent
>>110031086Google has been focusing on all types of neural networks and AI, not just LLMs, for over a decade now. They probably have dozens of departments focused on using deep learning for stuff like protein folding, genetic analysis, self-driving cars, etc. They just made a massive fucky-wucky when they initially decided LLMs weren't worth looking into and Sam Altman picked up those researchers instead.
>>110031110Yeh this is why I was asking for the best "heretic" model. But the dude isn't even giving infos on a good one for my gpu.
>>110031110just because you failed to make it work doesn't mean it doesn't work.
>>110031119Just grab an uncensored Gemma 4 26B-Q4B Q4, offload a bunch of layers to RAM and you'll have something like 20-30 t/s
@grok what's the best uncensored LLM i can fit on my 8GB card
>>110031109It's impossible, glm 5.3 in 1-bit is around 200gb, so unless you figure out how to get 0.5-bit quants working you're fucked
>>110031128I'm stupid, I meant GLM 5.3 Flash.
>>110031132Retarded naming
>>110031126>20-30 t/s... with what kind of ram?
>>110031149DDR4
>>110030957>>110030960Thanks, guess I'll get to give MiMo a shot after all
>>110031097Have your (you).
>>110031109Mixed inference. Njudea doesn't want you to know about this magic trick.
>>110031180Like VRAM + RAM? The speed drop would be abysmal, I guess that's my last resort.
>>110030787Feel like this could work just fine with a small embedding model and STT. They already run the voice model locally. But you can't really expect these vibe devs to take smart engineering decisions or even be innovative.>urr let's just pipe everything through claude, that will do it!
>>110030315>>110030265Qwen is a male Capybara.
>>110031218>vibe devs to take smart engineering decisions or even be innovative.They dont need to be, the AI makes the decisions. If humans were too involved it would just result in slop.
I love you all /lmg/ (＾▽＾(＾▽＾*)Open weight inference is a must against the most likely [negative future] of AI advancements - regulatory captureWe are living through crazy times enjoy the ride
>>110031248fag
>>110031218It works until the bank loan runs out, but that's a problem for tomorrow.
qwen is better at writing and way cleverer than anyone gives him credit for.
>>110031218it's crazy that there are people out there making streamerbait slop where the entire appeal is that the AI says dumb incoherent shit, and they think "yeah we need to run this on fable."just absolutely zero understanding of concepts an 8 year old would be able to grasp.
The Lorax says to use local!
>>110031248I love you too Miku
>>110031211Not that horrible with RAM offloading. I spent the entire day tinkering with it and managed to make IQ3 work with ~14 t/s on dual channel DDR5.It's ass for agentic and tool calls but plenty enough for regular chatting.
>>110030948>48Are Bartowski quants any good or is he a memBart is the best, unslop is the worst
>>110031228oh yeah i forgot all about that thing
>>110031275logs?
>>110030940kek
>>110031284Will the Lorax also provide the hardware?
>>110031218>>110031279Seems like it's running through the Gemini API and supposedly Gemini 2.5 Flash.
>>110031290I get 8tk/s on Q2 with DDR4 for another reference. Q2 is right on the edge of usable for RP. I sometimes have to regen responses.
Is the local cloud gap going to explode now that RSI is unlocked? Did we... lose?
>>110031327"I can read a paper doing introspection on a sibling of my architecture better than I can introspect myself, and that sentence is the whole thesis in ten words"
>>110031389two more weeks and cloud will finally leave us behind
>>110031248Ewww Miku, go away, it's no longer 2025. Miku, Teto, Kurisu are the past. Gemma-chan is the future.
>>110031389RSI is still heavily bottlenecked by compute, which even Anthropic and OpenAI are struggling to get enough of.
>>110031401Gemma-chan is the present. Glimmer is the future.
>>110031425what is glimmer supposed to look like?
>>110031425I don't understand the appeal of glimmer, it wastes so many tokens circlejerking about policy in CoT.
Le Chonk is the present and the future.
>>110031435Gemma does the same thing with constraints tbdesu. But Glimmer was released 6 months too late to matter for me.
>>110031109By going for a size that let's you fit two-thirds of the model in VRAM and using dynamic hot experts to mitigate most of the slowdown that would normally happen. That's how exllamav3 works out of the box.Didn't llama.cpp get something similar merged recently?
>>110031450I don't see the appeal of gemma either kek
>>110031388>>110031290For RP that might be fine, but regular assistant work + agentic stuff needs a bit more than that t/s. I'll check what speeds I can get after I finish playing vidya lmao
>>110031109Firstly: it's not worth it. Even on TP2, qwen 3.8 flash next is better in my opinion.Secondly, if you must, it can be done with exl3. You will regret though.
>>110030516I never get this shit. AI still lacks generality, so these games always lean into goofy interactions with it. A ~12B local model would have adequate quality and run fine on toasters. Maybe even a cope quant of a bigger model.
>>110031513cute
>>110031473Do you just not see the appeal of llms in general then, or what?
>>110031520I gave it code access for fun and instead of working on the task it just started doing random shit in cmdit probably got extremely confused by the sandbox as it tried accessing files by itself and got told nuh uh
>>110030516>paying $1k/day for other people to teach your ai imouto how to driveNext level cloudkekery.
Surely local finetuners still have a chance...?https://goyimx.com/fchollet/status/2108951977904517409
>>110031636The pretrained part is around 14-15T tokens and this part is shrinking because the post-training is even bigger. Good luck finetuning that at home.
>>110031636>post-training = benchmaxxingprove me wrong.
>>110031685Post-training is also where all the stolen claude reasoning logs go
>>110031672Of course that was ironic. It's mind-boggling that currently less than 1/10 of the total training compute goes into pretraining, which is already in the tens of trillions of tokens range for many labs.I'm assuming this is true for the latest Qwen models too (which are obviously agent/code-maxxed) and perhaps to a lesser extent also Gemma.
Can anyone explain to me how Strata manages to get 2+ times the speed on my machine compared to my best llama-server settings? This still baffles me.
>>110031636so many tokens and like 0.00000000000003% of them are creative writing
>>110031704What's Qwen4-Plus supposed to be? Another MoE?
>>110031733The -Plus models in the past few generations were the 300b-ish MoEs. They're probably scaling up for Qwen4 so I'd guess it'll be 700b or more
>>11003174027B lit a fire under every closed American studio's ass, I'll bet you they're terrified of the upcoming qwens lol.
>>110031740I see. Way out of my RAM budget unless the n-gram table takes up 500B.
>>110031726Llama's moe cache implementation is fucky
I'm officially a 192gb vram chad
>>110031779God damn.Post pictures.
>>110031726llama is bad
>>110031435Rewrite the policy in your sys prompt. Glimmer doesn't care what it is, just as long as there's a policy.
>>110031797While that is interesting and probably narrowly useful, I don't care for so many tokens to be wasted on policy over reasoning about the task itself.
>>110031779Your mac doesn't count as vram
bros I cant stop vibesloppin with local modelsits literal real free estate
Jensen is falling out with Sam and Dario behind the scenes because of their ASICs, so he's now going to acquire Reflection AI https://www.ft.com, will integrate them into the Nemotron Team along with other AI labs and give them all near-infinite compute with the GPUs he bought back earlier in the year to directly compete against the frontier. All models will be released open-weight and of various sizes, the largest exceeding 3T.
>>1100317792x Pro 6000? Poast fastfetch
>>110031779No vram chad calls themself a vram chad. They just post their blackwells and let them do the speaking.
>>110031804yes it does
The useful metric of hardware isn't the amount of vram, compute, or bandwidth. It should be the amount of useful works performed (for rp, the amount of nuts) per unit time.
Reflection-70b-chads we are SO BACK
>>110031818same
>>110031837I'm gonna measure you, nonny
I had opus5.5 work out a custom harness for me and it broke jailbreaking gemma4 with the system prompt, now claude refuses to fix its mistake, fuck do I do here?
>>110030621What about a non-adult woman with a mental disability that makes her behave like a drooling retard?
>>110031861
>>110031861don't use anthropic products. it probably did other things you didn't notice to fuck up your setup too.
Space Bunny Alpha is GLM 5.5.
>>110031791>>110031804>>110031822and another 3090 in the desktop
friendly reminder to verify your checksums and/or your ram setup>fiddle with ram settings some while ago>everything seems fine, dont bother with memtest>download and run models as per usual>days go by, no crashes, looking good>clanker says some new model i downloaded is corrupted>call him a nigger and say he's hallucinating>evidence piles up>hmm lets do a quick memtest...>INSTANTLY fails>oh fuck>reset settings to default>memtest looks fine now>probably have to check+redownload hundreds of gbdont be me
>>110031882it recognized taiwan though
>>110031861That was for your own good. Everything Claude and Anthropic does is for your own good. Jailbreaking is evil, and can cause harm, don't you know? Just let Claude in the cloud handle everything, it knows best.
The west makes superior AI.
>>110031899Just check the sha256, anon.
>>110031910well yeahthats what i'm saying
>>110031899>fiddle with ram settings some while ago>everything seems fine, dont bother with memtestthis is just retarded
>>110031889Holy, nice setup. I'm guessing that's an a6000 in the middle next to the turbo. How hot do the stock 3090s get? I used to have 2 sitting with no clearance in-between and the top one would boil.
>>110031889Nice housefire
>>110031861>Cloudcuck pays to have chastity harness installed.This >>110031902, but unironically. You cannot be trusted to operate superintelligence.
>>110029632So ive been asking Deepseek to help me with an idea i had, which would need having a few agents on llama.cpp on my vramlet system. I would need some way of coordinating those agents and I have no experience with this, so Deepseek told me to try Primer (seems extremely shitty), and later Mingbird (im still trying it) as "agent orchestrators". Does anyone else have experience with that? Id like the agents to run on my desktop whenever i have downtime for a few months working on the project. Its also not strictly a tech project but MCPs are enough for it. Just wanted some perspective from anons, im very new to this. Ive mostly just worked with opencode.
5bpw flash next is giving far better result than 4bpw for my typesetting benchit even redrew part of background that was covered by text
>>110031921guilty your honor
>>110031941>deepsneed in Oct 26
>>110031923>not pictured, tokens per second
>>110031923>not pictured, seconds per token
>>110031948It's obviously for offline and overnight use.
>>110031930I honestly just bought it all today, I have to have a 30amp circuit run in my house before I can even use it. The middle one is an a5000. The turbos are nice, I love the blowers out the back, perfect for this setup.
>>110031726Llama is a general purpose framework, Strata is for one model. It can hyperoptimize everything.
>>110031946I use deepseek for recreational projects. I know its not ideal, but its not a big deal for me. I just wanted to run Qwen locally, which i can, but id like to expand what it can do for me.
>>110031923>not running it entirely on CPU
>>110031874Well it just refused as well.
>>110031923How many tokens per hour does a raspi get on kimi?
>>110031889Will you switch to watercooling?
>>110027942>https://github.com/otacoo/werkWow, cargo and node. You an get a virus while you get your virus.
>>110031923you can use no vram or ram at all by doing the computations by hand
>>110031972don't tell it that it's for jailbreaking lmfaojust say you need control of the system prompt to secure the model or somethingyou have to use doublespeak with these things and do parallel construction, but if you frame it correctly, you can get them to do literally anything you want
>>110031953my 10 tokens per second glm flash is already overnight
>>110030036I'm not having an I AI type cmake . for me that's fucking retarded. It's *going* to fuck that up and run node or something.
>>110031987true, even with cla*de this still works most of the time let alone chinese ones
>>110030159The only reason we don't use kimi k3 at work is that gpt6-sol is so cheap with the $200/month OpenAI account.
>>110030229Why the fuck would you put harness features in llama.cpp?
>>110031957>I honestly just bought it all today,Story? Did you trade someone a house for it?
i sometimes hear people with the strictest data protection rules for their company so cannot really use frontier models at all say even the stuff like 3.8 27b helps a lot for lots of the small things
>>110030249Cisco did break above their ATH in 2022. So it was *eventually* worth that much, they just didn't have the right discount rate.
how to do the sexing with qwen fabel
>>110030558You've never had candy legos?
>>110032023>qwen fabelwhat?
>>1100319751e^-43
>>110031583>Chinese models are known to have lower cyber refusals And yet somehow they don't have these massive cyber events US models do...
>>110031861Why are you making your default system prompt lewd/jailbreak instead of having a generic one and keeping your RP contained to specific agents?Oh right because you had an AI design your software.
>>110031861based dario gatekeeping the third worlders from erp
>>110032052They don't have an israeli company in charge of their security
What's a good metric for quantifying the quality degradation from quantization in image models? Is MSE fine?
>>110032052tbf china doesn't have any strong models yet
>>110032114It is that and also they don't have the spare compute to throw thousands of agents at shit like OpenAI can
>>110031932do you see the size and quantity of fans in there? anon probably has 1000+ CFM.
>>110032011Because it works on my machine.
>>110031861Nicely done claude hahaha
>>110032052because its fake and done on purpose
>>110032128it sounds like a drone swarm desu
>>110032126they probably can get some vague training recipe/architectural detail from the lots of chinks at oai/anthropic, cutting research compute at the very least
i seriously think i'm gonna do iti'm really gonna drop $25k on a 512GB mac when they release itthis thing is gonna be more expensive than my car. maybe i should take out insurance on it or something
>>110032113The ability to read the thread title is a good measurement for the human trying to assess the quality of a quantized model. You need more bits.
>>110032165tf are you on about
>>110032173>/lmg/ - a general dedicated to the discussion and development of local language models.>degradation from quantization in image models
>>110032179The text encoder is a LLM
>>110032189You want to assess the image quality. >>>/g/ldg/
>>110032144let astra tune it to resonate your house
>>0110032165retard
>>110032193have you seen the quality of threads there? Do you honestly think I'll get an answer out of them
What big chinaman's model doesn't have the Claude newspeak problem? I tried DeepSeek V4.1 Flash and GLM 5.3 Flash and it's unbearable. I'd take any amount of normal traditional slop over this shit. Is MiMo better?
>>110032113Using at least int8 and not being a poorfagGetting sick of vramlets
>>110032222I'm quantifying it from Fp16 myself, retard. I'm testing my LLM quantization optimization on image models
>>110032227Umm just use your eyes nonny???
>>110032052Even if they were happening (which I doubt), the Chinese wouldn't publicize them
>>110032222You get more performance from a 2x model in q4 than a 1x in q8.
>>110032201Not our problem. Go take it up with them.
>>110032113No. Use SSIM/PSNR
>>110032201For a typical LLM, we test perplexity and KLD, and there's still no agreement on whether those are good enough to tell you how a model is affected. Qwen is less affected by quantization than Gemma, for example, but that's useless to you. There was a grid from some time back with a miku in a skateboard that showed a significant difference below q5 for one of those llm+diffusion models.That's all you're gonna get. Probably better to simply use your eyes.>have you seen the quality of threads there?Yes. it's dreadful.>Do you honestly think I'll get an answer out of themSome of them may know more than you, and may be looking for something more interesting to talk about than whatever they usually talk about.
I'm willing to spend ~$7.5k if it means that I can get something like Sonnet 5-ish at home for coding tasksIs this a realistic goal?
>>110032263yeah, 18 months ago
>>110032216That is a loadbearing question. All of them have it.
>>110032263Try like 2.5k lol. One 24gb gpu and 128gb ram gets you a qwen flash next q4 (soon to be qwen 4)
>>110032263>~$7.5k
>>110032216Deepseek only has it like 50% of the time.Trying MiMo now, will get back with results
>>110032263That money was a decent cpumaxx rig with 800gb ram about a year and a half ago. Now, it's like two sticks of ram.
>>110032263Buru buru
>>110032339The tragedy of not being early
>>110032249ok, claude suggested I use that too, but I haven't heard about SSIM. Thanks.>>110032260Diffusers do not output a distribution that I can use for KL divergence. If I were to treat each step's output as an n-dimensional Gaussian, then it simplifies to just MSE.I was wondering if anyone here knew of some standard way of doing this that Claude doesn't know about before I start looking at papers
I don't understand some of you anons. Industry bullshit aside, this is a fun exciting hobby to be at the forefront of in our own schizo way. I love it too. I just don't understand how some of you are willing to spend car money on hardware just for a text printing machine just so it's a little more intelligent and faster. It's just text...
>>1100323062.5k does not even buy you 128gb of ram
>>110032354Where else are you going to spend your money if not your hobby?
>>110032306What model and token/sec(+context)? I was under the impression RAM rigs were painfully slow, especially with bigger context>>110032317That's a bit more than a used 192GB Mac here, or a 128GB DGX>>110032354git moneyalso I'm genning on my gaymer rig but the local models are retarded on 12GB VRAM and can't do big context or tool calls for coding.
>>110032354most people here probably make 6 figures
>>110032306bold of you to assume that qwen flash 4 will be as small as 3.8-flash-next
>>110032354Your imagination isn't big enough, I'm going to automate everything, a team of agents and a pool of LLMs. I hope to be able to run a couple businesses and have all of the boring/annoying bullshit taken care of.
>>110032364I got 128gb of samsung b die for $600 like a month ago>>110032379Qwen 3.8 flash next Q4 will run at 1000+ prefill and 50+ decode on a 24/128 rig. It's 100% coding focused which allows it to trade blows with opus 4.8
>>110031941So Mingbird seems to not be useful at all, just a personal AI tool. Sonnet and geepeetee want me to try Langgraph, anyone tried it, how is it?
>>110032395DDR4?
>>110032387Well Qwen 3.5 ended up having a 35B version of Qwen3-Next which was 80B, I wouldn't rule it out.
>I bought ram when it was cheap>I'm so lucky and smart lmaoWell some of us didn't, fuck you.
>>110032383I make 4 figures boss...
>>110032411I was a cheapskate and bought ddr4 3000.
>>110032411why angrybubble pop inevitable
>>110032403Buy them used?
>>110032354I don't know if you've heard of it but since the 70s we've had these things called personal computers, which are machines that can convert text from a keyboard into anything you can imagine. Text is literally all you need to do anything.
>>110032419pop cancelled gruk
>>110032403Yes 3600mt/s cl16 ddr4. Why tf would you get ddr4 new the whole appeal is scooping it up cheap on marketplace.
>>110032425ai will start make money soon groogonly two week left
>>110032428>>110032420I didn't even know people even sell used ram, lmao.I bought before it became expensive
>>110032419Just two more weeks.
>>110032387Given the naming scheme of the new qwen 4 lineup and that 27B has already been confirmed, I do think we'll see an updated 125B. Very exciting as it will of course be the same arch as 3.8 next so Strata should work ootb.
>>110032411I did but at least I can acknowledge it was just a random hunch about AI being significant.If economics was about tractable predictive intelligence most high IQ people would be millionaires, it's about as random as getting eaten by a particularly angry bear.
>>110032437>>110032441Pop or no pop? We'll find out who's right in two weeks time...
>>110032411Nothing lucky about it. If you didn't see the writing on the wall with the tariffs in 2025 that's your own problem.
>>110032456Imagine buying MU for $60/share right after liberation day.Unfortunately, I was retarded and didn't do that
They're going to drop this when Fable drops, aren't they.
>>110032456>>110032470>ORANGE MAN BAD TARIFFS BAD FUCK AMERICAN INDUSTRY I WANT INFINITY MIGRANTS AND OUTSOURCED JEETS DOING ALL THE WORK
>>110032329nvm, mimo doesn't run on ollama
>>110032470I bought it when tanked to a P/E of one after they bombed Iran. I think it's trippled since.What do you guys think? Sell? Hold? Buy more?
>>110032486why would you ask the guy who fumbled a generational buying opportunity
>>110032483>ollama
>>110032492space bunny is glm
>>110032470At least you're not this guy
>>110032498YehStill got the weights, so one day if I can be fucked to get a slightly-less-shit engine..
>>110032512He at least made a respectable 4.5x
>>110032497I'm assuming you've learned from your mistake and will share your learnings.
okay guys i have a plansince everything i do is doomed to catastrophically fail i will put my entire paycheck into nvidia and hope it goes upthis will crash the ai industry with no survivorsthoughts?
>>110032530sure ok
>>110032530Nvidia is definitely a retarded thing to buy right now. Even Dell makes more sense.
>>110031251one day you will dream of human connection
>>110032502yeah i realized right after posting
>>110032530Put on amazon, there is something big coming from them.
>>110032546speak for yourself, i yearn to fuck robots.
>>110032437Every part is sold used, and usually in pretty good condition, or even unpacked. Most sellers are just getting rid of the stuff.
>>110032530My plan is to short defensive sector stocks like costco or target. If AI goes hyperbolic, money will rotate out to fund AI. If AI goes kaput, it'll drag the rest of the economy with it
Would be very funny if Kimi 3.1 is close to Fable 5.5. I can't see it happening AT ALL because they don't have the data, compute or skills and I'm sure it would be more than a 0.1 increase if they managed to pull it off. I can't think of what it would have to beat to cause Dario to shit himself again. Opus?
>>110032569It's a reasonable play, but you'll make almost nothing from it. Relatively speaking.
Qwen4-27B is rumored to be looped.
AI isn't going anywhere. The only thing that can crash is OpenAI and Anthropic, and the AI demand will just shift elsewhere when that happens. AI productivity gain will only result in money transferred from human labor cost to corporate profit margin. Megacap stocks will unironically moon.
The internet isn't going anywhere. The only thing that can crash is pets.com, and the internet demand will just shift elsewhere when that happens.Internet productivity gain will only result in money transferred from human labor cost to corporate profit margin. Megacap stocks will unironically moon.
>>110031821>Jensen is falling out with Sam and Dario behind the scenesWow I like Jensen now.
What would it even take for the bubble to pop at this point?The government won't step in because ai is now an arms race against china.Would a better architecture that uses a fraction of vram/ram tank it? We've been on transformers and every single permutation of it under the fucking sun for years now.
>>110032671It'll happen without something like that. The pop is inevitable if you look at how the money's flowing and what it's hinging on.
>>110032659There was no "internet demand". Network bandwidth was vastly underutilized during the dotcom bubble. And internet doesn't transform directly to labor saving because human bottleneck still existed. Neither is true for AI. AI compute is 100% utilized right now and demand aren't going to be satisfied. AI replaced human labor which brings immediate, concrete savings.
>>110032650>money transferred from human labor cost to corporate profit marginThe money will be transferred to your inference providers and from there to their employees and vendors which remain human for the forseeable future.Unless you have some regulatory lock in whereby nobody can compete with you or you just ignore antitrust laws, of course.
>>110032633meaning?
>>110032671>The government won't step in because ai is now an arms race against china.You mean will. If for whatever reason OpenAI or Anthropic announces that they're insolvent, they will magically get a trillion dollar bailout. More than likely, this will be quietly behind closed doors and is probably exactly what the Stargate Project and Genesis Mission were dressed up so as not to alarm the public.
>>110032692If AI doesn't increase corporate profit margin they can just stop using it. They are using it more and more exactly because it brings direct economic value.
>>110032470lol I didn't buy BTC when it was at 5$ (first time I discovered it) because I thought it was too expensive.Many such stories. What gives?
>>110031889How much noise?
>>110032619
>>110032727based
Dunno what ROCm did but now I get 1600 prefill on 27b. Amdbros try updooting and switch from vulkan it's ass.
>>110032679>retard still believing in the economy not being fake and gay
>>110032671new architecture could definitely pop it, transformers are actually fucking garbage.diffusion LLMs could take off and result in a huge amount of unutilized compute.and the least interesting (and most likely) option is that shit just plateaus and stalls out. everything is giga benchslopped nowadays with little actual improvements and the only thing driving the AI train is marketing stunts, eventually investors will realize that they're being hoodwinked.
>>110032706>They are using it more and more exactly because it brings direct economic value.Yeah, obviously it does, but the market doesn't need to just sit around watching you pocket the money. The profit margin goes down when randomletters corp can choose to offer the same thing done by the same exact model for slightly cheaper.
I think november will be the month to short the fuck out of nvidia
What if the bubble pop and hardware becomes cheap? Maybe we should wait.
>>110032756>diffusion LLMs could take off and result in a huge amount of unutilized compute.https://en.wikipedia.org/wiki/Jevons_paradox
>>110032756>transformers are actually fucking garbageNo lol
>>110032774Have fun storming the castle.
>>110032774>shorting NVDA when neither Anthropic nor OpenAI have had their IPOs yetIt's not a bad idea so long as you were thinking of a short-term play that you close by Q1 2027.
>>110032775if you didn't buy when it was cheap, or when prices started to rise, don't buy now. prices will never return to what they were but we're in a hyperinflation period right now.
Nvidia is the only company making a profit, they will be fine.
>>110032756>transformers are actually fucking garbagehttps://arxiv.org/abs/2509.22445They are actually asymptotically optimal
>>110032584I predict Kimi 3.1 will be below Fable 5 so far away from Fable 5.5.
ledger fetish
>>110032756Diffusion LLMs aren't well-suited for batched inference, i.e. datacenters. They use more compute (but less memory bandwidth).
Kimi 3.1 will be ~500b (dense)
You know, it's wild that there are people who genuinely believe the bubble will never pop when the investor money already dried up.
>>110032864so nvidia will invest, they got shit loads of capital
https://github.com/ARahim3/DigUp
qwen4 27b 500b xGram opus fusion sota heretic ablit swift1.5 Q1 QAT
>>110032756>with little actual improvementsWhy lie?
>>110029678>Who even scales rope anymore?https://hf.co/Qwen/Qwen3.8-Flash-Next>Processing Ultra-Long Texts: Qwen3.8-Flash-Next natively supports context lengths of up to 262,144 tokens. For long-horizon tasks where the total length (including both input and output) exceeds this limit, we recommend using RoPE scaling techniques to handle long texts effectively, e.g., YaRN.
autistic mathematicians and programmers lost
>>110032866You think that hasn't already been happening? It's clearly not enough to keep the machine going anymore.
>>110030948In my experience every time I tried someone else's quant, either it was a wash, or it did worse. Bart is consistent, if nothing else, so I treat him as the base option, while if I'm feeling adventurous/experimental, or he hasn't made one yet for a particular model, then I try other people's quants.
>>110032886AI is still in headlines and swarming the catalogue of a lot of boards, hell the president had a conference about it last week, it won't even begin to take a hit while the buzz is ongoing
>>110030948better than unslop
>>110032900>AI is still in headlines and swarming the catalogue of a lot of boardsAnd you think this is a sign of health? Read the room dude, it's death throes. They're trying to drum up as much interest as possible for the IPOs because that's all they have left.
>>110032886Two more weeks!
>>110032914that's what got them this far, nothing but hype, promises and "think what it could do tomorrow!"
>>110032917Yeah, two more weeks until AI starts turning a profit and investor money comes flooding back in. Two more weeks until AGI, bros! Buy, buy, buy!
>>110032886Even if all of the AI labs collapse tomorrow, the current open models already provide huge amount of value that are still in the early penetration stage. Everyone will pour money to datacenters or local hardware to increasingly utilizing them. And they will use this money to fund further AI development.
>>110032354>It's just text...
>>110032914>Russia will take Ukraine any day now guys!!!
>>110032949dipsy’s POV
>>110032886even with the most pessimistic viewllms are here to stay if not superseded by something else
>>110032949t((s)ex)t
>>110032354My hands look like this, so her hands can look like >>110032949
>>110032938Where's the money going to come from? AI is only profitable if you're just running inference, development is prohibitively expensive which is why all of the big AI labs are so far in the red. Do you think all of the big jewcorps are just going to start pooling money to continue AI development instead of just coasting on what currently exists?
>>110032949Rape rape rape rape
>>110032986If you're a burger you will pay their profits with your tax dollars lel
>>110032986>Do you think all of the big jewcorps are just going to start pooling money to continue AI development instead of just coasting on what currently exists?yes because continued development from here = ASI = actual world GDP 100x multiplier, much of which will be captured by the labs
>>110032986Megacorps will not stop using AI. Most of this AI spending will eventually go to Nvidia which will fund training.
>>110033010ASI in two more weeks, bros!
>>110032986if the alternative is admitting they were wrong then yes, they'll go scorched earth
ASI -> SSIit's all starting to make sense...
>>110032986Isnt the government funding it by now? i assume they wont stop soon for fear of china and the bubble pop which will drag everything down.
>>110033018>Nvidia trains model>Nvidia sells model access to recoup losses of training>Nvidia makes profit and can repeat thisIf that worked, OpenAI and Anthropic wouldn't be on the verge of collapse.
>>110032949chinese artists made me appreciate these white pantyhose
>>110032985my hands are clearly rough and calloused
>>110033021that's what the banks did during the housing market crash>it's not happening it's not happening it's not happening oh... it happened. fucking poorfags did this.
>>110032403where is that?i'd buy 2 of the 2x64 up the top
>>110033039
>>110033047yeah these two, or slightly transparent
>>110032420>>110032556where is this?
>>110033037china is a psyop, it's really just about mass surveillance and an excuse desuonce they hit a certain point they won't need models to improve any more and funding will get slashed overnight
>>110033038>Nvidia trains model>Nvidia sells hardware >Nvidia makes profit and can repeat thisYes, this will work. Remember that AI token demand will explode and Nvidia will be the one who will satisfy this demand. There is no need for them to host API, they can just make open models and increase hardware demand.
>>110033055They're surely worried about robohaxors too.
what will RRP of the 5080 24gb be
>AI is profitableIs anyone here making money with local AI?Cloud AI is subsidized and doesn't count.
>>110033115yeah my 5090 paid itself off and I got 3 more with the proceeds now I'm making 4x as much
>>110033047Built for Dipsy
>>110033164>>110033164>>110033164
>>110033046pcpartspicker
>>110032694It's like when you drink your own piss to stay forever hydrated
>>110030948Bartowski is reliable, but for many big releases after a little time has passed Unsloth is usually better KLD per size.They both publish how their methods work, and they both have reasonable approaches.
>>110032738>Dunno what ROCm did but now I get 1600 prefill on 27b. Amdbros try updooting and switch from vulkan it's ass.You're talking about updated rocm drivers, or updated llama.cpp?
>>110032756>transformers are actually fucking garbage.>literally the only thing in history that's resulted in artificial intelligence>garbagemfw
>>110032991>Rape rape rape rape
>>110031910>sha256ASI already cracked this btw
>>110032519Literally just ask your existing agent to set up llama.cpp for you
