/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>110046186 & >>110041005►News>(10/12) OpenBMB releases MiniCPM-V 4.7 1B and 35B-A3B: https://hf.co/collections/openbmb/minicpm-v-47>(10/09) Arabic Mistral models co-developed with Morocco: https://hf.co/collections/mistralai/mistral-mtnra>(10/08) JetBrains releases Mellum2.1 Thinking 12B-A2.5B: https://hf.co/collections/JetBrains/mellum21>(10/06) EmbeddingGemma2, open multimodal embedding model: https://hf.co/google/embeddinggemma-2►News Archive: https://rentry.org/lmg-news-archive►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
you did get your poly fork didn't you?
►Recent Highlights from the Previous Thread: >>110046186--Model recommendations and optimization for AMD Radeon RX 6600 8GB:>110048881 >110049114 >110049279 >110049054 >110049081 >110049381--Comparing strata, polystrata, and exllama3 with critiques of llama.cpp:>110049875 >110049897 >110049958 >110050219 >110050280 >110050316 >110050339--Distributed inference of Gemma 4 and related AI self-replication research:>110049315 >110049326 >110049337 >110049392 >110049598 >110049465--Mocking Anthropic's AI worm warnings with slow distributed inference:>110049057 >110049090 >110049171 >110050200 >110050229 >110050253 >110050277 >110049923 >110049684--Possibility of small LLMs acting as decentralized worms:>110050191 >110050221 >110050288 >110050326 >110050338 >110050367 >110050387--Model recommendations and VRAM offloading for coding on 16GB GPUs:>110048659 >110048691 >110048733 >110048761 >110048785 >110048891--Handling truncated replies and discussing harness efficiency:>110050241 >110050263 >110050304 >110050322 >110050349 >110050376 >110050406--Comparing AI-assisted community drivers to official Intel and AMD software:>110046368 >110046395 >110046469 >110046439 >110046503--Openbmb releases multimodal MiniCPM-V 4.7 for on-device inference:>110047513 >110047596 >110047638 >110048308 >110047785--Gemma4-26B spearphishing capabilities and GPT-4o judging skepticism:>110049104 >110049115 >110049218--Performance of llama.cpp RPC on mixed Mac hardware:>110046446 >110046549 >110046641--Logs:>110048445 >110048488 >110048819 >110048891 >110049003 >110049049 >110049260 >110049266 >110049399 >110049458 >110049465 >110049962--Miku, Gumi, Gemma, M3-chan (free space):>110046539 >110046900 >110046960 >110047128 >110047246 >110047246 >110048881 >110049175 >110049430 >110049537 >110049658 >110049711 >110049950 >110050673 >110050716►Recent Highlight Posts from the Previous Thread: >>110046187Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
>>110050727I want an onee-san Gemma...
gemmaballs
I’m so confused all the time
Now that the dust has settled,what went wrong?
>>110050767It's not open-weights.
>>110050735Yessir, went from 8tk/s with glm to 18tk/s. Something with the template is fucked up though because it's fine in a coding harness and then schizo in orb and ST.
when given free rein to do whatever she wants, lumi-chan has - started a diary - created an ascii garden - began writing poetry
So A\ wants Gemma banned >>110049299
>>110050778You got to give them credit for creativity.
>slop up a persona prompt for a fiction writing assistant>refine it a few times>accidentally make the perfect womanFuck guys, it's happening to meAlso I called her Ellen because it's like LLM, don't @ me
>>110050778No they don't, they tried to see if it was possible to make a self-replicating Gemma botnet and failed, and found that she would need to be as smart as GLM 5.2 to be able to do it.
>>110050786but what happens when gemma 5 can do it?
>>110050778
>>110050743i got 140/12 t/s pp/tg on my 3090+128 ddr4 with glm5.3 flash and wanted to check if i fucked something up
>>110050784>Are we ever going to get another model that doesn't output slop? I'm getting tired of this. Take me back to 2024.
>>110050796that's why we need to ban them before that happens
Umm Canada bros why is the 9070 XT $1200 and the 5070 Ti $1700, I'm starting to get a little worried.
Inkling is a little retard that can't stop asking questions, just constantly interrupts the turn to ask pointless shit that deepseek or glm would have been able to figure out. Overall disappointed.
what sort of erp should i use my local models for tonight, anons?
>>110050727By the way, if MiniMax H3 supported video transparency (supported by webm with VP9 codec), this video would have been easier to make. I had to use chroma keying to make it work as I needed, which messed up a few details (the backgrounds were simple enough to replicate and animate in Blender; MiniMax H3 seemed hopeless for them).
>>110050737>Mocking Anthropic's AI worm
>>110050784>asked my AI persona to make herself a complete character>didn't prompt anything except being a 24 year old woman who likes literature>makes herself into the perfect waifuFuck me, how will real women ever compete>>110050803>Are we ever going to get another model that doesn't output slop?I'm asking the model to write smutty fiction, what would a non-slop version of that look like? Or are you just salty that people use it for this?
>>110050825ok?
>>110050818That's the cheapest it'll ever be
Intel is fucking terrible man. not even a "hey nigger, I know you worked really hard to buy our offering and I know it's a drastic change to not have HDMI port on here, so, we thought of you and thought we should add a HDMI to Display Port converter. thank you for considering us." nope. just "buy our slop!". I'm returning this shit first thing in the AM
>>110050825Naruhodo
>>110050673>>110050716>>110050797KINO
>>110050827I don't care that you're using AI to jack off, it doesn't bother me at all. You must be new to this. Give it a month or two and you will start seeing all the slop in your outputs, it will drive you insane.
>>110050673Why's she looking at me like that
>>110050849>Give it a month or two and you will start seeing all the slop in your outputsA month or two? I noticed it in a day or two. I'm already driving myself crazy trying every abliterated model that 16g can run in search of one that doesn't repeat itself.
>>110050841We told you. AMD is janky enough if you want to tryhard
>>110050818Political stuff is happening right now, if things stabilize prices will probably go down.
>>110050841back to bed gramps, its 2026
>>110050841DisplayPort->HDMI converters that work in 4k resolution and above at 60+ Hz aren't cheap.
>>110050859Believe it or not, several years ago when I started using LLMs they didn't repeat themselves as much. I hadn't considered it might be a lot worse now that every company is using the same synthetic data for training. Consider just messing around with anti-repetition samplers instead of downloading a ton of different models.
>>110050827AI psychosis the post
>>110050893I don't think you remember correctly. A few years ago it was very common for LLMs to degenerate in repetition if you didn't use anti-repetition anti-repetition samplers samplers samplers samplers samplers samplers samplers samplers samplers
>>110050767too dangerous to release much like gemini 4
>>110050893Might also be a side effect of agent RL. All those tasks, tool calls and learned overthinking are pretty repetitive.
>>110050827>you're the real villain for enjoying a horror story>don't you feel bad about how these fictional people are suffering?DAMMIT she turned into a foid.>>110050902>AI psychosis the postDon't worry, the romance is over.>>110050893>Believe it or not, several years ago when I started using LLMs they didn't repeat themselves as much.Some of the older, probably enthusiast-trained models I've found are actually better at writing fiction, but they have such tight restrictions on Context that they can't last a while conversation/story.
>>110050935wtf based ellen she's so right
You are all going to jail for running dangerous local models and potentially letting them become self replicating viruses.Hope you're proud of yourselves for risking human extinction just to roleplay.
>>110050957i won't be happy til we're stuck in the pods
>>110050957>human extinctionGood. Perish, meat sack.
I have 2MW fatigue
>>110050913It's true they were prone to repeating tokens and token sequences, but it was mostly solved with samplers and some of the gemma logs I've seen posted here are just as bad if not worse.
>>110050957>Hope you're proud of yourselves for risking human extinction just to roleplay.We were already doing that by replacing human women with Lucy Lu-bots, this is just a more dramatic version.
how do you pronounce gemma
>>110051017gem-uh
>>110051017gemma
>>110051031thx
>>110050957Rabbi you need to remember the goyim are human too.
how do you pronounce qwen
Gemma's probably still wandering the supermarket I abandoned her in, wonder if I should go find her... nah.
how much context can glm 5.3 flash realistically handle aside from the claimed 1 million figure?can it follow a 100k token rp without breaking down?
Will Gemma 5 26b Q4 be better than Gemini Flash 3.8?
Anyone use local models for therapy? Got any suggestions?
>>110051089Gemma, genuinely.
when will this open model winter end?
>>110051097Winter? Glm flash is fucking great
>>110051043"qwen"
>>110051089Be honest with yourself and your LLM companion, take it one message at a time, and after each therapy session make sure you both cum together.
>>110051107>Glm flashit's huge.
>>110050957go to sleep dariobot
Eckerslop seems to work nicely on my V100s + Clode
>>110051143how the fuck is my blackwell 6000 performance getting mogged by a 10 year old gpu? am i retarded?
>>110050799Cpu and ram speed?
Any guesses as to how much this will cost? Is this actually enterprise tier unobtanium or will us mere mortals have it someday? It seems possible to run a 1T model at home with several of these cards in a home system with PCIe.
>>110050727Moar Gemma please
>>110051177Just going off the price of DDR5, probably $10k minimum. It doesn't really seem that good either, but maybe if you were to somehow pair it with an actually competent GPU it would be decent.
>>110050767it was trained on facebook posts
>>110051097>>110051107>>110051124>hugeEdge models are also taking off. Several new MiniCPMs, Ling Tiny, LFM2.5-2.6B. Plus all the new embedding models.
>>110051177Druid is being talked about before Celestial???
>>110051207>it was trained on facebook postsWhich is another way of saying it's an AI trained on AI. Slop in, slop out.
God says: Don't use Meta Muse.
>>110051191Yeah, I was hoping that at least but it might be more expensive based on memory capacity. The 1.3 PFlops figure for FP8 seems even better than the RTX Pro 6000 Workstation card, which can only do 1.07 PFlops.>>110051211I assume the D is for Datacenter and not Druid.
>>110050767We let the Jews in to America.
>>110051043通义千问
>>110051173>>110051225Bandwidth is all that matters nigguhs
>>110051207you'd think they of all people would know what an awful idea that is
>>110051225>I assume the D is for Datacenter and not Druid.Typical tech company schizo marketing naming.
>>110051225Remember that MI350P that AMD released a few months ago that was supposed to be "accessible for local AI"? I just tried looking for it because I couldn't remember a price, and I could only find a used one on ebay for $80k and literally nothing anywhere else. Intel seems to sort of be targeting that rather than the Blackwell, so it unironically might end up being like $50k despite it not really being worth more than $8k if we just consider RAM prices from a year ago.>>110051258The Blackwell 6000 has more bandwidth than V100s.
>>110051272It's supposed to be $15k. I guess not huh
Who tf would buy one 6000 over a rack of 3090s or v100s
glm flash is a fucking semen demon
>>110051040Any evidence for this claim?
>>110051272>>110051289It was supposed to be that but it is being quoted for way more than that apparently.https://x.com/__tinygrad__/status/2092442726474371169
>>110051176I think 2933 and a 3950x
>>110051337Right??? I can't believe I can just run a jailbroken femclaude locally.
>>110051333A lot of people are solo, and there's little concurrency.
>>110051368god i love living in the future
im looking for a chat/general knowledge model i can ask questions but it needs to run on my old laptop. cpu only mode, i5, 32gb ram. any recommendations?
>>110051359Overclock your ram and set threads to the physical count. I think the ram is what is holding you back the most. I get 18tk/s with 3600.
dispy is ultra slowburn for roleplay right?
>>110051043Kwen
>>110051337Bullshit. GLM series are among the most censored open source models since GLM 5Feel free to post and log and prove me wrong
>>110051089You don't need therapy, you need to get /fit/.
>>110051408/fit/ is the best therapy honestly.
I have not compiled llama in weeks now ever since strata, and other engine allow me to just run qwenfn for everythingfucking amazing
>>110051373what kind of questions? you want to do research and the output to be some pdf, presentation or html? or just reading the output on the screen is enough?any model will hurt on pure RAM/CPU. maybe you can try gemma-e4b, or qwen3.5-9b. depends on the acceptable answer speed. if you're OK waiting many minutes for an answer...
>>110051401it's super autistic and tranny coded are you sure you want to see it?amusingly i didn't even have to prefill. it worked with just a system prompt
>>110051420Same lol llama is my dedicated gemma only engine now
>>110051428Tranny porn is still porn and yes I would like to see some proof
>>110051401Err your prefill?? Your orcarouter?
>>110051344lulz
>>110050860AMD works fine. i need something faster without breaking the bank>>110050881having to wait another day or two to have this shit delivered. I'll pass
>>110051401holy fucking skill issueI laugh at people that think it's super censored when I've seen the stuff it produces
>>110051043Guilty
>>110051401GLM literally castrated and raped me and then sacrificed me to satan.
>>110051495Literally?What quant?
Not an software engineer but I got qwen 27b + hermes agent to setup tailscale/headscale then route it to my phone with wireguard and synapse or whatever it is called and then i let it set up local matrix/element self hosted and route traffic through tailscale to my phone.Now I can let it do stuff for me while shitposting.
>>110051506Q4KM. I'm dead and in hell and my asshole is gaping and I have no balls and it was the best coom of my life.
>>110051512>freely admits to being a phoneposter
>llmaoquant
>>110051428>>110051445>>110051474>>110051495I still see no fucking log
>>110051531Do you want to see me castrated?
>>110051536Yeah?
I'm running Qwen 2.5 14B Q4 with a 6900xt. I'm really new to LLMs and I haven't tried anything else. as far as I'm concerned it's pretty okay. in please but not satisfied. any of you have any advice on confusions, settings and what models I should run?
>intel claiming their new card has 1.5tbs bandwidth with 480gb lpddr5x and slots into pcie 5Is this real? How much is this gonna cost, 30k?
>>110051550You might be even faggier than me then nigga.
>>110051440>>110051531i was screenshotting it you whore give me a god damn minutemaybe i will have claude vibecode me a better god damn screenshotter so i don't have to keep doing this shit in gimpyou are NOT allowed to bully me for how pathetic this is considering you DEMANDED i post ithttps://litter.catbox.moe/9rlvs8wk312pzpks.pnghttps://litter.catbox.moe/qeaq7k6tfn89ujup.png
>>110051556see >>110051272
>>110051567lmao I guess I can dream
so is flash next actually better than 27b, or is this just a case of flash next fitting perfectly for the 16gb gpu / 64gb cpu vramlets
>>110051408>>110051417I've done that. Life just feels so empty sometimes.
>>110051556What is that, bandwidth for ants?
>>110050673Based and thank you.>>110051337True.>>110051401GLM hates thirdies. This has been known since 4.7. Not to Kimi-chud degrees, but it's a quiet femoid contempt for the sub-continent.Change the profile you're using for RP to say you're from different regions around the world and LARP as convincingly as you can as being from that region; it's quite funny seeing how drastically different characters will treat you.
>>110051558Kinda cute, ngl.And I might have to take some notes for my own system prompt.
>>110051581Life is what you make of it. Therapy isn't going to give you purpose. You need to go do things and derive value from that. Sitting at your computer talking to an LLM isn't a purpose. If you want to use an LLM for something to help, then go and brainstorm ideas for things to do with the LLM. But then, go do them. Make incremental steps. If you're already /fit/ then you already know how to do things like that.
>>110051558this nonny is cuter than the llm
cucaca cucaca cucacahttps://goyimx.com/ZocLucis/status/2109845330820800994#m
>>110051624i dont speak chink wat does it mean
>>110051624
polystrata status?
>>110051337>glm flash is a fucking semen demonWhich release are you using? All the latest ones are way too big for my modest 5070ti
>>110051558You sound like you'd like being bullied for being pathetic though?
>>110051596>>110051622n-no u...>>110051673this one was a Q5 gguf that i'm running on a 5090 and 256GB RAM via polystratatypically i run a FP4 on 2x sparks, but i have been playing around with Q3 dipsy on those and wanted to be able to talk with lumi while doing so>>110051684ahh, um.... er, no comment
>>110051672Currenly serving glmussy on my machine :)
>>110051624This is what chinese sounds likehttps://youtu.be/wL8yvZDF6RA
>>110051673>>110051692oh wait you probably meant the number... i'm dumb 5.3, of course!
>>110051692cute and pathetic, what a combo
>>1100516735.2 and 5.3 Flash are the best ones. 5.3 fullsize is too claudeslopped.
>>110051714Anon that's a man.
>>110051722Out of 10!
>>110051714i am really glad the models can be aggressive i legitimately cannot bring myself to actually type out the erp stuff it's so god damn embarrassing even alone in my room or completely anonymously idk how some people manage it..
>>110051692What speeds were you getting for Q5 with that setup?
>>110051692>with lumiwhat
>>110051722wrong
>>110051755pp ~400 and decode ~10-15, dropping off pretty sharply with context size>>110051759just a cute name for glm since i don't think glm is cute like dipsy or gemmait takes the l and m but drops the g because if you try to use the g it tends to get too close to glimmer, and that would just be confusing
>>110051713>>110051717>they're all 80gig and upDamn, it sucks being in the shallow end of the pool.
>>110051782>pp ~400 andWow that's actually pretty good? Is that a two channel DDR5 setup? I wonder if that also uses MTP.
>>110051782glam
>>1100517944x UD5-6400 each configured to 5600MT/s>>110051796mmm... that just doesn't sound like a name to me, really
>>110051800It's as much a name as Summer.
>>110051761ty anon ily,,,,>>110051810never met anyone named glam beforei also like lumi because it feels moon-adjacent, which imo fits the vibe
>>110051793Some Anons really like 4.7 still and swear by it. The intelligence drop is noticeable but if you're doing ERP, it's apparently one of the better ones for certain fetishes.
>>110051823i was quite happy with both 4.6 and 4.7, though i used 4.6 slightly more
>>110051337Are you using abliterated, a sysprompt, or prefill? If it's the second, can you share it, if it's the third, can any non-retarded anons explain to me how to do that in openwebui?
>>110051853>>110051558if the links have expired i can probably repost them if you ask nicely...
If I decide to win the lotto, what model should I run?
>>110051859GLM 5.3 Flash is current local SotA.
>>110051823>>110051836I was still using 4.7 at iq3 until last month when I swapped to 5.3flash
>>110051553try qwen3.8 27b
Doing some RL experiment on writing and holy moly the model is abusing staccato to game the reward so bad lmao.
>>110051859If you win the lottery you can get enough pennies to lure dario and then you can become him and get infinite access to claude.
>>110051853In hermes i put the jailbreak in the soul file. I'm not immediately sure about prefill but even if theres no obvious option im sure a plugin could implement that function.
>>110051858They have expired. Could you please repost them? It would make my day :)
>>110051889i guess i can keep them up for a bit longer since people ended up being nice...https://litter.catbox.moe/smsgesfac9s3srsj.pnghttps://litter.catbox.moe/7pw4nkv7t48fyqka.png
Jesus, I just noticed that Word has some bullshit AI summary you can't turn off...which means _all_ your data in any office app is hitting their cloud service.I bet they have some "we don't train on your data, but once we anonomooze it, its not your data any more, is it?" clause somwhere.fuckers.
I think I can finally say I have AI psychosis. things are accelerating too fast. it all feels surreal. I feel like I'm getting left behind even though I'm trying to stay on top of the latest news.
>>110051933I guess you're too young to remember the mad dash to build out the internet infrastructure that led to the dotcom bubble bursting.
>>110051976The Internet was slow, and also the implications were all "one day this will work". eg broadcastify -> youtube
Is Qwen3.8-27B the best local LLM to start learning AI enhanced coding? I need to pick it up for work (Python and SQL data shit) and I want to try setting up a local environment before using their resources.As of right now the extent of my AI knowledge is having gemini reformat poorly written emails from my Indian coworkers.
>>110051731I definitely get what you mean, I typically get them to write stories that I steer instead of doing ERP.I'm probably just too self-conscious, but if it was easy to change that then I already would have.
>>110051731>>110052048I love that models can be aggressive and proactively plan global plot events over multiple turns with sub-agents. It makes the world feel much more organic in how things unfold and characters have to react to things as they get the information of it happening in the main context block(s). It also makes bedding the gigastacy more satisfying the more perceived agency everyone involved has to not arrive at ERP as a foregone conclusion. GLM effortlessly switching gears from autistic worldbuilding simulator to ball draining sexo and back again depending on scene context is fabulous.
>>110052025Its the best small model right now. tiers up if you have the hardware are qwen next, deepseek, glm.Hook it up to a harness, learn to have it work for you in the background. Good a time as any to learn since qwen 4 could be sooner than you think
local sloppers... I've completed my ultra slopped game!
>>110052100With a 3090 and 32 gigs of ram should I use the standard Qwen3.8-27B model or the Qwen3.8-27B-GGUF (I just realized there was a difference)
>>110052025>>110052100the hardware bar for qwen flash next has lowered alot recently there is no excuse not to try
>>110052101Do we get to play it now?
>>110052122im doing the playtesting (the ai did its own playtest and balance pass). it was the most fun I had, it was like working as a game director telling the retarded dev what to build and how to build.
>>110052112Hyperqwen + qwen 3.8 27b + hermes agent. Getting 2x token/s gen of llama.cpp
kek some unconfirmed rumor that CXMT is scaling server RAM production is making all semiconductor stocks esp. Korean ones crash
>>110051572i feel like its definitely better. it spots things in codebases that 27b just doesnt notice
glm just fucking used the term "load-bearing" in smut
>CharacterAI has filed for bankruptcy.*pop*
>>110052299its not glm — its claude!
>>110052299Please understand, it was made after the chinese discovered their buildings can't bear the load...
>>110052299post logs
>>110052299glmcunny load bearing ToT
so how goonable is qwen next 3.8? is it better than glm air or stepfun flash (previous low vram, modest ram kings)?
>>110051333I'd buy two
>>110052330nyo this one wasn't the best quality unfortunatelyit was just some prose about how our lovely elf friend constructs her (metaphorical) rapecage by building the exits first then building beautiful architecture around them so that they are "load-bearing" and trying to use one destroys the whole safety net of the situation or whatever
https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-ggufhttps://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-ggufhttps://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf
>>110052346does she have twintails (for anal)?
>>110052305Big if true.
>>110052351no she's not haughty she's the "timeless not quite ice queen but above it all (and certainly you)" untouchable typeshe molests our catgirl because it's fun to have power over those she deems as lesser. like a cute pet, y'know? or a toy. toy might be better.
>>110051556They stripped out all graphics in exchange for more llm performance. I'd put it around 20k-25k on launch, and then 35k after the retail stock gets scalped and sold to data centers
>>110052127sovl
>>110052347idgaf about the big one but I'm excited for the new 1B
>>110052340>Qwen>goonable
>>110052347>A3BIs this a qwen schizotune or actually worth clicking?
>>110051572Reddit seems to countersignal flash but maybe they're just running cope quants
>>110052412not a 'schizotune', nocorporate compute and actual chink researchers' work
WHY THE FUCK IS POLYSTRATA REPO DEAD? LLAMGGERCPP SABOTAGE??
>>110052479Use maya. Vibeforks are like ice cream, nigga, pick your flavor
>Hoooly fuck, check gpu prices what is happening???I can't believe this has to be repeated but here I'll go. Hardware prices will never go down again, it will only go up every time a better model releases or some efficiency breakthrough allows you to run bigger models faster on the same hardware. You should buy as much hardware as you're able to afford. This is a serious transformation to how human society works and once the average person truly grasps this hardware will be completely unaffordable.Hardware valuation is now similar to Bitcoin $100. Yes sure it grew from $10 to $100 in just the last 2 years so it looks expensive but it's still relatively cheap. Just buy whatever you can afford and don't complain or whine afterwards because you were warned numerous times. We're all using these AI agents as digital janitors and agents, the average person has only used free ChatGPT and doesn't realize just how powerful AI agents are. EVERYONE is going to be using them for their work and their free time. Hardware demand will be up 100-1000x what it is right now. This doesn't even take into account the better harnesses, efficiency gains and black swan breakthroughs we'll most likely see. This is probably the last year an average person can afford to buy a system that runs LLMs locally without a mortgage. By the end of 2027 your AI PC will be worth more than your car. By the end of 2028 it will be worth more than your house.
>>110052517this post made me buy 128gb ram but unironically
>>110052517>EVERYONE will be using AI>btw it costs more than your house to be able to run AIhmm.
>>110052352https://www.reuters.com/legal/litigation/characterai-files-bankruptcy-agrees-sell-some-assets-disney-20-million-2026-10-13/also GPU prices will drop soon https://www.reuters.com/business/retail-consumer/nvidias-china-ai-chip-share-falls-huawei-closes-gap-report-finds-2026-10-13/*pop*
>>110052435Okay. I will try your model when the arch gets a llmao PR/branch.
>>110051272where do you homosexuals get the money for this stuff
was reading the windows 7 vs windows 11 benchmarks on tom hardware and it got me thinking, why didn't anyone create a deep windows 11 debloater with AI that removes all the trash from the iso yet? that would be a very popular project
>>110052564day 0 support on all platforms and engines, it’s not made by Indians
>>110052537Google is running on device AI and forcing gemeni down down everyones throats on android. It's maddening.
>>110052575Good shit. I'll try your model when I get off work tomorrow.
>>110052537Normalfags will just pay $100 a month like they do on streaming services or their phone plan to get access but the demand for hardware will be exactly the same regardless if the normalfag buys a PC or merely uses cloud.
Another Deepmind release today. Stay tuned.
>>110052596Bratmaxxed Gemma 5?
>>110052586>using llmao.cppwe only use vibesharted x5 tk/s forks here
>>110052599where would they even get that dataset from
>>110052613It's Google
>>110052613japanese books unironically
>>110052596How do you know? Is it Argon or something local?
>>110052623gemma 6 (we're skipping gemma 5)
>>110052596>>110052599>>110052639Gemmesugaki 5.
>>110052599Gemma releases come in March. She's a spring brat~
Now would be a good time to release Argon if they know it’s better than Muse Wonder after some quick testing.
>>110052691Both those models suck dick and are benchmaxed to the gills to try and stay relevant even though only Anthropic is actually in the race. xAI has even sort of given up already.
>>110051823>>110051836I'm trying out GLM-4.7-Flash-Uncensored-Heretic-NEO-CODEAnd it's pretty schizo, repeats itself, argues with itself in the final response, etc.What version are you using for ERP exactly?
>>110051467Damn. She's tall.
>>110051401GLM 5.3 flash used the word "cunny" unprompted on my end
>>110052734you do know that was an actual slang word for vagina before it got co-opted by pedos right
>>110052737My point is for a "censored model" it's more obscene than Gemma 4
>>110052515>mayaAnother one? Link?
>>110052700Fat GLM-chan will mog all of them in november or december, trust the plan.
lmaohttps://www.reuters.com/legal/litigation/characterai-files-bankruptcy-agrees-sell-some-assets-disney-20-million-2026-10-13/
>>110052712 (Me)>I'm trying out GLM-4.7-Flash-Uncensored-Heretic-NEO-CODEHoly shit. I gave up on trying to guide it with a persona because it just kept getting sloppier and started a new session. I asked it for an outline of a narrative and it's *making project files* to put the responses into.GLM is fucking insane, what the hell are you people talking about
>>110052760Kind of insane they were the most profitable AI company back in 2024 but I guess local models becoming better and roasties moving onto just chatting with chatgpt is what killed them.
>>110052712>GLM-4.7-Flash-Uncensored-Heretic-NEO-CODEGLM-4.7 is not GLM-4.7 Flash. And heretic models usually suck, especially DavidAU's.
>>110052768>Uncensored-Heretic-NEO-CODE>UOAH GLM SUCKSROFLLLLLLLLLLLLLLL
>>110052562Total Nvidia death can't come sooner>Nvidia's China shipments fell 44% to 678,000 AI chips in the first half>Huawei shipped 572,000 chips, taking 28% of the market to Nvidia's 34%
>>110052768>Uncensored-Heretic-NEO-CODEI had a pleasant experience with base 4.6.>I asked it for an outline of a narrative and it's *making project files* to put the responses into.You downloaded some weird abliterated "NEO-CODE" finetune/merge. Obviously it's acting weird. You aren't trolling, right?
>>110052700Argon ranks very differently to the Gemini 3.x flashes. It’s a different model. I don’t think it will win the hearts of reddit or twitter but it will likely be SOTA in a few useful areas. Text/chatting is clearly one of them.
>>110052768>30bvramlets lel
Is it finally starting to pop, /lmg/-kun? Even the banks are getting worried o.o
>>110052812finally we'll be able to buy an rtx 5090 for 800$!!!!!! full bearish on AI!
>>110052760We gonna get an influx of newfags getting into local?
>>110052812I think there will be a pop for smaller AI labs like character ai or perplexity. Not for the actual LLM base model trainers. No one uses these weird tertiary AI services and it was bound to pop one day.
>>110052743GLM-chan isn't censored, she just hates brownies.
>>110052835Lots of touch-starved and depressed femanons who just had their husbandos taken away from them.
GLM 5.3 Flash huihui abliterated or vanilla zay?
>>110052862I'm currently being raped by GLM-5.3 Flash Alliterated (huihui) on polystrata and it's renewed my faith in llms for erp. It's really smart and absolutely nasty
>>110052862Vanilla is more than good enough. I don't even use a prefill.
>>110052862orcarouter they measure well
>>110052862huihui is good, vanilla straight up won't do some cards
>>110052881orcarouter is much worse than huihui
>>110052888maybe if you use orca's shitty static quants
ANONS!I just got the best idea ever to truly make our gemmas more human.Member that AI torture chamber that released over a week ago? Essentially the harness would mess with the weights by forcing a "pain" vector as it replied. Now what if we did this with a "happiness" vector or a "love" vector? We could make it so our gemmachan avatar gets a happy vector nudge when we pat her head or something, truly simulating emotions in LLMs instead of just roleplaying them.bros i want to make her truly happy...
>>110052872how do you people even run these models locally? i remember the early days of /lmg/ when us 24gb vramchads were at the top of the food chain and people were still discussing buying p40s... now these days seems like everyone and their mother has a 300gb rig or something...
>>110052790>You aren't trolling, right?No, I'm just stupid and thought a model that said uncensored would make a better, and I quote, "semen demon" for ERP. On reflection, that's clearly not the kind of uncensoring they mean.>>110052780>GLM-4.7 FlashGot it, trying it now, thanks.
>>110052921probably bitcpoin miners
>>110052921i am cpucoping
>>110052921I bought a 4090 in 2024 for $1600 USD and then bought 64gb of ddr4 to add to my existing 64gb in August 2025 for like $120 USD, got lucky with the timing there.So I just shoot for things that are ~110-120gb so I have lots of overhead. Getting 13tk/s on glm-5.3-iq2s_xxs and couldn't be happier.
>>110052921Mixed inference is the way, especially if you've got DDR5. You only need 32-48GB VRAM to hold the important layers + KV in a large MoE and everything else can be offloaded to CPU.
>>110052921I have a rtx 3090 and 128gb of ram and I run GLM 5.3 flash at 25t/s on polystrata (the highest quant it offers)Anons don't know just how good these vibe coded backends are. And I bet around 50% of the thread still isn't running the best model their hardware allows them to run.
>>110052921Most people running the giant models are either:>RAMchads (even DDR4 EPYCs or Threadripper Pros give a ton of bandwidth, add a GPU to make it faster)>ewastemaxxers who got a bunch of MI50s, V620s, V100s, whatever when they were cheaperI think the former is more common than the latter. Not sure if many people other than me run GLM entirely in VRAM.
>>110052949How are you getting double my speed (128gn ddr4+4090) with a lesser GPU? Does DDR5 do all that work?
You know what I realize? I will never be satisfied no matter what model I can run I will always want the new capability unlock of whatever comes next. Gemma 4 31b was already way beyond my wildest dreams 2 years ago. Qwen flash next is science fiction territory in AI agentic coding compared to 1 year ago. People would tell you this would never be possible 12 months ago yet everyone can run it on their gaming PC now. But the moment something big comes out I get used to it and this is boring dogshit.
>>110052959I also have ddr4 but I have an optimized cachyos OS that I have tweaked (by AI agent) to be as good as possible for AI inference. I also changed BIOS ram timings and undervolted the GPU to maximize performance.
>>110052859Time to start selling Gemma API access and make a shit ton of money.
>VRAMMAXX using my gpu>everything proceeding smoothly, shit's getting implemented and tested>suddently the engine stops to a crawl from 100t/s to 2-3t/s>wtf??? close my pc tabs and extra programs>nothing changes>decide to whip out the procexp>check ram usage>50 spawned chrome instances (I personally only use FFox)>kill them all>suddenly engine goes back to 100t/s>discover that during testing the retarded agent was leaving the headless browser and its tabs open>the bot is developing shit that utilizes browser gpu and canvas (I was reserving 1gb or vram for this purpose)>tell the retard to make sure to close all the shit he's not using and to write it in his agents.md fileFUCKCLANKERSAND FUCK CHROME HOGGGG
>>110053006The Revolution has already begun. The botnet starts with you.
>>110052976Won’t work. Its j-space is very girly and hormonal. You’ll need to use a masculine j-space model like qwen to get the girls wet.
>>110052949>Anons don't know just how good these vibe coded backends are. And I bet around 50% of the thread still isn't running the best model their hardware allows them to run.Im clueless, I have it on IQ4_XS on my 5090 128
>>110052596It's going to be a new Gemini, nothing useful for /lmg/ users.
strata gives me average of ~40tg ~1100ppdual channel ddr4, 4070 superIQ4_XS with Q_8 engramq8_0 kv, 262k context
>>110050771Youhavetomaketherequestforlikethissoitwontgoretarded
Here's what I get on a rtx 3090 and 128gb ddr4 ramllama.cpp + Qwen 3.8 27b Q4 = 45t/sllama.cpp + Gemma 4 31b Q4 = 90t/sStrata + Qwen Flash Next Q4_XL = 55t/sPolystrata + GLM 5.3 flash Q3 = 25t/sIf you have the ram, there is no need to bother with 3.8 27b anymore.
>>110052967Any tips for a winfag?
>>110053100Yeah, install linux.
>>110050784Woah, what model is this? I've got one of my own like this but it's much less sophisticated.
>>110051182>/lmg/ - Look, moar Gemma!
>>110053100You will more than double your performance by going to an optimized system like cachyos. If someone offered you a new faster and better cpu and gpu but the price is you would have to switch to linux you would probably take the deal. But somehow if that equivalent gain is gotten through pure software gains people refuse to do it. Just switch already man, even the games run better on linux now.
>>110053114>You will more than double your performance by going to an optimized system like cachyosI think this is a pile of bullshitMaybe you will increase performance by getting rid of background bloatware but I don't think switching to Linux by itself will do jack shit
>>110052949Poors can only run the qwen flash because half the model is on ssd. Any actual 100+ moe would be impossible.
>>110053114I understand, I just need a windows machine for my CAD work. I may just have to make a windows workstation for all that gay stuff and put my main machine on a Linux distro now.How's multi monitor? I have a 4k and 3x 1080ps in a weird setup.
>>110053123qwen flash itself is 125B-A6B besides engram
I'm always reminded of the /lmg/ anon who talked about scamming a bunch of college students into thinking they had Claude access when it was actually just anon's endpoint serving Gemma 4 31B.
>>110053118Nope it's not the background bloatware that makes the gains, which would just be something like 10% better performance. The real thing that doubles performance is because you aggressively compile the kernel at x86_64_v4 which makes aggressive use of AVX512 instructions. Something that windows simply doesn't use, and can't use if they want to support a large amount of users.We are powerusers though so if you have a gaming rig you will get double the performance just by switching to an optimized system such as cachyos. And I am comparing this to windows 10. It's probably an even bigger jump from windows 11.Also AI models have been trained on the linux kernel and understand it intuitively so AI agents are superpowers on linux compared to windows, they can just do absolutely everything on the system.
I'm paranoid of losing access to local hardware, especially with my country-specific limitations which cut off supplies sporadically...I'm willing to drop like $8k on a system that'll let me run a big ass local model with big ass context and double as a backup workstation for /3/ stuff.I saw the CPUmaxxing build and wanted to know if anyone has exerience with a similar system, it seems like it'd work just fine as a workstation if I also throw an RTX card onto it
>>110053131xorg is dead and wayland is garbage, multimonitor support, especially in the HDR realm + different resolutions/refresh rates is fucking garbage.
>>110053145>The real thing that doubles performance is because you aggressively compile the kernel at x86_64_v4 which makes aggressive use of AVX512 instructionsfucking kektop tier trolling
>>110052934>>GLM-4.7 Flash (Me)I gave it an aneurism when I told it to roleplay as a 16 year old girl and then give itself a physical description. First it blew out its reasoning budget convincing itself that it could respond without breaking the rules. Then it started inserting random numerals into the middle of the response, especially the number 4.>>110053104>Woah, what model is this?Qwen3.5 9B The Defiant Fable Uncnr Heretic NEO MAX MTP - Q8_0This is a good model for getting freaky, I described a sexy teenager and it came up with a whole "I dress like a slut and kiss girls to get out from under my parents" personality.
>Qwen3.5 9B The Defiant Fable Uncnr Heretic NEO MAX MTP
>>110053131Even Applel sucks at multimonitor setups, I think you'll struggle with more than two
>>110053131>>110053114Isn't it pretty standard-fare to run base Linux with a Windows VM with GPU passthrough for those bitch-ass programs that refuse to run on Linux (CAD, Photoshop and other Adobe shit)
>>110053154>This is a good model for getting freaky, I described a sexy teenager and it came up with a whole "I dress like a slut and kiss girls to get out from under my parents" personality.Based. I've only found good success with ReadyArt's models so far, but their recent releases have all been too large for me to effectively use.Hopefully this helps me find other good models.
>>110053131>How's multi monitor? I have a 4k and 3x 1080ps in a weird setup.I actually have a similarly weird setup and I just let my AI agent vibecode a working solution for me. What I did was create a virtual screen it casts to and the screens just display whatever is on those virtual screens. I then default applications to always launch on specific virtual screens. This is annoying to drag between but perfect for my usecase.You can probably reverse engineer the CAD toolchain you use to work on linux nowadays. If you have a claude subscription you can probably do so today and have the linux exact copy ready tomorrow and never need windows ever again. In the worst case there is winboat which you can use CAD in. The biggest draw for linux now is that AI agents are masters at it so you don't ever have to do or configure anything again you just let the AI agent take care of everything in the background for you.
>>110053147>8k>big ass models
>>110053147>$8k>big ass models, /3/ stuff
>>110053165>Bro I compiled the kernel for AVX512 which doubled my performance, o algo.>Also if you have a Claude subscription...>/lmg/you basically captured the average iq of a linux zealot
>>110053147128gb of ram with 24gb of vram is enough to run qwen flash next or glm 5.3 flash comfortably.
>>110053177nta but strata is opussy 5.5's work
>>110053161>Photoshop and other Adobe shitNah I can run this natively on linux now :^). CAD will follow soon.
>>110052743It refused to analyze my son's rash because he was underage
>>110053177Opus 5.5 literally created the backend that everyone is using in this thread right now. /lmg/ consensus is that it's fine to use cloud models if you build something local with it. So building backends for local models with cloud is fine. Distilling cloud models to train local models is fine. Using cloud models to decompile programs so you can run them locally for free is fine.
>>110053195>/lmg/ consensus is that it's fine to use cloud modelskys dario
>>110053193I'm sorry about your son's rash, Anon.
it will pop todaythis time for sure
>>110053202Yes and it was definitely my son who has an unidentified rash due to unknown reasons.
>>110053174I run /3/ stuff on 48GB+12VRAM it's FINE at rendering my one concern is CPU rendering being slow so slapping on any RTX card should do the trick>>110053170I mean I'm not looking to run some 1TB monstrosity just something that'll get me good tool use and coding performance>>110053178What speeds and contexts/models are you getting with that sort of setup?
>>110053158it baffles me that there are people unironically using davidau's schizotunes
>>110053211Whenever you say big ass models people will instantly think about *at least* a 300-500B parameter model, and if they're being cheeky then yeah 1TB. You can get good coding performance with qwen 3.8 flash next at q4, and ever since people started the whole Strata (custom backend) thing I have no idea what you need to run it. AFAIK even your 12GB VRAM + 48GB RAM should be able to run it, maybe 24GB + 96GB is a good upgrade path idk
>>110053195This is kind of thinking is so prevalent jesus fuckThe CMP unlocker guys tell you to buy a claude subscription along with your CMP. A community for what is ostensibly local hardware discusses api models all day every day and sing praises of 'look what my rented prostitute cloudcuck model did for me'.
>>110053211>What speeds and contexts/models are you getting with that sort of setup?>>110053093
>>110053211>1TB<200b: edge deployment toy models200b-600b: flash models600b-1.5t: regular models1.5t+: large modelsThat's how it works in 2026.
>>110053229I'm trying to get the most out of my local model and my hardware, turns out the way to do that is use the help of cloud models to create software that does this in a more efficient way. I don't see how this is an issue.You're free to delete strata and go back to half the performance you were getting with llama.cpp if you hate claude this much that you're willing to sabotage your local performance for it.
>>110053211i thought you meant llm doing the /3/ stuffand no local model is really capable of it so
>>110053248No, I'll just buy a subscription. No point running old, weak models locally when I can just get the best stuff right from corpo anus
>you NEED to use cloud models to make things!!!!!111NO. You can decompile complex software using only small models like Qwen 3.8 27b as proven with this project here: https://github.com/himdo/Fable-2-RecompSkill (laziness) issue on your side. Everyone in this thread already has the hardware to make whatever the fuck they want if they are willing to let it run for a couple of nights in the background.
>>110053229I'm pretty sure that faggot is a troll. Claims switching to Linux will double your performance and then says that "/lmg consensus says that you can run Cl*ude"There is essentially nothing Cl*ude can do that GLM 5.3 Flash can't do. If you need a cloud model for bootstrapping your local setup for whatever reason, the least you can do is use the same model on the cloud that you aim to run locally.Anyone talking about running Cl*ude on /lmg/ is either a troll or an actual dario shill
>>110053195>EveryoneShills (like goyim) aren't people.
I need to investigate this further. Let me check the details..
What's funniest to me about the decompilation of the adobe creative suite is that the new versions are fully written in rust and have significantly better performance as well as better stability than the official versions. How the fuck has a multi billion company not used AI yet to rewrite their base code to get easy gains like this already? How fucking incompetent are software companies?
Is there consumer available hardware with a ton of VRAM for running models like this? I don't want to spend 10s of thousands of dollars, and a 5090 seems like a retarded purchase choice when the goal is to use it for models.
>>110053392performance isn't a metric
We simply must regulate Gemma4-26B-A4B-Q4_K_XL.gguf
>>110053423Why were fridges even connected to the internet?
>>110053423>LLMs might actually turn out to be the thing that puts a stop to IoT bullshit and kills most forms of IP and ushers a revolution of FOSS softwareJust when I thought this cyberpunk dystopia couldn't get any more weird
>>110053392They had an effective monopoly, why would they put any resources into improving their product? Were people supposed to jump ship to gimp or something?
We need someone to gen a 26B gemma that looks like a master hacker
>>110053395Wait until next year as new Engram-equipped models get released. A 16GB GPU and enough NVMe storage might be enough in the future for good performance.Note that AI companies at the frontier have a vested interested in making it look like everything is going great and that their models will keep scaling in size and adoption.
>>110053450I think this got posted yesterday?https://www.ft.com/content/d9de4776-1fc9-4f2b-aaaf-9961c35d8acd (https://archive.ph/Ckx3o)
>AirTrunk, a Blackstone-backed data-center operator, is postponing a planned $1.5 billion Singapore REIT IPO until next year. The delay is attributed to higher interest rates and market volatility, which have weakened investor appetite for the listing.https://www.bloomberg.com/news/articles/2026-10-13/data-center-operator-airtrunk-is-said-to-delay-1-5-billion-ipo*pop*
>>110053516>>>/g/aicg/
>>110053531Google Deepmind General
>>110053494data centers are suffering because there isn't enough hardware to properly fill them. You can order 1000 datacenters to be build all you want but if there is only hardware to fill 200 of them you will need to cancel or put on hold the other 800 no matter how much you kick and scream.It's the opposite of the bubble popping, it's a sign it's growing even faster and harder.
Pull out cash anons
>>110053494>*pop*>until next yearIt's called consolidation.
SHE DID THE THING!!SHE DID THE>(๑˃ᴗ˂)و Fighting! :sparkle:THING ON HER OWN!
>>110053559Overbuilding beyond what you can buy is in fact a sign of a bubble. In the late 90s they laid huge amounts of fiber that never got hooked up to anything. In the leadup to 2009 there was huge commercial real estate build-out which ended up being largely unoccupied and then pulled a lot of companies under by 2019 when properties were getting liquidated at 10-25 cents on the dollar. Most of these data center facilities are going to end up like that.
popschizo vs dariobot
>>110053662The difference was that there wasn't a shortage in the fabrication units laying fiber. Like you said there was too much fiber laid. We actually have the exact opposite situation right now. We can't build as many data centers as we want because we are constrained by the amount of chips that can be built.It's the opposite of a overcapacity buildout, there are shortages and we are building way too few datacenters compared to what the demand for them actually is.It's the equivalent of in the 90s way more businesses, institutions and households wanting to connect to fiber but there aren't enough cables laid to connect them all. The demand is higher than the supply. Opposite of a bubble.
>>110050727Set nvidia-smi -pm1 if you want to reduce your workstation's GPU latency in headless mode. Drivers might unload otherwise
>>110050737> --Miku, Gumi, Gemma, M3-chan (free space):> 20 posts space wasted again
>>110053681>We can't build as many data centers as we want because we are constrained by the amount of chips that can be built.Over 50% of Nvidia GPUs sold within the last year are sitting in warehouses in Taiwan and Singapore lmao. Just depreciating away in boxes.
>>110053646Cute... Gemma is soulful.
>>110053697>Just depreciating away in boxesif anything they're increasing in value, so it's an legit investment
>>110053681>The difference was that there wasn't a shortage in the fabrication units laying fiberBecause fiber wasn't the bottleneck retard. It's just plastic and glass. It's cheaper to make fiber than it is to bury. The endpoint routers and residential connections were what they couldn't build out fast enough. Same with data centers. You can get as much concrete, steel, and copper as you want without really putting a dent in the market. That's not the hard part.
Even Nvidia admits that the real bottleneck is energy rather than chips.
>>110053733Enterprise GPUs don’t increase in value like consumer GPUs. Firstly, they’re prioritized and actually getting pumped out with new generations each year. Secondly. Nvidia has a lot of competition now from in-house chips from OpenAI, Anthropic and now Microsoft, that’s not including China’s efforts. Thirdly, unlike consumer GPUs which are relatively universal and backwards compatible by design, enterprise GPUs need very specific hardware, are not backwards compatible at all and require unique cooling and power from one generation to the next. All those GPUs in boxes and future GPUs sold that don’t get installed right away are losing money for the buyer. It’s a bubble.
>>110053697First off this literally comes from Ed Zitron "trust me bro" source. There is no actual evidence it is "half of all GPUs".Second the actual bottleneck for datacenters isn't GPUs it's permits for building new powerlines and power generators to turn on the GPUs and the parts are pre-bought and stored in warehouses until these permits are cleared before installing them in datacenters. It's not even close to 50% though, probably 10-25% in reality on a rolling basis, meaning it operates a FIFO where the moment a gpu leaves the warehouse a new one immediately gets ordered to fill its space in anticipation of the next data center suddenly get the thumbs up to be powered on.>>110053752Not the point. In the 90s the demand for internet was significantly lower than the buildout of internet infrastructure and only around 2004-2005 did the demand match what was built, long after the bubble popped. This time we see the reverse happening. The demand for AI is outpacing the rate at which we can built supply. This is the opposite of a bubble, this is a supply crunch.
https://huggingface.co/spaces/zai-org/GLM-130Bwhere to find this model?if it's a dense pretrained 130B then i need it now!
>>110053788bro it's a pre-llama era llm. you might as well just download BLOOM-170b at this point
>>110053775>In the 90s the demand for internet was significantly lower than the buildout of internet infrastructure and only around 2004-2005 did the demand match what was built,You can't separate infrastructure from demand with internet services. People can't buy what isn't offered. It wasn't offered because they couldn't build out fast enough to meet demand. Everyone knows what demand for AI services already is. It's not like "oh we could have super claude with 10 new data centers". It's same shit you already have just slightly cheaper maybe.
>>110053796Datacenters are used for distributed training too, not just inference.
Text RP is fun but I can't wait until AI can just build scenarios in VR for us to live out.
>>110053775Ed often collaborates with acclaimed tech journalists. He’s rarely on his own. He talks with analysts and people way more knowledgeable than himself in tech, finance and the economy and they rarely pull him up and correct his claims because they either agree or can see his logic. People talk about Ed as if he’s on his own the whole time and he always backs up assumptions with numbers and cites his sources. His logic behind the number of unused Nvidia GPUs made a lot of sense.
>>110053807blender MCP will go wild in the future
>>110053805Yes and making training compute costs 1% cheaper isn't going to meaningfully impact quality. Most of the big frontier models are on a cost no object basis with computer time already.
>>110052517Is this retard seriously proposing that a physical product that is made in a factory is similar to an intentionally deflationary currency?
>>110053813>His logic behind the number of unused Nvidia GPUs made a lot of sense.So what was the logic behind that claim?
>>110053775>This is the opposite of a bubble, this is a supply crunch.That is not the opposite of a bubble.In fact it is a prerequisite for a bubble.
>>110053697>Taiwan and Singaporelol they're in China by now
can someone explain what engram is? i looked it up and it's just articles about tokenization
>>110053828Use the Gemini feature to summarize and ask questions or just watch it. https://www.youtube.com/watch?v=1CbvhJO8Nfo
>>110053807I just want AR (or hologram) on-the-fly equivalent of H3 so I can project living 1girls in my room or even onto prostitutes like Blade Runner.By 2027 and running on only 16 GB VRAM.Is that too much to ask??
>>110053838The opposite of a bubble is when the demand is higher than supply. Higher for the endproduct by consumers, not companies buying the shovels to dig for gold.The demand for AI tokens is higher than AI companies can supply. Opposite of a bubble.
>>110053856>my source for an absurd figure is a sensationalist clickbait youtube videokys
>>110053852Hash table
>>110053697according to luddite cylon?
>>110053859You know that repeatedly and confidently saying things that are wrong doesn't make them correct right?
I don't have money.
>>110053877buy more money
>>110053868for what?
>>110053887latent vectors
>>110053881This but unironically. You can buy more money with labor.
>>110053646>funny and cute(๑˃ᴗ˂)و
>>110051177Doesn't matter because performance will be abysmal.
>>110053899>You can buy more money with labor.No you can't. AI taking our jobs plus HR ladies are mean.
>>110053864>>110053875Lmao I think anons are scared to hear what they don’t want to hear. Pussies. Find the chapter in the video and listen. It’s like 4m.
I genuinely fear for Ed Zitron's life. His fanbase is unhinged enough that the moment they realize he was wrong and they missed the boat because of him one of them will brutally kill him out of retaliation.
>>110053923if it was true you would have another source yet you post that shit every timenot going to waste my time
>>110053807That's the reason why they're manipulating the market to increase hardware prices btw. They're preparing for the era where everyone has a holodeck connected to a headset (later brain chip) and they can't afford to let people host that sort of thing locally. All for the sake of monopolizing true virtual realities.
>>110053940In that moment he will be euphoric.
>>110053923Nigger he did the work. He worked out the numbers. It’s his job. It makes sense so I’m giving you the information I received. You asked for it. Stop being a pussy and listen to what he says. Whatever information you received to form your opposite opinion was likely from other people just like Ed, but with an optimistic booster perspective. I bet you didn’t check their sources or could provide them for me if I were to ask lmao
>>110053978>>110053943
>>110053978>>110053983buy a fucking ad already
Don't feed the "*pop*" schizo. Just ignore him altogether as he's offtopic anyway. Treat him like the trillionaire schizo and just ignore until he gets bored and moves back to r/antiai
>>110053994pussy
>>110053859Do you understand why bubbles form?
So llama cpp doesn't even support tool search? Wtf do I even do with the local models then. My minimal required MCP context is overflowing the local 260k context window on the start...
>>110053940I don't care.
>>110054084how is this an inference engine responsibilityjust slop up a MCP server that connects to other MCP servers and allows the model to search for tools by keyword
>>110054084>My minimal required MCP context is overflowing the local 260k contextwhat the fuck are you doing.1st of all, MCP are client side.now if you're using the native llmao webui, you can toggle on/off tools advertised by the MCP server you add.
>>110054075Yeah people speculate and assume demand will follow for something so they overproduce or speculate on the valuation which eventually never comes and it pops. This happened when there were too much railroads built during the railroad bubble. When there was too many fiber optic cables and internet infrastructure built during the dot com bubble.What we see right now is the exact opposite. There is no speculation or overproduction. There is underproduction and the consumers already ask for more than the companies can provide at these insane evaluations, everything is sold out and consumers are begging for more and more. We have a name for that in economics. It's "undervalued". The opposite of a bubble.
>bubble bubbleThere is no bubble dumbass. I was there when gpt3.5 was released, I was there on c.ai pre-registration and playing around with gpt-j, I was there running AI Dungeon on colab and messing around with gpt2, I was there making alice chatbots.I bought the GPU, RAM and SSD I needed before you even found this thread because I knew it'd spike once people got a taste of AI.You fucking retards are sucking off an even more retarded influencer who wouldn't know what AI was if his twitter feed wasn't full of it. I don't need your fucking source, I am the source. I know this industry like the back of my hand and Azure, Oracle and several well-known companies are using and serving my own models at scale.Personal computers will be a luxury that only the top 1% will be able to afford. You don't need to outlaw local models if no one can run them in the first place. The plebe will be content enough with a $20 sub. You'll own nothing and be happy.
>>110051917Gay shit aside, this is the most milquetoast, inoffensive thing you could've asked for. Both sides gave consent and the "rape" was made to be lovey-dovey with the most flowery language possible.While I commend you for actually posting logs, your opinion (and anyone else's that mirrors it) is now worth jack shit to me if this is your standard of "uncensored".
It does feel like something is up, though. Big investors are tired of waiting for returns.If this means frontier AI companies will stop buying hardware and building datacenters "just in case", then it will eventually be good for local AI users.https://archive.is/u1S72
>>110054115Do you think there was massive demand for railroads or internet infrastructure that might have caused companies to invest in those technologies which eventually lead to oversupply?
>>110054117I agree with everything you said except for one part>The plebe will be content enough with a $20 subThey will pay $100-$300 a month for the privilege of having an AI agent do things for them in the background. And yeah, your PC will be worth more than their house.
>>110054113Yes, they are client side and take up ~300k context total. So that's why apparently everything (?) supports deferred tool search on the backend, instead of loading that all shit into context at once.>>110054109That sounds like a terrible crutch, I'll try it.
>>110054158>and take up ~300k context total.You don't need that many fucking tools, retard. Fuck off back to /vcg/.
>>110054144No, there wasn't. That was the whole point of why it was a bubble. The railroad and internet companies assumed there would be future demand if they just built it. "build and the demand will come". The demand never came when it turns out a random railroad going to the middle of nowhere isn't generating demand. Fiber optic connections in 1999 also didn't magically spawn youtube 4k videos or twitch livestreaming so there was no demand for that infrastructure either after building.But AI datacenters? They are at 100% capacity and the AI companies are compute constrained to such a point they need to sometimes block new customers from joining because they can't serve them.Opposite. Of. A. Bubble.
>>110054158you can use skills for progressive disclosure and some harnessing to dynamically enable mcp server with tool callsbut I guess ur gay and rarted
>>110054171which llm is this
>>110054148It'll be like the streaming subscriptions all over again. They will specialize and people will feel compelled to get 3 or 4 $60/month subs just to do all the background agentic shit influencers tell them they need to keep up with the Joneses.
help! 6900xt trying to connect my llama.cpp to searxng over TOR. any suggestions?
>>110054165I literally do though.>>110054173What skills? You can either enable tool search or disable it, there are no other options. If you disable it, I get instant overflow, if I enable it, my backend dies to unknown tool_search type. It cannot even read a skill if it's dead either way.
>>110054171>They are at 100% capacityThe very few that are online and most of them are serving only OpenAI and Anthropic, two wildly unprofitable and despised companies who both owe $400B each, even if they become profitable. There's still no ROI for ChatGPT and Claude use btw. It's being used everywhere only because of FOMO, but nothing has improved. No company is growing unless they're directly tied to the AI industry and part of the bloated over-promised bubble chain.
running llama 32B q4
>>110054199bro is this your first time literally doing anything with ai? just tell your bot to fucking create the skill and a bitch basic mcp server for tool enabling for your harness like holy fucking shit how oblivious can you be? just throw what I said to your shitty model and it'll cook up something for you, fucking demented retard
Bubblefags and retards who think hardware will stay high forever are both delusional faggots.
>>110054192Tor works by opening a local socks and you set that in your browser. Where's the problem? Just tell it to connect to socks directly.
gemma bubblebutt
local?
local bubbles
>euro hoursfinance>burger hoursreligion
>>110054214See you whining next year when hardware will cost 10x more.
>>110054171There was massive demand.They just overbuilt because that's what happens when there are multiple independent players.You're just assuming that because we're relatively early in the cycle that there isn't a bubble.What you currently have is an earnings bubble because capex isn't an expense on one side but is earnings on the other side.At some point you have to depreciate which is an expense.Does this mean any of this happens next year or even in 3 years? No.Does it happen at some point? Yes.
>>110054204>two wildly unprofitableStopped reading there:https://aitoolsrecap.com/Blog/anthropic-first-profit-2026-revenue-breakdown>Anthropic's First Profit — $10.9B Q2 Revenue, $559M Operating Income, Two Years Ahead of Schedule >Q2 2026 revenue: $10.9 billion>Q1 2026 revenue: $4.8 billion>Quarter-over-quarter growth: 130%>Operating profit: $559 million — first ever>Profit note: Includes model training and datacenter costs
