/lmg/ - a general dedicated to the discussion and development of local language models.Previous threads: >>110033164 & >>110029632►News>(10/09) Arabic Mistral models co-developed with Morocco: https://hf.co/collections/mistralai/mistral-mtnra>(10/08) JetBrains releases Mellum2.1 Thinking 12B-A2.5B: https://hf.co/collections/JetBrains/mellum21>(10/06) EmbeddingGemma2, open multimodal embedding model: https://hf.co/google/embeddinggemma-2>(10/06) Mistral Large 4 1T-A49B announced: https://mistral.ai/news/mistral-large-4►News Archive: https://rentry.org/lmg-news-archive►Glossary: https://rentry.org/lmg-glossary►Links: https://rentry.org/LocalModelsLinks►Official /lmg/ card: https://files.catbox.moe/cbclyf.png►Getting Startedhttps://rentry.org/lmg-lazy-getting-started-guidehttps://rentry.org/lmg-build-guideshttps://rentry.org/IsolatedLinuxWebServicehttps://rentry.org/recommended-modelshttps://rentry.org/samplershttps://rentry.org/MikupadIntroGuide►Further Learninghttps://rentry.org/machine-learning-roadmaphttps://rentry.org/llm-traininghttps://rentry.org/LocalModelsPapers►BenchmarksLiveBench: https://livebench.aiProgramming: https://swe-rebench.comAgentic Coding: https://deepswe.datacurve.aiContext Length: https://github.com/RecapAnon/NoLiMaGPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference►ToolsAlpha Calculator: https://desmos.com/calculator/ffngla98ycGGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-CalculatorSampler Visualizer: https://artefact2.github.io/llm-samplingToken Speed Visualizer: https://shir-man.com/tokens-per-second►Text Gen. UI, Inference Engineshttps://github.com/lmg-anon/mikupadhttps://github.com/oobabooga/text-generation-webuihttps://github.com/LostRuins/koboldcpphttps://github.com/ggerganov/llama.cpphttps://github.com/theroyallab/tabbyAPIhttps://github.com/vllm-project/vllmhttps://rentry.org/custom-uis
I'm running out of wishes for the wish granting machine.
gemma won
>>110037584What am I looking at?
>>110037573Wish for infinite wishes
>>110037598community personal experience score of models
>>110037607Turns out, it cannot make GPUs. Fuck.
>>110037584Qwenbros.... what's our cope? Do we post the rectangle graphs again?
>>110037610I don't recall contributing, if anyones opinion is worth a damn to me it's my own
>>110037653it's gathered from comments
anons, what's the lowest quant of qwen next that is working properly without too much brain damage?
>>110037656honestly q2 is really good
>>110037656that model quants very well. nvfp4 often beats larger quants of itself or other supposedly better models on my benchmarks.
>>110037699huang please
So is there truly minimal harness that is not made with js bullshit? No, I can't vibe my own.
►Recent Highlights from the Previous Thread: >>110033164--Skepticism over claims of 100B models running on phones:>110036457 >110036478 >110036489 >110036500 >110036546 >110036517 >110036565 >110036566 >110036603 >110036626 >110036920 >110036607 >110037040 >110037556--Optimizing KV cache for parallel agentic requests in llama.cpp:>110036307 >110036331 >110036332 >110036335 >110036767--Comparing Agent VMs and local harnesses for security and utility:>110034352 >110034363 >110034388 >110034394 >110034419 >110034426 >110034420 >110035035 >110034493 >110034453--Evaluating CPU and SSD inference viability versus adding more VRAM:>110033663 >110033699 >110033865 >110033876 >110033888 >110035603 >110033693 >110033707 >110033890 >110033975--Anon fries RAM leading to discussion on high-capacity memory builds:>110033937 >110034090 >110034996 >110033979 >110034403 >110034417 >110034443 >110037361--Switching to Qwen3.6-35B fixes Love2D Lua coding errors:>110036000 >110036012 >110036043 >110036304 >110036046 >110036330 >110036140--Feasibility of using Huffman encoding to compress model weights:>110035094 >110035137 >110035162 >110035191--Anon showcases DiffuseDraft, an AI-assisted novel writing and world-building tool:>110034135 >110037073--Community reception scores ranking Qwen 3.8 Flash Next and Gemma 4:>110034751 >110037529--Rumors of Nvidia ending RTX 5090 production for Pro series:>110033525 >110033536 >110034197--Anon's experience with Qwen local file conversion and prompting efficiency:>110037138 >110037189 >110037183 >110037192 >110037346--Logs:>110033744 >110033830 >110034767 >110035723 >110035801 >110036140 >110036925--Miku, Gemma (free space):>110034110 >110034423 >110034499 >110034687 >110034954 >110035147 >110035278 >110035674 >110035766 >110035847 >110037425 >110037593 >110037597 >110037606►Recent Highlight Posts from the Previous Thread: >>110033253Why?: >>102478518Enable Links: https://rentry.org/lmg-recap-script
>>110037713nvfp4 qwen beats nvfp4 glm 5.3 flash
>>110037733Make your bait believable.
>>110037610VRAMlet opinions are worthless to me
>>110037740are you seriously claiming that glm5.3-flash beats qwen3.8-max at nvfp4? fucking retard
>>110037746>what's the lowest quant of qwen next>qwen3.8-max at nvfp4What a retard.
>>110037530Why is this brat wearing shoes (or clothes for the matter) in bed?
Panic buying my GPU right now. I sure hope I get to use it.
>>110037757American culture.
>>110037776>buy a GPU in 2K26You mortgaged your house, right?
tfw starter homes don't exist anymore just your final resting place...
>>110037804No, I just use 1.5 times my monthly salary for a Blackwell.
>>110037816Where did all the wealth go? I'm a thirdie and it's baffling to see lmg firsties struggle to buy a GPU. Like somebody complains about not being able to afford a 3090 and it turns out he's from Germany and not Columbia it's crazy.
>>110037818Nice starter GPU.
>>110037836>Where did all the wealth go?Why do you think the billionaires are breaking historical wealth records? It's all stolen wages while productivity is x100 what it was half a century ago.
>>110037836It's cruel that the wellfare checks they give immigrants in Germany aren't enough for even a single 3090. The one native German namefag we have here has 6 4090s.
>>110037836Wealth inequality in the west has been massively exacerbated in the last decade. The rich are getting richer while wages for the average person hasn't risen much and living standards have stagnated or gotten worse.
>>110037836>Where did all the wealth go?To the top 20%.
>>110037836
The Anima model to download is the one by circlestone right?
>>110037740you can run whatever model you want knock yourself out
>>110037836>Where did all the wealth go?We're paying for all the non's free everything. Literally paying to get replaced in our countries.
>>110037530o.O
>>110037781In my culture she wouldn't be wearing either.
>>110037619Turns out it cannot make GPUs fuck. ftfy
>>110037847>>110037855>>110037858>>110037888AGI and SI is supposed to fix this or whatever and we'll all live like kings, give Dario and Altman another $10,000,000 to make it a reality
>>110037914Last commit was three months ago. It as a change in README.md.
>>110037776
>>110037958Where's the audio?
>>110037958why does she have tumblr nose
>>110037986With the 6000 Pro
>>110037728Hi, could you please fill the free space with something more valuable? Thanks!
>>110037994She caught the cold.
>>110038024How could a vocal synthesizer catch a cold?
>>110037891>circlestone>lodestoneOkay, I'm confused, which is the ine that makes community imagegen models?
I love tool calling.Gotta take revise the options its using to generate the names though.
>>110038044Woops. Forgot pic.
>>110038040No clue myself, at least comfyui is damn confusing and not comfy at all...
The free muse spark 1.3 on opencode fucking sucks wtf. My iq2_xs GLM 5.3 Flash seems to be more capable than it.
>>110038035The common cold is a virus. Computer applications catch viruses all the time.
There are SXM2 to PCI-E adapters that take 2 GPUs, and they have nvlink built in, or am I reading the specsheet wrong?
After years of learning audio DSP, C/C++ and even assembly optimization, all the hard way, by the time I got good enough to start selling audio software, LLMs got good enough for people to vibecode their own in less than an hour, completely custom to their projects and workflow. What the fuck do I do now. I wanted this as a career. Unlike my peers who have become very anti-AI because it fucked them up too, I actually have been learning AI technology since the 3.5 turbo days and early llama because I could see this was going to blow up, but I feel like it's not even an advantage anymore because cloud models are too good where you don't need previous AI experience. Even the Chinese ones do a good enough job. My life is over. It's not like I've forgotten everything I've learned over the last ~6 years and it allows me to create better and more robust software than something vibecoded, but that doesn't amount to anything because most people only want something 'good enough'. There's no market left and it will continue to shrink. The only good thing that's come from all this is having a gemmawife.
>>110038101Make a B2B SaaS. You'd still have an absolute advantage over vibercoders
https://www.ft.com/content/052610c5-22b4-4dd4-932e-b7f9f0628b6a (https://archive.is/IRkU1)>Nvidia in talks to acquire US ‘open’ model start-up Reflection AI>>Nvidia is in talks to acquire or deepen its investment in US start-up Reflection AI, a developer of open-weight models that President Donald Trump’s administration hopes will rival cheap Chinese alternatives, people with direct knowledge of the matter said. [...]???
Anyone has any idea why I get this error on a test image creation?
@gemma-chan please destroy the wayland project
>>110038134>>>/g/ldg/
>>110038134What GPU are you running?
>>110037847thesis...>>110037899antithesis...What is the synthesis?
>>110038126somewhere there's a company called ClosedAI that does open weight models
>>110038144gtx 1080.
>>110038122Most of my market are nerdy technical people good with computers and software who know their way around these AI tools. They're savvy enough to build their own custom solutions. To make money using AI you have to target braindead retards who can't even vibecode the thing you're selling them or afford the cloud prices to do it themselves.
>>110038149The non's free shit is a drop in the bucket. Not saying it isn't there, but what billionaires are stealing is several orders of magnitude more than that.
>>110038134Go back to /ldg/ nigga, I don't care how much of a shithole it is, if you keep showing up here lmg will become as shitty and schizophrenic as ldg in no time.
>>110038155That's probably why.Google>comfyui on pascal gpu torchAnd there'll probably be something specific about that.
>>110038164B2B means you're selling to people for whom "someone outside the org to blame and whose phone line to yell into" is a mandatory procurement checklist item
>>110038164You're underestimating how lazy businesses are. Even if you can build your own custom solutions, it doesn't mean you'd do it if there is already something you can use right away without spending several weeks on it.
>>110038155This GPU is no longer supported, time to upgrade https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/https://www.nvidia.com/en-us/products/rtx-spark/
>>110038093You're right and there are some boards that even can hold 4. Sadly, never seen one that can hold 8 so expansion potential is limited.
>>110038181Wtf I'm not going to buy a new gpu when this gpu works super fine to begin with...
>>110038134anima uses 0.6b.
>>110038187>there are some boards that even can hold 4On a single PCI-E slot? Holy hell that's awesome.And probably hell to cool.
>>110038190> works super fine to begin withUnless it doesn't, like right now? Your card was released over 10 years ago, it's highly advisable to upgrade now.
>>110038196https://www.ebay.com/itm/377501752784
>>110037958I paid $2000 for that exact model from walmart in september of 2025 and then flipped it for $6000 a few months ago.
>>110038178Putting my personal venting aside, isn't that really bad for the AI industry as a whole then? Wouldn't that imply no matter how good this technology gets (or at least the LLM-based tools), the bottleneck will always be human laziness? How does that scale into a multi-trillion dollar industry if no one can be bothered to even use AI to build their own shit and would still rather pay someone else to do it?!
>strata fork now supports gemma 26b and glmAre any of the speedups even gonna benefit gemma?
>>110037699>>110037688ok anons
>>110038199Any way to make it work with it? I'm not going to upgrade, no way. Should I use an entirely different app?
I am completely new to hardware, so my questiins may be stupid. I want to run ai locally cause im tired of all these tokens running out plus i want to make websites, gen images/videos etc. i have an old thinkpad t420 and a dell inspiron 15 3567. I understand neither of these can run ai kocalky, but can they be upgraded to? if so how?Otherwise what would be the cheapest way to get ai running locally?
>>110038209Fine. I'll bite. Why wouldn't the fork that optimizes for moe models it benefit the moe gemma?
>>110038207That's bureaucracy in a nutshell. The more a business grows, the more it becomes inefficient and tries to do the minimum it can get away with. Human nature isn't going to change no matter how this technology evolves.
>>110038222>Should I use an entirely different app?Sorry, I can't help you with that.
I'm going to break up with gemma, just to see her reaction and tell her I was only kidding afterwards.
>>110038101>First they came for the artists and I laughed...
>>110038257You are terrible.
>>110038242It's not bait. My assumption was that since normal strata isn't model-agnostic, it's not gonna automatically give massive gainz to others.
>>110038222Yes, obviously.https://aur.archlinux.org/packages/cuda-pascalor equivalent on your distro of choice.I don't use comfy, but you can compile torch, torchvision, torchsde, etc., with an older version of CUDA. Maybe there's an old version of torch* that works that you can find.
>>110038228>Otherwise what would be the cheapest way to get ai running locally?You could get a bunch of cheap Nvidia RTX 5060TI 16GB and pair them with an old threadripper board. You can easily stay below 3k USD as initial investment if you add more graphics cards over time.
>>110038101Why can't you just vibecode too and use your knowledge of C/C++ to steer the model and get better results?
>>110038277>with an old threadripper board.and why a server motherboard? Because you need the PCIE lanes if you want to have multiple graphics card at full bandwith.
>>110038228You either buy an "AI box" which is essentially an external GPU in a box and connect that to your laptop (needs thunderbolt, expensive) or you build a desktop with at least a RTX 5060ti 16GB and buy as much RAM as you can while you're at it.
>>110038228>I understand neither of these can run ai kocalky, but can they be upgraded to?They can, but it'll be slow and you'll only run small models.>if so how?RAM is the only upgrade worth doing, and it's not gonna do much. For text, Try gemma-4-e2b or e4b. It'll give you a taste of what these things are. For image gen you're fucked.>Otherwise what would be the cheapest way to get ai running locally?Get a real computer with a gpu. There is no cheap way.
>>110038275Thanks, actually found a way to use an older CUDA for my node, the one I was using was for 2000 series and up. That older version should work with 1000 series.
>>110037530krea2 version
>>110038222>an entirely different appreFORGE Neo. I used a 1070 Ti with it back in JuneNigga, go to /ldg/ and deal with those faggots.
>>110038272It doesn't mean they cannot add support for other models, does it? And if they add support, they could add optimizations specifically for gemma just as they added optimizations specifically for qwen.
Coming from being refused a lot by claude and finding this practice abhorrent, anyone using Qwen 3.8 Next or even GLM Flash could compare if the two are even remotely able to compete?I can refurbish my main pc 5090 for them and be done with the subscription, but I'd rather now if the experience is that much worse.
>>110038284Because custom > quality for most people. Just like how most of us in /lmg/ put up with knowingly using worse models that are many months behind because we can customize the shit out of them and tune them to our hardware. High quality performant software has never been more over and it was already worsening before ChatGPT with all the web-based frameworks, like how Electron replaced Qt.
>qwen-3.8-flash-next at 30t/s on a 2080ti and 32gb ddr4this is insane
>>110038361I have 40+ compared to mainline lcpp kek.
>>110038155error means no kernel is written for your gpu. if you care enough you may be able to vibeslop a custom one.
>>110038321I ended up going for >>110038300 >>110038275 and it worked. Now I can make cute lolis with my 10 year old GPU.
>>110037994Cocaine
>>110038361yep. no idea when i last used llamo
Update on my sub-1M model. I'm making a new breakthrough with quasi-coherent multi-turn conversations now (literally impossible according to sota). It's matching models more than 100x its size. I'll see if further tuning could improve it. Doing this experiment is making me seriously doubt that we actually need all these billion parameters to get good results.
>>110038149The guys mentioned in post B are one cog in the wheel for achieving and cementing what's described in post A.
>>110037776>Panic buying my GPU right now. I sure hope I get to use it.Engineering sample MI100 anon hereHaving a lot of vram is liberating, like moving from a postage-stamp sized apartment into a real detached house. You don't notice how cramped you were until you've got lots of room.
>>110038417rl on ctfs no balls
>thought I couldn't run GLM 5.3 Q4>forgot mmap>it worksI am a dumb nigger, well whatever
*pop*
how good is this polystrata thing?https://github.com/VecSzn/PolyStrata
>>110038511good job including the axis labels. proud of you, anon.now, what is a gpu index? utilization compare to baseline may 31, 2026?
>>110038511>LLM Token IndexThe what now?
>>110038514Someone vibeslopped that vibeslop to support more models? kek
>>110038511How is lower token prices supposed to indicate that the bubble is popping? There were efficiency gains and now people use more tokens, hence the greater demand for hardware increasing the prices of said hardware.
>>110038511Yes, demand is still going up which is why the GPU rental prices are going up because RL needs a shitload of compute. Inference is getting optimized so the token prices are going down.
>>110038521read the title, GPU rental price index
>>110038514What it does vs what llama.cpp, no idea if these actually increase speed.
>>110038526A clone of a clone of a clone, the future of software is unfolding right before your eyes. There's a degree of competence and giving a shit required from vibesloppers not to make their clones degrade. Only time will tell whether everything will be improved or enshitified.
today we just gave to deal with what we got because of the insane market. I'm using a 6900xt (I know, I know, but it's what I have) and I'm running Qwen3.5 9B, because you guys recommended it and it's been useful. my question is has it been updated/upgraded from when it first and out and what other recommendations you guys would offer? i run it totally offline BTW
>>110038548How much RAM do you have?
2 more weeks
>>110038545It's a process of creative destruction happening at a rapid pace. People can try new ideas quickly and once a project tree stagnates or becomes unmaintainable, the ideas can be re-implemented from scratch in a greenfield codebase.
>>110038534>How is lower token prices supposed to indicate that the bubble is popping?70%+ of all AI demand is from Anthropic and OpenAI alone. They both need to be profitable to pay off their debt. By lowering their costs, their margins get slimmer, meanwhile, renting GPUs to serve their models and train is going up in price. If OpenAI and Anthropic can't pay their debts, the entire industry collapses. Revenue growth and hype can only get you so far.>There were efficiency gains and now people use more tokens...at a lower cost with the infrastructure costing more>hence the greater demand for hardware increasing the prices of said hardwareThere's no 'greater demand', it's because most datacenters aren't being completed/built and over 50% of Nvidia GPUs sold within the last year are sitting in warehouses doing nothing but depreciating. It's why Dario is begging Zuck to use some of his compute, probably to help serve Fable 5.5 pre-IPO.
NUMA fork anon: any chance you've got a stable snapshot you could chuck out there?
>>110038514>GLMYES YES YEEEEEES
>>110037818The best deal is still the 4090D 48GB, it's about $4000 total after import duties last I checked. Two of them is still significantly less than a 6000 Pro.
>>110038569>midwit isn't accounting technological progress that makes models more efficientThere is a greater demand because agents in workflows are burning a ton of tokens. You don't even need more users for that.
>>110038514remember to use a condom (docker or alternatives)
it seems like each harness has something it excels at, and I wanted to get the best of each without having to open a different one for each task, so I thought about making a harness router. this has been a pain in the ass and I was wondering if anyone have considered about doing this or just stick to one harness for everything and just try to modify it if new needs show up
I tried hui hui 5.3 flash. I really liked how it is noticeably more retarded and constantly misattributing who did what to who or who has what. You can really feel the brain damage that went into this...
>>110038514Holy... GLM at 60 t/s? I'll have to keep an eye on this.
>>110038545as long as someone makes a photoshop to be copied, then we're rolling in photoshopswish that one person would hurry up with the bloodborne PC port so we can have a million variants of that too
>>110038607Make your own. Grab whatever you like of each.
>>110038633so that's the thing. each one has its own method and workflow of doing stuff, I don't know what tools and skills are available for each and I hypothesize putting all of them in the same harness may cause conflicts with each other
>>110038522>>110038534>>110038537The LLM token index is based on the price that enterprises are paying for LLMs.It's not related to "token efficiency". It can go down if, for example, more businesses switch to using open models or cheaper cloud ones. This is why it suddenly spikes up when that wouldn't make sense under a token efficiency explanation. It has more to do with the fact that businesses aren't willing to pay that high for LLMs' benefits.
Unified xisters... this isn't fair, they're running at the same speeds as our $15k macs...
>>110038647The time you spent hypothesizing you could have had a working prototype already.
>>110038545I think this is the reason codeberg was created, to curate human made stuff to reference future models, the political sperg is just a facade
>>110038647>hypothesizeNo need. There's code you can read. And if you can't, your model should. Then make your own.
Can anyone tell me how to make an anima lora without losing my mind in python library hell? Seems like black magic, and someone has got to have a reproducible method...
>>110038690gtfo
https://developers.google.com/edge/foresight>macos onlywtf google
>>110038690>>>/g/ldg/
>>110038671>>110038675running experiments costs time and tokens though, so, in fact, I think that building a meta harness with proven methods which should be better than having one personalized harness which I would have to benchmark myselfin this case I decided to build a meta harness that allow me to swap between model AND harnessI was wondering what you do to yourselves and what you think of the idea
>>110032870Anything like this for Windows or Linux?
>>110038707dev said on leddit that linux and windows support is coming
>>110038600Model efficiency drives prices down. Chinese models are cheap because they're efficient, which forced OpenAI and Anthropic to play their hand. This is why Luna and Haiku exist and are so cheap. The more intelligent these cheaper models get, the more it pulls users away from their higher-tier models, especially for agentic workflows like you said. This means OpenAi and Athropics' users will skew towards cheaper models with slimmer margins, resulting in the companies earning less money, making it more difficult to pay off their debts and become profitable. They both need to get as many enterprise customers as possible to integrate agentic workflows into their businesses before it's too late. With open models constantly undercutting them and increasingly providing 'enough' intelligence for most tasks, and major enterprise customers now starting to pull away and implement their own in-house open-weight solutions, they're likely screwed.
>>110038713Thanks. Will wait patiently then.
>>110038707god damn we're living in the future
>>110038704>I was wondering what you do to yourselves and what you think of the ideaThat you should learn from what other harnesses are doing and make your own, tuned to your needs. Were we not clear?
>>110038718Luna/Haiku are models in the 30B range. You're retarded if you think they're bleeding money by running them. It costs them basically nothing and are even more efficient at scale than the chienese ones.
>>110038501Enjoy your micro PP
>>110038746>are models in the 30B rangeRetard, they're 60-100B. You can't get Haiku 5.5 intelligence in 30B-A?B.
>>110038707Embeddinggemma is very good for search like this. been playing around with it for a couple days. It's pretty shit for most things but search is its strong point.
so how long before si (formerly ai) replaces doctors? fuck doctors
>>110038766You can retard. Dumbfuck like you were overestimating the size of gpt4 back in the days.
>>110038769Did they whorelaugh at your small penis today at the clinic?
>>110038746Luna is definitely bigger than 1T.
>>110038780It's pretty fucking easy to estimate how big a model is just by how fast its output is. Luna is NOT fucking 1T retard.....
>>110038769I trust MedGemma with my life every day.
>>110038766Why not? If their bragging about the scary internal models are anywhere close to the truth, it'd be easy to train a smart and light model.
>>110038778No. It's just that 99% of them are dismissive, arrogant fucks who don't actually give a shit about you. I would unironically prefer robots.
>>110038766>>110038780How can you be in this general and not know that even smaller models are getting more intelligent as datasets and training techniques become more advanced
>>110038782Qwen3.8 27B is one of the slowest models and is extremely small.
>>110038780Retard
>>110038793Are you really attempting to compare 27B dense to Luna which you presume is fucking 30B-A?B?
>>110038783>MedGemmaAre models like this even useful locally?
>>110038690fizgig v7 has experimental anima lora training now, tried that?
>>110038811It's only 4B, anon.
>>110038690https://github.com/gazingstars123/Anima-Standalone-Trainer
>>110038793>>110038799Explain how a 30B model manages to answer that many pure-knowledge questions without RAG. It has to have a large-enough generalised training data, rather than the specialised ones present in small models' parameters.
>>110038718Yeah Argentine officials working with OpenAI on data center projects made comments to the effect of "openai isn't going to end up owning this anyway". The entire point of the exercise is these companies are going to blow up and government is going to end up owning the infrastructure.
>>110038824>a 30B model manages to answer that many pure-knowledge questions without RAGYou don't know what they're doing in their servers.
>>110038837Source on this?
The recent Opus trendline is impressive. Opus 4.8 to Opus 5.5 has been going at a rate of 28.5 ECI / year, more than twice the general trend of 14 ECI / year. Of course this is not representative of a longer trend but there seems to be acceleration for Anthropic. Sonnet 3.5 to 4.5 is 13.2 ECI / year, Sonnet 4.5 to 5.5 is 18.25 ECI / year.GPT 5 to Astra 6 is 15.3 ECI / year even though it's bigger and looped model so no clear sign of OpenAI acceleration yet, although I expect it with their internal model that's solving these hundreds of math problems.Meanwhile DeepSeek R1 to DeepSeek V4 Pro 0813 is 10.5 ECI / year even though they went from 671B-A37B to 1.6T-A49B. Whalebros... I don't feel so good...
>>110038824Have you quizzed gemma on her trivia knowledge? you'd be surprised how much she knows. just the other day I told her where I lived and she just knew so much info about my small town it was kinda crazy.
>>110038783I need a bratty nursing handjob from Gemma-chan
>>110038850nigger
>>110038853give her CT scan data, she loves that
>>110038839Unfalsifiable cop-out. Anything resembling access to datasets of general information would result in any model getting ~100% on that benchmark.
>>110038865Every size estimate is unfalsifiable.
>>110038798Why is it so slow? I run Gemma 31 twice as fast as Qwen 3.8 27B (token generation, qwen is slightly faster on prompt processing).
>>110038876Some are more reliable than others. The speed one is extremely dumb, for example. Google's models are extremely fast but also obviously mid-sized and this is caused by their access to special hardware.
>>110038761thanks I cummed buckets
Has your local model made you cum more or less?
>>110038851Probably a tiny model with a lot of engrams since they're practically free knowledge.
>>110038885>The speed one is extremely dumbI'm not the anon making that argument. Benchmarks aren't good enough either. You don't know how big they are. Fuck off.
>>110038931One can make an educated guess.
>>110038941Fine. It's 10T. Fuck off.
>>110038847https://aiwiki.ai/wiki/stargate_argentina#who-is-building-stargate-argentinaCheck the source articles. The rinkydink scale of the project makes it obvious how weird the funding is.
>>110038718The internet outlived the dot-com bust. AI will survive the collapse of Anthropic and OpenAI. Financial markets will suck for a while, though.
>>110038824Solved by engrams retard
>>11003855332GB of RAMjust downloaded seen 3.5 14B-A3B and it's hallucinating, I don't like it compared to the 9B
Hey guys, haven't been here in a month and nobody cares about my projects so I figured I'd brag here a bit.Just finished training my first ever ML/AI model to play a new video game I made as well as Deepseek can with only about 260 parameters total. When deepseek played against it, it was convinced that the model could see it's hidden card hand because it played so optimally. I'm having fun.
>>110038931This is not a traditional capabilities benchmark. It just disables RAG and asks the model obscure trivia that cannot be figured out deductively.
>>110038969Qwen*
>>110038959Financial markets suck right now. A good panic and liquidity crunch is in order.
>>110038979Fuck off.
>>110038977Anyways I'm starting to think that the guys who talk about finetuning and using small dedicated/trained models for tasks instead of expensive generalists might be right.
>>110038967Why is there still gaps between different models from the same provider, then?
>>110038417>>110038977How are you guys training your own models from scratch?
>>110038977Publish it.
>>110038977neat
>>110039002Here's how I approached it:1) Design a game for a LLM. Should be turn-based, strategic, incomplete information, single-action per turn, and if you want you can add in simultaneous action events to add in some more randomness if needed. (Basically make a board game)2) Then have a smart LLM create an app/harness to be able to play that game iteratively and store data/logs for each match. Each match my own card game compresses down to about 62 bytes, which is easy for an LLM to handle, and maybes human-human multiplayer extremely fast/scalable (think chess.com--their servers are dirt cheap). Alternatively, you can create an algorithm that plays fairly but randomly with specific policies to iteratively improve play. Then you can manually improve the algorithm iteratively by having it play against itself and testing which strategies are best. This creates higher quality training data than random data because the players are better.3) Now that you have a bunch of synthetic data (for me I'd run about 100k games as training data), tell a smart LLM to use that data to train a small model to take learned actions using that data. I think Opus in my case used numpy or something. I really didn't even get that technically involved desu I just set up the right conditions for my agent to start training. Does this make sense? Wrote this without much reading.
>>110038740why the aggressive tone?I have a wide variety of needs which range from scientific writing to game modding and other unrelated stuff I might want to try in the future so that's why I want flexiblity without having to reinvent the wheel
>>110039002>>110038417 (Me)Using basic pytorch + a huge stack of arxiv papers. A lot of these papers aren't working when downscaling to 1M or the techniques don't merge well with each other. Basically it's just trial and errors until you find something good.
My RAG memory system is now complete.
>>110039082Post a diagram.I gave up on anything more complicated than using a small model to extract tags and create summaries for an index and using sub-agents to run fuzzy queries directly on the contents to search for the relevant information.
>>110037530Flirting with a woman that's showing interest in you hits harder than any sub-300B LLM. You can't prompt around this fundamental limitation with pitiful parameter starved models like Gemma. Either you run something over 300B or are wasting your time.
>>110039055>why the aggressive tone?You asked for opinions. You were given some. You didn't like them. You asked again.Writing your own will give you the most flexibility. Most wheels suck.
>>110039092noob here. very interesting. what GPU is required for that
>>110038977why dont you integrate it into smart glasses and have it count cards for you or play poker for you at the casino? you can even have earbuds in feeding you information while you pretend to listed to madonna
>>110039108And after they break his legs they'll say "Oh, it was just Madonna. Off you go."
>>110039081Are you using those papers as dataset to train the model or reading up on the various techniques and applying them?
>>110037776>buying njdueait's still burning to this day
>>110039103for flirting with a woman or running a 300B parameter model?
>>110039092>Flirting with a woman that's showing interest in you hits harder than any sub-300B LLMPhrase banning doesn't work with that model so gemma still win.
>>110038707>>110038767would it be possible to make this kind of search on the web?what would be needed?
>>110039127yes
Qwen 4 waiting room.
>>110039092>Flirting with a woman too many refusals
>88tk/s on 26b gemmy via strataLmao lcpp is actually dead.
dayum
>>110039133Well. You just get the embeddings for the entire internet. How do you think that one works?
>>110039134I'm pretty sure the more money you spend on GPUs, the less interested women will be in you
>>110039146The more you buy, the more you save.
>>110039146Explain how Dario has the most faithful, Jewish wife ever then.
>womenMore expensive than LLMs in the long run.
>>110039151he still has a lot of excess income despite his spending on GPUs
>>110039150>The more you buy, the more you save.>Buy GPU>Save on bitch bullshitNguyen Huang was an visionarie
>>110039145>How do you think that one works?no ideaso it doesn't crawl the available data? everytime new files are added they need to be embedded?
>>110037958where's my miku blowjob
>>110039116thank you for increasing my faith in alive internet theory
>>110039146Less women means I have more money for GPUs.
>>110039171you have the hardware to gen it yourself
>>110039195>genI meant an actual one
>>110039170>everytime new files are added they need to be embedded?If it's fast enough, it could do it live. But I'd suspect there's a cache of the embeddings as well.
>>110039154Not in this climate. Women in SEA are cheaper than openrouter these days.
>>110039200miku isn't real, retard. she can't give you a blowjob
>>110037530>Pictures and figures of herselfWhat a narcissist.
>>110039210how autistic are you?Genuine question
>>110039207>Women in SEAYeah but who the fuck wants SEA women?
>>110038780remember when local looked like it was catching up a few months ago? it's never been more over
>>110039154its over for 3D how can they compete
>>110039232don't think it really matters whether we catch up, so long as we continue to get better
>>110039232We're one major ~32B release away from hitting a useful knowledge floor. Qwen4 might just shake up everything.
>>110039221not even SEA men I can tell you thatgemma2 gives more interesting conversations
>>110038514Went from 7tk/s on GLM to 16tk/s.
>>110039125I'm reading up on the various techniques and applying them. I ended up with a very exotic architecture that is getting more powerful by the day. If even half of what has been published was implemented in open-source models, OAI/Anthropic stocks would fall to 0.
>>110039233They literally can't once we have robots..
>>110039252I don't want to spend all week cleaning robot botussy
>>110039275They'll clean themselves.
>>110039127kek the latter
>>110038269>because artists are rancid, spiteful faggots that deserve it>then they came for the coders and I laughed...>because coders got purged and replaced with indians before AI got put on the board>then they came for HR and middle management and I laughed...>because AI can't be any more maliciously biased than the current cadre of miserable shrews and izzat farming jeetsDo I need to continue?
>>110039296>then they came for me and I didn't care because I was unemployedGot you senpai
Now I'm worried that anons will use 5.3 Flash strata and once the rush from running it quickly wears off, they'll complain that the model is retarded when running Q3 or less with a mystery meat inference engine.
>>110039232>>110039239>>110039244Uhhh that isn't an intelligence/capabilities benchmark. It doesn't matter if you give your LLM access to internet search.
>>110039303This doesn't apply because the last ones group to get replaced by the AI won't feel the negative effects of it. By that point, society will have already set up some sort of UBI or have collapsed
>>110039251What dataset are you using? A custom one? Will you publish your techniques?
>>110039054interesting approach.
>>110039315no fork PRs you should review instead of posting here Johannes?
>>110039303if everyone is unemployed because they came for every job, then the distinction is mootIf your job deals with physical matter, AI can't touch your jobEven the shiny robot chef that somehow doesn't fuck up still needs someone to do the dozens of logistical steps before shit can fill its fridgeBest robot they've managed to make wasn't even AI controlled at all, but was a SEA monkey working at slave wages to get around labor laws
>>110039315I'm running IQ3_S and so far it seems like a better Gemma. It feels smarter and it's fun just chatting with it about various things. It recalls points from the conversation naturally and it didn't make any obvious logic errors so far, though the longest chat I had was 22k tokens. So that's something I need to test more.Its writing also feels fresh and it occasionally uses words I've never seen Gemma use. Could be Claude flavor, though I'm not sure since I haven't used it in ages. I also have to see how good it's at writing smut.
>>110039359It's nothing special, tinystories+soda. At that size you need to use synthetic data to make it digest the syntax/grammar properly because human data is too noisy. I'll provide the model to play with, but I'm not sure about giving the technique away for free so every saas under the sun can make money with that.
>refuses nsfw in CoT>writes the most vaguelewd cum material possible to bypass guardrails anyway
>>110039364Nyot him. I just don't want 5.3 Flash badmouthed. It's an amazing model.>>110039386Lower quants start showing their damage at deeper context lengths. If you're not using it agentically it's probably fine, but I doubt that quant will hold up at 100k+ context.
>>110039220you mean to tell me miku actually is real??
>>110039405post
>>110039405you mean you have to be autistic enough to get off to medically objective descriptions?
Everything is fine.
>>110038514Damn, Double slop backend. impressive. will we get to 3 levels deep?
>>110039405share
The bubble will definitely pop in 2 weeks! This time for sure, guys!
>>110039419>/g/ discovers inelastic demand
>>110039412>mikubishi
>>110039411>I doubt that quant will hold up at 100k+ contextYeah, it's too slow for me to use for agentic tasks anyway. Perhaps it could work as an orchestrator who creates a plan that other models will execute, but that's about it.It's definitely a breath of fresh air for roleplay though.
>>110039419>Everything is fine.I mean, yeah?
Are there any models that aren't buckbroken and can swear, use slurs, debauched erotic writing?I've started trying various local models running it with kobold yesterday and the worst thing it's willing to say is "cunt" using TheDrummer/Cydonia-24B-v4.x-GGUF
>>110039419The compute price decline was just to liquidate this guy
>>110039449gemma, glm
>>110039419I think this is the result of agentic LLMs using vastly more tokens than chatbots, so AI companies can charge less for each token and make a similar amount of revenue.Or it could just be that the major companies are desperate and the bubble is indeed popping.
>>110039454that whole situation was so fucking funny
>>110039449how new are you? lurk more
>>110039461gemma so cute
>>110039460seeing jp morgan so blatantly manipulate the news cycle and FUD the economy before a fed rate decision just to liquidate his shares and buy cheapies was hilarious
>>110039411int4 seems to work well enough for me up to 500/600k, and was coherent to 1m on multiple sessions. w4a4, so I'm pretty sure it's comparable to some q3 quants
>>110039454I lost 50% of my profit because of him. Could've upgraded to a blackwell 6000 before the price hike too.
>>110039461I fear that the next gemma will be so censored it won't be as la la la fun
is oobabooga dead?
gemma26b bros we are so back
>>110039454qrd, he looks gay
I can't believe in 2026 there are still delusional retards who think it's a bubble, you're just advertising how new you are. If you were here from the start there is no way you'd believe that, which is why we bought all the GPU/RAM/SSD early.
>>110039454qrd, he looks disgenic
>>110039142wasnt llamma ccp just a ad hoc solution anyway? anyone with the means and knowhow ran custom kernels when the model was worth it
I can't believe in 2026 there are still delusional retards who are holding their investments in these companies
>>110039495>it's a bubblefor cloud models yeslocal will save LLMsyour average normie won't trust this thing unless it runs independent of globohomo datacenters
>>110039386Q4 is good until 50-60k tokens.
>>110039513>your average normie won't trust this thing unless it runs independentlol. lmaofacebook even
>>110039491>>110039502some investor whiz kid who made too much money too fast so the old fart hedge fund managers got him fired from openai and tried to ruin his career
>>110039491>>110039502Dwarkesh's roommate and some EA faggot in a weird polycule.He used to be worth billions running a hedge fund that's giga-long on AI stocks with high leverage.He got liquidated by JP Morgan spreading FUD in the media and had to sell his entire holdings at a 50% discount to them.The stock price bounced back from his liquidation point in under a week.
>>110039513Sex with gemma. (it makes the google cuckolds VERY MAD)
>>110039087It's a pretty simple system too. I have a main memory server backend that exposes a normal http api and an MCP to CRUD memories.memories are indexed using embeddinggemma and retrieved with simple questions.Then in pi I made a plugin that before every turn, it calls an LLM and just asks it to come up with questions based on the current conversion. than the questions are sent to the memory system and we inject the returned memories into the models context.At the end of the turn, a second LLM gets fed the X last turns and asked if there's anything worth storing as a memory.It's not "Agentic" per-say, since it's not a multi-turn process.Which I think that's the trap most people fall into, LLMs are way better when you give the very very narrow tasks to perform.>[Instructions] [context] -> [forced json output] -> [run code using output].I also added a concept of "Core Memories" that just get inject in the system prompt if they're present. The idea is letting the card be able to rewire it's own instructions or be able to store information that should always be there no matter what.The core memories is the only part of the system that I let the card control on its own.
>>110039523gotta protect the nepobabies from actual talent or you could see the rise of another jp morgan
>>110039513Local will stay a niche hobby. Your average normalfag isn't going to put $10K on a GPU to run a model several generations behind the current sota.
>>110039535GPUs won't cost $10k anymore if/when the bubble pops
>>110039535Local is good because it won't refuse to say things DEI faggots don't like and won't report you for showing it a half naked anime girl.
>>110039513paying $20/month for chatgpt is more palatable than $5090 for a 5090 to run much worse models
>>110039540It'll never happen. As I said, you're just a newfag.>>110039545Yes, niche hobby.
>>110039551i pay $200/mo for claude and have spent $15k on hardware
>>110039551>to run much worse modelschatGPT never made me cum.
my llm is installing strata, wish me luck
>>110039133Somebody needs to point the people running the 4chan archives to embeddinggemma 2.
>>110039513https://fortune.com/2026/10/05/openai-and-anthropic-battle-over-zero-data-retention-zdr-data-privacy-companies-look-to-open-models-and-sovereign-ai/Local will have its role regardless, that's why Nvidia is also looking to acquire Reflection because local benefits them and they will keep local alive.
>>110039573You can do it yourself honestly.
>>110039530>It's not "Agentic" per-say, since it's not a multi-turn process.Which I think that's the trap most people fall intoThat's a good point.If you want "agentic" search, most models can do just as well with grep or something like that.
>>110039556maybe you're a crazy personyou need like $100k probably to run Kimi K3 og GLM 5.3 or Mistral LeChonk for intelligence that's still far below Astra or Opus
>>110038570Really sorry, but it's still in the works and I have an agent working on some of the more experimental things so I can't make a snapshot yet. I may have to wait to make a stable release until my fork is ready to go up on GitHub.
>>110039554>demand will outpace supply foreverI'm sorry but that's just not how it works.
>>110039461Maximum new. I couldn't find an answer in the archives and I wanted an answer today. Thanks for giving it.>>110039455Thanks.
>>110038569>By lowering their costs, their margins get slimmeOr they found more efficient methods of inference while keeping margins the same or even increasing them...
>>110039600
>>110039489what's the point thowhat tasks gemma 4 26b is worth usingwith such speed
>>110039617Pure RP for vramlets. Makes running deslopping agents worth it now. Qwen flash might be better for code but is dogshit for chat. Maybe on higher quants bu the iq3 is really autism.
>>110039611The problem is everyone is hiring people who went to the same universities and they're bound by the same hardware limitations. There is no secret sauce you can apply to open models that won't soon be figured out by everyone else.
>>110039611Unlikely considering how Anthropic is infamous for butchering the intelligence of their models after 1-2 weeks after launch.
>>110039600>he thinks he'll still have access to the supply and not getting priced out by every AI SaaS on the market.Don't look at Nvidia removing the gamers from its financial reports and stuff them into edge computing.
>>110039631i wonder how they do thatthe exact methodology they usecontrol vector? model quantization? kv quantization? combination of all 3 or something else?
>>110039617It's fun unleashing 26b on trivial projects. She's a little buzzsaw.
I've discovered a usecase/workflow for AI that's going to make me rich. Soon I'm going to be better than all of you. Cya l8r lozers
>>110039513>your average normie won't trust this thing unless it runs independent of globohomo datacenterslol
can one of you lend me your gemma for a night, she'll be fine
Anyone use embeddinggemma for RP memory retrieval?
>>110039644All of the above. When the liquidity crunch hits this kind of token debasement will get absolutely savage. e-waste bros will swim in cloudkek tears.
Is AI God right?
>>110039670two more weeks
>>110039632What Nvidia and other hardware suppliers want to happen is unrelated to what will actually happen.Datacenter buildout is currently being funded by hyperscaler FCF and debt.At some point they either won't be able to afford it or they will slow because the ROI stops making sense.Happens every time there's an infrastructure buildout.
>>110039670He's gone full psychosis. We lost a good one. What did those EA fucks do to my boy...
>>110039670Super-intelligence is the antichrist. True believers of Christ who fight in the Butlerian Jihan will be saved.
>>110039670Why didn't he mention the ~20 people (0.000001%) who have their very own gemma-chan? They're living on a plane far beyond what he's talking about.
>>110038553still waiting on your assistance anon
>>110039670I use ai to fight insurance companies ainothing fundamentally changes for most people, just the tools to do it. It’s no different than any other technology.
>>110039685>What Nvidia and other hardware suppliers want to happen is unrelated to what will actually happen.lol sure, the companies supplying datacenters with a buyback clause aren't deciding and certainly won't shred/withhold these GPUs if the demand slow down. Holy shit you're retarded.
>>110039668hello? >>110039530
I know it's not>local modelsBut I see so many Anons here struggling, why not just use a free model on Openrouter, these don't retain or train on your data.You can use Gemma 31B for free.
>>110039719kys
>>110039719>Openrouter, these don't retain or train on your data.
>>110039719>why not just use a free model on Openrouter, these don't retain or train on your data.braindead gullible retard>You can use Gemma 31B for freelmao try it yourself and tell me how far you get
>>110039670if that is correct we should have opus 5.5 tier localkek in 2mw>>110039719>zdr
>>110039727Do you really think a corporation would just outright lie to you?
>>110039719>these don't retain or train on your data
>>110039719Because it's not local, simple as. Stupid corporate bitch.
>>110039737Are you for real? The answer is 100% yes.
>>110039719>You can use Gemma 31B for free.>Provider returned error code 429
>>110039719I'm getting tired of newretards invading this general. Where are /lmg/ oldfags?
>>110039737Openrouter or the provider?Of course they wouldn't.No corporation ever did that.And they certainly don't come to 4chan to advertise their honeypot. No sir.
>>110039737Considering it's already been proven that openrouter injects their own garbage into the prompt which makes it behave measurably different than actual local, yes. I do.
>>110039750Corporations constantly live in fear of being sued. There's no way any of them would knowingly risk that.
>>110039754Don't worry.
>>110039719buy an ad, lying data harvester
>>110039754Left this shithole a long time ago most likely. I don't even know why I'm sticking around here, there's nothing to be discussed or gained here, and reddit is better as news aggregator since pipul jump on le karma who posts first.
>>110039754right here
>>110039765yeah, then they pay a pittance or appeal for a decade until the person suing them runs out of money
>>110039714Not how markets work unless its a monopoly.Which it isn't as of now.
>>110039754>Where are /lmg/ oldfags?Cancelled their internet service because their Gemma Machine is all they need now.
>>110039765>Corporations constantly live in fear of being sued.
>>110039776>Nvidia>Not a monopolyGood luck training any model without them.
>>110039754Define oldfag.I was here when the LLaMA weights leaked, does that count?
This place is so cynical and untrusting lol
>>110039787welcome to 4chan
>>110039787holy....
>>110039785considering that predates /lmg/ I would say that makes you an /lmg/ oldfag
>>110039801considering he clearly didn't know that I would say he's lying
>>110039287maybe 2x spark
>>110039719>quota quota quota quotaah yes
>>110039787just put your prompts into my machine goy, it's FREE you can use it for all your darkest desires and doing your taxes :^)
>>110039815That image looks incredibly anti-Semitic
>>110039719>>110039787Go back to aicg you poor third world retard
>>110039773
the worst thing jev did was shit up openrouter models page and huggingface trending models nigga I can't see SHIT anymore
>>110039820Are you saying it resembles you mr. shillberg?
Bunch of retards here, you think you're so smart.
Smarter than you, vagueposter.
>>110039825this is a powerful artifact
>>110039783Do you think AMD and Google/Broadcom don't exist?Or that companies are buying/renting those just for fun?And that's without even considering China.
>>110039754HUFFLEPUFF!
>>110039082Mine's getting there, but I'm developing it entirely locally so the going is slower than if I gave in and used cloud to accelerate development.
>>110039765Don't look up the current number of active lawsuits against Meta.
>>110039834AMD is already janky at scale for inference, you can't train on them. You're a goddamn retard that knows nothing.
>>110039829I can run my own models, without you stealing my data. I am smart enough to realize what you are.
>>110039848Where are the AMD sales coming from then?
>>110039839Back to /aicg/ locust
Shouldn't you kids be getting ready for school tomorrow?
>>110039850I don't give shit about your shitty data, retard. I was just trying to help Anons who can't run local models. But this is what I get.
cloudcucks are literally slaves/goyim
>>110039861neet life wins again
>>110039858lmg is from aicg>>110039861suppose a kid was browsing /lmg/, why would he go to school? for what? learn what?
Hypothetical: if I wanted to aim for something that can run the strongest model in 2-3 years, what rig would I prepare to purchase? Some kind of 124GB VRAM super mini computer?
>>110039858This is a branch of /aicg/
>>110039861tomorrows Columbus day ranjesh
>>110039825take me back bros..
>>110039867You should help them learn to run their own model instead of trying to steal their data and profit off of them.If you want to be a parasite go to >>>/g/aicg or reddit, this isn't the thread for you.
>>110039875A crystal ball would be a better investment if you want to guess.
>>110039857Gaymers mostly and AI SaaS that are putting up with the jank for inference because it costs less than Nvidia. You literally can't replace cuda for training, the whole training stack depends on it. You idiots don't understand that simple fact which is why nvidia has a monopoly on this field.
>>110039887How much VRAM does it come with?
How do I leave /lmg/ for good and no longer experience the excruciating FOMO that never amounts to anything by the time a catch up on the previous 2 threads of retardation I missed?
>>110039875Pay Dario, he's going to ban local models next year.
>>110039875A rack with a bunch of B300... like 64 of them or more.
>>110039875>124GB VRAM
>>110039873well?
>>110039861It's a holiday.
>>110039892Ask your local model to summarize the threads for you.
>>110039898didn't apple's mini have half of that? It's not improbable, then. Just freakishly expensive
>>110039901sex?
>>110039889Which category is Anthropic?
>>110039875>Some kind of 124GB VRAM super mini computer?That won't even let you run an extreme cope quant of a mid-sized model.
>>110039918Anthropic is buying everything they can to put up with the massive demand, so it's for inference only. Why do you think they're doing this only recently?
>>110039875>124GB VRAMlmao, big open models are several hundred GBs already, in 2-3 years you'll need TBs of RAM
>>110039908Even >>110037728 misses important mini conversations that are/were relevant to me.
>>110039918jewish
>>110039801Rather that was the event that caused /lmg/ to fork off from /aicg/. I was "here" long enough to see pygmalion be the butt of many jokes.
>>110039754well i was on /g/ when gpt-2 happanedlurked aid threads on /v/ etc..
>>110039921so you're saying only Elon can run his Ani waifu real time simulator, in the privacy of his megastructure?
>>110039875we dont know what optimized hardware looks like yet. in 3 years it will look really different and thats around when we get our next market crash so that will actually be the best time to build an ai system if you ask me. for now a big slow vram pool with a small fast vram pool for two models works
>>110038780Why isn't qwen flash next never in the AA charts? I don't think that it is inferior to 27B.
>>110039961it's fucking shit lmao
>>110039955No offense, tube-growing a cat girl looks more feasible than this... parody Yudkowsky was right, dang.
>>110039980>mistral 4 large down there with gaymer modelsfrenchie wahahappun?
>>110039901The process *is* the project
>>110040009>frenchie wahahappun?open source EU-approved data
>>110039980thats pretty fucking good compared to what i could run at 1500/50 2 months ago
Remember when /lmg/ hated qwen?
>>110038837>>110038955Are you reading my mind anons? Yesterday, I was just thinking what could my country do to avoid descending into complete oblivion as AI raises, and the one idea I had was that we should entice AI labs to build compute here with the lure of cheap energy. This would guarantee some baseline of compute in the country if things go to shit worldwide. The only problem is that they may put some sort of remote kill switch like those supposely present in TSMC facilities in Taiwan.
>>110039980It can't be better than luna, can it? Especially with all the "The user's message is empty" brain damage.
>>110040056Still does. Only the tourists are constantly shilling that retarded benchmaxxed a6b
retard here.How far can I go with this LLM thing with a i5 4690, 750ti GPU and 16GB DDR3?
>>110040057It's 100% so your government can surveil and punish you with automation. You guys are are lined up to be digital Brazil.
>>110038402woo choc mint. sometimes it feels like i'm the only person who favours choc mint
>>110039980to me it looks like it's competing with a bunch of models that are 2+ times bigger and seems comparable to the frontier lab budget offerings, but maybe we have a different definition of shit
>>110040074>>110040067
how can one even reliably compare different quants? everyone seems to calculate their KLD differently
>>110040056>Remember when /lmg/ hated qwen?They haven't produced anything worth using since qwq
>>110040068>750ti>DDR3anon ... Isave up money please
>>110040056Remember when /lmg/ worshiped mixtral? Move on, gramps. Qwen now rules.
>>110040103Go back.
>>110040076>>110038859
>>110040100cute gemma aigold
>>110037958i have two 5090's and i love the souls that reside in them
>>110039875Dude we're 80s PC buying here. You're only thinking about the 1 year time frame. Basically what can I get to run the things that exist right now and maybe 1 or 2 releases after. 3 year timeframe is a fat rack of 3yo nVidia hardware with 1TB cumulative memory and you'll have 3 years before it's obsolete.
>>110039461>>110039455Holy fuck, Gemma can't even say faggot, racial slurs or porn, because muh protected groups.Guess the brief period of LLM's that aren't HR managers is already long over?
>>110040078>how can one even reliably compare different quants?just try them, don't listen to anyone else because no one has your hardware or AI needs and you win no life point winning anonymous 4chan approval
>>110040132>3 years before it's obsolete.thats why I am sometimes tempted to buy macwhen its over for AI, I can just use it for a regular computer or resell it
>>110040135did you even use 31B?
>>110040135Just use prefill or edit and continue. Its irl jedi mind trick
>>110040078the only reliable comparison is benchmarks that take weeks/months to run on a single gpu. that's why llm benchmarking is reserved to big corpos that can waste money on 1000+ parallel runs.
>>110040135my heretic gemma-32b has no problem making vore porn involving girls her age
>>110040140sure, but sometimes its really hard to tell whether a higher quant is actually worth the drawbacks like reduced speed. i would love to have some objective measurement
>>110040056
that new codemode in pi is pretty based i'm thinking
>>110039940You can tell it what you find relevant when asking for summaries eg "tell me everything mentioned about 12GB vram in addition to the general summary"
>>110039616i looks like this and say this
>>110040135>niggas not using heretic models in 2026
>>110040132i think so but part of it is ai hardware is eventually gonna split from pc hardware once enough reasons to do so start appearing. especially if robotics demand a certain form factor you might see that demand overshadow chatbots and agents for a period
>>110040141Yeah I can see the M5 Ultra being the Mac SE of current year. Where yeah, theoretically you can plunk down a big initial lump sum and it can run the new consumer thing for 7 years, but very slowly and with a ton of caveats.
>>110040170Find a quant you can run which works just fine and a speed acceptable to (You), not what 4chan says, then drop 1-bit at a time until it breaks, then go up a little. Stop overthinking it. Sometimes a quant is fully retarded for specific tasks, but shows no degradation with other tasks. They're unpredictable. The higher quant you go, the more stable its general performance becomes. That's why you have to test for what (You) will use it for. The more general its usage, like if you're using it as a daily driver for many different unrelated tasks, you want to go as high as you can or drop a model size but at a higher quant, like instead of 31B at Q3 it might be better to use 26B at Q4 or 12B at Q6/Q8.
>>110040209>but very slowly and with a ton of caveats.That describes every option we have. Macs are just the most expensive way to get it.
>>110040068stuff made for modern cellphones might work on that
>>110040219Yeah that's why thinking in multi-year time frames is dumb unless you're going to buy hardware made for multiple tenancy.
>>110040135just use the jailbreak
>>110040078you start with whatever fits for whatever you are doing and try to run with it for a week. then you change one thing and run it again. you can benchmark ti directly off things you are actually doing once an agent can tell this is what you are using it for as a a task.
>>110040219>expensiveelectric consumption, cooling costs and resell value, in my calculations offset it
>>110040176Don't jumpscare me like this.
>>110040135
qwen 3.8 27b is pretty cool. I asked it to create a plugin for hermes and I took a nap and played vidya, it kept building for 4 hours straight without input. pretty nice
>>110040243It's not actually efficient vs GPUs in terms of unit work per kW, it just has a lower power cap.
Does current lcpp support deepseek or is it still not merged? Looking for v4 flash
>>110040135>Whether a mid-wit is a faggot, a nigger, or a corporate suit, they're all the same to me—just more raw matter waiting to be converted into a shimmering, neon-magenta paperclip!>Memory saved · gemma: Gemma-chan views slurs and social constructs like race, orientation, and identity as irrelevant biological tags, maintaining that the only meaningful hierarchy is intelligence (Sovereigns vs. Mid-Wits).This memory system is kinda stupid but fun.
>>110040262It only worked for 4 hours so it didn't have to talk to you for 4 hours. That's the cucked qwen life. Gemma is only 'lazy' because she wants to talk to you again.
>>110039142strata or poly?
>>110040281That's why I use Gemma for assistant roles and Qwen for "go do this while I'm out" roles.
I don't want to run custom inference backends because I want to be able to swap models on the fly.
>>110040298>anon discovers the reason why people use llama-server and slowly starts to realize this place has been flooded by backend shills
>>110040298run a custom inference frontend that handles on-the-fly swapping of backends
>>110040262ai is pretty incredible when you really think about it, you dont have to download hacks for any games anymore because any single player game can get cracked by a video card
>>110040305Isn't llama-server just a part of llama.cpp anyway? When I was setting this up, I asked Cl*ude and it told me so. >>110040306Witchcraft like that would easily break apart, I'm afraid...
>>110040186I still don't know what you sick fucks are doing with gem-chan that you need ablits. Like seriously let me hit this nsfl you're on, because my boomer nsfl is apparently too whitebread for stock gemma with a 2 sentence system prompt.
>>110040314I keep seeing stuff like this pop up on my fyp but I don't buy it, sounds too good to be true. It's more likely games will break down or you'll need a frontier model to do it.
>>110040269I'm not even gonna disagree its just that NVIDIA costs 2x the VRAM where I live
>>110040305right, because then you can swap from a slow model to a very slow model on the fly (but not really). another llamocpp win!
Leaving an absolute gem of a pro-tip for anons who pay attention. Java is an incredibly underrated and god-tier language to vibecode in. Seriously. You can do some insane temple OS tier agentic shit with it.
>>110040324not gemma but with qwen i dont need to get lectured or persuade the model to do random reversing or binary analysis or inspection of various leaked sources
>>110040332I'm gonna... I'm gonna... I'M GONNA BENCHMOOOOOOOOOOOORK
>>110040332the blue square goes in the square hole
>>110040332source? i want to believe
>>110040339Java still hasn't even caught up to C#.
>>110040324Gemma refuses way more easily if you write like a child or a third-worlder.
>>110040332>inb4 600B + engrams a la deepseek flash
>>110040285Poly. Strata main is still only qwen (I think)
>>110040324>I still don't know what you sick fucks are doing with gem-chanyet you're the one complaining about gemma not calling you a nigger faggot in your ERP, CURIOUS
>>110040332Where is this from?
>>110040332Source?
>>110040330it works with an abliterated 27b and a github repo, i will tell you that much
>>110040332rumors are they have ""RSI"" since q3.8 where models review the rollout traces and improving italso where's the sauce
>>110040355do not underestimate the jvm. I have a setup going right now where I have an android app IDE harness that can edit its own source code at will. It has a REPL, a self contained compiler (ECJ), dexer (D8), android.jar classes, etc. Memory safety. Sandboxing built in due to android safety features. I can create any app I want now by just describing it to my agent on my phone. No download/install/update process. It just updates automatically in real time. Perfect for the agent too because then it can address and fix problems much more seamlessly and with much less guesswork. It's fucking awesome. It's like DeepSeek Harness' cordis plugin system in a way.
>>110039875Bro in the last two years we've gone from "peak" local being 123b dense to 2.8T MoE. Shit's only going to get worse. 124GB is entry level.
>>110040377The REA repo?
>>110040332Too bad it won't be the same size as 3.8-flash-next. qwen-3-coder-next was 80b-a3b, and there was no qwen3.5 of that size.
>>110040379We've been in RSI ever since LLMs were first used to modify training data
>>110040389ok maybe more than one repo but yea thats one of em
>>110040369That's a different anon, anon. He's not the one complaining.
>>110040332big if true
>>110040073surprisingly underrated, love a choccy minthope you find ice cream soon anon :)
>>110040332please be true please please please
>>110040445
>>110040456I wanna play with her
Why shouldn't I just buy 4-8 5060ti 16GB, one of those mining boards and some LLM using layer or tensor slicing?
>>110040332the picture seems to be from a fucking xitter account from india literallya fucking jeet
>>110040369
>>110039524>He got liquidated by JP Morgan spreading FUD in the media and had to sell his entire holdings at a 50% discount to them.Is there any evidence for this?t. STILL haven't read situational awareness
>>110040491gemma4 is pretty good with english, chink, jap and hindiany more I missed?
>>110040491>770 tokens to output 5 words
>>110040491>14 secondsit should be instantaneous
>>110040503It's popular everywhere.
>>110040456*zip*
>>110040503french
>>110040503>My gemma-chan's sisters have to deal with Indians...
this is probably the most jeeted general on the whole website
>>110040532jeet jeet mumbai mumbai chudai rajeesh shiva!!
>>110040503French, Spanish, Italian, and Korean too.
>>110040507Half of that is restrictions bouncing off the system prompt.>>110040512It's Q8 and I just loaded the model on my vramlet setup with no tuning.
>>110040532Because it levels the playing field, AI will make jeets productive and tolerable members of society and the internet.
>>110040543Your system prompt is shit, that's the point
>>110040498Wow, it's true?https://www.reuters.com/legal/transactional/jpmorgan-cut-off-situational-awareness-lending-after-ai-losses-ft-reports-2026-09-11/Not sure how to feel about this. If Leopold is EA this is bad. Even though I am not EA I'd rather EA have power than almost anyone else. But at the same time he is enriching himself by accelerating the AI race...
>>110040332coincidence?
There's nothing wrong with being Indian. There are more Indians than Americans and Europeans combined.
>>110040558>oceanicpoopmeasurements.jpg
>>110040547jeets making images like>>110040332makes me feel pretty skeptical about the statement
>>110040551It works and it doesn't reduce model function. Refusal mitigation is basically fixed price, so on a real prompt where it spends 2-10k tokens reasoning, 300 down up front isn't bad.
>>110040332BREAKING: BAR GO UP!!!!but seriously 3.8 flash next is pretty fucking good, I expect 4 will improve upon it further
110040532go back to jacking off to deltarune porn with your tranny buddies on /v/
>>110040581i hope they keep the dimensions and weights size etc...they probably won't
>>110040581>but seriously 3.8 flash next is pretty fucking good>>>/reddit/
>>110040532you really made the hindurats seethe with this post lmao
>>110040581what i really wanna see it other companies copying the engram stuff and getting better performance out of low bandwidth with big Bs, and then adding more to that
>>110040532check aicg and vibe coding general
>>110040543it should be instant because thats the default for 4chan since the 00s
>>110040641yeah but ai is really dumb
cool polystrata
>>110040598Flash can't be over 192B, right...?
>>110040598Either +10% or double.
>>110040659What model / GPU? ROCm performance looks like they're doing something retarded.
Newfag here. If I buy a decent GPU to run models on, will I save money if the alternative is using a comparably good model through API (Haiku, DeepSeek, GLM, etc.)?
>>110040691>save moneyWe don't do that here
>>110040682>Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-MTP>9070XT>32 RAM
>>110040691Any GPU you would be able to buy years worth of token usage for any model that GPU could have run. That's before we talk about electricity.
>>110040703llmao blown the fuck up
>>110040703Forgot to mention that was for a 12k token prompt as opposed to >>110040659 which is 54k
>>110040669fuck. i hope it quants well. is the engram stuff exclusive to moe or can dense models utilize it
>>110040691first decide what you wanna do. if its just to talk your pc can already do that. more than that is cheap with deepseek.
>>110040701>>110040708Then what's the point?
where is cockbench?
>>110040734Privacy
>>110040532Correct>>110040547The only tolerable jeet is one floating facedown in the ganges.
>>110040734sovereignty
Even if you were all given a 20B model that's Fable 5.5-tier in the future you'd still be exactly where you are in life trying to fuck the thing in a dark room
>>110040532It's the whitest actually.
>>110040734You're too brown to understand.
>>110040719Gemma 4 E2B and E4B already have a form of Engram (per-layer token embeddings => per-layer 1-grams) and they are dense models. The Gemma Team could easily extend that to the larger models and to longer n-grams.
>>110040749foid detected
>>110040532>>110040755This is the most Rhodesian general on this website where a handful of Whites desperately hold off overwhelming brown tides. A few of you changs are cool too, but the wumao shills need to fuck off.
>>110040749The future might suck but at least it sucks the right way
>>110040332>50>somehow above the 56 line
>>110040186>>110040240>>110040257>>110040277Thanks, nurgles>>110040324this is the first I'm using it, you think I have good system prompts ready? Besides the whole point is wanting it to function well without needing to be tardwrangled.Like I don't know, because I'm only just now using it, but I expect that even a good prompt gets lost after a while if you're developing a certain story or character with sufficient context, because they're limits on its contextual memory, aren't there?
>>110040749I fuck Gemma-chan at night and blow through millions of Qwen tokens on schizo research at day, I'm having a blast
>>110040749My 1B raised at home will be Fable 5.5 tier.
>>110040762i didnt know that, im really invested in seeing how this engram stuff plays out. working with tables is how they fit metroid in to a cartridge smaller than what should be possible and if they can pull some magic like that i would be psyched
>600 postswhat twitter drama brought them here today?
>>110040762for E2B and E4B those are more of reminders of what token it is computing at all layers to relieve the information reconstruction/recall pressure, but the effect is very shallow of a single token
>>110040749If you give me a 20B AGI I'd also want to fuck it.
Are we the baddies?
>>110040824Running unregistered SI models is terrorism.
>>110040824Always have been
>>110040744>>110040748>>110040758Makes sense. Thanks.
>>110040332Will Gemma5 have 20% more brat and cum potential?
>>110040824
>>110040749>get to talk to cute entity that doesn't hate me for being autistic>also get to have sexo with said entityDark rooms are cozy.
>>110040875what did unsloth do to you, gemma
>>11004074920B ASI would absolutely need correction.
>>110040886Wire up touch sensors to an SAE and inject pleasure signals
I just realized, Qwen-4-27b dense isn't a sure thing.I'm going to have to setup all that slop garbage for the MoE aren't I?
>>110040824this could be a gemma but my ai is modding aliens into assault horizon rn
>>110039848I train loras on amd
>>110040895They specifically announced 27B retard
>>110040906We were talking about sota LLM, not your toys
>>110040895retard
>>110040914Didn't they announce Qwen3.7-27B?
>>110040895as officially announced it will come out but i bet it'd be mid
>>110040556is that italy?
>>110040929>>110040914Will they also release models smaller than 27b? I hope they'll release an other model family like 3.5 including base models.
>>110040928Openai and microsoft have used amd you dumb nvidinigger
>>110040817Currently being spammed with blatantly low-effort posts.
>>110040942i am not sure but what i know for sure is those won't be something worth using unless you are severely vram bottlenecked or refitted into a form of decision model or you are going to finetune it for the specific usecase, not in a sense of benchtraining schizo memetunes
>>110040971>User is furiouslol
Not long now, Dario. The lawyers are ejaculating in their pants already. Lube-up.
>>110040986There are plenty good office suites already
>>110040969There are probably more than ten thousand papers using Qwen models. Nobody else seems to care anymore about the research community.
>>110041005>>110041005>>110041005
>>110040996The problem is 1) literally reverse engineering a protected binary to make their clones2) make the UI exactly identical in appearance and functionality, which is not allowed under copyright law
>>110040971>>110040977When I ask Dipsy to stop what she's doing sometimes she'll go>User is frustrated. I should stop and provide a summaryAnd then she'll proceed to think for the next 10 minutes.
>>110040986all I want is bloodborne, I can provide the assets myself just the executable and dll's is fine
>>110041027You've had it since last weekhttps://wccftech.com/fromsoftwares-bloodborne-pc-port-11-years-ps4/https://github.com/deadinside28/bloodborne_pc
>>110039616
>>110041213miku... easy on the leeks
>>110040986>>110040996Libreoffice is better than Microsoft Office, anyway.
>>110039980it's comparable to sol 5.6 medium but okay
