It seems to me like too much money to spend when you can get a subscription or even use ir for free.
>>110244832resale value for one thingprivacy is anotherno refusals, no TOS and nothing off limitsneed more?
>>110244862i get that, but the price.... shit. Is like buying a car ffs.
>>110244913Wait for the market to crash, then these will go for half the price once they're no longer scarce unicorns.
>>110244832freedom privacy for one telling the corporate surveillance state to fuck off is another
>>110244832A used server and 2 gpus are not as expensive as you think
>>110244970And would you be 100% sure that he models that you run wont send information to those who created them?
>>110244989Well i am trying to figure out what is an amount of money that will deliver practical performance. Coding and research at a decent level. But i am not spending 25k on it.
>>110244913if you’re buying $30k ai cards you probably run a business that generates money with ai somehow.
>>110244832Cloud AI has Cloud restrictions, it won't tell you how to make molotov cocktails and if you call it a dumb clanker it will shut down the conversation.
>>110244832they want to make the nasty.
>>110244832Building your business around a service that might end any day looks like a bad idea.>>110245030You just described the business model of Mistral.
>>110244862open models are absolute garbage right now, so no, that's not justified currently
>>110245099Yes, i was thinking about that too.Nobody knows if these fuckers will say tomorrow that the price is now 10x to use chatgpt or grok.If you think about it, what other route do they have to make their business profitable?
>>110245135Open models are always just 3-6 months behind paid models, what are you talking about...
>>110244832I am just waiting. a few tokens a second isn't the worst thing, but it isn't great either.t. 64GiB RAM + 16GiB VRAM let.
>>110245149Chat gpt is saying to me that 4 x rtx a6000 48gb is the best bang for the buck, but is 20k we are talking about here.
>>110245145Open models you can run with 2 TB of VRAM, yeah. Assuming OP's model you need 20+ of those lmao.
>>110245141Inference is cheap to run.The "reasoning" on the other hand is expensive as hell.They would either need a lot of scientific breakthroughs to make running the models cheaper.Or they would need access to far cheaper electricity.I think they're going to jack up prices because that's 1000x times easier.
>>110245177ya. my whole build is old as fuck and sucks. I can't fathom paying more than $5k for a new computer so I'll waitfag it or look into subbing, but I don't have any strong want or need. work pays for some shitty Agentic nonsense and at home I have to time to just fuck around and snail cat it out if the free services or local shit is useless.
>>110245237Jesus. Where do you even get 2Tb of Vram these days?
I want unlimited, unrestricted, and reliable.The local model will never change unless I want it to, if it works right now it will work this way forever. And I still retain the option to change the setup to something else at any point.With any vendor I am beholden to their decisions on their timeline at their price and so far, that has already eroded away in the free space as popularity/usage increases so it's not exactly something I can depend on.
Cheese pizza generation
constant cucking of frontier modelswas also spooked when claude changed all their pricing that time
>>110245258HBM cards no more DDR memes
>>110245258Hire 500+ bums to raid a data center
>>110245334
>>110245252Reasoning is inference, it's an easy way to burn a lot of tokens fast but it's still just generating tokens which is very cheap relatively.Training new models and to a lesser extent doing reinforcement learning are expensive, and obviously building 500 massive data centers, but when these companies say "profitable on inference" assuming they aren't outright lying that means running the models pays for itself which means even if the big companies that spent billions training them go under it will still be profitable for whoever gets their data to spin up a couple AWS instances and keep offering the serviceAnd since open source models are only a few months behind the frontier, unless they manage to actually solve distillation and make some huge breakthrough they have very little power to set prices in the market, even if OpenAI wants to charge 5000% markups to pay off their massive investments theres very little they can do to prevent any random startup with a couple million in funding from buying some GPUs and offering Deepseek or GLM at a small fraction of that cost.That's (part of) why they're all about the safety angle and legal control of hardware these days. They don't care that you personally can buy a GPU and run some barebones local model, they know 99.99% of people won't be doing that anyway, they want to prevent other players from entering the market. A lot of them, meaning employees and lobbyists, do actually believe in the danger as well, though I'm somewhat skeptical it is anywhere near a primary motivation for the CEOs and other people holding the money and making the real decisions.
>>110245355But that would just make the business impossible. There is no way they can make profit of AI then.So we get to a paradox, what is going to happen to all the hardware they have bought in a couple of years? The space and energy for data centers are limited.
>>110244832picrel
>>110245506anon i dont get itwtf
>>110245355Reasoning is not inference.The early forms of LLM's were inference plus a little more.That resulted in a lot of silly errors.So they started running multiple inference passes in a loop.Now the frontier "models" are huge agent swarms reasoning with each other.
>>110245619Inference is just the process of generating tokens, that's what running an LLM is called. Everything an LLM does is inference, reasoning is just a specific case of generating tokens where it thinks to itself in a little scratchpad in its context window before starting on the actual task. Recent frontier models have a lot deeper layer counts and sometimes process layers multiple times so they can hold more "thought" in between tokens but it's still just doing token prediction which is inference and reasoning requires many tokens basically by definition.
>>110244832Can you believe I passed on one these at $7,500 because I thought I could get it for little bit lower if I waited?
>>110245772maybe it will become true within a couple of years. Who knows?
>>110245273>>110245135>>110245030
>>110245893add this one >>110245332sloperators are motherfucking retarded
>>110245258
>>110245978>prebulit for ceoat prebuilt prices too
>>110244832Lower latency for playing video games with characters that are controlled by AI.
>>110244832I'd rather have qwen 3.8 27b living in a VM in my basement look over my taxes than give The China my info
>>110244832video generation
>>110244832there's no value in buying the hardware now, but if you already have it, it's totally worth the privacy and unlimited tokens to use local models
Local models are uncensored, and for more serious work, even images, you have to do hundreds of inpainting/ControlNet passthroughs to get it to look the way you want it, and that will burn credits quickly.
>>110245978the margins on this shit must be insane
