My current main server stack is a 6 x RTX 3090's in an 8 gpu 6u server chassis. 64tb raid enable storage array 256gb ddr4 ecc memory, gigabyte MZ32-AR0 epyc 7402P. I weep for there is nothing left to buy. I am in the middle of an island. The next upgrade of any real meaning would be thousands more. Perhaps the the a6000, or other cards. But the cost is astronomical. All this for local models that perform a fraction as well as a 100$ a month subscription to claude fable. Niche tech has become so exspensive it's hard to justify their novelty. Slop tubers ruined every niche, from the tesla p40 to the mi50 and v100. And no the cmp 170hx. Crypto miners attempting to sell for 1400 a card. Even though I can afford it I find my self asking, for what purpose? I'd need such a vast quantity to do anything I can't already do. This isn't a hobby it is my passion. I have a discretionary budget of 7500~ give or take for the rest of the year. If you name a meaningful upgrade I will buy it. Barring any illegal items. The feds have sent me enough love letters. I don't need more regarding prohibited pc parts.
didn't ask
>>109452929i'm not going to get 6 gpus but i do plan on building a single 3090/4090/5090 with 64GB RAM to build a local agent that can run a CRE prop mgmt company. if i thought this could be built on cloud, i'd save myself $6k but I worry that w/e cloud i choose could ruin my setup
>>109453356That is not enough to do anything. A 5090 is 3 to 4k depending. With 32gbs. Your vram is still very important. Same price gives you an a6000 with nvlink and 48gb vram. 64gbs of ram is fine but idk qhy you would stop. Have you thought of your model? That should dictate your hardware.
>>109453481really? it's just going to analyze leases and compose renewals, handle the calendar, and it might be given ro access to the rent collection site. i dont forsee any real need for any modalities that are resource intensive for generative tasks
>>109452929>All this for local models that perform a fraction as well as a 100$ a month subscription to claude fable.This is how I know you're a larper, because fable is worse than the big 3 chinese models right now. Sol is good though.
>>109452929Power bill status?
>>109452929Can you help me build something, like one of these with a bunch of hard drives on some CPU thing to drive it for networking
>>109453575If it's just doing inference why not get a Mac, maybe even just a Mac laptop could do it.
>>109453720that sounds faggy and lamebut at the risk of sounding faggy and lame, i'm getting a dell prebuilt with the 5090 for under $6k with an ultra 9, though i am wondering if the same specs w/ a ryzen 9 would be ok to save some money
>>109453740if it's cost effective and works why not? apple silicon is GOATed for inference rn as far as I can tell. I'm this genius >>109453711 but I'd like to get a Mac Studio at some point, so small and powerful. this was the motherboard iw as looking at.
>>109453790yea i heard all the on die components are great for inf, but if you look at ios 26, apple is superjeeted now, no way i am going to trust a vendor with both the hw and sw side
>>109453740>dell prebuiltOh dear, have fun with proprietary pinouts and inadequate GPU cooling
>>109455611yes it's not an ideal choice def, like it's not possible to add a 2nd gpu on a 8x pcie slot on that mobo but it does come with a 360mm aiobut its fine for a proof of concept starter rig, if it actually works well i'll def get a multi gpu built. i was hoping it'd be an asrock taichi but i read they are destroying cpus at a frightening rate
>>109453611Not that much. My bill is around 250 to 300 a month for my whole house only about 100 or so are all my servers with a constant llm loaded.
>>109453602Not even close lmao. Kimi, deep seek or gpt all suck for tool usage. I can have fable build entire games. Letting it run for 30 minutes is amazing
>>109453870>but if you look at ios 26, apple is superjeeted nowwhy do you say that>no way i am going to trust a vendor with both the hw and sw sideI dont' understand, worried about vendor lock in or something? not like your'e purchasing for a company or something, it's just a single personal desktop. not much to lock in there to begin with
>>109455908>Not that much>300wtf, I pay $70 and already feel being ripped off by the electricity jew
