I need more VRAM, Like, a lot. Don't fucking ask why but I need local enterprise AI. It doesn't even need to be on the cheap. My department has just been authorized one hell of a budget.But again it needs to be local and can not talk to the internet. What do>>>>
>>109158099if it's just LLM buy bunch of mac mini or mac studio with huge ram and run local on it. If it's something like image gen, then well, mac won't do
>>109158099ask ai
>>109158125y'all are better than ai until I get this nigger running
>>109158099Okay. If money is no object, then buy it.
>>109158141I have 1,6M
>>109158099sudo systemd-run --quiet --scope -p IPAddressDeny=any -p IPAddressAllow=127.0.0.1 sudo -u ai ./llama-server
>>109158099To run any of the big boy models you'll need at least an H200 HGX node, which is extremely expensive on its own, but in order to rack it, cable it and power it you'll need a ton of other equipment. If your company already has a server room with rack space and someone who knows how to hook it up and configure everything then just convince them to buy an HGX node.
>>109158099Sounds like you got everything you need. Buy Nvidia TI series cards, the datacenter ones. The DCGLs that work in series.https://resources.nvidia.com/l/en-us-gpu#referrer=vanityGet in touch with them, leave overhead in your project for contingency, and enjoy having your own personal sexbot. Get the minimum requirements for DeepL or Minimax Minimax is the best open weight model but not the best model. It's the one you can run no questions asked. https://www.aimadetools.com/blog/how-to-run-minimax-m3-locally/Look at the requirementsSetup VRAM needed Hardware CostFull model (estimated) 400-800GB 4-8× A100 80GB or 4-8× H100 $30K-80KMulti-node Distributed 2+ servers with NVLink EnterpriseIf you have 30K available in your project, get the needed GPUs to fill the minimum or recommended VRAM capacity using A100 series cards or whatever the Nvidia salesman can hook you up with.Good job anon, you hit the fucking motherload.
Setup VRAM needed Hardware CostFull model (estimated) 400-800GB 4-8× A100 80GB or 4-8× H100 $30K-80KMulti-node Distributed 2+ servers with NVLink Enterprise
>>109158181Oh yeah this is the other thing: Make sure you have the power capacity for this, because 4-8 H100s or H200s WILL put strain on any current closet you have if it's not IT rated. H200s draw 1000W each, H100s draw 700W each. Contact an electrician and ask them to upgrade the service to the room this is going in to account for 4-8 of these.
>>109158193Yup this is true. My company needed to have the electrical reconfigured and rewired to accommodate adding some AI compute capacity.Those fuckers use a LOT of power.
>>109158099Pomni owes me sex
>>109158210Oh and Lots of power means lots of heat. The room needs to be really well ventilated and the HVAC needs to be extremely robust.
>>109158193Thank you good sir I am on it. Our firm can more than handle it, just being a dinosaur corpo technically owned by black rock things... go slow. But we can and will do it (eventually) Two 8X H200's servers will be on order before the fourth. Im not an IT guy but it's not like those counts get paid to sit around. Again thank you for checking my "market research" box in my fucking docuserv account. We are a go.
>>109158223Right, that too. This equipment has listed in it's specs the "heat load" of the hardware. It's usually on the power supply or listed with the power supply specs. Your HVAC load needs to be upgraded to fit the specifications of the Heat Load or it WILL turn into an oven in the room. Thankfully, we fixed it at my place of work with Minisplit units, fairly cheap, can operate in parallel.Oh I almost forgot one other thing: Also account for the fact you will need an additional server for database operations and queries to your new cluster. Don't forget to account for that.
>>109158221based and Jax pilled
>>109158242based, good luck, also remember our consulting fees are cheap
>>109158252I mean, the other departments are lucky and get to use public frontier models. We are not for.... reasons.
Sam, post from your normal 4chan Premium+ account.
>>109158154You're not gonna buy shit with that. 1.6m is like the fee Nvidia demands just for a quote.
>>109158099No amount of VRAM is going to unabstract himI'm sorry :(
>>109158609>him
>>109158099If its for LMs, then Mac Studios linked together
