I'm not a gamer, I would just be getting one for the local LLM/finetuning capabilities. Worth it or should I hold off for something in the pipeline?
yes
no
MaybeNigger
boomer just found out about nvidia stock 6 years too late
No Nvidia card is worth its price at the moment.
>>108440089This thread has nothing to do with Nvidia stock. It has to do with bang for buck really. From what I've seen the 5090 doesn't seem much better than the 4090 so 4090 still seems to reign supreme in sub 3k category.
>>108440076>yes>>108440084>no>>108440086>MaybeI don't know. Can you repeat the question?
>>108439888>>108440159How about 3090 24Gb? I want to run a local llm too.
>>108439888
>4090/3090 for llmsAm I wrong for thinking a commercial card would be better for this? I do stable diffusion on an 8Gb 2060 super and its great, only slow when upscaling to higher resolutions.
>>108440301which model? and how long does it take for image gens
>>108440301the 4090 is obviously much more capable. it really depends on what you want
>>108440463even if they have the same VRAM?
get a blackwell 6000
>>108440490you need at least 3 of those if you want decent results with local LLM. oh, and make sure you have at least 128 GB of ram, too.
>>108440499yeah
>>108440417The Asus Dual, I think there's a v2.0 in there somewhere. Takes about 55 seconds for a 1024 x 1024 image using Invoke with a ~6Gb model
>>108440499so you really can't get shit unless you spend 20-30k?
>>108439888It's hard to get a 4090 without getting scammed, if you're spending that much might as well just get a 5090
>>108439888Get a couple 3090's instead.
>>108440573Get a Mac Studio, 6 grand for 256GB.
>>108440631Apple gives me hemorrhoids. any non apple alternatives that you could run linux on?
>>108440695Get a decommissioned Threadripper or dual Xeon server with as much memory you can afford, min couple grand. You'll be limited to a few tokens/s though.
>>108440710Would a Mac Studio be faster?
>>108440714Shitloads. >>108440631 is actually the most sensible option money wise.
>>108439888>LLM/finetuning capabilities>finetuningYou need hundreds of VRAM to finetune even something like a 7b parameter model, not 24.
>>108440718>Mac Studio,doesn't Nvidia also have a box like the max mini with unified memory?
>>108440724The Spark? Less memory for pretty much the same price.
you don't need to finetune your own llms, they're all trash and theres no real use case for any of this. Don't fall for the meme. With the exception you're getting paid to do it and your company is fell for the meme
>>108440767But i thought pewdiepie fine tuned an LLM to be as good as ChatGPT
>>108439888Get a 5070TI instead, about 350 tps faster than the 4090 for ai workloads. 4090 is not worth the cost and is inferior for AI performance.
>>108440794I think memory is the bigger issue than speed here.
>>108440767but can't i finetune it to be more based or something
>>108440796if Memory is the goal then: DGX spark. Spark = 128 gigs of vram at 3800-4000 USD with a tops through-rate of 1050.
>>108440796Or wait for 6x series 6x series rumored to have of a tops of roughly 5000-6000 on the 5070 equivalent. ... Assuming if we ever see them materialize at this point due to component availability, I wouldn't waitfag personally but that's just me.
>>108440820Spark is 4800 USD now. At that point just get >>108440631. Double the VRAM and still decent performance.
>>108440813look into OG dolphin Mixtral / or BlackDolphin.It's already "based" guardrail-free.
>>108440813>>108440856Just go to /lmg/ and actually look at the recommended current models for particular use cases. We have a general for that for a reason.
>>108440864but I want to ERP
>>108440865There's literally uncensored models listed there for that... At least look before posting.
>>108440855Dogshit mac evirons with no support for anything built for Nvidia which is most AI stuff still. + trash mactarded "jeetgineering" t. Louis Rossman. Even apps written native for ai on macs hit the wall with a laughably bad tops of 50-75 at best. Enjoy waiting 7 - 20 minutes for your single 1024x1024 SDXL local genned image x.x.
>>108440885>Enjoy waiting 7 - 20 minutes for your single 1024x1024 SDXL local genned image x.x.You could have started by saying you're clueless.
>>108440865Literally any local llm running on lama or whatever backend of choice like lmstudio chained to SillyTavern front end gui then.
>>108440885>OP is talking about LLMs>bring up SDXLok?
>>108439888It was absolutely worth it when it launched, I highly doubt its worth it now given the prices of everything
>>108439888this thing cannot finetune, you need at least a blackwell 6000 for local finetuning. a 4090 with at least 64gb of ddr5 is a pretty average local llm rig. just ask on /lmg/ about this shit.
Anyone have one of those modded 22gb 2080tis from China? I heard they cheaped out on some of the board components so they aren't reliable.
>>108440301I know nothing about it but grok told me the commerical cards that hold big amounts of ram are actually slower than 5090 for image generation, their up side is the ram which they use to train way bigger models thou
