lower your tone if you don't have 1tb ram
>>109358732use case?
>>109358763three chrome tabs
>>109358763Three virtual machines running After Effects
>>109358732lower your tone if you need over 8gb ram
>>109358814I thought 16 was the minimum in [insert_current_year]
>>109358763Firefox
>>109358763Gaming. Simple as.
>>109358763LLMs
>>109358763compiling some homosexual code in rust
>>109358814Dual channel 8 is probably comfy but still living on the edge.>>109358834Hopefully a few years because that's what I'm stuck with.
>>109358732>8x128Come back to me when you have 4x256
>>109358996just how slow do LLMs run on standard DDR5 like this as opposed to faster VRAM?
>>109358763so my fizz buzz programs run longer before they run out of ram.
>>109361726>his cpu only supports quad channel memorypoint and laugh
>>109358763worse programming
>>109358763Windows 11
>>109361745My rule of thumb for LLMs is they need to read twice the size of the model in memory per token. So multiply the model size (after quantization) by two and divide by your bandwitdh to get a rough estimate of token/s.My Ryzen Threadripper 7970X with 4 channels of 6400 MT/s will do 200 GB/s.My RTX 4080 Super will do 736 GB/s.So it's a pretty big difference, but of course a consumer GPU doesn't have 192 GB of memory to play around with. And another way to look at it is that you can bleed into the RAM with less of a performance penalty if you have 4 or 8 channels of high speed RAM, compared to a 2-channel consumer CPU/motherboard.
>>109364362>twice the size of the model in memory per token"in memory ACCESS per token", you don't need 140 GB of RAM to run a 70 GB model, but you will read 140 GB of RAM in the process of running the model for one iteration.
they make 128gb ddr5 dimms now? when I got my workstation in 2024 64gb was about how far you could go, maybe 96, and that's at 4800 (rdimm ofc)im such a poorfag for only having 256gb ddr5-4800 registered... (worth 10 grand now when I bought it for like 1)
>>109358763Running a dual react native app
>>109364362Miss me with the fancy talk I just want to know if I can fuck it.
>>109367041>DDR5>poorfagnigga i still run DDR4 and DDR3, you call DDR5 being a poorfag??
I might have 1tb across all my devices. Does that count?
