Will Cuda forever remain king?
>>109462860KEKED, still no dynamic VRAM for room on comfyui as far as I know.It's like they know they lost on the AI front and aren't interested in playing catchup.If you want to genAI, just..stay away from AMD, and I say this as my gaming pc has a 9070xt, I tried, it works, but that's it, it just works, but it's behind older Nvidia cards on AI by a lot at this point.
>>109463088>9070xt>look inside>16 gb vramBrutal, why didn't you buy an rx 7900 xtx 24gb?
>>109463088comfyui is telemetry cancer, use sdcpp instead
>>109463158Because he's primarily using it for gaming, like me. Yes, it sucks that AMD doesn't have high VRAM cards besides the dated 7900 XTX.If you want both a gaming card and a card for LLMs / generative AI, then unfortunately he's right, you're better off with NVIDIA.Hopefully, the next gen AMD cards will come with 24 GB VRAM instead of 16 GB.>>109462860It works.
>>109463170>comfyui is telemetry cancerwtf, why???
>>109463199>dated>7900 xtxit came out in 2022, its only 4 years old, the rtx 3090 everyone loves is more than 6 years old
>>109463225>its only 4 years olin amd time it's almost good for eol
>>109462860Tried it on my 6600xt and it keeps running out of memory crashing the entire systemWould not recommend
>>109462860it's getting there.At least with TheRock 7.14 (which I think they said would become the basis for ROCm 8) you can now install ROCm and pytorch specific to your card's model using python pip into a vdev instead of worrying about your linux OS package manager.Triton is closing the performance gap. The lack of native fp8 support on pre-RDNA3.5 cards can be worked around by using the actually superior int8 convrot format.>>109463088comfy is lazy when it comes to AMD hardware and usually prefers to just disable shit instead of actually fixing it. I think they've actually fixed the crashes with the triton backend but they haven't reenabled it by default yet so int8 is shit unless you enable triton backend with the flag. int8 makes a huge difference on 7900xtx.
>>109462860Still doesn't fucking work on windows.
>>109462860zluda will fix ai on amd in a few years, the software is only optimized for cuda so they just need cuda on amd which is what zluda does, same deal as running windows games on linux>>109463170https://rentry.org/IsolatedLinuxWebService though I use sd neo over cumfart, best to isolate any of this shit from the network in general
comfy is very slow on amd, back in the sd1.5/sdxl days a11111 was like 3 times faster than comfy and comfyfag refused to implement amd specific optimizations and called them useless gimmicks lol. i don't think it's different now
Ignoring jewvidia? how is ryzen 395 with 128GB quad channel is doing btw?Is this cost efficient solution for llms? Or mac is better (but cost more)?
>>109462860A conservative estimate is that ROCm is 50-60% done catching up to cuda, AMD has done some great stuff, AI performance has doubled in a year on the same cards just because of better software
>>109463199There is the r9700, it is a 9070XT core with 32GB ram, the price doesn't make sense but is cheaper than a 5090.
>>109466287For CPU only you're looking upwards of 256GB of ram to be worth it, for 128GB you can cluster 4 GPUs and it will give you actually usable speeds.
>>109462860Only recently it stops segfaulting on my 9070XT in torch lmao. Hip SDK on Arch is like 10-20 GiB of storage space.
Local AI with AMD on windows is pure garbage, it just doesn't work. If you want to at least test it, you have to switch to linux. Although dont expect too much since you lstill will be limited by lack of cuda support. That was my experience at least. Plus side you won't have to deal with adrenalin on linux.
>>109462860I've done ample testing from the 8GB cards all the way up to the 24GB ones over the past 3 generations.ROCm with AMD drivers is behind CUDA.ROCm with Mesa drivers is slightly ahead of CUDA.Vulkan is and has been ahead of both for years now.OneAPI would be the best is Intel could get it's shit together, but like all good things Intel, Lip Bu Tan seems to have scrapped it for whatever this Battle Matrix crap is.
>>109466403how do you gen with vulkan?
>>109466457What kind of gen? You can do txt, img, vid, TTS, STT, interrogate, vision and a few others just with Kobold.You can also often just raw CPP workflows.Most Chinese models come CPP capable ootb and they're the highest quality for local use.The only time I really use ROCm anymore is for HordeAI donation while idle.It should be noted that Vulkan on AMD or Intel is faster than CUDA on Nvidia, but CUDA on Nvidia is faster than Vulkan on Nvidia.
>>109466287This guy makes a lot of Strix Halo content> https://www.youtube.com/@donatocapitella/videos> https://github.com/kyuz0
I finally figured out how to stop comfyui from randomly killing my gnome desktop session>--reserve-vram 4.0haha of course that didn't work>just run your desktop off the iGPU so the 7900xtx is dedicated to comfy/gamesactually works perfectly. still get the occasional random comfyui segfault but at least that's better then having my entire desktop logged out when the gpu crashes out on no free vram.
>>109466403>ROCm with Mesa drivers is slightly ahead of CUDA.Explain. I used ROCm and it sucked to the point I started looking for an nShitia card because I want to make deep fake vids of me banging a Terminator 3 Claire Danes.
>>109466868>>just run your desktop off the iGPU so the 7900xtx is dedicated to comfy/gamesIs this really something people DON'T do on default? Like you are literally wasting free vram if you have an integrated graphics cpu, wtf???
>>109463158gaytracing is needed for modern games, and only 9000 series have good one.
