I got oobabooga kinda working with my new 7900xtx. I can't load any gguf models but exl2 works well, 30t/s with rocinate 1.1 8bpw and 32000 context.
I keep getting this error with gguf:
ggml_cuda_compute_forward: RMS_NORM failed
CUDA error: invalid device function
current device: 0, in function ggml_cuda_compute_forward at /home/runner/work/llama-cpp-python-cuBLAS-wheels/llama-cpp-python-cuBLAS-wheels/vendor/llama.cpp/ggml/src/ggml-cuda.cu:2368
Do I have a setting enabled somewhere that makes it look for a cuda device? I did a brand new install with rocm and downloaded rocinate straight from huggingface.