>>109274416
nta but speaking of thinking process i once accidentally got the thinking process of gpt 5.6 sol medium - NEVERMIND gpt 5.6 terra xhigh instead
to install packages.
- CUDA 11.4 and 470 ceiling vs 610/580 supports newer CUDA 12.
- 9 MiB display overhead is not equal total usable VRAM.
- Total 12051 means 237 MiB unavailable; 470 maybe worse.
- On host with server running monitor.
- optional arch check specific `sm_86`; quick command.
Need be cautious `nvidia-smi` 470 can total 12051 vs actual 12288 from host perhaps reserving 237. yes.
We can mention compare same model exact flags. Their flags perhaps include duplicate `--no-mmap`, no impact. Their 610 result 60 t/s perhaps perhaps 13.3? They said 13.3 (what is that? maybe 13.3? Probably speed). Current is 50t/s. We calculate ~15.6% less:
(60-50.7)/60=15.5%. Good.
Need perhaps include concrete performance fields:
- prompt speed 595-712 tok/s
- generation 49.7-51.2 tok/s
- GPU utilisation 99% => GPU bottleneck (not CPU fallback). It used all layers given `-ngl 10000`.
- `-ncmoe 10` etc unchanged. Great.
We can say VRAM savings may be illusory: nvidia-smi total is driver reported allocatable pool, total less on 470. The `9MiB` maybe display allocation and process accounting not compare with host due chroot. Need mention `/dev/nvidia-uvm` we created works though not shown? After modprobe manual, device nodes likely nvidia-modprobe not created? User claims run server successfully GPU active, so UVM perhaps via CUDA created nodes? Wait after modprobe host initially ls didn't show UVM, but then user didn't post nvidia-modprobe/mknod follow up. Yet CUDA server successfully used GPU 99%. It must have UVM device somehow created by application or they performed commands unsaid. No need.
had to cut the rest cuz character limit
if anyone at lmg is works at chinese labs, mail me at ... uh... i-i dont have an email thats safe to share..
