Gonna share my setup for local vibecoding on normie hardware, because it seems like there's no way to google this and get answers that aren't ai generated. someone write this down:
prerequisites:
I'm doing this with a 16GB 5070 Ti. it's a regular gaming machine
If someone is doing similar with a 16GB 5060 Ti, I'd love to hear from you.
also this only works if you're already a programmer.
method:
host is arch linux running llama.cpp-git, using a Q4 quant of unsloth/Qwen3.6-35B-A3B-MTP-GGUF
$ llama-server -m $MODEL -c 0 -fa on --jinja --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0 --repeat-penalty 1 --presence-penalty 0 --spec-type draft-mtp --spec-draft-n-max 2 -np 1
then I spin up a virtual machine using the images from arch-boxes on github so that I can run pi.dev in an environment that I can set on fire. ssh into the VM with port forwarding to expose llama-server.
I use elpapi42's observational-memory extension in pi to maintain better performance across compactions.
results:
that specific model with MTP I get 70-100 tok/s. Plain Qwen3.6-35-A3B without MTP nets 18-20 tok/s. both are 262k context length, and plenty fast enough for coding
it does about 80% of my programming tasks without issue but sometimes struggles with using libraries that have had recent changes (because rolling release distro vs. 1 year old training data), but I can usually `git clone` the new docs in and ask pi to read them before trying again. that gets me to 90% of tasks done without problem.
other models are available, but I haven't been able to get anything that runs as fast as that. it feels like i'm doing something wrong, so someone tell me if this is a skill issue.
Qwen3.8-27B at Q3 runs at 5tok/s