>>109198317
>what are the right settings for pi for it to respect llama server's context limit
In ~/.pi/agent/models.json:
{
"providers": {
"llama.cpp": {
"baseUrl": "http://127.0.0.1:8080/v1",
"api": "openai-completions",
"apiKey": "no-key-needed",
"models": [
{
"id": "llama.cpp model",
"name": "llama.cpp model",
"reasoning": true,
"input": ["text"],
"contextWindow": 131072,
"maxTokens": 131072
}
],
"compat": {
"supportsDeveloperRole": false,
"supportsReasoningEffort": false
}
}
}
}
As far as I can tell, the context size set here is only used for the % display at the bottom and for deciding whether to compact right after the assistant finishes its turn. During the assistant turn it just keeps going until it gets a context limit error from the backend and then compacts at that point.
>and for reasoning effort to actually have any effect?
Dunno. I set the effort limit on the llama-server side instead.
>whats a good frontend for raw chats? using curl gets autistic fast
mikupad
Or, curl but use jq to process the inputs/outputs so you can write the prompt as normal text in your favorite editor
