norboten · cheat sheet

A model follows only the instructions that fit in its context window — cheat sheet

ollama, ai-services
# what the server holds
ollama show support-bot --modelfile
ollama show support-bot --parameters
curl -s 127.0.0.1:11434/api/show -d '{"model":"support-bot"}' | jq '{parameters, system: (.system|.[0:80])}'

# build / rebuild from the file
ollama create support-bot -f /srv/support-bot/Modelfile

# how much of the prompt was read
curl -s 127.0.0.1:11434/api/chat -d '{"model":"support-bot","stream":false,
  "messages":[{"role":"user","content":"test"}],"options":{"num_predict":1}}' | jq .prompt_eval_count
#   compare with the same request and "num_ctx": 8192
sudo journalctl -u ollama -o cat | grep 'truncating input prompt'

# precedence: request options > model PARAMETER > OLLAMA_CONTEXT_LENGTH / defaults
grep -rn num_ctx /opt/support-chat/

# Modelfile
FROM qwen2.5:0.5b
PARAMETER num_ctx 4096
PARAMETER temperature 0.2
SYSTEM """…"""