· about 35 minutes · runs on ubuntu-26.04-automation · unrated
An unrated lab. It runs on your machine with no account and no network, and everything about it — the faults, the checks, the hints and the reference solution — is in the repository. An attempt on it is recorded on your profile and never moves a rating: only rated labs do. Rated and unrated labs.
The operations team's assistant runs Ollama with qwen2.5:0.5b on this 3 GB machine. After a
round of "tuning" it stopped answering entirely: some requests come back with a 500 and a long error
about a buffer, others never come back at all, and the service's restart counter keeps climbing.
The tuning is all in one place, and every line of it was meant well: long documents need a big context window, eight people use the assistant, and a memory limit keeps a runaway model from taking the machine down.
What the team actually needs:
What is expected, and graded:
You have root through sudo. Everything must still hold after a reboot.
The machine is checked, rebooted, and checked again. A check passes only if it passes both times.
| Check | Objective |
|---|---|
| 01_a_question_gets_an_answer | Size a model server's context and parallelism to the memory it has |
| 02_two_people_with_4096_tokens_each | Size a model server's context and parallelism to the memory it has |
| 03_a_memory_limit_with_room_to_work | Cap a service's memory without starving it, and tell an out-of-memory kill from an allocation failure |
Where the lab's hints send you, level by level, as you ask for them (h, then l opens a journal section in the TUI).
4 questions on the same topic, in the lab's Theory tab. They never affect the lab's grade. Three of them, to answer here:
A model file is 400 MB. Ollama is started with OLLAMA_CONTEXT_LENGTH=32768 and OLLAMA_NUM_PARALLEL=8. Why does loading need far more memory than 400 MB?
The key-value cache is allocated for context length times parallel requests: 262,144 tokens here
Besides the weights, the runtime reserves a key-value cache sized for every token of every parallel slot. On the lab VM qwen2.5:0.5b at 32,768 × 8 tried to allocate 3,072 MB for that cache alone; at 4,096 × 2 the whole server peaked at about 770 MB. The weights are loaded once.
https://github.com/ollama/ollama/blob/main/docs/faq.mdx
The journal of a service says "A process of this unit has been killed by the OOM killer" and "Failed with result 'oom-kill'", and the machine has gigabytes free. What is the likeliest cause?
The unit's own MemoryMax= (its cgroup limit), not the machine, ran out
systemd puts each service in a cgroup; MemoryMax= caps it, and the kernel's OOM killer acts within that cgroup when the cap is reached ("constraint=CONSTRAINT_MEMCG" in the kernel log), however much the machine has free. systemctl show -p MemoryMax -p MemoryPeak shows the cap and the peak.
man 5 systemd.resource-control (MemoryMax=) · https://docs.kernel.org/admin-guide/cgroup-v2.html
Ollama answers a request with HTTP 500 "failed to allocate buffer for kv cache", and NRestarts does not change. How does this differ from an OOM kill?
The runtime asked for memory it could not get and gave up cleanly; nothing was killed
An allocation failure is reported by the program: the load fails, the server stays up, the client gets a 500 with the error. An OOM kill happens to the program: the process dies mid-request, the client sees the connection drop, and systemd restarts the unit (NRestarts rises).
man 5 systemd.resource-control · https://github.com/ollama/ollama/blob/main/docs/api.md