[ norboten ]

Data model

Norboten keeps data in four places, and each place holds only what belongs there:

Where What Survives
PostgreSQL (server) accounts, credentials, tokens, attempts, ratings, play sessions and frames, verified questions, events everything; backed up nightly with a restore check
Redis (server) live frames in flight, rate-limit counters, a short cache nothing needs to — losing it costs a reconnect and a cold cache
The learner's machine (~/.norboten) lab VMs, images, sessions, grade reports, recordings, theory progress, the sign-in token until the learner deletes it
The repository labs, question banks, journals, docs, recordings shipped with the site git

The schema is SCHEMA in api/src/norboten_api/db.py: plain SQL, applied on every start, every statement idempotent (CREATE … IF NOT EXISTS), under a PostgreSQL advisory lock so two API workers starting together do not race. A column that holds data is never renamed or dropped, so an older API image runs against a newer database — which is what makes a rollback safe. The one exception came before any account existed: the password and email columns of the earlier sign-ins, and the login_codes table, went on 2026-09-16. Times are timestamptz in the database and Unix seconds in Python.

PostgreSQL

credentials 1──* tokens            users 1──* attempts
     │                               │
     └──────── user_id ──────────────┤ 1──* ratings (one per topic)
                                     │
                                     ├─ nick shown on play_sessions 1──* play_batches
                                     │
                                     ├──* rated_attempts ──────┐
                                     └──* rated_quiz_sessions ─┴─ closed ─▶ one attempts row

events

user_id is a random 32-character hex id created by a GitHub account's first sign-in. It is the key everywhere and never changes; the nick is a label on top of it and is chosen once.

credentials

Who an account is: a GitHub account, and nothing else. Created by the first sign-in with that GitHub account (POST /auth/github/poll from a terminal, POST /auth/github/exchange on the site). There is no password and no email column (both dropped 2026-09-16), and no GitHub token is ever stored.

Column Type Notes
user_id text primary key
github_id bigint GitHub's numeric user id; unique (credentials_github_idx); never changes
github_login text the login as GitHub last reported it, refreshed at every sign-in; the public profile links to it
created_at timestamptz
discord_id text the linked Discord user, or null; unique (credentials_discord_idx)
digest boolean the weekly learner digest; false until the learner turns it on, and only possible with Discord linked (PUT /auth/preferences); read by the digest job (norboten_api/digest.py) on the server
discord_error text why the last digest could not be delivered (Discord's 50007), shown on the account page; cleared on a new link

tokens

Opaque bearer tokens. The client keeps the token; the server keeps only its SHA-256.

Column Type Notes
token_hash text primary key; hex SHA-256 of the token
user_id text credentials.user_id, ON DELETE CASCADE
kind text web (the site) or cli (a terminal); either 90 days when remembered, 12 hours when not. mcp and mcp-refresh are an MCP client's
label text for cli, the machine that signed in, e.g. tower.local (Darwin)
created_at, expires_at, last_used_at timestamptz last_used_at is updated on each authenticated request

Index: tokens_user_idx (user_id) — "where am I signed in" and revoking all.

pending_sign_ins

A sign-in or a Discord link that has started and not finished (norboten_api/pending.py), in the database rather than Redis so that a restart strands nobody mid-sign-in. Every row is used once — it is deleted as it is read — and expires.

Column Type Notes
id text primary key; random; the terminal's poll handle, GitHub's state, the one-time code
kind text device (a terminal), web (the site), once (the one-time code the site exchanges), discord-link
data jsonb device: GitHub's device code (never sent to the terminal), the polling interval, the terminal's label, remember; web: the page's own state, remember, where to return; once: the GitHub id and login, remember; discord-link: the account and whether to join the server
expires_at timestamptz 15 minutes; a once code 60 seconds

Index: pending_sign_ins_expiry_idx (expires_at).

users

The public profile. Created by POST /me once a signed-in account chooses a nick.

Column Type Notes
user_id text primary key
nick text unique; ^[a-z0-9][a-z0-9_-]{1,18}[a-z0-9]$
country char(2) ISO 3166-1 alpha-2; drawn as a flag
created_at timestamptz
seed boolean a generated sample account, labelled wherever it is shown

attempts

One graded lab attempt or one finished theory run. Written by POST /attempts, and by a rated attempt when it closes (rated_attempts); never updated.

Column Type Notes
id bigserial primary key
user_id text
kind text lab or quiz
lab_id text the lab id, or the quiz topic for quiz
started_at timestamptz when the faults were applied (the clock's start)
duration_seconds integer
score_percent smallint 0–100
passed boolean
rated boolean practice attempts are recorded and never rated
within_limit boolean false when a rated attempt ran over its clock (then it is a loss)
difficulty smallint from the lab manifest, never from the client
topics text[] from the manifest too
rating_delta jsonb {topic: change} this attempt caused; empty for practice

Indexes: (user_id, started_at desc) for a profile, (lab_id, started_at desc) for analytics.

ratings

The current Glicko-2 rating per learner per topic. Upserted after each rated attempt.

Column Type Notes
user_id, topic text composite primary key; topic is one of the nineteen slugs in cli/src/norboten/topics.py
r, rd, sigma double precision rating, rating deviation, volatility (start 1500 / 350 / 0.06)
games integer
updated_at timestamptz

Index: ratings_topic_idx (topic) for a topic board. The overall rating is not stored: it is the inverse-variance-weighted mean of the topic ratings, computed when read.

rated_attempts

A rated lab attempt between being issued and being judged (docs/lab-spec.md §13). Created by POST /rated/attempts; closed by the last fact record, by …/abandon, by starting another one, or by the hourly sweep once it expires. Closing writes one attempts row with rated = true.

Column Type Notes
attempt_id text primary key; 24 random url-safe characters
user_id text
open boolean false once closed
expires_at timestamptz issued + three times the lab's clock, at least an hour
data jsonb the rest, read whole: lab, image, nonce, the attempt key, the received records, the outcome, each check's verdict, the rating change
created_at timestamptz

Indexes: (user_id) WHERE open — one open attempt per learner is looked up on every start — and (expires_at) WHERE open for the sweep. The key is stored because the server verifies with it; it signs this attempt's records and nothing else.

rated_quiz_sessions

A rated theory run (docs/quiz-spec.md §6), in the same five columns as rated_attempts: the run's id as attempt_id, its owner, open, expires_at (an hour after it started) and data — the topic, the questions it will ask in order, each served question with when it was served and what came back, the outcome and the rating change. Closing it writes one attempts row with kind = 'quiz' and rated = true. Indexes as for rated_attempts.

play_sessions

A recorded or live terminal session. Created by POST /play/sessions.

Column Type Notes
session_id text primary key; 16 random url-safe characters
user_id, nick, country text the nick and country as they were when it started
lab_id, lab_title text
width, height smallint the terminal, for the player
started_at, last_frame_at, ended_at timestamptz live = not ended and a frame in the last 25 s
frames, commands integer running counts
passed boolean set when it ends, if known
seed boolean
expires_at timestamptz started_at + 7 days; purge() deletes past it hourly

Index: play_sessions_recent_idx (last_frame_at desc).

play_batches

What the recorder flushed every couple of seconds. Deleted with its session.

Column Type Notes
session_id, seq text, integer composite primary key; a retried batch is ignored (ON CONFLICT DO NOTHING)
at double precision seconds into the recording
events jsonb asciicast v2 events `[time, "o"
commands jsonb [{at, text}]
changes jsonb [{path, diff, truncated, command}] — diffs taken inside the guest

events

Append-only operational events, as JSON: chat (answered, blocked), progress (anonymous, opt-in), stuck_point (opt-in telemetry). No question is stored: the banks are files in the repository (a questions table from before 2026-09-15 is dropped on start).

Column Type
id bigserial primary key
kind text
payload jsonb
created_at timestamptz

Index: events_kind_idx (kind, created_at desc).

Redis

api/src/norboten_api/live.py. No persistence (--save "" --appendonly no), 256 MB with LRU eviction.

Key or channel Type TTL Written by Read by
play:<session_id> pub/sub channel POST …/frames and …/end publish `{type: batch end}`
rate:<bucket>:<who>:<window> counter the window + 5 s every rate-limited endpoint the same
`board:<topic overall>:<limit>` string (JSON) 30 s GET /leaderboard
readyz string 10 s GET /readyz the same

Rate limits: chat 20 a minute per client (NORBOTEN_CHAT_PER_MINUTE); starting a sign-in and exchanging a one-time code 10 a minute per client each (NORBOTEN_SIGN_INS_PER_MINUTE); starting a Discord link 10 a minute. The client is the first X-Forwarded-For address Caddy sets.

The learner's machine

~/.norboten, or NORBOTEN_HOME.

Path What
lima/<version>/ the pinned Lima
vms/ the lab VMs (LIMA_HOME): disks, snapshots, serial logs and sockets
images/<id>/<arch>/ golden base images, with the metadata they were verified against
labs/<id>/<version>/ labs pulled from the registry
sessions/<lab>.json a lab session: state, image, attempts, hint levels, clock
sessions/<lab>.last.json the last grade report, with every check's result per pass
plays/<lab>-<stamp>.cast, .log.json recordings: asciicast v2, and the commands and diffs
journals/<id>.pdf exported journals
progress.json theory results per topic
credentials.json mode 0600: the cli token, its expiry, which API issued it

Nothing here is sent anywhere unless the learner signs in (graded attempts, rated theory runs) or streams a session.