3 Internals
Thomas Forgione edited this page 2026-09-10 18:10:09 +02:00

How it works

The screen

papote takes the whole terminal — the alternate screen, like an editor. The conversation is above, a boxed input below it, and the model and hints on the last line. A scrollbar appears on the right as soon as the conversation is taller than the pane.

Every frame is drawn from scratch from the messages. That is what makes a resize a non-event: the whole conversation simply re-wraps at the new width, mid-answer included. An earlier version printed into the terminal's scrollback and did its own cursor arithmetic; resizing during an answer corrupted the display outright, which is what prompted the move.

Rendering as it streams

Markdown cannot be formatted as it arrives — a table is not alignable until its last row is in, **bold stays literal until it closes. So the answer still being written is re-parsed and re-rendered on every frame.

Completed messages are cached as rendered lines and only rebuilt when the width or the message count changes, which is what keeps a long conversation cheap to redraw sixty times a second.

The pipeline is:

pulldown-cmark events  →  a small block tree      (md.rs)
block tree             →  ANSI lines              (render.rs)
ANSI lines             →  ratatui spans           (ansi.rs)

The renderer emitting ANSI and a parser reading it back may look like a detour, but it keeps the markdown renderer a pure function of blocks and width — testable on its own, with no terminal in sight — and it is what allowed the whole display to be swapped out without touching it.

Each frame is bracketed in synchronized output (DEC private mode 2026) so the terminal shows it in one go. It matters more than it sounds: when a new line arrives at the bottom the whole conversation shifts up, so every cell on screen changes at once, and a terminal free to present mid-rewrite shows half the old screen and half the new. Terminals that do not know the sequence ignore it.

Streaming and cancellation

The HTTP request runs on its own thread and feeds deltas through a channel, so the screen keeps redrawing and the keyboard keeps responding while the model is talking. Doing it inline would freeze the redraw and make Ctrl-C unreachable.

Cancellation is a flag the reader checks between chunks: it lands at once while tokens are flowing, and only when the server next says something if the connection has stalled.

Where the numbers come from

The two figures on the bottom bar are lifted from the last chunk of the stream, so they cost nothing extra:

  • usage.total_tokens needs stream_options: {"include_usage": true} on the request. That is standard OpenAI; a server that does not know it leaves the field out and papote shows no count.
  • timings.predicted_per_second is llama.cpp's own addition and arrives without being asked for. predicted_* is the generation phase; the prompt has its own prompt_* counters, which papote ignores.

The size of the context is deliberately not shown, because there is no portable way to learn it. /props reports n_ctx: 0 behind llama-swap — the answer describes the proxy, not the model. The only other source is the --ctx-size buried in the launch arguments that /v1/models exposes, which means parsing a command line and would break the first time that format changes. The absolute token count says enough.

Errors

reqwest's own message stops at error sending request for url (…) and buries the reason, so papote walks the error's cause chain. An expired certificate reads as invalid peer certificate: certificate expired rather than as a shrug.

A reverse proxy answering 401 with a full HTML page has its body dropped — the status line already says everything it does.