# millivolt documentation > Self-hosted, cross-protocol inference proxy with a live dashboard, SQLite history, and an operator plane. One Go binary; no hosted dependency. # Documentation millivolt is one self-hosted service for routing model traffic across OpenAI Chat Completions, OpenAI Responses and Anthropic Messages, seeing what happened, and controlling the operator surface. Start with the path that matches what you need to do. ## First ten minutes 1. [Install and first run](install.md) from Compose or a source checkout. 2. Send one request on a client protocol and find it in the [dashboard](dashboard.md); the [capture gallery](gallery.md) shows every surface in both themes. 3. Read [configuration](configuration.md) before changing the example or a setting. 4. Read [security](../SECURITY.md) before exposing the listener. 5. Keep [operations](operations.md) nearby for health, storage and the known limits. ## If you want to ... | Task | Start with | | --- | --- | | Install and send a first request | [Install and first run](install.md) | | Configure a setting, request override or provider alias | [Configuration](configuration.md) | | Look up a setting's default or a CLI flag | [Settings reference](settings.md) | | See every dashboard surface on both themes | [Capture gallery](gallery.md) | | Expose the listener behind TLS or a network boundary | [Reverse proxy](reverse-proxy.md), [security](../SECURITY.md) | | Debug a 429 or a 5xx | [Routes and debugging](routes.md#error-storm-protection), [pause and limits](routes.md#pause-and-limits) | | Back up or restore | [Settings backup and restore](containers.md#settings-backup-and-restore), [image-only backup](containers.md#image-only-backup-and-restore), [online database backup](operations.md#storage-and-accounting) | | Upgrade or restart | [Versions and images](containers.md#versions-and-images), [reload, restart and shutdown](containers.md#reload-restart-and-shutdown) | | Integrate a client or translate between protocols | [Protocol](protocol.md), [adapters](adapters.md) | | Drive the operator plane from an LLM | [MCP server](mcp.md) | | Read or search the docs from an LLM | [llms.txt](https://millivolt.llm4people.dev/llms.txt), [docs.json](https://millivolt.llm4people.dev/docs.json), [MCP server](mcp.md#tools) | | Drive it from a terminal | [Operator CLI](cli.md) | | Contribute a change | [Contributing](../CONTRIBUTING.md), [architecture](architecture.md), [support](../SUPPORT.md) | The [operations overview](operations.md) owns health, storage and the runtime cost model. Measurements live in [performance](performance.md); ownership and invariants live in [architecture](architecture.md). Project pages are [support](../SUPPORT.md), [releases](https://github.com/LLM-4-People/millivolt/releases), [license](../LICENSE) and [third-party notices](../THIRD_PARTY_NOTICES.md). ## Verification and limitations The generated [settings reference](settings.md) lists every setting and proxy flag with its type, default, accepted values and reload behavior. Every page also has a Markdown twin, and `llms.txt`, `llms-full.txt` and `docs.json` index the same sources for an LLM or the `docs_list`, `docs_search` and `docs_read` MCP tools. [proxy.example.yaml](../proxy.example.yaml) is the commented deployment example. `internal/config` owns defaults, validation and Settings metadata; local `proxy.yaml` is not a public reference or a test fixture. Current commands live in [Contributing](../CONTRIBUTING.md#checks). Runtime limits live in [operations](operations.md#known-limits); measurement caveats live in [performance](performance.md#limitations). The [architecture map](architecture.md) connects invariants to their shared implementation and regression coverage. Past development notes are not an operating guide or a capacity guarantee. --- # Install and first run Install millivolt from the published image or a source checkout, set the operator credential, and send a first request on each client protocol. The [quickstart](../README.md#quickstart) is the short form of the same procedure; this page carries the full path. ## Before you start The container path needs Linux with Docker Engine and the Compose plugin. No Go, Node or source checkout is needed for it. The source path needs the Go toolchain required by [go.mod](../go.mod). > [!IMPORTANT] > One shared operator credential protects the dashboard and operator plane only. > Inference is transparent: it forwards your upstream key and does not use the > operator token. There are no user accounts. Keep the listener on loopback, or > put the network or ingress boundary in front. With the token unset, the > dashboard denies every request. Read [Security](../SECURITY.md) before network > exposure. ## Install with Docker Compose ```sh curl -fsSL https://raw.githubusercontent.com/LLM-4-People/millivolt/main/compose.yaml -o compose.yaml export MILLIVOLT_OPERATOR_TOKEN='replace-with-a-long-random-value' docker compose up -d ``` Open [http://127.0.0.1:8080/](http://127.0.0.1:8080/) and sign in with that token. If the container exits, run `docker compose logs --tail 50`. The Compose file publishes on loopback only. It keeps history and settings in named volumes, runs non-root with a read-only root filesystem, and accepts `MILLIVOLT_PORT` for an unused host port. For bind mounts, pre-create directories writable by UID/GID `65532:65532`; see [persistent state](containers.md#persistent-state-and-configuration). For image selection, backups, updates, shutdown, or protected ingress, see [container operation](containers.md#containers) and the [reverse-proxy guide](reverse-proxy.md). The published image is `ghcr.io/llm-4-people/millivolt:main` for amd64 and arm64. Version tags identify releases; pin a digest when exact bytes matter. See [versions and images](containers.md#versions-and-images). ## Install from a source checkout Install the Go toolchain required by [go.mod](../go.mod), then: ```sh git clone https://github.com/LLM-4-People/millivolt.git cd millivolt test -e proxy.yaml || cp proxy.example.yaml proxy.yaml export MILLIVOLT_OPERATOR_TOKEN='replace-with-a-long-random-value' go run ./cmd/proxy -config proxy.yaml -listen 127.0.0.1:8080 ``` For isolated development and checks, see [Contributing](../CONTRIBUTING.md). ## Environment and defaults The Compose inputs are environment variables; their defaults live in [compose.yaml](../compose.yaml), not server configuration. Supply them consistently through the environment or an ignored local `.env` file. | Variable | Meaning | | --- | --- | | `MILLIVOLT_OPERATOR_TOKEN` | The single operator credential. It is read once at boot from the process environment and never from `proxy.yaml`, so rotating it requires a real process restart. A nonempty value shorter than 16 or longer than 512 bytes fails the boot. Unset or empty leaves the dashboard denied. | | `MILLIVOLT_IMAGE` | Image reference; defaults to `ghcr.io/llm-4-people/millivolt:main`. | | `MILLIVOLT_PORT` | Loopback host port; default `8080`. | | `MILLIVOLT_STOP_GRACE_PERIOD` | Container stop grace; default `30s`. | | `MILLIVOLT_MEM_LIMIT` | Container memory ceiling; default `1g`. | | `MILLIVOLT_PIDS_LIMIT` | Process and thread ceiling; default `512`. | `docker compose config` shows the resolved deployment. A source deployment configures the same service through `-config PATH` (default local `proxy.yaml`), `-listen ADDRESS` and `-db-path PATH`; the generated example and the Settings schema are the field reference. See [configuration](configuration.md). ## Send a first request Point a client at the proxy and supply the upstream base URL and key on each request. The proxy registers no providers or keys. `X-Proxy-Format` selects a different upstream protocol; absent, the request passes through. See [protocol](protocol.md) and [adapters](adapters.md). OpenAI Chat Completions: ```sh curl -sS http://127.0.0.1:8080/v1/chat/completions \ -H "X-Proxy-Base-URL: $UPSTREAM_BASE_URL" \ -H "Authorization: Bearer $UPSTREAM_API_KEY" \ -H 'Content-Type: application/json' \ -d "{\"model\":\"$UPSTREAM_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"hello\"}]}" ``` OpenAI Responses: ```sh curl -sS http://127.0.0.1:8080/v1/responses \ -H "X-Proxy-Base-URL: $UPSTREAM_BASE_URL" \ -H "Authorization: Bearer $UPSTREAM_API_KEY" \ -H 'Content-Type: application/json' \ -d "{\"model\":\"$UPSTREAM_MODEL\",\"input\":\"hello\"}" ``` Anthropic Messages, using the native upstream's own auth header, prefix and API version: ```sh curl -sS http://127.0.0.1:8080/v1/messages \ -H "X-Proxy-Base-URL: $UPSTREAM_BASE_URL" \ -H "X-Proxy-Key: $UPSTREAM_API_KEY" \ -H 'X-Proxy-Auth-Header: x-api-key' \ -H 'X-Proxy-Auth-Prefix;' \ -H "anthropic-version: $ANTHROPIC_VERSION" \ -H 'Content-Type: application/json' \ -d "{\"model\":\"$UPSTREAM_MODEL\",\"max_tokens\":1024,\"messages\":[{\"role\":\"user\",\"content\":\"hello\"}]}" ``` The semicolon in `-H 'X-Proxy-Auth-Prefix;'` is important: curl sends a present empty header. `-H 'X-Proxy-Auth-Prefix:'` instead omits it, leaving the proxy's `Bearer ` default. A response means the proxy answered; check the status code for the upstream outcome. The request appears in the dashboard. On a shared Compose network, use `http://millivolt:8080/v1` instead of loopback. `/v1/chat/completions`, `/v1/responses` and `/v1/messages` are all available. Each can reach Chat, Responses or Anthropic upstreams; only a Chat client can target the Cursor bridge. Read [upstream compatibility](adapters.md#client-and-upstream-compatibility) and [cross-protocol translation](adapters.md#cross-protocol-translation). ## The dashboard The dashboard is the site root, [http://127.0.0.1:8080/](http://127.0.0.1:8080/). It asks for the operator token once per browser tab, keeps it in sessionStorage, attaches it to gated calls only and drops it as soon as the server rejects it. The same credential gates `/metrics/*`, `/admin/*` and `/mcp`; transparent inference does not use it. See [operator access](routes.md#operator-access) for the canonical open-path list. ## What to read next [Documentation](README.md) routes by task. For the next step, read [configuration](configuration.md) before changing settings, or [the dashboard tour](dashboard.md) to find the request you just sent. --- # Dashboard guide The plate below is the complete dashboard. Use this guide to follow its charts, explorer, request drawer, operator console and Settings without hunting through the interface. These are real application captures from explicitly authorized local history. Recorded timing, token and cost values are retained; identifiers are replaced, and request content, retained headers, debug captures and operator state are removed. Settings and the theme tray show the public example, not private configuration. No requests, holds, limits or debug sessions were created to make the pictures. The capture session blocks external requests and disables live SSE, so its **offline** indicator describes a static screenshot session, not service health. The images are not latency guarantees, capacity measurements or independent bills. Every capture in this guide and in the [capture gallery](gallery.md) is one state photographed on both primary surfaces and split along the diagonal. The upper-left half is Instrument Black, the lower-right half is Instrument Clear. Data, scroll and any open surface are identical across the cut. ![The millivolt dashboard at 1920 pixels, split diagonally between the Black and Clear surfaces](images/overview/dashboard.png) ## Error storm banner When enabled in Settings, error storm protection adds a live banner below the header. Click an incident to inspect the number of requests with selected errors, failed upstream attempts, error category, queued requests, next retry time and recovery progress. It names the affected provider or exact provider/model, shows its safe error category, failed upstream-attempt percentage and window, queue count and recovery probe state. Provider incidents require all models active in the same window to have elevated errors. The banner shows that active-model evidence; its attempt percentage differs from the Errors chart's affected-request rate. It clears when protection recovers and can be hidden independently in Settings. See [configuration and behavior](routes.md#error-storm-protection). The static captures below show no banner because the capture session has no active incident; the banner stays hidden until protection fires. ## Timelines The KPI band remains global. Explorer filters scope the timeline and request log together. The timeframe control in the card head changes chart history, not the global KPI band. It offers a zoom ladder from one minute through one year plus **All time**, and the calendar presets today, yesterday, this week and this month. Its panel draws the stored coverage as a rail and disables any window wider than the stored history, so a selection can never promise more history than exists; **All time** means exactly the retained history, and a calendar preset clamps its start to the oldest stored request. The selection persists per browser. Legend controls show or hide individual series; hover a bucket for its precise values and time interval. Counts, sums and percentiles come from the server. A window with fewer than four plotted points paints a waiting message instead of the chart: one to three buckets are too few to read a trend, and the totals strip still tells the story until a fourth point arrives. A span with no stored history says so; a span inside history with no requests says that instead. ![Timeframe panel with the stored-coverage rail, the zoom ladder and the calendar presets](images/charts/window.png) A failed chart request never blanks a good payload: one transient error should not hide the history you were reading. The context row states which of the two situations applies. With nothing drawn it reads `chart unavailable - retrying`; with a last-good payload still on screen it reads the window and `stale, retrying`, so old buckets are never presented as a current reading. Either way the next successful request clears it. The **In-flight** KPI tile separates sending work from parked work. Its summary names two totals that fit the narrowest tile - active and queued - and the composition rail under the value splits every live state by color: sending, pending, operator-paused and throttled. Hover the rail, or focus it with assistive technology, for the exact per-state counts. Queued covers both the admission queue (per-key concurrency, pacing, provider caps, operator holds) and the retry send gate, so a provider's 429 pacing window shows its parked requests as queued instead of as idle streams. The request table's `throttled` pill marks the same state per row. ### Overview **Overview** is the summary: it carries no plot at all. The tiles are the whole surface and they take the card. Every headline metric reads its period value first, then its measured companion fact, then its evolution sparklines: - requests / tokens: the request count over the blended token volume. - tokens in/out/cached: one in / out / cached triple, with the input share of all tokens and the cache hit share of input underneath. Each share line names what it compares. - spend with the server's blended per-Mtok price (the same blended-price rule as the KPI band, scoped to the viewed period and cost-reporting requests only). - the health pair (errors in the error red / rate-limited requests in the 429 tone, over the request denominator). - the merged timing tile: average latency / average speed over every measured request in the period (server-computed from the same samples the percentiles rank, never an average of bucket percentiles), each metric's low-high sample range underneath. Each tile carries one sparkline; combined tiles draw one self-scaled line per metric half in exactly the half's color. Sparks span only traffic-bearing buckets - the same empty-bucket removal the plots apply - so window padding never squeezes the data into a corner. The metrics picker in the card head chooses what shows: one menu of checkable rows, checked metrics render and unchecked ones leave the band (it reflows), and the selection persists per browser like every chart view. Every selected tile always renders in both layouts: the band wraps into as many whole rows of 170px tracks as the card needs, and the page scrolls past them in the base layout; the picker's count reports the selection itself, so it can never disagree with the band. The wide layout is the 1200x750 one-screen lock: the page locks to the viewport with no scrolling, the explorer compresses to one gallery row, and the timeline column floors at 572px so the 5-metric selection wraps into at most two rows. Every tile keeps one fixed size whatever it carries - spark or not, one share line or two, and the no-data skeleton swaps in without moving the card - and the band sits at its natural height at the top of the card. Only the timeframe control applies. ![The overview preset with five summary tiles and their sparklines](images/charts/overview.png) ### Traffic **Traffic** compares request and error counts. Bars represent bucket totals, not smoothed rates. Empty intervals may be omitted in this preset; the chart explains that below its legend. ![Traffic timeline with request and error counts from recorded history](images/charts/traffic.png) ### Errors **Errors** compares the count of affected requests with an error-rate line on a separate percentage axis. A request that only encountered HTTP 429 is not an error. A recovered failure can still contribute an error signal. ![Error timeline with affected-request bars and a separate percentage axis](images/charts/errors.png) ### Token usage **Tokens** compares input, output, reasoning and cached usage. The upstream's reported fields determine which splits are available. The compressed vertical scale keeps large prompt/cache volumes and small outputs readable together. A cache-hit line on a separate percentage axis tracks cached prompt tokens as a share of input per bucket. The totals strip adds each side's share of the blended total, the cache share of input, and the in:out tile: the raw pair (one line, halves in the token bars' colors) with the input share riding underneath - a percentage reads the same in every locale, where a small comma-decimal ratio looked like a count. ![Token timeline with input, output, reasoning and cached usage](images/charts/tokens.png) ### Speed and latency **Speed + latency** compares generation speed (first token to last token) with time to first token on separate axes. TTFT is measured from each request's upstream attempt send, so a retry has its own TTFT. One percentile dropdown controls both series; the legend does not repeat it. Choose p50, p95 or p99. Each statistic needs at least four valid samples; smaller sets and missing measurements remain unavailable. A bucket plots only when both measurements exist at the selected percentile, and those unmeasured buckets compact out of the timeline instead of stranding gap points in dead space - the context row below the plot says when that happened. Period readouts use the period's samples, not an average of bucket percentiles. ![Speed and latency timeline with independent axes and percentile control](images/charts/speed-latency.png) ### Provider-reported cost **Cost** shows reported spend alongside request counts. Values below $1 use cents, including tooltips and totals. Missing provider cost does not establish a zero bill, and there is no hardcoded per-model price table. ![Cost timeline with spend and request-count axes](images/charts/cost.png) ## Explorer Changing the dimension in the rail changes the grouping, not the active filters. Selecting a card adds or replaces that dimension's exact filter; selecting it again removes it. Filters from other dimensions remain active. The breadcrumb chips remove individual selections or return to all traffic. The gallery ignores its own dimension's filter so alternatives remain visible; the timeline and request log still use the complete selected scope. Multi-valued dimensions, such as tools and errors, can overlap rather than partition requests. | Dimension, in rail order | What it groups | | --- | --- | | Providers | Server-derived upstream identities after configured aliases. | | Models | Model groups produced by the configured canonicalization rules. | | Clients | Explicit client labels or inferred SDK/User-Agent identities, retention-bounded before the ring and store. | | Conversations | Conversation IDs from explicit sessions, tracked body params (`k:` identities) or automatic grouping, with declared ancestry when resolvable. Selecting a conversation selects that leaf, not its descendants. | | Tools | Observed tool names; one request can belong to several tools. | | Time | Request start in the server's local timezone: weekday night 00:00-08:00, work 08:00-16:00, evening 16:00-24:00, or a single weekend group. This is separate from chart history range. | | Status | Coarse outcome classes and live states. The Requests dropdown instead selects exact HTTP codes or named live states, grouped into live states and HTTP status; it stays a native select so the operating system picker is the touch affordance. | | Errors | Distinct observed error identities, including absorbed failures; one request can have several. HTTP 429 alone is not an error. | | Keys | SHA-256 hashes of upstream credentials, never raw keys. Labels show only an eight-hex-character prefix; identical short labels do not establish identical full hashes. | ### Providers Provider cards compare request share, reported cost, tokens, cache use, timing and health. Error and HTTP 429 badges count distinct affected requests and omit zero values independently. A request can contribute to both badges. ![Provider explorer with pseudonymous groups, recorded costs and health signals](images/explorer/providers.png) ### Models Model cards group configured model variants for comparison. The shipped rules fold provider decorations (vendor namespaces, tags, quantization, service tiers, release channels and build stamps) into the base model, so one model served by several providers is one card. Model tiers, fine-tunes and parameter counts stay distinct. Selecting a card scopes the timeline and request log together. Model grouping is configuration-owned; stored model spellings remain available for exact export and deletion filters. Adapters can normalize those spellings before storage; see [request metadata](routes.md#request-rows-and-details). ![Model explorer with pseudonymous model groups and recorded usage metrics](images/explorer/models.png) ![Provider explorer filtered to one provider, with the removable scope chip above the rail and the selected card marked](images/explorer/scoped.png) The [capture gallery](gallery.md#nine-ways-into-the-traffic) carries all nine dimensions. This guide shows the two that move the most. Conversation ancestry is explicit, not guessed from timing or names. The authorized snapshot has no parent declarations, so these screenshots do not fabricate an agent tree. See the [client protocol](protocol.md) for relationship headers and the limits of missing, conflicting or ambiguous declarations. ## Request inspection ### Request table The request table combines recent finalized records with current in-flight state during ordinary live use. Rows show status, retry information, latency, throughput, token/cache usage, tool calls, duration, queue time and reported cost. Columns size to their content and keep a per-column floor: a value wider than every value seen before grows its column, and a later shorter value never shrinks it back. A live row finalizing from `streaming` to its recorded status therefore cannot pull the status column back; a final value wider than its in-flight placeholder can still grow its own column once. When the card is narrower than the learned columns, the table scrolls inside its own region. The region is keyboard-scrollable and labelled, the identity cell carries the full `client · provider · model` value in a hover title, and compact status pills and the actions button abbreviate by design. Its status selector and older-history paging operate on the current scope. ![Request table with recorded statuses, token usage, latency, throughput and cost](images/requests/table.png) ![Request table with one row expanded into its absorbed retry attempts](images/requests/retries.png) ### Request details Click a row or its detail button to open the inspector. The header keeps the request identity and the previous/next controls. The summary below it shows the live or final status, method and path, client, provider and model, plus duration, TTFT, tokens, speed, cost and retries when captured. The full field inventory is classified into four sticky groups, each with the number of cards it contains: - Request: identity, time bucket, request and response sizes, client/runtime, provider metadata, and captured model and parameters. - Content: conversation and parent ids, turn and size composition, prompt and response previews. - Usage: input, output, reasoning, answer, cache and generated tokens; timing, throughput, cost, transport decomposition, and tools. - Diagnostics: queue and rate limits, absorbed retries, in-band upstream responses, response headers, and the Debug capture when one exists. Only the selected group is shown; its cards behave like the previous single page. Each card header can collapse or expand its section, and both the active group and collapse choices survive live updates until the inspector closes. Copy actions sit on the values that matter (identifiers, conversation ids, paths, models, headers, error detail and captured bodies), on field-list cards, and on the header's JSON action for the whole record or the selected attempt. One polite status line confirms each copy or reports a denied clipboard write. Long values wrap. Captured prompt, response and Debug bodies open in a bounded code surface with a line count, wrap toggle, copy action and expand/collapse control; a long single line scrolls inside that surface instead of widening the inspector. A final upstream failure appears as a bordered alert above the cards with its type, code, message and copy action, plus the HTTP status when the failure was an HTTP response. A rate limit the provider announced in its own type or code counts in the rate-limit card even when the status was not 429; a request that ended on one still appears as an alert, and a bare 429 stays the flow-control handshake it is. On narrow viewports the inspector fills the width, stacks each key above its value, keeps touch-sized copy and close targets, and respects the bottom safe area. Previous/next controls follow the scoped table order. Explicit zero and false parameter values are retained instead of treated as missing. While a request is in flight, pause and throttle state appear as explicit rows. ![Request drawer showing pseudonymous identity, the summary readout and recorded request parameters](images/requests/details.png) ### Tokens and performance Open Usage for token splits, cache hit, generated-token composition, timestamps, the adopted attempt's send anchor, throughput, reported cost, finish state and transport timing. Absorbed retry attempts appear under Diagnostics when the selected request has them, each with its status, provider error detail, the attempt's own provider request id, and captured upstream server, model, processing and rate-limit facts. An in-band upstream response that arrived after a committed success is disclosed separately rather than presented as a retry. Clicking an absorbed attempt, a retry sub-row in the request table or an entry in the drawer's retry list opens that attempt's own detail view. ![Request drawer diagnostics group with queue and rate-limit facts and four absorbed attempts](images/requests/diagnostics.png) These captures deliberately omit request content and Debug payloads; ordinary opt-in captures remain sensitive. ![The same request drawer showing token and performance measurements](images/requests/performance.png) ## Operator console The header's operator readout is the one control surface. It stays in the header rail at every width, including the phone header; the theme tray and Settings sit behind the actions flyout there. It reads `steady` when no hold or capture is active, no limit policy is enforcing, no request is parked by the storm/quota gate and no restart is running, then one segment per active hold (`paused 1`), capture session (`captures 1`), enforcing limit policy (`limits 2`) and gated request (`gated 3`). A stored policy whose schedule window is closed enforces nothing, so it is named separately as `off schedule N` instead of being counted as a cap; the server owns that verdict and reports it per policy, so the header cannot re-derive a different answer from the stored minutes. The wording names the behaviour, because `scheduled N` read as "the cap is on a schedule and working" when it meant the opposite. While a rebuild runs, `restarting` comes first, with ` · step N` when the server reports a real step. The counts are live state: a hold that expires, a stopped capture or a cleared limit leaves on the next state repaint. A hold takes the warning tone because it is the one state that stops live traffic; a restart takes it too, with a pulsing dot. If the live stream was open and then drops, the line keeps its last known words and appends ` · offline`; if the operator poll instead stops answering, it appends ` · stale`. Both mark the words as last known rather than current. The header readout is not a live region; it updates when operator state, connectivity or restart status changes, and the trigger's accessible label carries the same words plus the action the trigger performs in its current state. The open console's head line repeats those words without a second total and is a polite live region. Activating the readout opens the console. On a wide viewport it is an anchored popover under the trigger: 520px wide, and only as tall as its content up to a 560px cap, scrolling inside it. At 800px or narrower it becomes a bottom sheet with a definite height of `min(88dvh, 100dvh - 16px)`, leaving a strip of page above it for outside-press dismissal. The height is definite rather than a cap: the body and the pane grow into the remainder, so a tall composer scrolls inside the pane instead of being clipped by the panel. On a landscape phone (800px or narrower and 500px or shorter) the header, rail, pane and ledger tighten and the sheet gives up its top gap, sitting flush with the top edge so only the 8px bottom dismissal strip remains; a very short viewport turns every fixed band into space the form loses. The section rail is one horizontally scrollable strip and fades its trailing edge while more sections remain; at 560px or narrower, where six chips genuinely cannot share one row, it becomes a two-row grid. That threshold is the content, not the device class: a landscape phone is 640-844 wide and keeps the single-row strip. Escape, an outside press, the close button, or opening the theme tray or Settings closes it; opening the console closes the theme tray and Settings first. The session remembers the last section, so reopening returns where the operator left. Opening the console itself performs no action. The rail names six sections: overview, limits, pause, capture, export and system. Limits, pause and capture carry a count badge: for limits the number of rules enforcing right now, for pause the active holds, for capture the running capture sessions. A rule outside its schedule window is installed but enforcing nothing, so it does not raise the limits badge. The overview is the rule register: one row per hold, capture and every installed limit, each jumping to the section that owns it, with an off-schedule limit marked `off schedule` rather than reading as a live cap, plus the queue drain control when requests are actually parked. A request parked by the storm/quota gate is named in its own block instead: that gate runs before admission, so a drain cannot release it, and the block points at the controls that can. The counts are not repeated there, because the head line and the section badges already state them. Limits, pause and capture hold rule rows and one composer; export holds the download builder, and system holds the restart and Settings cards. ### Rule rows Every rule is one calibration line: a kind tag in its state color, the human scope, the caps or schedule, then a live column for occupancy and actions. The scope sentence is generated from the selector the server enforces, so an empty dimension reads inside the sentence instead of as a separate checklist. Every rule kind offers the dimensions its own matcher honours: a limit and a hold both scope on providers, models, clients, key hashes and conversations, because the scheduler matches both against one request identity, while a capture scopes on clients, providers and models. - A limit row names its budgets and schedule plus any remaining request and token budget; who set it rides the row title rather than a third line. - A hold row names its remaining time, its queue cap or `queue unlimited`, and its queued count. - A capture row names its retained count, its run time, and its duration or `until stopped`. A row is the one edit target: activating it opens its composer in edit mode. The action column offers copy, which starts a draft from the same scope and schedule with no budgets, and the terminating action for that kind: remove, resume or stop. A copied limit that still matches its source selector is refused and named, because a limit's identity is its selector: applying it would edit the original rather than add a second rule. The edit action is not repeated there, because the row already performs it. On a narrow phone the row takes two lines so the scope can be read in full instead of being clipped; the budget wraps beneath the scope rather than being dropped, because a limit's scope identifies it while its budget is what it does. A rule list that outgrows the pane offers a filter once it holds enough rules to need one, so a console with two rules never spends a row on an empty box. The filter matches the text each row already renders, which gives limits, pause and capture one search rule and makes a new row field searchable without a second list of what to match. A filter that hides everything says how many rules are in force and offers to clear itself, so an empty result never reads as an empty policy. ### Composer anatomy Limits, pause and capture share one composer: a titled bench, one scope strip, budget or duration controls, a live preview sentence, and a footer with cancel and the one commit action. The composer opens above the rule list, so its form and its commit are in view the moment the operator asks for one. A phone pins that action row to the pane's bottom edge. The scope is one wrapping strip of dimension chips rather than five stacked rows. An unset dimension is a dim chip you click; a set one carries its values inside the same chip. One picker opens at a time, below the whole strip, with a filter only once the list is long enough to scan, the known values as checkable rows, and a free-value field except for key hashes. Provider, model, client and conversation scopes accept a typed value; the key scope lists only SHA-256 hashes already present in the live record ring. The preview states what the commit will enforce. Limits add concurrency, request and token budgets with their windows, an optional anti-burst ramp, and an optional day-of-week schedule with server-local from and to times. Burst control and schedule are peers on one row, each opening its own body below. A window is a native select: the list is long and the track is one number wide, so the current choice is stated in one line and the rest is the platform's own picker. The ramp is offered only on a single-provider rule without a schedule. Equal from and to times mean all day, and a window that starts after its end spans midnight. Pause adds everything, new-clients and named clients/providers modes, a duration or until-resumed choice, and an optional queue cap. Its head also carries a one-click bounded hold (`hold all 15m`), which writes the same all-traffic document through the same gate, ledger and undo path. It is bounded by construction: an indefinite hold is still available, but only through the composer where the operator chooses it. Capture starts a scoped Debug session and adds client, provider and model scopes, a duration or until-stopped choice, and the retention and byte facts for the current configuration. ### Confirmation and the change ledger Removing a limit, resuming every hold, stopping every capture, draining the queue and restarting open the shared confirmation dialog. It states the consequence, focuses cancel first, and names the action on its confirm button. Individual stop, and a resume with nothing queued, commit directly; resuming a hold that has parked requests opens the shared confirmation first, because it releases that backlog to the provider. An edit commits when its composer is applied. All are recorded. The console footer is a bounded session ledger. It shows the newest change and offers Undo while that change has a defined inverse. Undo applies it once and appends an `undid: ...` receipt; the next change becomes the new undo target. The ledger has one owner: the console footer. Up to eight changes are retained for the page session. Changes without an inverse, such as a download or a restart, are recorded without an undo action. The ledger is browser memory, clears on reload and is not an audit log. A rejected write is named in the open pane, not hidden behind a closed panel. ### Export The export section owns record exports. Choose all history or matching records. Matching records offers provider, model, client, conversation and status in the same one-strip selector the composer uses, then errors-only, debug-only, newer-than and older-than, plus an exact error type field. The receipt and its download stay in the same view as the filters that shape them. Provider, model and client merge the server's known vocabularies with whatever the live record ring holds, so a quiet proxy or a fresh restart still offers them; all four accept a typed value as well, so a dimension the server has never seen is not a dead end. Status is the one list-only dimension, because a status code is not a name anyone can type reliably. Both modes preview through the same server count: all history is the same predicate with an empty filter. The receipt always names the number of records the download will carry, the download stays disabled until a nonzero count is ready, and the ledger repeats that number. A mode switch invalidates the count rather than quoting the previous mode's total, and filter mode with nothing selected asks for a filter instead of counting the unfiltered set. Downloads are gzip JSON artifacts. Configuration and database backups stay in Settings, and the console links there. ### Request detail A request's detail drawer carries the two console prefills that record can seed: `hold this scope` and `capture this scope`. Both close the drawer and open a draft scoped to that request, so holding the client of a slow response does not require reading the value, closing the drawer and retyping it into a picker. They are prefills, not commits: the composer still gates every write, so no confirmation is added. The open is deferred one task, because the press that triggered it would otherwise reach the header's outside-press closer and dismiss the console it had just opened. ### System The system section owns restart and the console's link to Settings. It explains the rebuild and handoff, lists the four steps as they actually start (`rebuild from source`, `finish in-flight streams`, `flush metrics to disk`, `start the new process`), and disables restart while a run is active or the build cannot restart in place. A restart asks for confirmation and is recorded in the ledger without an undo action. Queue bounds, retry budgets, storage, retention and dashboard cadence are Settings fields, not console policies; the section links there instead of duplicating them. Static captures of the operator console publish to `docs/images/console/` from the same authorized history workflow as the rest of the gallery. The retired menu screenshots were removed with the menus they pictured. Every section is recorded in its neutral starting state: no draft is committed, no rule is selected and no filter is armed. ![Overview section with no policy in force](images/console/overview.png) ![Limits section with no policy set](images/console/limits.png) ![Limit composer with its scope chips, budgets, window and live preview](images/console/limits-composer.png) ![Pause section before any hold exists](images/console/pause.png) ![Capture section with no session start](images/console/capture.png) ![Export section previewing the whole-history record count](images/console/export.png) ![System section with restart available and the step list](images/console/system.png) See [routes and debugging](routes.md) for exact admission, filtering and failure behavior, [containers and data](containers.md) for restart, and [operations](operations.md) for persistence, before using these controls. ## Theme ![The theme tray split between the Black and Clear surfaces, with the two-pane comparator and all eight palettes](images/theme/tray.png) Theme has two surfaces and eight palettes. Each palette owns a complete surface family (base, panels, rail and borders), its accent family, and its shadow and veil hue, so a switch changes the shell rather than only a few accent edges. Clear is the light family; Black is the near-black family. The palette ids, labels, order and defaults live once in `js/theme_boot.js`. `js/theme_state.js` owns normalization, the stored pair, the one apply path and the browser chrome color; `theme.js` owns only the dashboard tray. The public documentation site publishes `css/themes.css` and both scripts byte-for-byte from the dashboard and renders the same eight palettes and two surfaces in its own header tray. Each origin keeps its own choice under the `millivolt.theme` key, so the site and dashboard do not share browser storage. Instrument (the default), Slate, Midnight, Nord, Forest, Rose, Ember and Contrast are available in both modes. Contrast is the maximum-separation palette: black with signal yellow in Black, white with black in Clear. The theme tray stays open while choosing and shows a two-pane comparator: the committed pair on the left and the candidate on the right. Both panes use the same token-faithful miniature with a three-color series legend; the captions outside the decorative panes name the committed and candidate pairs. The preview line stays muted while it reads "Preview: same" and takes primary ink only for a real candidate. Hovering or focusing a palette or surface updates only the preview pane and captions. Clicking, Enter or Space commits immediately and persists the pair; there is no apply step. Arrow keys, Home and End move focus and the preview cursor without committing, and the focused row keeps the group's single tab stop so Tab leaves and returns to it. The current surface's Clear/Black radios sit under a visible Surface label in the caption bar. Pointer activation commits without writing the polite candidate status; only keyboard focus announces a preview. The trigger's arrow keys open the tray on the saved palette; while open they move the preview cursor. Reset restores Instrument Black; Close only dismisses the tray. On touch or coarse pointers the comparator, preview line and hover hint are hidden, leaving the current caption, the surface control, all eight palette rows, Reset and Close. A window 800px wide or narrower uses the same compact picker inside the header actions flyout even with a fine pointer, and the flyout rather than the tray or palette list is the one vertical scroller. At a viewport height of 760px or below the comparator, preview line and hint are hidden. In windows wider than 800px the tray scrolls so every action remains reachable; narrower windows keep the header flyout as the single scroller. At 360px or below the caption bar stacks into one column with a full-width surface control, and the palette hints are hidden while names and swatches remain. Reopening the tray or flyout brings the saved palette row into view instead of restoring an arbitrary previous scroll offset. The choice is stored in this browser only and is applied before the first paint, so a reload does not flash the default. Charts follow the selected palette; status colors follow the selected surface, and the browser's theme color follows the selected palette's rail. The setting is presentation only: it does not change proxy behavior, stored history or the configuration revision. ## Settings Settings uses the same schema, validation and defaults as configuration files. Search finds fields across categories. Changed values are revision-checked; every row whose field is restart-bound carries its own `restart` marker instead of only a category banner. Closed vocabularies render as selects from the schema's `choices`, and duration/byte values are parsed and checked against the schema's numeric band before Apply. A read-only config disables its field controls, not only Apply. The category rail right-aligns its row counts beside the label; on a phone it becomes one horizontally scrollable chip strip so the field pane keeps the height. Switching categories resets the field scroller, and the footer status is a live region with a distinct error tone. Structured fields replace their whole map/list value rather than recursively merging nested entries. The generated [configuration example](../proxy.example.yaml) documents every field, including categories not pictured here. The [capture gallery](gallery.md#the-patch-panel) records five categories, including the backup and restore sheet. ### Dashboard cadence Dashboard settings control log-window size and polling/aggregate refresh cadence. Live rows and the in-flight gauge remain distinct from periodic chart and explorer aggregation. ![Settings dialog showing public dashboard configuration fields](images/settings/dashboard.png) ### Queue and retry Queue and retry settings bound admission and retry behavior. Their help text distinguishes queue waiting, provider retry hints, retry budgets and capacity. Tune them for the upstream and workload; the picture uses the public example. ![Queue and retry Settings with their public values and field descriptions](images/settings/queue.png) ### Storage Storage settings control the SQLite writer and its bounded queue/batches. Metrics persistence is best-effort under sustained overload; a successful inference response does not prove its record was stored. The dashboard reports process-local storage drops. The category also carries both database deletion controls, which live here and nowhere else: Delete matching removes only the records matching its provider, model, client, status, errors, debug and age filters after a live count preview and a confirmation, and Delete everything permanently deletes all metrics history, the live ring and the entire durable database. Neither action can be undone. ![Storage Settings with bounded writer, queue, persistence and deletion controls](images/settings/storage.png) ### Provider compatibility The example enables Grok, OpenCode Zen and Cursor compatibility profiles without credentials or upstream registrations. Their headers and metadata mappings are editable operator-selected snapshots, not guaranteed current provider contracts. See [adapters](adapters.md#bundled-compatibility-profiles) before relying on them. ![Provider Settings showing the public Cursor compatibility profile](images/settings/providers.png) See [Contributing](../CONTRIBUTING.md#documentation-screenshots) before refreshing the gallery. Never publish private configuration, bodies, credentials or raw history copies. --- # Capture gallery 37 plates across 7 stations, captured 16 November 2026 UTC from 100,158 recorded requests with identifiers replaced and content removed. Every plate is one state photographed on both primary surfaces and split along the diagonal from the bottom-left corner to the top-right. The upper-left half is Instrument Black, the lower-right half is Instrument Clear. Nothing else changes between the halves: the same data, the same scroll, the same open surface. Values are a static sample from recorded history, not a benchmark. The offline indicator describes the capture session. ## The whole bench The default view at 1920 px and at phone width; every plate carries the Black and Clear surfaces across one diagonal.
The millivolt dashboard at 1920 pixels: nine global readings above a provider explorer, a traffic timeline and a request log of recorded requests.
01 · The whole bench Global totals stay fixed while the explorer scopes the timeline and log; upper-left Black, lower-right Clear.
The dashboard at phone width, with the KPI band stacked and the explorer reduced to a dimension picker.
02 · 390 pixels wide Below 1200 pixels the locked bench becomes a scrolling page and the nine-item rail becomes one picker.
## Six readings of one ruler The plotted presets, the summary tiles, and the window picker that bounds them.
The Overview preset: five tiles for volume, tokens, cost, health and speed, each with a share line and a sparkline.
03 · Summary tiles The overview preset answers what the window contains before any series is drawn.
Traffic timeline with request and error counts from recorded history.
04 · Traffic Requests and errors share one time axis; the totals strip carries the window rate.
Error timeline with affected-request bars and a separate percentage axis.
05 · Errors A 429 is a rate limit, not an error; the two series stay separate.
Token timeline with input, output, reasoning and cached usage.
06 · Tokens Cached reads are drawn beside billed input, so a high cache share is visible rather than inferred.
Speed and latency timeline with independent axes and the p95 percentile selected.
07 · Speed and latency The percentile control applies to every measured rate in the preset.
Cost timeline with spend and request-count axes.
08 · Provider-reported cost Spend is whatever the upstream reported, plotted beside the request volume that produced it.
The timeframe panel open over the timeline: a coverage rail, a zoom ladder from one minute to one year, and calendar presets.
09 · The window picker Windows wider than the stored history are disabled, so the picker cannot promise history that is not there.
## Nine ways into the traffic The dimension rail, its cards, and one scoped reading.
Provider explorer with pseudonymous groups, recorded costs and health signals.
10 · Providers The rail counts distinct values; each card shows its share and the signals that matter for that dimension.
Model explorer with pseudonymous model groups and recorded usage metrics.
11 · Models The model dimension applies the configured grouping rules before it counts.
Client explorer with pseudonymous client groups and their recorded traffic.
12 · Clients The client dimension answers which caller produced a share of the window.
Conversation explorer with main and sub counts and recorded request totals.
13 · Conversations Conversation filters select exact leaves; the rail reports main and sub conversations separately.
Tool explorer with recorded call counts and the traffic behind them.
14 · Tools The tool dimension counts calls the relay observed, not calls a client claimed.
Time explorer with recorded buckets and their request totals.
15 · Time Time is a grouping like any other, so a bucket can be filtered and compared with the rest of the window.
Status explorer with final HTTP classes and their recorded counts.
16 · Status The status dimension separates what the client received from what the upstream first returned.
Error explorer with canonical error classes and their affected request counts.
17 · Error class The error dimension groups by the proxy's canonical class, not by provider wording.
API key explorer with pseudonymous key groups and their recorded traffic.
18 · API keys Keys are hashed in the copy, so a group is stable without exposing a credential.
The explorer filtered to one provider, with the scope chip above the rail and the selected card marked.
19 · A scoped reading A card toggles an exact filter; the chip removes it, and the timeline and log follow the same scope.
## Every crossing The request log, an absorbed retry, and the drawer that explains a single crossing.
Request table with recorded statuses, token usage, latency, throughput and cost.
20 · The request log Each row is one crossing; the columns are the facts a later review needs first.
The request log with one row expanded into its absorbed retry attempts, each carrying its own status and timing.
21 · An absorbed retry A retry the client never saw is still a recorded upstream attempt.
Request drawer showing pseudonymous identity, the summary readout and recorded request parameters.
22 · The request drawer The drawer opens on the record, not on a reconstruction of it.
The same request drawer showing token and performance measurements.
23 · Tokens and performance Usage is grouped by the question it answers, so timing and tokens do not compete in one list.
The request drawer's diagnostics group, with queue waits, rate-limit handling and absorbed attempts.
24 · Diagnostics The diagnostics group is where a transparent retry stops being invisible.
## The operator plane The five policy sections and the composer behind them.
The operator console overview with the live line, section rail and the change ledger.
25 · Overview The overview is the register of what is in force, not a second settings form.
The limits section with no policy set and an empty rule register.
26 · Limits An empty policy surface states the state instead of hiding it.
The limit composer open with its scope chips, budgets, window, schedule and live preview.
27 · The limit composer A draft is local until it is committed; the composer previews the effect before a write.
The pause section before any hold exists.
28 · Pause A hold parks new work for a scope without changing a route.
The capture section with no session running.
29 · Capture A capture session retains bodies for a bounded scope and time; the panel states both.
The export section previewing the whole-history record count and the download receipt.
30 · Export The receipt counts what the chosen mode will write before the file is produced.
The system section with restart available and the four restart steps.
31 · System A restart is a state machine with named steps, not a button with a spinner.
## The patch panel Five categories of the public example configuration.
Settings showing the public dashboard configuration fields.
32 · Dashboard Every field is generated from the schema, including its help and the restart badge.
Queue and retry Settings with their public values and field descriptions.
33 · Queue and retry Retry policy, error classes and pacing sit together because they act on the same wait.
Storage Settings with the bounded writer, queue, persistence and deletion controls.
34 · Storage Deletion is scoped and previewed; the storage facts come from the running store.
Provider Settings showing the public Cursor compatibility profile.
35 · Providers Provider data is a field map in configuration, not a branch in the router.
Backup Settings with the download and restore rows and their scope choices.
36 · Backup and restore A restore is inspected before it is applied; the sheet says what will be replaced.
## One bench, two surfaces The shared theme tray on Black and on Clear.
The shared theme tray in both surfaces: two comparison panes, a surface switch and eight palettes.
37 · The theme tray Palette, surface and order come from the same registry the site publishes; the seam is the surface switch made visible.
--- # Protocol and client integration The client-facing interface accepts OpenAI Chat Completions, OpenAI Responses and Anthropic Messages; a Chat request can also target the Cursor Connect bridge. The upstream-facing protocol can differ from any of them. Per-request headers select the destination, credentials and optional native adapter. The [compatibility matrix](adapters.md#client-and-upstream-compatibility) maps the supported combinations; [cross-protocol translation](adapters.md#cross-protocol-translation) documents the canonical model and its loss policy. No endpoint/API-key registry is required. Read [Security](../SECURITY.md): routing credentials do not authorize operator access, and the whole dashboard plane is gated by `MILLIVOLT_OPERATOR_TOKEN` ([operator access](routes.md#operator-access)); only `/healthz` and `/readyz`, `/favicon.ico` (and the other origin-root brand/PWA files), the `/session` login handshake and the relay stay open. ## Endpoint behavior The dashboard, embedded assets, `/admin/*` actions, `/healthz`, `/readyz`, the `/session` login handshake, the `/mcp` MCP endpoint and registered `/metrics/*` routes belong to millivolt. Their methods and meanings are listed in [routes and debugging](routes.md#operator-and-data-routes). `GET /v1/models` and `GET /models` use the discovery path described below. `POST /v1/chat/completions`, `POST /v1/responses` and `POST /v1/messages` select the client wire protocol. Other paths reach the relay catch-all and require valid routing headers; the client chooses the upstream endpoint and model. A known client path is translated only when `X-Proxy-Format` selects a different upstream protocol. An absent `X-Proxy-Format` selects passthrough, as does an explicit format equal to the client protocol. Without a configured request rewrite, a passthrough request's body bytes are relayed unchanged; configured request overrides, a sub-conversation strip or ensured tools can still rewrite the upstream body (see [request overrides](configuration.md#request-overrides)). `X-Proxy-In-Format` overrides the protocol named by the path, so an otherwise unknown path is translated when that header names the inbound wire; without the override, paths outside the three known endpoints are never translated. A translated request addresses the outbound protocol's canonical endpoint (`/v1/chat/completions`, `/v1/responses` or `/v1/messages`) instead of forwarding the client's path, unless `X-Proxy-Path` overrides that endpoint. A same-protocol request on one of those three endpoints normalizes to the same canonical spelling, while one on any other path keeps the path the client chose. The path is matched in its escaped spelling, so an encoded spelling of a known route (for example `/v1%2Fchat%2Fcompletions` or `/v1%2Fmodels`) is an unknown path: on a same-protocol passthrough, or when no inbound override is present, it is relayed unchanged and is never normalized, translated or treated as model discovery. When `X-Proxy-In-Format` selects a wire that differs from `X-Proxy-Format`, the request is a translation like any other and addresses the outbound protocol's canonical endpoint. A base URL with a path joins by comparing the first client segment's decoded spelling with the base's last segment, so an encoded client spelling of that segment can be dropped from the joined upstream path. Forwarding a body is not the same as understanding every field in it. The normal relay does not rewrite an unfamiliar JSON shape just to observe it, but usage, first-token timing and quality classification depend on recognized formats. Native translation is explicit and narrower than arbitrary HTTP forwarding. There is no automatic provider failover or model-selection registry. ## Routing headers Only `X-Proxy-Base-URL` is required by the proxy. The upstream may require a key. Routing validation and request construction live in [request_target.go](../internal/proxy/request_target.go) and [request_upstream.go](../internal/proxy/request_upstream.go). | Request header | Meaning | | --- | --- | | `X-Proxy-Base-URL` | Absolute HTTP(S) upstream base; no userinfo, query or fragment, including a bare trailing `?` or `#` (a percent-encoded `%23` remains path data). | | `X-Proxy-Key` | Upstream key; takes precedence over `Authorization`. Without it, a Bearer credential or the supplied Authorization value is used. No credential is also allowed. | | `X-Proxy-Auth-Header` | Key's upstream header; defaults to `Authorization`. The chosen name joins the credential class for that request even when its own spelling looks harmless (`X-My-Gate`): it is redacted from retained headers and skipped by request overrides like a real credential. | | `X-Proxy-Auth-Prefix` | Prefix before the key; defaults to `Bearer `. A present empty header selects no prefix. | | `X-Proxy-Path` | HTTP relay: replace the incoming path before joining it to the base URL. Cursor uses its [native path rule](adapters.md#cursor-connect-bridge); model discovery replaces its default models endpoint. | | `X-Proxy-Query` | HTTP relay: nonempty query override; otherwise preserve the incoming query. Model discovery uses it for the upstream discovery call; Cursor ignores it. | | `X-Proxy-Headers` | HTTP-relay header overrides as a JSON object of string arrays, e.g. `{"x-example-version":["2026-01"]}`. `accept-encoding` and `range` are proxy-owned and rejected with 400. A JSON `null` member is treated as an omitted header. Values are JSON-decoded, so a raw obs-text byte (0x80..0xFF) that is not valid UTF-8 reaches the wire as U+FFFD instead of its original byte; send the UTF-8 spelling if the byte matters. Not applied to model discovery or the Cursor bridge. | | `X-Proxy-Timeout-Ms` | Positive milliseconds. HTTP relay: a fresh headers-and-body deadline per attempt, excluding queue/hold/backoff waits. Cursor applies it only to send header waits; model discovery uses its separate configured budget. | | `X-Proxy-Format` | Upstream format: `openai` (Chat Completions), `openai-responses` (Responses), `anthropic` (Messages), or `cursor` (agent.v1). Absent means passthrough; configured request rewrites can still change the body. See [supported combinations](adapters.md#client-and-upstream-compatibility). | | `X-Proxy-In-Format` | Optional client-protocol override: `openai`, `openai-chat` or `chat` (Chat Completions), `openai-responses` or `responses` (Responses), or `anthropic` (Messages). Aliases normalize to the canonical wire. Normally unnecessary because the request path selects the protocol. | | `X-Proxy-Client` | Observability/client label; one that only repeats the user agent's product name records the fuller user-agent identity instead. Otherwise inferred from client SDK/User-Agent metadata. | | `X-Proxy-Session` | Explicit conversation ID; otherwise a configured tracked body param or automatic grouping applies. | | `X-Proxy-Parent-Session` | Direct declared parent; requires an explicit own session on the same request. | | `X-Proxy-Max-Concurrency` | Positive provider-plus-key scheduler-group cap. Each admission updates the shared group's policy; later admissions, including headerless ones using config defaults, can replace it. Not the persistent operator limit policy, and not applied to model discovery, which returns before inference admission. | | `X-Proxy-Limit-Concurrency` | Sticky provider-wide in-flight cap. | | `X-Proxy-Limit-Requests` | Sticky provider-wide request rate as `COUNT/WINDOW`. | | `X-Proxy-Limit-Tokens` | Sticky provider-wide token rate as `COUNT/WINDOW`. | | `X-Proxy-Refresh-Token` | Opt-in stateless expired-token refresh for supported mechanisms. See [refresh](adapters.md#token-refresh). | Every `X-Proxy-*` routing, limit and format header is read with first-value semantics: when a request carries the header more than once, the first value is authoritative and later values are ignored. The rule is deterministic across the whole control prefix, including `X-Proxy-Query` and `X-Proxy-Headers`. ### Upstream allowlist matching `allowed_base_urls` is empty (allow any target) or a list of entry spellings; each entry admits exactly one authority plus that entry's path and its children. Both sides are compared as written, with one normalization: the scheme's default port is filled in when either side has no port. Entries are validated at load into the only spellings that can ever match, and those spellings are the contract: - the scheme must be literally lowercase `http://` or `https://`; - the host must be lowercase ASCII as written, with an optional numeric port in 1..65535 after a hostname; - no userinfo, query, fragment or trailing slash; - a port-less entry admits only the scheme's default port, never every port on that host. What the byte comparison does not do: lowercase the request's own host, strip a trailing dot (`example.com.` and `example.com` differ), normalize a leading-zero port (`:080` differs from `:80`), or apply IDNA folding (the punycode or Unicode bytes must be spelled the same on both sides). A path prefix is compared on the escaped path, so percent-encoding that decodes the same is still different bytes. The joined upstream path is not cleaned either. `.` and `..` segments from `X-Proxy-Path` or the incoming path travel verbatim, and an upstream that normalizes request paths resolves them after the proxy has already allowed the base. `X-Proxy-Path: /../../admin` under an entry for `http://host/v1` is sent as `/v1/../../admin` and reaches `/admin` at such an upstream, outside the allowlisted path prefix. The authority stays allowlisted either way; only the path prefix can be left. Client identity labels are retention-bounded before the record enters the live ring, analytics or durable store: classified client and language 256 bytes, user agent 1024 bytes, and `client_ip` 128 bytes. Longer values are cut on a UTF-8 boundary and end with `…`; normal labels are unchanged. `client_ip` records the first `X-Forwarded-For` element when that header is present and the socket peer otherwise; the value is diagnostic, never verified, and a directly reachable caller can set it. Finalized and pending record text is retention-bounded at the record boundary before it enters the live ring, pending view or durable store: - path 1024 bytes, method 32 bytes; - provider label, model, provider model, provider request id, provider server, finish reason, error type, error code, error message, request tool choice, response format, service tier and last turn role 512 bytes each; - each tool name 128 bytes with at most 64 retained in order; - debug session id 128 bytes; - conversation identifiers 512 bytes before the namespace prefix (retained identifier 514 bytes). Absorbed-attempt provider model, provider request id, provider server, error type, error code and error message use the same 512-byte bound; provider labels and attempt error messages are bounded at the same record boundary. Longer values are cut on a UTF-8 boundary and end with `…`; normal values are unchanged. These record bounds do not alter debug capture bodies, retained headers, content previews or request-parameter JSON. One caller setting `X-Proxy-Limit-*` affects every key/client of that provider. Absent headers preserve limits. A present `X-Proxy-Max-Concurrency` or `X-Proxy-Timeout-Ms` value follows Go's HTTP field parsing: leading and trailing optional whitespace is removed before the handler reads it, so a whitespace-only value arrives empty and is treated as absent, while a padded numeric value is read as its trimmed spelling. `0`, `off`, `none` or `unlimited` clear a `X-Proxy-Limit-*` policy. Windows accept Go durations from 1 second through 24 hours, plus `1d`. Anti-burst and its ramp duration are operator API policy (`/admin/throttle`, the dashboard and MCP), not header-driven; a header update changes only the three caps and preserves the stored pair. The ramp is inert unless `anti_burst` is true, the concurrency target is at least 2 and `ramp_duration` is above 0, and its duration uses that same window grammar. Malformed limit values reject the request without applying any of its limit headers. A header update edits only that provider's single-provider, unscheduled rule; operator rules scoped by model, client, key hash, conversation or schedule are untouched. The operator console shows every policy, its schedule and its source. Request rates are strict sliding windows over the configured length, so no window ever admits more than the configured count; token rates remain continuously refilled budgets. See [Pause and Limits](routes.md#pause-and-limits) for charging and window behavior. Supported proxy-control headers never forward upstream. Injected names must be valid, unique case-insensitively, and not reserved control, framing (`content-length`), hop-by-hop or Host names; values must be valid HTTP field values. Ordinary hop-by-hop and Connection-nominated headers are removed. Precedence depends on the request path: | Path | Header precedence, lowest to highest | | --- | --- | | HTTP relay, including Anthropic translation | Forwarded values → extracted authentication → configured provider headers → explicit injection → matching request overrides. Configured/injected authentication can override the extracted key. | | HTTP model discovery | Constructed authentication → configured provider headers. Explicit injection and request overrides are not applied. | | Cursor run/model discovery | Configured headers → matching request overrides (runs only) → required protocol/framing headers and extracted authentication when a key is present. Explicit injection is not applied. | The provider label is server-derived from the base host's registrable domain; IPs retain their authority/port, and nonregistrable hostnames remain hostnames. Optional provider aliases apply afterward. There is no supported `X-Proxy-Provider` override. `Host` is the one name with a different mechanism rather than a different precedence: it is accepted in a provider `headers` map and written to the request's host field, not its header map, because `net/http` sends the former and ignores the latter. A configured host, port included, is the authority the upstream sees on both paths above. The explicit `X-Proxy-Headers` map and an `X-Proxy-Auth-Header` still refuse `Host`, and no request override may set it. ## URL and auth examples For the HTTP relay, a base ending in `/v1` and request path `/v1/chat/completions` do not produce `/v1/v1/chat/completions`: the join removes the shared leading segment. A path override still joins to the selected base. Use `X-Proxy-Query` rather than embedding a query in the base URL. For a native Anthropic-compatible endpoint, use its documented base/model/key and set translation plus the appropriate upstream auth/path/version headers: ```sh curl http://127.0.0.1:8080/v1/chat/completions \ -H "X-Proxy-Base-URL: $NATIVE_BASE_URL" \ -H "X-Proxy-Key: $NATIVE_API_KEY" \ -H 'X-Proxy-Auth-Header: x-api-key' \ -H 'X-Proxy-Auth-Prefix;' \ -H 'X-Proxy-Path: /messages' \ -H 'X-Proxy-Format: anthropic' \ -H "X-Proxy-Headers: {\"anthropic-version\":[\"$NATIVE_API_VERSION\"]}" \ -H 'Content-Type: application/json' \ -d "{\"model\":\"$NATIVE_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"hello\"}]}" ``` The example assumes JSON-safe model/version values. The semicolon in `-H 'X-Proxy-Auth-Prefix;'` is important: curl sends a present empty header. `-H 'X-Proxy-Auth-Prefix:'` instead omits it, leaving the proxy's Bearer default. See the [official curl header guide](https://everything.curl.dev/http/modify/headers.html). An unauthenticated local upstream needs no dummy API key. ## Relay, retries and model discovery Ordinary requests relay the original body bytes unless a configured request override, sub-conversation strip or ensured-tools rule rewrites them; the request is buffered up to `max_request_bytes` before metadata extraction and sending. Larger requests are rejected with HTTP 413 without an upstream send. A decoded output cap above 1,000,000 tokens is rejected with HTTP 400 the same way, before any upstream send and at both trust boundaries - the original body's decode and the re-decode of a translated body - for any spelling (`max_tokens`, `max_completion_tokens` or `max_output_tokens`). A body that combines the Responses `max_output_tokens` spelling with a Chat `max_tokens` or `max_completion_tokens` spelling is rejected with HTTP 400 before an upstream send because the wire families are ambiguous. More than one top-level member that folds to the same cap name is also rejected with HTTP 400 before an upstream send, including case variants and escaped spellings: the relay sends every occurrence, so accepting duplicates would let the validated and recorded value differ from the bytes the upstream reads. A valid request is also rejected with HTTP 400 before an upstream send when it carries more than 10,000 entries in a `messages`, `tools`, `functions`, `input`, `stop`, `stop_sequences` or `modalities` array, more than 10,000 members in a `metadata` or `stream_options` object, more than `request_tool_calls_max` aggregate `tool_calls` members across every message (default 1,024), more than 10,000 distinct top-level members, or more than 10,000 distinct `logit_bias` keys merged across duplicate top-level objects. Those preflight bounds apply to native and translated requests alike. A translated request also passes through its protocol codec, which rejects with HTTP 400 more than 10,000 entries in a Responses `include` array or in a tool `allowed_callers` array, or more than 4,096 blocks in an Anthropic `system` list. A native passthrough without a configured body rewrite does not apply those codec-only collection bounds: the body is relayed unchanged, so there is no materialization to bound. A configured `request_overrides` rule, a sub-conversation strip or an ensured tool can still rewrite that body, but those rewrites are not a codec decode. Content composition has two independent limits. A content array above 4,096 parts is not rejected for that reason on the byte-transparent passthrough path: within the aggregate budget it still relays unchanged, but composition skips it, so it contributes no dashboard composition counts. On a translated request the codec applies its own limit: Chat drops the oversized content with a warning, while Responses and Anthropic reject the request with HTTP 400; `translation_loss: reject` turns any recorded loss into a 400. Composition reads the Chat `messages`/`system` and Anthropic `messages`/`system` shapes; a byte-transparent Responses body's `input`/`instructions` items contribute no composition counts (their model, stream flag and output cap are still recorded). Separately, the aggregate content-part count over every message content array, a top-level `system` array and every Responses `input` item's nested `content`/`output` array is capped at 16,384. Each Responses annotation array is capped at 4,096 members, and their aggregate inside those content parts is capped at 10,000. Anthropic citation arrays are capped at 10,000 members per array, and every citation array in one document shares a 10,000-member aggregate. Each nested content array is capped at 4,096 blocks, and the nested content blocks of one document share a separate 10,000-block aggregate; the citation and nested-block aggregates are independent, so their combined total can reach 20,000. A document block's citations configuration object is reported as a dropped field instead of being decoded into text citations. A valid request above any of these limits is rejected with HTTP 400 before an upstream send. Invalid JSON is passed through on the nontranslated path. Syntactically valid JSON whose strings carry raw invalid UTF-8 is bounded the same way: valid UTF-8 and JSON escapes decode to no more bytes than their spelling, while the standard decoder replaces each invalid byte with a three-byte U+FFFD. A body whose decoded strings would exceed its uploaded size by more than a small internal allowance is rejected with HTTP 400 before an upstream send; smaller invalid sequences remain accepted on the passthrough path, where a tracked value that fails the identity rules is still dropped rather than rejected. Anthropic translation requires the whole request body to be valid UTF-8 and rejects a raw invalid byte before decoding it. The proxy can retry transient transport failures, 429 and 5xx before returning the final response, and retryable in-band error events on an open stream before any generation content was relayed. Durable quota/billing 429s follow `quota_pause_mode` instead of the transient retry ladder: `off` (default) surfaces them immediately; `retry` parks the triggering request and every new request to the same OpenAI-wire provider behind a recovery gate that closes after `quota_pause_recovery_successes` consecutive successful probes (the default 1 closes on the first 2xx); a provider `Retry-After` on the quota response paces the next probe like any other 429; `manual` installs an indefinite provider pause that only an operator resumes. Quota re-sends never burn the transient retry budget. Provider retry hints and operator holds can substantially extend total wall time. With `suppress_client_retries` the proxy declares itself the client's sole retry authority: every error response the relay returns then carries `x-should-retry: false`, the official OpenAI SDK convention (python, node, go and ruby honor it from source), which overrides the SDKs' 408/409/429/5xx auto-retry defaults and any `Retry-After`, including a provider hint that would otherwise relay verbatim. No response exists to carry the directive on transport-level failures, clients that ignore the header (the Vercel AI SDK / opencode stack, whose retry policy is status- and body-text driven) keep their own policy, and mid-stream SSE errors never auto-retry in official SDKs and cannot carry new headers. Success responses are untouched, and the setting hot-reloads for new responses. One classifier owns every error decision: the retry ladder, the storm observer, the health counts and the client-facing class all read the same canonical class (`rate_limit`, `quota`, `unavailable`, `unreachable`, `timeout`, `content`, `client_error`, `proxy`, `canceled`, `upstream`), computed from the status, the provider's own structured type/code, and - for a transport failure with no response at all - the transport kind, which is `unreachable` or, for a timed out send, the narrower `timeout`. Transport evidence outranks the proxy-minted arm, so a failure with no provider answer is never filed as a proxy fault and never advised against a retry. Message text plays no part except for that transport timeout narrowing. A rate limit announced by a non-429 status (a gateway answering `tier_limit_exceeded` or `resource_exhausted` with a 400) is therefore retried, counted in the rate-limit series (and, because the request still failed, in the error rate), and labeled `rate_limit` instead of `client_error`. With `client_error_class: header` every error status the relay returns also carries `X-Proxy-Error-Class: ` and a class-derived `X-Should-Retry: true|false` before the status line commits (the body is never rewritten; relayed errors stay verbatim), while `suppress_client_retries` still wins with a flat `false`. The directive is the class's own retry verdict, not a re-reading of the status: | Class | `X-Should-Retry` | Why | | --- | --- | --- | | `rate_limit` | `true` | transient flow control at any status | | `unavailable` | `true` | transient capacity or server failure | | `unreachable` | `true` | transport failed with no usable response, including a fresh send whose peer closed the socket | | `timeout` | `true` | the send or read deadline fired | | `content` | `false` | a 2xx with nothing usable; retried on the quality/thinking budgets instead | | `quota` | `false` | waiting does not clear it; `quota_pause_mode` owns the reaction | | `client_error` | `false` | the caller's own fault | | `proxy` | `false` | millivolt's own failure | | `canceled` | `false` | the local client left | | `upstream` | `false` | unclassified upstream failure, final by default | Two settings extend the built-in vocabularies with gateway-specific spellings, additively: `rate_limit_error_classes` and `quota_error_classes` (plus the existing `retryable_error_classes` for in-band transient errors). Entries are matched case-insensitively against the parsed envelope's type and code; validation rejects a spelling that names a built-in of a different class, so a built-in classification can never be moved or weakened (the one exception is `retryable_error_classes` naming a built-in client-fault spelling, the documented override for a gateway that mislabels transient failures). Reload applies to new classifications, and the published predicates render per call, so they re-read stored history through the same effective vocabulary. A failed response's reason is read from the provider's own body, in two layers. The OpenAI/Anthropic envelope (`{"error":{"message","type","code"}}`, `{"type":"error","error":{…}}`, or a plain-string `"error"`) always wins and supplies type, code and message. When the body carries no such envelope, the failure path still recovers a human-readable reason from the members gateways commonly use: - a top-level `detail`, `message`, `msg`, `reason` or `error_description` string; - a FastAPI validation `detail` array (the first item's `msg`); - Cloudflare's `errors` array (`message`, with its numeric `code`). Nested lookups are depth-bounded, a wrong-typed member is ignored rather than stringified, and every reason is truncated to the same record bound as an envelope message. A body that is valid JSON but carries no extractable reason stays classified as `provider_error` with no message, and an empty body stays unclassified - only the status remains. This recovery runs only where the response is already known to have failed; the success paths keep treating those same bodies as non-errors, so a 2xx completion carrying a top-level `message` is never mistaken for a failure. An error body that cannot be read at all does not erase what the status already decided. A `429` keeps both its status and the rate-limit series, because the status alone is flow control; other statuses fail closed to a `502`, and a provider-decided type survives a later body-read or translation failure instead of being replaced by the proxy's `stream_read_error`. SSE pacing may insert comment keepalives. A known queue/hold wait may commit HTTP 200 before the final upstream outcome, so a subsequent failure must be signaled in-band. Bounded terminal/quality handling can replace a degenerate completion with an error. Relay framing follows SSE's own join rule: the data lines of one event are joined with a newline, so a multi-line (pretty-printed) payload stays one event and a synthetic blank separator is inserted only when the event's joined data is already one complete JSON document, or before the `[DONE]` sentinel. The completeness check is incremental, so one long event costs linear time in its bytes rather than quadratic in its line count; a bare scalar counts only when it is exactly `true`, `false`, `null` or one well-formed number, so a prose line that merely starts with one of those bytes (`the cat`, `3 apples`) stays open for its continuation lines. Two single-line events a provider sent without a separator therefore stay two events instead of merging into an unparseable payload. Non-streaming quality retries may make another upstream request; the adopted attempt's response is spooled under the fixed 1 MiB quality buffer before the client status line commits, so the retry is delivered as one ordinary response with no post-commit keepalive gap or partial payload. A stream truncated before any generation content was relayed is re-sent on the same committed connection, while a truncation after relayed content surfaces the in-band error for the client to retry. A trailing partial line that a rescue or the degenerate terminal boundary drops was never delivered, so it neither counts as relayed content nor as the record's answer content or first-answer time, and it does not veto the re-send; only a delivering path feeds and writes it, exactly once. Only the truncation class is decided before that feed, and only when the analyzer's non-committing tail classifier says the line carries no completion terminal marker: a final Chat `[DONE]` or non-null `finish_reason` without a trailing newline is fed rather than dropped. A fed tail is delivered only when it supplies a complete answer or tool call; an empty or reasoning-only completion resolves through the degenerate boundary, which withholds or replaces the terminal bytes. A terminal tail is classified before the pre-content rescues, so it is never re-sent. A delivered tail is written as its own complete SSE event: a pending event is closed first and a blank boundary follows, on clean EOF and on a read error. Answer content that the source places after its finish marker is therefore relayed as a later event, and a client that stops at the finish reason may ignore it. A terminator-less partial line that carries neither a completion terminal marker nor a completion is not delivered, so the stream reports truncation rather than a completion or provider error the client never saw. A standalone in-band error line is not a delivery terminal and is dropped the same way, and a `[DONE]` on a Responses feed is not its terminal marker, so it is also dropped as truncation. A translated streaming request whose 200 response body is not SSE is still relabelled `text/event-stream`: the upstream bytes are never converted into SSE frames, and the relay surfaces the non-SSE body as a truncated-stream error rather than translating it. A provider error event streamed in-band on the 200 is retried the same way while nothing content-bearing was relayed: the same availability and timeout vocabulary the status path treats as retryable (500 `server_error`, the 503 overloaded/unavailable family, the timeout spellings, the equivalent Anthropic/gateway spellings including `bad_gateway`, `gateway_timeout` and `upstream_error`, and the `engine_overloaded` capacity code) is dropped before the wire and the request re-sent, so the client never sees the failure. A specific capacity code overrides the generic `invalid_request_error` wrapper, because the code names a transient overload rather than a malformed request; authentication, permission, not-found and too-large types stay final. The `retryable_error_classes` setting extends that vocabulary with operator-supplied type or code strings for gateway-specific spellings (durable quota/billing classes are rejected at load); rate limits in-band (no `Retry-After` exists there), durable request/billing classes, unknown types, Responses-API feeds once any of their events was relayed (per-response sequence numbers), and errors after relayed contract bytes relay verbatim for the client to handle. Mid-thinking rescues (`thinking_retries`) extend the re-send to requests that die or end without an answer during the reasoning phase: a truncation or upstream read error after reasoning-only output, a cleanly finished reasoning-only completion (the withheld terminal region is replaced by the fresh attempt), and a retryable in-band error that arrived after reasoning-only output. The fresh stream appends behind the already-relayed reasoning, which is auxiliary display text; answer content, tool calls, refusals, `finish_reason: length` (the client's own token cap) and Responses-API feeds (per-response sequence numbers) are never rescued. An exhausted budget surfaces the `reasoning_only` class as a provider error (`upstream_error`) on both surfaces: in-band on the streaming connection, and as an HTTP 502 error envelope for non-streaming requests. Stream accounting has two accepted limits. A source that never reports a numeric tool-call index is counted per id-bearing fragment; a provider that repeats the same id across continuation fragments can overcount the calls, and the fallback is unchanged until such a provider shape is observed. Provider reasoning tokens are not cross-checked against output tokens: a reasoning count greater than the reported output is accepted, the derived answer count is `max(0, output - reasoning)`, and the record keeps the provider's own totals. A future observed shape that violates the relation needs a targeted validation rather than a universal rejection. Neither transparency nor exactly-once upstream execution is unconditional. Before relay or translation, the upstream request is subject to the configured `request_overrides` (see [request overrides](configuration.md#request-overrides)): matching rules set, replace or remove upstream headers as the last header stage, and their body fields set `max_tokens` / `max_completion_tokens` on Chat Completions or Anthropic client bodies before translation, whatever the target (a Responses client body is not rewritten), fail closed. A configured `sub_conversations` strip also removes tracked body fields before relay ([tracked body params](#tracked-body-params)). Only the upstream request changes; the client-visible contract is unchanged. ### Scheduling and timing The provider-plus-key group controls simultaneous upstream sends and its waiting queue. A separate limit policy can apply to the whole provider or narrow to models, clients, key hashes and conversations, with an optional recurring server-local schedule; a provider-only rule combines all keys and clients. Queue capacity and wait limits can reject work locally; the configured `queue_retry_after` duration (for example `2s`, never a bare integer) supplies the caller's retry hint. Operator-hold time is excluded from the ordinary queue-wait limit. A hold's own queue cap still applies. For the HTTP relay, `upstream_timeout` limits waiting for response headers, not the streaming body. `X-Proxy-Timeout-Ms` instead gives each upstream send a fresh deadline covering headers and body. Neither is a total end-to-end deadline across queueing, holds, retries and backoff; client cancellation is the way to stop the whole request. The request-body progress bound applies only while the body is being read. Reaching EOF or closing the body clears the connection read deadline, so a long-lived response stream is never cut by the upload guard. The [Cursor bridge](adapters.md#cursor-connect-bridge) instead applies the tighter configured/per-request timeout only until send headers arrive, without a body deadline on its parked run. Model discovery ignores the per-request timeout and uses `models_discovery_timeout` for the whole discovery operation. Buffered whole-document reads (translated non-streaming responses and discovery pages) use `whole_document_read_timeout` when the send has no deadline of its own and `whole_document_read_max` as the ceiling even against a longer `X-Proxy-Timeout-Ms`. A shorter caller deadline still wins. Configuration validation keeps `models_discovery_timeout` below the ceiling, so its configured budget is untouched. Provider Retry-After/reset hints are a floor on the wait, not a replacement for adaptive exponential backoff. A long hint is not clamped by the ordinary adaptive cap and can therefore outlast it. A short hint does not reset the doubling, so a 1s "retry shortly" cannot re-hammer a struggling upstream. When a 429 carries no header hint but its bounded error body contains an RFC3339 reset instant (for example "... resets 2026-11-03T23:11:00Z"), that instant is used as the same bounded hint: a provider-specific code such as `tier_limit_exceeded` then waits for its window instead of exhausting the retry budget in seconds and surfacing to the client. A finalized request retains its absorbed attempts and final outcome in one record. Every retried or final attempt stores the same upstream-response information: the provider request id, server, processing time, echoed model, rate-limit state and the redacted response headers. A provider-side request id therefore matches to the exact upstream attempt that produced it, including failures the client never saw. Transport failures have no HTTP response, so their attempt entries leave those fields empty. An in-band 429 is the one attempt entry that carries the in-band flag: when a keepalive already committed the client's success 200, the upstream status cannot replace it, so the record describes the committed 200 and keeps `{status_code:429,at,in_band:true}` to preserve the distinct rate-limit signal. The record likewise keeps the committed 200 for every other in-band failure, including a failed retry re-send whose wire status never reached the client. Only a 429 mints one; any other in-band provider error stamps the record's own type and status instead, and a failed body read behind the commit keeps the provider-decided type rather than replacing it. The entry carries no retry or error detail, is not counted as a retry, and is excluded from retry disclosures and error enumerations. Recovered upstream failures can count as errors even when the final status succeeds. A final, retried or in-band 429 is a separate affected-request signal. A client that disconnects before a final outcome is recorded as status 499 with `client_disconnected`; a plain 499 is a flow event, but one carrying a structured error counts as the upstream error it observed. That is the provider's in-band stream failure the client aborted around, or a truncated stream whose completion marker never arrived - an abort must be classified by its outcome, never by which relay write happened to fail. See [metric interpretation](routes.md#dashboard-data-and-request-inspection) for duration, first-token latency and throughput. ### Error storm protection Enable `storm_enabled` in Settings to add shared provider/model protection to existing admission and retry pacing. Requests wait in the proxy with the caller connected; this is not a client SDK queue. The proxy keeps the existing bounded request bytes and credentials only for the request lifetime. Queues are in memory, cannot survive a restart, and stop waiting when the caller cancels. Each exact recorded model name within its resolved provider has a rolling `storm_window`. A model qualifies when its sampled attempts meet `storm_min_samples` and its selected-failure percentage reaches `storm_error_percent`. Provider-wide protection requires **every active model** to qualify, as well as the aggregate sample/rate threshold. Only models receiving requests within that same window are active; an idle model or an unused model in a provider catalog cannot affect that decision. This scope combines credentials and clients. Dashboard model-display rewrites do not merge storm scopes. The detector samples successful upstream attempts and selected failures, counting absorbed retries separately. Incident details also count distinct logical requests with a selected failure in the window; retries move the same request's last-failure membership instead of adding another request. Unselected statuses, non-retryable HTTP account/auth rejections, recognized durable quota envelopes, caller cancellations and fired per-send deadlines are excluded - quota outcomes never become storm evidence, even while a quota gate is open. A selected 429 is a storm failure sample but remains a separate rate-limit signal in dashboard request health. Configurable response/stream failure detection observes a 2xx attempt once when its relay completes, or before an existing quality re-ask. It never adds a replay after meaningful output or native tool execution. An open scope holds new sends until its cooldown expires, then admits one shared probe. Failed probes multiply the cooldown, with configurable jitter and a cap; valid larger upstream retry hints remain effective. Recovery needs the configured number of consecutive successful probes. The existing provider-plus-key queues, operator holds and provider Limits still apply. The HTTP relay checks storm readiness before acquiring key concurrency, then claims a probe only at the actual send boundary. Readiness never reserves a probe while waiting for a key slot. Native parked runs remain resumable; fresh Cursor HTTP handshakes use the shared native send owner and can retry before a run starts. The quota pause (`quota_pause_mode: retry`) is a separate provider-wide gate in the same scheduler: the first durable quota/billing 429 from an OpenAI-wire provider opens it immediately (no sample thresholds), the triggering request absorbs the 429 and re-sends per the same recovery cooldowns, and every new request to the provider parks at admission. The gate closes after `quota_pause_recovery_successes` consecutive successful probes (the default 1 closes on the first 2xx) and the queue drains; a provider `Retry-After` on the quota 429 is a floor on the next probe, bounded by the shared hint ceiling. Parked sends obey `storm_max_queue` and `storm_max_wait`; beyond them the proxy answers HTTP 429 locally. The gate appears in the storm banner as a quota incident with an operator **Resume now** action; a manual-mode hold appears on the pause surface instead. The gate is in-memory: a restart drops it and the provider's next durable quota 429 re-arms it. Manual-mode holds persist like every operator pause. The ordinary HTTP retry budget remains `max_retries`. Selected failures during an active storm may use up to `storm_max_retries` additional attempts across the logical request, including quality re-asks. Cursor runs the same verdict: its fresh handshakes take the ordinary `max_retries` budget and, with protection armed, the additional storm budget, and a retried success leaves the group at one probe exactly like the generic ladder. Exhaustion returns the final upstream failure. Retrying inference can repeat upstream execution or billing even when no output arrived; this feature does not provide exactly-once processing. See the [HTTP retry constraints](https://www.rfc-editor.org/rfc/rfc9110.html#section-9.2.2). `storm_max_queue` bounds callers waiting at storm gates across the process; ordinary scheduler queues have their own limits. `storm_max_wait` bounds each storm wait, not total request wall time. Scope capacity and oversized scope identifiers also fail closed. Local storm admission rejection returns HTTP 429 with `queue_retry_after`; a streaming connection already committed by keepalive comments receives an in-band error instead. Successful retries remain one finalized request record with its absorbed attempts and accumulated queue wait. ### Model discovery `GET /v1/models` and `GET /models` are constructed discovery services, not ordinary inference records. The proxy normalizes recognized model lists, translates native formats, completes supported pagination, and optionally enriches missing fields. Unrecognized successful list bodies remain verbatim; primary discovery budget/errors fail explicitly, while optional enrichment can fall back to the complete base list. Discovery can apply limit headers without consuming an inference slot. Its time/byte/page/entry limits are configuration-owned. Primary and enrichment entries share one `models_discovery_max_entries` budget (default 10,000); enrichment degrades to the base list when the remaining budget cannot hold its metadata source. The rendered list is bounded by `models_discovery_max_bytes` plus a fixed per-entry normalization allowance (512 bytes for each configured entry) and is encoded without HTML escaping, so literal `<`, `>`, and `&` keep their wire size. A rendered list past that derived bound fails closed before any response bytes are written. Discovery addresses the format's own models endpoint regardless of translation: `/v1/models` on the shared base join, Anthropic's pagination query, or Cursor's native unary model service. The translator's inference path is not reused because both inference endpoints are POST-only upstream. An explicit `X-Proxy-Path` or `X-Proxy-Query` still replaces the endpoint and query, while the incoming request query is not forwarded. A primary upstream non-200 response, including a redirect, becomes a 502 discovery error. Discovery authenticates with the resolved target auth header and prefix, the same construction as a relayed request, and has no protocol-specific fallback: an Anthropic endpoint that requires `x-api-key` must be called with `X-Proxy-Auth-Header: x-api-key` and an empty `X-Proxy-Auth-Prefix`. The ordinary HTTP relay returns upstream redirects rather than following them. Forwarding also changes transport headers and may decompress an upstream gzip response; do not compare transport framing as a byte-identical contract. ## Main and sub-conversations ### Automatic grouping An explicit `X-Proxy-Session` takes precedence, then a configured tracked body param (see [tracked body params](#tracked-body-params)). Without either, the tracker partitions requests by client label and credential hash, then compares total user/assistant/tool turn counts against its open conversations. A nonexpired conversation with the greatest previous count no larger than the new count is the nearest match. A reset or idle gap can start another conversation, and the configured open-session cap evicts the oldest tracked conversation in that partition. This is a heuristic for clients that send conversation history, not a content comparison or a universal agent identity. Parallel tasks with similar turn counts can be ambiguous. Tracker state is process-local, so use explicit session IDs when predictable continuity matters and keep the client/key namespace stable. Automatic grouping never establishes a parent/child relationship. ### Tracked body params The request body is client data, and one field can double as the conversation identity. When `sub_conversations` names the request's classified client, the tracked field the client already sends (exemplar: opencode's `promptCacheKey`) supplies that request's identity in the `k:` namespace: behind an explicit `X-Proxy-Session`, ahead of automatic grouping. Params match only top-level body fields, checked in order. The first present field decides: a present value that is not a JSON string counts as absent and the next field is checked, while a present string that cannot be decoded or fails the identity rules drops the identity for the request - no later field is consulted. A top-level field that occurs more than once is ambiguous and drops the identity the same way, leaving the request on automatic grouping. The body is relayed unchanged unless the entry sets `strip`, which removes the entry's configured fields from the top level of the client request body before relay or translation. It applies to any JSON body, including a Responses client body. A request translated to the Anthropic wire gains a top-level `prompt_cache_key` carrying the tracked value. That field is a proxy tracking extension, not an Anthropic cache control, and only upstreams that tolerate unknown top-level fields accept it. An absent or dropped identity is never a client error. A tracked identity never declares a parent: `X-Proxy-Parent-Session` still requires its explicit session pair, and the session wins identity. The identity rules, name grammar, caps and the settings surface live in [configuration](configuration.md#sub-conversation-tracking). ### Declaring a parent A parent sends `X-Proxy-Session: main-task`. Its child sends its own `X-Proxy-Session: worker-1` plus `X-Proxy-Parent-Session: main-task`, using the same client label and API key. Send IDs without the stored `s:` prefix. When declaring a parent, both headers must occur once, contain nonempty valid UTF-8 without controls, differ from each other, and fit 512 bytes after trimming. Self-parent/invalid declarations return 400. The parent header is proxy-only; it neither edits the body nor instructs an upstream to create a child agent. The server resolves observed relationships within `(client, key_hash, conversation_id)`, including history outside the selected view. The explorer reports main/sub/unresolved counts and role/parent metadata. A missing parent is **not observed**, not an invented main. Conflicts, cycles or ambiguous reused IDs remain unresolved. “Main” means no parent declared in observed history, not independently verified orchestration. Parent links preserve the full representable client/key namespace. Conversation filters remain exact leaves, never descendant filters. A changed credential can change the namespace. Historical parentage cannot be inferred from absent data, message similarity, or ordinary conversation-continuity fields. --- # Upstream compatibility, adapters and token refresh The client-facing protocol is separate from the upstream protocol. Select an adapter with `X-Proxy-Format`; do not infer one from a provider label. See [protocol](protocol.md) for routing/auth headers and [operations](operations.md#known-limits) for current limitations. ## Client and upstream compatibility millivolt accepts three client wire families (Chat Completions, Responses and Messages) plus the Cursor Connect bridge, and translates between them and the supported upstream families. The client protocol is selected by the request path, or explicitly by `X-Proxy-In-Format`; the upstream protocol is selected by `X-Proxy-Format`. Translation runs only when both are known and differ. An absent `X-Proxy-Format` never enters the IR, and an explicit format equal to the client protocol also stays passthrough. Passthrough is not an unconditional byte-identity promise: configured `request_overrides`, a `sub_conversations` strip, or ensured tools can still rewrite the upstream body before it is sent (see [request overrides](configuration.md#request-overrides)). Without any of those configured, the body bytes are relayed unchanged. | Client path | Client protocol | `X-Proxy-Format` | Upstream protocol | | --- | --- | --- | --- | | `POST /v1/chat/completions` | OpenAI Chat Completions | `openai` or absent | OpenAI Chat Completions, passthrough when both are Chat | | `POST /v1/chat/completions` | OpenAI Chat Completions | `openai-responses` | OpenAI Responses | | `POST /v1/chat/completions` | OpenAI Chat Completions | `anthropic` | Anthropic Messages | | `POST /v1/chat/completions` | OpenAI Chat Completions | `cursor` | Cursor agent.v1 | | `POST /v1/responses` | OpenAI Responses | `openai` | OpenAI Chat Completions | | `POST /v1/responses` | OpenAI Responses | `openai-responses` or absent | OpenAI Responses, passthrough when both are Responses | | `POST /v1/responses` | OpenAI Responses | `anthropic` | Anthropic Messages | | `POST /v1/messages` | Anthropic Messages | `openai` | OpenAI Chat Completions | | `POST /v1/messages` | Anthropic Messages | `openai-responses` | OpenAI Responses | | `POST /v1/messages` | Anthropic Messages | `anthropic` or absent | Anthropic Messages, passthrough when both are Anthropic | xAI/Grok uses the ordinary OpenAI-compatible relay, not a separate native adapter. Other providers and local model servers can use that same path when they expose a compatible API; there is no fixed provider registry. Native translation covers the fields listed under [cross-protocol translation](#cross-protocol-translation), not every endpoint or arbitrary provider protocol. Clients must support custom routing headers or middleware that adds them. For native formats, supply the adapter selection and the upstream's required authentication, version and path headers; the [Anthropic example](protocol.md#url-and-auth-examples) and [Cursor login setup](#cursor-login) show these choices. Provider profiles enrich headers, metadata and ensured tool signatures but do not select an adapter or grant provider access. ## Cross-protocol translation Cross-protocol translation goes through one canonical request, response and stream model in [format/ir](../internal/format/ir/ir.go). Each wire family has one decoder and one encoder; the IR is the single source of truth for roles, content blocks, tool calls and results, reasoning artifacts, stop reasons and usage. Same-protocol requests never enter the IR: they relay unchanged unless a configured request override, sub-conversation strip or ensured-tools rule rewrites the upstream body first. Carried wherever the target protocol represents them: text, images (URL, base64 or file id), documents, audio where the target accepts it, system/developer/user/assistant turns, function tools, tool choice, parallel tool use, tool calls and results, structured outputs, reasoning effort and summary, stop sequences, sampling parameters, metadata, usage and cache-token accounting. A malformed Chat `stop` value is rejected before the upstream send on a translated request. Absent, null, a string and an array of strings are the accepted shapes; any other type returns 400 instead of silently dropping the stop filter. An array element that is null or not a string is also rejected: Go would otherwise decode it as an empty stop sequence. The same element rule applies to Anthropic `stop_sequences`. Native Chat passthrough is not decoded: the body is relayed unchanged, so `stop: 5` reaches the upstream and its response is returned. An empty string and an empty array still mean no stops. Protocol-only artifacts are preserved for the same family and reported when a target cannot consume them: - Anthropic thinking and redacted-thinking blocks keep their opaque signatures. A Responses reasoning item has no Anthropic signature: a translated request or non-streaming response emits its visible text with an approximation warning instead of a `thinking` block, while a translated stream opens a `thinking` block with an empty `signature`. The proxy keeps the reasoning text and does not invent a signature; resending an empty-signature block is a client choice and the target may reject it. - OpenAI Responses reasoning items keep `encrypted_content` and item ids. A cross-family encrypted reasoning artifact is never replayed into another provider's field. A non-streaming surface drops it with a warning; a stream has no warning sink and drops it silently. - A Responses reasoning item exposes two independent text channels: the summary parts and the content reasoning text. A translated stream emits both as they arrive, so text the source places in both channels appears in two target reasoning blocks. The proxy does not deduplicate across channels: a native client accumulating the source observes the same two channels, and merging them would corrupt genuinely distinct summary and content text. - Responses namespace tools are flattened, not dropped. A client may group function tools under `{"type":"namespace","name":"mcp__context7","tools":[...]}`; the target receives each member as an ordinary function whose name is `__`, and a returned call names the group again in the client's `function_call` item. The flattened spelling is truncated to the target name class and suffixed with a hash when it is longer than Chat's 64-character limit, and two members that flatten to the same string receive a numeric suffix, so every declared tool stays addressable. - Provider-executed tool definitions (web_search, file_search, mcp, image_generation, tool_search and similar) are optional to the client's remaining tools and are dropped with a `tool.` warning when the target cannot execute them; `translation_loss: reject` fails the request instead. - A freeform `custom` tool is dropped the same way. Its input is a grammar, not a JSON schema, and the client expects a `custom_tool_call` item back, so declaring it as a function would hand the model a tool whose call the client cannot dispatch. - Provider-executed blocks such as server tool use and web search results have no cross-protocol equivalent and are dropped with a warning. - A Responses `max_tool_calls` request field has no cross-protocol equivalent. A native Responses relay forwards it byte for byte, so the provider still caps its own tool calls. A translated request drops it with a `max_tool_calls` warning and `translation_loss: reject` turns that loss into a 400 before any upstream send. The translated turn therefore has no provider tool-call cap unless the client also sets one the target protocol understands. - A Responses item `phase` is a wire hint with no target equivalent. A non-streaming request or response records one `input.phase` or `output.phase` warning. A translated stream keeps one `output.phase` warning only in the decoder's diagnostic state; no client header or stored record carries it. The phase text never reaches the client wire. - A `response.output_text.delta` for a part the source never announced is dropped, but the matching done event or the terminal document recovers the full text, so only the incremental gap is lost. A Chat `[DONE]` without a finish reason maps to `response.incomplete` for Responses and to a clean `message_delta` with `end_turn` for Anthropic; both are accepted limits with no reproduced client failure. - An Anthropic target derives its terminal from content when the canonical reason maps to `end_turn`: a non-empty refusal becomes `refusal`, a tool call becomes `tool_use`, and a refusal wins when both appear. The non-streaming response reports the approximation in `X-Proxy-Warnings`; a stream has no warning channel. - A canonical output block with no Responses content part, such as a file or audio block carried from Anthropic, is dropped from the output with one `output.` warning. Text around the dropped block is preserved. - Responses provider-executed tool calls in the shared tool-item vocabulary (`custom_tool_call`, `local_shell_call`, `mcp_call`, `computer_call`, `tool_search_call` and their peers) are tool activity: they count toward a `tool_calls` finish even though no cross-protocol content block exists for them, and a translated record still counts them (retaining streamed names) when the target wire drops the item. Provider-executed generation artifacts (`program`, `program_output`, `image_generation_call`, `code_interpreter_call`) are output presence instead: a completed turn that carries only one of them is not an empty completion, while its finish stays the source's stop rather than `tool_calls`, because the item is generated source or media rather than a model tool invocation. Extending the tool vocabulary waits for a demonstrated client need. - `previous_response_id`, `item_reference`, `background`, `conversation` and prompt templates require provider-side state and are rejected when the target cannot honor them. `store: true` is honored by Chat Completions and Responses targets and rejected by Anthropic. A Chat or Anthropic request translated to a Responses target sets `store: false` when the source never chose storage, because the Responses default retains data for at least 30 days while the source protocol has no such opt-in and the proxy has no retrieval path. Explicit source values are preserved. A Responses source's implicit storage cannot be honored on Chat or Anthropic, and no response id is returned for later retrieval. - Every system or developer message before the first emitted conversation turn, together with the Responses instructions, becomes the top-level Anthropic `system` prompt. A mid-conversation system run stays in `messages` only after an emitted user turn and before an assistant turn or the end; any other position is lifted to the prompt with one approximation warning. A later developer turn has no Anthropic role and is lifted to the prompt with an approximation warning, which reports the positional change. - An Anthropic target drops leading assistant turns because the first message must use the user role, reporting one `messages.leading_assistant` warning. A tool result that answered a dropped leading call is dropped with one `messages.orphan_tool_result` warning, so that result never reaches the wire. A source-supplied result that matches no call in the source history is content, not an artifact of the drop: it passes through unchanged as a `tool_result` even though no preceding `tool_use` exists, with no warning. Pairing happens before the drop: a result that names a dropped call stays paired to it and is dropped with that call rather than retargeted to a later same-name call, because the name may answer the dropped call and retargeting could answer the wrong tool. A system or developer turn before the first emitted turn is still lifted to the top-level prompt even when a dropped assistant or an empty user turn preceded it. - Consecutive same-role turns are merged. A merged user turn keeps Anthropic's results-first rule: when the later turn would otherwise put text before a `tool_result`, the results move to the front with their relative order preserved and one `approximated:messages.tool_result` warning reports the reorder. An already-ordered or text-only merge keeps source order and warns nothing. - A non-clean non-streaming terminal can carry a missing, null, whitespace or partial tool-call argument value the source never finished: a Chat finish other than `stop`, `tool_calls` or `function_call`, or a Responses `incomplete` or `cancelled` terminal, including a `completed` response that still carries an incomplete reason. The proxy drops only the unusable call with one `tool_calls.arguments` or `function_call.arguments` warning and keeps the text, refusal and source finish. A Responses `failed` terminal is not covered by that rule: it returns the provider error envelope and discards the output, so no block or warning is produced. A clean terminal still maps an omitted, null or whitespace value to the empty object, but fails closed on a partial or otherwise invalid value. Structural failures such as a missing name or id remain fatal on every terminal, even when the arguments are also unusable. - A streamed Responses `function_call` that omits `call_id` uses its item id as the target tool id. When both identities are absent, the block starts with a stable `call_proxy_` id instead of an empty id, so Chat does not write an empty tool call and Anthropic does not reject the block. An announcement that carries both identities accepts either spelling on the terminal item. A terminal item that uses the other identity kind, an item id after a call-id announcement or the reverse, matches a unique open announced call with the same name. A terminal entry that repeats an announced call whose block already closed is skipped before open matching when its identity and name agree; an id-less entry also requires its effective argument value to agree. An id-less repeat is skipped before its array position is resolved, so a closed call announced at a foreign position cannot claim a different open call. Formatting differences and object key order do not affect that agreement, but numeric literals are compared as written, so `1` and `1.0` do not agree. A value above the canonicalization size, nesting or member-separator bounds keeps the byte-exact rule, as does a partial or otherwise invalid value, so formatting and key-order differences in such a value do not agree. A named terminal item matches only an open announced call with the same name; several same-name candidates are disambiguated only by the announcement at the terminal position, and no same-name candidate starts a new call instead of taking a differently named open call. An id-less terminal item then matches a unique open announced call with the same name, then the announcement at its own position among same-name candidates, then a lone or first still-open id-less announced call in output order, so a reordered terminal array completes the announced block rather than synthesizing a second call. If no open call matches, an id-less entry whose name and effective argument value agree with a closed id-less call is skipped as that call's repeat. An unannounced id-less call at a colliding announced position cannot be distinguished from a repeat and is dropped rather than assigned to a different call. An identified terminal call also matches only a same-name candidate whose identity kind is compatible: a candidate keyed by the same identity kind must carry that identity, while a cross-kind candidate stays eligible under the mixed-spelling rule. Position alone never moves arguments to a conflicting identity. The non-streaming decoder still rejects a call with both identities absent. - A legacy Chat `message.function_call` is treated as answer-shaped for retry safety because the deprecated shape can represent either a tool invocation or a final answer, and it is not counted as a tool call. A Chat source that uses only that shape therefore records no tool-call count even though the call translates. - Anthropic `metadata.user_id` is limited to 512 characters. A longer Chat or Responses `user` value is rejected before any upstream send rather than truncated, because truncation can collide two distinct identifiers. Chat/Responses `service_tier: priority` maps to Anthropic `speed: fast` with an approximation warning; `fast` maps without one. A translated stream takes its tier and metadata from the source start event. A value that appears only in the terminal document is not applied retroactively, because the target `message_start` and earlier chunks are already on the wire. - An Anthropic document `title` is limited to 500 characters. A longer source filename is dropped with a `document.title` warning while the document bytes still translate; strict mode rejects the loss before any upstream send. - A response metadata map has no field on the Anthropic Message object, so a non-streaming translated response drops it with a warning. A translated Anthropic stream cannot report response-side warnings after its status line is committed. - An image or document inside an Anthropic assistant request turn is schema-valid because assistant request content shares the message content type. The request encoder keeps it and reports an `assistant.image` or `assistant.document` approximation warning; runtime acceptance of assistant media is not verified. Anthropic's response `Message.content` union has no image or document block, so the response encoder drops it with a `response.assistant.image` or `response.assistant.document` dropped warning. User media is carried without that warning. - A Responses `input_file` with inline `file_data` but no media-type declaration has one inferred from a data-URL prefix, a `.pdf` or common plain-text filename extension, or a base64 `%PDF-` signature. Unrecognized bytes are dropped with one warning while a file id or URL on the same part survives; a part with no remaining payload is dropped. Chat file parts keep their existing PDF-only default. - A Chat response carries one refusal field, so multiple IR refusal parts are concatenated in block order. An empty part contributes nothing and never clears earlier text. - A translated response has no audio representation: the IR has no audio output block, so a Chat `message.audio` or streamed `delta.audio` and a Responses `output_audio` content part never reach a target wire. The non-streaming Responses decoder reports one `content.output_audio` warning; the Chat decoder carries no audio block and reports no warning. Native same-protocol relay keeps the audio unchanged. Mapping a transcript to assistant text would be a fidelity feature, not a lossless translation. - A translated non-streaming response assumes a schema-valid source identity. A source that omits `id` or `model` yields an empty string on the target rather than an invented identity. The Responses envelope carries protocol defaults for request-echo fields (`parallel_tool_calls: true`, `tools: []`, `tool_choice: "auto"`, null `instructions`/`temperature`/`top_p`, `store: false`); a translated response cannot echo the source request's parameters without threading the request IR into the encoder. - A Responses output tool-call `call_id` has no schema length bound, so a provider id longer than 64 characters is relayed unchanged. If the client replays that call as request history, the Responses request boundary enforces its 1..64 character reference rule and rejects the request with 400. The proxy never shortens or drops the output id, because that would break the correlation the client already received. - Responses `include` options other than `message.output_text.logprobs` and a non-string Chat metadata value have no target equivalent and are dropped with a warning. The logprobs include maps to the target's request field, but translated responses do not carry provider logprob details. - Deprecated Chat `functions` and `function_call` are converted to the canonical tool list and tool choice; a legacy per-assistant `function_call` pairs with its `role: function` reply by function name. Chat `prediction`, audio output, non-text `modalities`, and `web_search_options` have no target equivalent and are dropped with a warning. - A request-history tool call without an id gets a generated id and reports the recorded approximation. Tool ids are pre-reserved once per request across every message and item before any message is decoded: a generated id can never collide with an explicit id anywhere in the request, including one that appears later, and a repeated explicit id fails closed with HTTP 400. The deprecated per-assistant Chat `function_call` keeps its function name as the id because that shape carries no id; a repeated same-name legacy call is inherently ambiguous and remains the documented exception. Pairing keeps the kind of reference. A non-empty `tool_call_id` answers the unanswered call with that exact id, so an explicit id is never shadowed by another call's function name; when no id matches, a generated call may still be addressed by the function name its client supplied in place of the missing id. A message `name` (with no `tool_call_id`) addresses a unique generated call. A reply with neither resolves only while exactly one generated call is unresolved and no other call is unresolved. Any reference matching more than one unresolved call fails closed, and an explicit call is never answered by its name alone. - A `json_schema` response format must carry a valid JSON object `schema` to translate; a schema-less, null, non-object or invalid `json_schema` schema is rejected with 400 because a cross-protocol target has no meaningful mapping for one. A native same-protocol Chat request is relayed unchanged, so the passthrough path still forwards it. - A non-function tool type and malformed tool arguments are rejected before any upstream send. `translation_loss` selects the policy. `warn` (default) drops the optional fields this proxy models but the target cannot represent, and reports each one in `X-Proxy-Warnings`. Provider-specific extension fields outside that modeled vocabulary are not inspected or reported; the passthrough path relays them unchanged. `reject` fails the request with 400 naming the fields before any upstream send, and it refuses every recorded loss, including a documented approximation such as a clamped temperature, a moved developer message, a generated tool-call id or a `json_object` response format, not only a field the target cannot represent at all. The header is proxy-owned: a provider response header of the same name is never forwarded. Its whole value is bounded at 1024 bytes; an over-cap list is cut at the byte budget, possibly inside the final field, and ends with `; truncated`. Response-side warnings are never fatal; a non-streaming response reports them in `X-Proxy-Warnings`, while a streaming response cannot add a header after the status line is committed and drops them. The field is described under [settings](configuration.md#finding-a-setting). A provider error envelope that arrives inside HTTP 200 is translated too, but the client status becomes 502: a translated client would read 200 as a successful completion. A non-200 translated error body is buffered before the client sees it, at most 64 KiB, under `whole_document_read_timeout` (default 2 minutes), so a provider that holds an error stream open can delay the client until that bound expires. A non-200 status that forbids a response body (1xx, 204 or 304) cannot carry the translated envelope, so it fails closed as 502 `upstream_error` naming the upstream status instead of committing a status whose error body the client cannot receive. Pre-commit failures (routing, request validation, translation, or an oversized translated body) answer in the client protocol's own error envelope. Non-streaming translation reads the complete upstream 200 body and the translated document under `translated_response_max_bytes` each (default 32 MiB); exceeding either fails closed with 502 `response_too_large` before any client bytes. Translation buffers the whole document, so the setting also bounds one in-flight translation's memory. Streaming translation is event-driven and bounded. Each target encoder owns its protocol's ordering invariants: Anthropic opens a block before any delta and closes it before `message_delta`; Responses emits `response.created`, per-item lifecycle events and one terminal event; Chat emits the role chunk, deltas, one `finish_reason` chunk and one `[DONE]`. The official Anthropic event flow closes every content block before `message_delta`. A nonconforming source that sends a content delta after `message_delta` is not given a runtime special case, because the terminal contract and the target encoder ordering still own the turn. A truncated upstream is closed with the client protocol's own error event, never a fabricated clean finish. The Responses encoder can emit `response.cancelled` for a canonical cancelled finish. The current `ResponseStreamEvent` union lists completed, failed and incomplete terminals, while the `Response` status enum includes cancelled; the event is kept as a forward-compatible extension and a strict client may treat it as unknown. Synthetic Responses error and terminal frames emitted on a committed socket after an accepted non-completion status restart `sequence_number` at 0 and 1; relayed events before that keep their original sequence. The degenerate terminal-region substitution emits the same error plus `response.failed` pair at 0 and 1 even when earlier streamed events carried higher numbers, because the relay does not retain the provider's counter. Both reset cases are documented contract gaps, not reproduced client failures; the terminal event is final for the turn. - A Chat stream chunk is decoded from its first choice only. A translated request never emits `n` above 1 because only one choice can be translated, so the limit is unreachable for compliant traffic; a noncompliant upstream that interleaves several choices in one stream would mix them in the translated stream. - A Chat stream must carry `chat.completion.chunk` documents. A full completion document, or a choice carrying `message` instead of `delta`, is rejected instead of becoming a successful empty answer; a chunk without `object` stays accepted when it carries a delta. - A source in-band error followed by the protocol terminal is deliberate: Chat and Responses clients receive the provider error and then `[DONE]` or the terminal event, matching the source order. The relay does not suppress the terminal after an error. - A streamed tool `arguments` value that is exactly `null` is emitted literally and fails the block-close validation, while the non-streaming decoder normalizes it to `{}`. The stream cannot rewrite bytes it already delivered, and no official provider shape emits this value, so the inconsistency is an accepted limit rather than a normalization path. - Anthropic `content_block_start` text is not counted as answer content, first-answer time or a preview. The Anthropic wire is exempt from the OpenAI degenerate substitution, so a completed start-text-only stream stays healthy and records no answer content. Official start text is normally empty; this is an accepted accounting limit. Streaming citations are not translated. A Responses `response.output_text.annotation.added` event is not decoded, and a streamed target does not emit citations or annotations; non-streaming responses do carry url citations. A citation stream event is a separate feature, not a silent claim of streaming parity. A non-streaming translated Responses message item carries `status: "incomplete"` when the response-level status is not `completed`, and `status: "completed"` otherwise. Function-call items always carry `status: "completed"` because every emitted call has validated arguments. The response-level `status` still owns the turn lifecycle. In a translated stream, `response.output_item.added` carries `status: "in_progress"` and each `response.output_item.done` carries the status at which that item closed: `"completed"` for a normal close and `"incomplete"` for an aborted item. The terminal response output preserves those per-item statuses; only an item still open at an incomplete or failed terminal is rendered `incomplete`, and a failed terminal omits an open call whose arguments were never validated. Parallel tool calls translated to Anthropic open their indexed content blocks concurrently (for example start 0, start 1, delta 0, delta 1, stop 0, stop 1). The blocks carry the index the streaming contract requires, and SDKs accumulate by index; a strict assumption of sequential block lifecycles would need a separate encoder design, not a per-request special case. A streamed Responses tool call is validated by its terminal class. A completed terminal fails closed on a partial function-call value. Incomplete and cancelled terminals do the same, because both are normal turn boundaries and an executable empty object must never close a call the model did not finish. On those two terminals a call with missing, null or whitespace arguments is also rejected unless the source explicitly sent `{}`. When validation fails, the target abort closes non-tool blocks but leaves an open tool block unclosed: Anthropic emits no `content_block_stop` over empty input, and Responses emits no done function-call item with empty arguments. The proxy error frame terminates the stream, so a client that keeps reading cannot execute an empty call. A failed, errored or truncated terminal instead drops only the unparseable call and keeps the provider error or truncation. On Anthropic and Responses, an already-announced open call is left unclosed on those three terminals: Anthropic emits no `content_block_stop`, Responses emits no done function-call item, and the proxy error frame ends the stream. The Chat target already streamed the announcement with empty arguments, and Chat reports a cancelled finish as `stop`, so that accepted limit is not distinguishable there. A Chat `[DONE]` without a preceding non-null finish reason is a clean transport terminator but an empty finish: a Responses target receives `response.incomplete` with `incomplete_details: null`, and the record reports `finish=other`. That degenerate classification is deliberate rather than a fabricated completion. ## Bundled compatibility profiles The generated [configuration example](../proxy.example.yaml) enables the Grok, OpenCode Zen and Cursor profiles and the opencode sub-conversation tracking exemplar (the `promptCacheKey` tracked param, strip on). `config.Example()` in [internal/config](../internal/config) owns the exact profile mappings, header values and that tracking entry; built-in `config.Default()` remains provider-neutral. The image installs the example into fresh config volumes. Existing saved configurations are not silently replaced. These are operator-selected compatibility snapshots, not verified current public API fingerprints or permission to use an account service. They contain no keys and do not choose the upstream URL or protocol adapter for a request. Review them for your integration and remove or adjust any unwanted profile in Settings. The Grok profile supplies client-identity/request-ID headers and model-modality enrichment through the model discovery endpoint. Model `version` is not context length, and the profile does not invent that mapping. Standard usage detection and USD-tick scaling already handle recognized cost fields; custom `cost_keys` must name USD values rather than bypassing that scaling. Streaming cost requires the upstream to report usage; missing cost is not evidence of a free request. See xAI's [model reference](https://docs.x.ai/developers/rest-api-reference/inference/models) and [cost tracking](https://docs.x.ai/developers/cost-tracking). The Cursor profile supplies CLI compatibility, allowed-tool and request-ID headers. Its ghost-mode field is a supplied wire preference, not a privacy or retention guarantee. Cursor's [public authentication documentation](https://cursor.com/docs/cli/reference/authentication) does not establish a stable public contract for this native bridge or its compatibility headers. Keep authorization and current service support separate from a successful local fixture test. The OpenCode Zen profile supplies the opencode CLI's client identity on the ordinary OpenAI-compatible relay: the CLI's composite `User-Agent`, `x-opencode-client` and `x-opencode-project`, plus the CLI's per-conversation `x-opencode-request` and `x-opencode-session` headers, which the `{{opencode-msg-id}}` and `{{opencode-ses-id}}` templates mint fresh per request in the CLI's 26-character identifier format (provider headers override client-forwarded values; the ids carry no session state, so replacing a genuine client's own ids is functionally neutral). The profile also sets `ensure_tools: [bash, read]`. A check on 2026-09-24 found zen's free tier gated on that wire signature rather than on TLS, header order or credentials: with the id headers in shape and a chat `tools` array containing the `bash` and `read` tool names, the gated free models served even under the anonymous `public` bearer. `ensure_tools` injects missing names as inert stubs. When the client sent no tools of its own, an absent, null, or `auto` `tool_choice` is forced to `none`, while an explicit `none` stays; a `required`, named, or malformed choice skips injection entirely, because forcing `none` would contradict the client and injecting would let the model call an undeclared stub. When the client already declares at least one tool, the configured names are added to that list and an explicit `required` or named `tool_choice` stays in force. A body that already satisfies the signature is relayed unchanged, apart from any configured request override or sub-conversation strip that runs before it. The stubs are Chat-shaped, so they are injected only into an OpenAI Chat client body on a Chat outbound wire; an Anthropic, Responses, Cursor, or arbitrary body is never rewritten. The same check found the signature also requires a streaming request (`"stream": true`): non-streaming requests to the gated free models get the provider's `FreeTierError` 403, so clients of those models must stream (the CLI always does). That observation is about that day, not an entitlement: the profile is not an account, key or permission, and there is no zen login helper. Zen's models are listed in [OpenCode's Zen documentation](https://opencode.ai/docs/zen/). Every profile is editable operator configuration, in the private config file or in Settings under Providers (each card edits the field maps, ensured tool names and upstream headers, with hot reload): changing a signature tool set or a header value is an edit, never a code change. To reproduce another first-party client's signature for a different provider, work from evidence top down: capture the client's own requests against a local logging upstream, strip one signal at a time against the real endpoint (TLS ClientHello, header order and values, body fields, credentials) until only the gate remains, then express the surviving identity as a provider profile - static headers directly, per-request dynamic values as `{{uuid4}}`, `{{platform}}` or a new mint placeholder in the single template expander, and required tool names as `ensure_tools`. The zen profile above is one worked example of that recipe, not a hardcoded mechanism. ## Anthropic-compatible Messages `anthropic` works in both directions: as an upstream selector for a Chat, Responses or Anthropic client, and as the inbound protocol for `POST /v1/messages`. Translation covers message history, system text, images, tools and results, thinking blocks, cache control, selected parameters, usage and streaming events. Fields with no target representation follow the [translation loss policy](#cross-protocol-translation). A configured [sub-conversation tracking](configuration.md#sub-conversation-tracking) value is injected as a top-level `prompt_cache_key`. That field is a proxy tracking extension, not an Anthropic cache control: Anthropic caching uses `cache_control`, and the injection is only safe for upstreams that tolerate unknown top-level fields. The client's own spelling of the tracked field is otherwise dropped by the allowlist. The caller must supply the native endpoint's correct auth/version/path headers; the [protocol example](protocol.md#url-and-auth-examples) demonstrates the mechanism. Stored parameter and conversation-composition metadata describe the translated upstream body, not an exact copy of the original request. For a native or translated Anthropic body, the recorded stop count, tool choice, parallel-tool flag, reasoning effort, response format and service tier come from the Anthropic spellings (`stop_sequences`, the `tool_choice` type/name and `disable_parallel_tool_use`, `output_config`, and `speed`) when the Chat spelling is absent. A body that combines the Responses `max_output_tokens` spelling with a Chat `max_tokens` or `max_completion_tokens` spelling is rejected with HTTP 400 before translation or any upstream send. If the caller omits all three output-limit spellings (`max_tokens`, `max_completion_tokens` and `max_output_tokens`), translation supplies `anthropic_default_max_tokens`, which can therefore appear in request details as a captured value. When a request supplies both Chat spellings, translation prefers `max_completion_tokens`; the proxy's request decode applies the same precedence when capturing the cap, so the captured value always agrees with the one the translated body carries upstream. A translated Chat body always spells its output cap `max_tokens`: it has the widest OpenAI-compatible support. OpenAI documents `max_tokens` as deprecated and its o-series reasoning models require `max_completion_tokens` instead. A native Chat client may send either spelling, and the decoder prefers `max_completion_tokens`, but a translated Chat request still carries `max_tokens` and can therefore be refused by a target that accepts only the newer spelling. An Anthropic target request forwards `temperature` (clamped to 0..1), `top_p` and `top_k` for older and compatible upstreams. The current Anthropic `MessageCreateParamsBase` request type no longer lists those sampling fields, so a strict current endpoint may reject them. The forwarding is kept as a compatibility choice rather than silently dropping client values; revisit it if a live current endpoint is verified to reject them. Non-streaming translation buffers the full response before committing status; a translation failure can therefore return a real error status. A translated stream is emitted in the client's protocol, not as OpenAI-shaped events, and it observes original-source custom cost privately before final accounting. Cache-inclusive native usage conversion has one owner; never add reasoning twice when it is already part of output. A valid native stop marker closes the turn. Missing completion/error state must not be converted into a fabricated clean finish. Legitimate empty/refusal/native tool turns are not permission to execute upstream tools again. General native multi-line event assembly still has limits; fixture coverage is not a blanket protocol-conformance claim. Native model discovery translates and completes supported paginated lists under the shared discovery time/byte/page/entry budget. Optional enrichment fills missing fields without replacing upstream-owned values; it draws from the same discovery entry budget and degrades to the base list when the remainder cannot hold its metadata source. Owners: [format/anthropic.go](../internal/format/anthropic.go), [format/models.go](../internal/format/models.go), [proxy/relay_transform.go](../internal/proxy/relay_transform.go), and [proxy/models.go](../internal/proxy/models.go). ## Cursor Connect bridge `cursor` is a text/tool bridge from OpenAI chat to the native agent.v1 bidirectional Connect/protobuf protocol. It currently requires an OpenAI Chat client: a Responses or Anthropic client targeting `cursor` is rejected with a clear 400 instead of receiving the wrong response shape. It is not a general multimodal bridge or a promise of a stable public provider API. Check the provider's current access terms and supported integration options before using account credentials; the project does not establish permission on your behalf. The run owns a bidirectional HTTP/2 transport, framed messages, heartbeat, context/KV replies and one ordered event queue. Malformed known nested messages, invalid usage or error terminal frames fail explicitly. A validated turn-ended marker plus EOF remains a supported completion convention; bare EOF without completion is not a clean success. The event queue carries an internal retained byte budget and the pending tool-call map internal count and payload byte caps; the KV blob store carries per-run byte and entry caps. The parked-run store enforces aggregate budgets across all parked runs: retained pending plus queued event bytes, retained blob bytes, and blob entries. It holds one per-run pending-plus-queued allowance free for the run being actively driven, and it rejects admission or growth that would cross an aggregate bound. The entry caps bound unique zero-length blobs that the byte cap cannot charge. Exceeding a run or aggregate parked bound fails closed with a stable descriptive error instead of retaining unbounded memory; an over-cap history fails the request and a refused park cold-starts the next turn. None of these bounds is operator-tunable. When the upstream requests a client tool, the proxy returns OpenAI tool calls and parks the native stream. A later matching tool-result tail can resume that same stream. Full route/auth/model/client scope prevents cross-identity reuse. Registration, expiration and close share the run-store owner; client cancellation ends active work, while a legitimate parked run lives until its configured TTL, upstream close, capacity eviction or process restart. For each new send, the tighter of `upstream_timeout` and `X-Proxy-Timeout-Ms` bounds waiting for response headers. That wait deadline is removed after headers; it is not an overall body deadline. Resuming an existing parked stream does not open another send. The default path is the native Run RPC; an explicit `X-Proxy-Path` is appended to the base with a leading slash, without the HTTP relay's shared-segment deduplication. Query overrides and the incoming query are not used. A lost park cold-starts from provided history. The quality-retry path may re-ask an empty continuation; a fresh Run is accepted only on HTTP 200, so any other response to the re-ask is a failed re-ask and the void surfaces instead of an acknowledgement being driven as a stream. Exhausted/invalid continuations surface errors rather than silently completing. Cursor's wire reports one combined output total with no answer/reasoning split, so the generic mid-thinking rescue (`thinking_retries`) does not apply here: a Cursor turn that produced nothing is the existing empty-turn re-ask's business. This is not exactly-once execution, checkpoint persistence, or a guarantee that an upstream keeps a parked stream alive. Output usage comes from validated native counters. Input may use checkpoint context occupancy or a documented estimate when unavailable; unreported cache and reasoning splits are not invented. Fused model spellings are decomposed by the existing protocol owner, not copied as raw native parameters. Stored model metadata uses the canonical base ID, not the caller's fused spelling. Model discovery uses the native unary usable-models service and returns the constructed OpenAI list. Context metadata is not guessed from a static price or capability table. Discovery budgets cover this path as well as optional enrichment. Owners: [proxy/cursor_bidi.go](../internal/proxy/cursor_bidi.go), [proxy/cursor_store.go](../internal/proxy/cursor_store.go), [format/cursor_run.go](../internal/format/cursor_run.go), and neighboring format/protocol tests. Keep protocol constants distinct from the configurable [compatibility profile](#bundled-compatibility-profiles); built-in defaults remain neutral and the example does not establish provider support. ### Cursor service and protobuf versions The service name, transport version and codec are different layers, not two protobuf implementations running together: | Layer | What millivolt uses | | --- | --- | | Cursor service | `agent.v1.AgentService`: bidirectional `Run` and unary `GetUsableModels`. The `v1` is part of the service namespace. | | Connect transport | `Connect-Protocol-Version: 1`. Run uses framed `application/connect+proto`; model discovery uses unframed `application/proto`. | | Protobuf codec | The limited handwritten encoder/decoder in [proto.go](../internal/format/proto.go), using the adapter's mapped fields. There is no Go protobuf APIv1 or APIv2 dependency. | The [Connect specification](https://connectrpc.com/docs/protocol/) defines its version header separately from protobuf. Likewise, Go's [APIv1/APIv2 distinction](https://go.dev/blog/protobuf-apiv2) and Buf's [JavaScript runtime v2](https://buf.build/blog/protobuf-es-v2) are library versions, not instructions to change `agent.v1` to `agent.v2` or protobuf syntax to proto2. The small codec avoids a generated-runtime dependency, but requires maintaining explicit field mappings and does not promise general protobuf conformance. A read-only check on 2026-09-06 of the installed Cursor CLI bundle `2026.08.11-e8db854` found `agent.v1` descriptors with proto3 semantics and `@bufbuild/protobuf` **1.10.1**, not v2. This is evidence about that inspected client artifact, not a claim about every client release or the current server. No live login or inference was used for this check. ## First-login helpers [grok-login.sh](../scripts/grok-login.sh) and [cursor-login.sh](../scripts/cursor-login.sh) obtain initial account credentials through interactive browser approval. Run them on a trusted Linux host or workstation, not inside the minimal proxy image. Neither a running millivolt instance nor the provider's CLI/desktop application is required by these scripts. Account eligibility, permitted integrations and model access remain the provider's decision; successful authentication does not establish subscription entitlement or a free inference allowance. The scripts do not write files or update client configuration. They print secret credential headers to standard output, and approval links/progress to standard error. Keep the terminal private: do not use shell tracing (`bash -x`), CI logs, screenshots or shared output redirection. Review a downloaded script before running it. These helpers have no command-line option parser; configure them with the environment variables below, not `--help` or invented flags. ### Prerequisites and obtaining the scripts Both need Bash, `curl`, `jq` and standard Linux command-line utilities. Cursor also needs `openssl` and a curl build with HTTP/2 support; check that `curl --version` lists `HTTP2` in Features, as described in the [curl HTTP/2 guide](https://everything.curl.dev/http/versions/http2.html). `xdg-open` is optional: without it, open the printed approval URL yourself, including from another browser-capable device. Source checkouts already contain both helpers under `scripts/`. Image-only deployments can download just the desired helper on the host, without cloning or rebuilding millivolt: ```sh curl -fsSL https://raw.githubusercontent.com/LLM-4-People/millivolt/main/scripts/grok-login.sh -o grok-login.sh curl -fsSL https://raw.githubusercontent.com/LLM-4-People/millivolt/main/scripts/cursor-login.sh -o cursor-login.sh ``` Use a new directory so downloads do not overwrite existing files. For a pinned installation, replace `main` in those URLs with the reviewed commit or release tag. Standalone downloads use `bash ./grok-login.sh` or `bash ./cursor-login.sh` instead of the checkout paths below. Do not pipe a download directly into Bash. ### Grok login From a source checkout: ```sh bash scripts/grok-login.sh ``` Open the printed verification URL, check the account and approve the displayed device code. The helper polls for approval and prints `Authorization: Bearer` with the access token, `X-Proxy-Refresh-Token` when supplied by the provider, and `X-Proxy-Base-URL: https://api.x.ai/v1`. This is an account-login token flow, not creation of a pay-as-you-go `xai-` API key. It does not select a model or add a native-format header; the ordinary OpenAI-compatible relay is used. For SSH or a headless terminal, suppress automatic browser opening: ```sh NO_OPEN=1 bash scripts/grok-login.sh ``` ### Cursor login From a source checkout: ```sh bash scripts/cursor-login.sh ``` Open the printed login link and approve the account sign-in. The helper creates a PKCE challenge, requests HTTP/2 when polling the account endpoint, and prints the access token, optional refresh token, upstream base URL and `X-Proxy-Format: cursor`. Use the printed format header to select the [Cursor bridge](#cursor-connect-bridge); a provider profile by itself does not enable translation. The same headless switch is supported: ```sh NO_OPEN=1 bash scripts/cursor-login.sh ``` ### Configure the client after approval Keep the client's API base pointed at **millivolt**, such as `http://127.0.0.1:8080/v1`, using your actual host/port or protected ingress URL. The helper's `X-Proxy-Base-URL` instead identifies the upstream; do not replace the client's millivolt URL with it. | Printed value | Client configuration | | --- | --- | | Access token after `Authorization: Bearer ` | Put only the token in an SDK's API-key field, or use the complete printed Authorization header if configuring headers directly. Do not add `Bearer` twice. | | `X-Proxy-Base-URL` | Add as a custom request header. | | `X-Proxy-Refresh-Token`, when present | Add as a custom request header if using [automatic refresh](#token-refresh); the client must retain rotated credentials returned by the proxy. | | `X-Proxy-Format: cursor` | Add for the Cursor helper only. | Select a model your account is authorized to use. Keep tokens in private client configuration or a secret store, not `proxy.example.yaml` or a Git-tracked file. Millivolt does not register these credentials globally. If your ingress consumes Authorization, follow the [separate upstream-key setup](reverse-proxy.md#credentials-and-access-control). ### Helper environment overrides These affect the login process only, not server Settings or the saved client. Use only origins and OAuth registrations you trust: they receive login material. Defaults remain owned by each script. | Variable | Helper | Meaning and default | | --- | --- | --- | | `NO_OPEN` | Both | Any nonempty value suppresses `xdg-open`; unset or empty permits it. The URL is still printed. | | `GROK_LOGIN_TIMEOUT` | Grok | Approval-poll budget in seconds; `600`. Use a positive integer. | | `XAI_AUTH_BASE` | Grok | Login origin; `https://auth.x.ai`. Does not change the printed inference base. | | `XAI_OAUTH_CLIENT_ID` | Grok | OAuth registration; defaults to the client ID shared with the proxy's xAI refresh implementation. | | `CURSOR_LOGIN_TIMEOUT` | Cursor | Approval-poll budget in seconds; `300`. Use a positive integer. | | `CURSOR_API_BASE` | Cursor | Poll origin and printed inference base; `https://api2.cursor.sh`. | | `CURSOR_LOGIN_BASE` | Cursor | Browser login origin; `https://www.cursor.com`. | Timeouts are polling budgets, not strict end-to-end deadlines: a current HTTP operation or sleep can extend elapsed time. Changing Grok's auth origin/client ID does not change millivolt's xAI refresh endpoint/registration, so a custom login can be incompatible with renewal. Cursor refresh targets the request's base URL and still requires a recognized refresh mechanism. These overrides do not establish support for arbitrary identity providers or compatible service clones. ### Login troubleshooting | Symptom | What to check | | --- | --- | | Required tool not found | Install the named dependency on the host running the helper, not in the proxy image. | | No browser opens | Open the printed URL manually. `NO_OPEN=1` deliberately disables automatic opening. | | Cursor reports unsupported HTTP/2 or keeps waiting | Check `curl --version` for HTTP2 support and inspect its diagnostics. The helper keeps polling non-200 responses until its budget expires. | | Approval times out, expires or is denied | Verify the account/browser approval, then rerun for a fresh login challenge. Never reuse an expired code. | | No refresh token is printed | The helper warns and omits that header. Millivolt cannot renew without it; obtain another login when the access token expires. | | Login succeeds but inference returns 401/403 | Check client headers, upstream base, account permissions and model availability. Login alone does not prove inference access. | | Grok or Cursor repeatedly returns `token_expired` | The client must adopt the returned access token and any rotated refresh token, then retry. Repeating the old credentials does not complete handback. | | Refresh is rejected | Recheck provider access and perform a new interactive login. Do not publish token responses or credential headers for troubleshooting. | ## Token refresh With `auto_token_refresh` enabled, a request may supply its own `X-Proxy-Refresh-Token`. The proxy can exchange an expiring JWT for supported mechanisms; it stores neither a credential database nor reusable refresh state. The JWT expiry check is not signature authentication. **Grok and Cursor use the same inference handback behavior.** A successful exchange returns HTTP 401 with error type `authentication_error` and code `token_expired`. The error object includes `access_token` and, when the exchange returned a different refresh token, `refresh_token`. The same values are in the error message. No inference is sent or recorded for that handback. The client stores the returned access token, adopts any returned refresh token, and retries the original request. Starting with **0.2.0**, Cursor no longer refreshes in place on inference requests. Clients that previously relied on that behavior must now handle the same 401 handback as Grok. Starting with **0.3.1**, credentials are in that error body, not `X-Proxy-Access-Token` / `X-Proxy-Refresh-Token` response headers. Update client handling before upgrading; a generic client that only reads those headers cannot complete automatic renewal. Retry only after adopting the fresh access token, not by blindly replaying an old key. Model discovery refreshes transparently instead of requiring the handback workflow, but does not return updated tokens to the client. Missing refresh tokens, unknown mechanisms, non-JWT keys or disabled refresh leave ordinary forwarding unchanged. Exchange failure falls back to the original upstream credential, allowing the provider to decide authentication. Use the [first-login helpers](#first-login-helpers) to obtain initial credentials. They are not proxy startup dependencies. Treat terminal output and returned tokens as secrets; never paste them into issues or commit them. Adopt a returned refresh token when present; if a successful exchange omits it, including when the provider echoed the same refresh token, retain the existing refresh token rather than replacing it with an empty value. The implementation lives in [tokenrefresh.go](../internal/proxy/tokenrefresh.go). Provider/account behavior can change; historic successful observations are not compatibility or authorization guarantees. --- # MCP server The proxy's own `/mcp` endpoint exposes millivolt's operator and observability API to an LLM, using the official [Model Context Protocol Go SDK](https://github.com/modelcontextprotocol/go-sdk). It is a client of a running proxy: it never starts, stops or restarts the proxy process. Its operator tools can reconfigure a running proxy, but only through the same credential and the same routes the dashboard uses. The endpoint serves 25 tools from one implementation. It is operator-gated and Bearer-only, enabled by default: `mcp_enabled: false` removes it from the process (the route stays reserved, answering 404 and never inference; restart to apply), and the limits are config fields in the `mcp` category. It forwards the caller's own `Authorization: Bearer` credential on its internal calls, so the dashboard session cookie is refused there. The token length band matches the proxy's own boot check (16 to 512 bytes). A credential the proxy would refuse to arm with is not a credential this server can use. Redirects are refused outright. No operator route redirects, and `net/http` strips `Authorization` only when the hostname changes, so a same-host different-port or subdomain `307`/`308` would replay both the credential and the full request body to the redirect target. ## URL placement A URL-capable client (an editor agent, a hosted connector, an MCP-capable CLI) needs no binary and no environment: it points at the proxy's `/mcp` endpoint and presents the operator credential in an `Authorization` header, the same credential the dashboard uses. ```json { "mcpServers": { "millivolt": { "url": "http://127.0.0.1:8080/mcp", "headers": { "Authorization": "Bearer the same value the proxy was started with" } } } } ``` A remote client uses the reverse-proxied HTTPS origin from [the reverse-proxy guide](reverse-proxy.md), with `/mcp` appended: ```json { "mcpServers": { "millivolt": { "url": "https://your-hostname/mcp", "headers": { "Authorization": "Bearer the same value the proxy was started with" } } } } ``` The header is the whole setup: the endpoint reads no environment, starts no process, and no session cookie or per-request state outlives the request. Every request is independently authenticated, so a missing or wrong Bearer is denied before any tool runs, and a request presenting only the dashboard session cookie is refused with 403, because the caller's own credential is what the internal calls carry. `/mcp/` look-alikes are reserved: an unregistered path answers 404 and is never forwarded to inference. The published Compose port is loopback-only, so it is not a remote origin; remote clients terminate TLS at the ingress as in the [reverse-proxy guide](reverse-proxy.md). The transport is stateless and request/response only, and its limits are part of choosing a client: - `GET` and `DELETE` answer `405` with `Allow: POST`. There is no standalone SSE stream and no session to delete. - There is no session id: the endpoint never sends `Mcp-Session-Id`, and no per-request state outlives the request. No session cookie is minted here either, unlike `/admin`, `/session`, `/metrics` and `/dash`, so an MCP client's cookie jar gains no dashboard credential; a client must not expect or send a session id. The admitted credential does key an in-process cache of the SDK server object, so repeated calls do not rebuild all 25 tool registrations; admitting a different credential replaces it. The credential is only the in-memory cache key: it is never logged or persisted. - SSE resumption is not offered: no event store exists, so `Last-Event-ID` replay is not available. - The server never sends requests to the client: there is no sampling and no elicitation, so a client that waits for a server-initiated request waits forever. - Every request must accept both types, for example `Accept: application/json, text/event-stream`. A wildcard that covers both (`*/*`, `application/*, text/*`) is accepted too; an `Accept` missing either type, or one absent altogether, answers `400`. - The accepted protocol versions are the bundled SDK's full set, newest first: `2026-07-28`, `2025-11-25`, `2025-06-18`, `2025-03-26` and `2024-11-05`. This build does not narrow them. A classic `initialize` is answered with the requested version when it is one of the legacy versions (`2025-11-25` and older, verified live) and with `2025-11-25` for any other requested version; `2026-07-28` is served only through the SDK's per-request metadata form: the request carries `Mcp-Protocol-Version: 2026-07-28`, matching `_meta` keys `io.modelcontextprotocol/protocolVersion` and `io.modelcontextprotocol/clientCapabilities` (`io.modelcontextprotocol/clientInfo` is optional), and the `Mcp-Method` header. `tools/call` additionally requires an `Mcp-Name` header equal to `params.name`; a form missing any required piece answers `400`. - The SDK's localhost DNS-rebinding check is deliberately off. It rejects a loopback connection whose `Host` is not loopback, and the documented reverse-proxied placement preserves the external `Host` while dialing the proxy over loopback, so leaving it on would reject that placement. The endpoint is Bearer-only and the operator gate re-validates the credential on every request, which is the stronger check for an authenticated endpoint. ## Tools `describe` is the one to call first for this instance's live schema; without it, correct SQL is guesswork. `docs_list` is the one to call first for how millivolt itself works: it indexes the same Markdown as the published site and needs no proxy request. The three docs tools read `internal/docs/docs.json`, which is generated from the same Markdown as the published site by `scripts/build_site.py`. Refresh the embedded copy with `python3 scripts/build_site.py --output site --write-manifest` after any page, title or navigation change. The tools never call the proxy, read storage, or use the operator credential. A full page is capped at 256 KiB by `internal/mcp/tools_docs.go`. | Tool | Purpose | | --- | --- | | `describe` | Live SQLite schema, the complete `requests`/`request_debug`/`meta` column reference, the exact health predicates, a value vocabulary for every filter dimension, the live known client/provider/model and tool names, the indexes, the complete result-limit set (the query row default and explicit `max_rows` ceiling, query bytes, log page band, default page size, explorer group and chart bucket caps, Prometheus line cap and capture byte budget), and the working SQL idioms for this build. | | `docs_list` | Every published documentation page in navigation order, with its path, title, description, public URL, Markdown URL and section count. Starts the docs flow; local only. | | `docs_search` | Ranked section search across titles, headings and body text, with path, anchor, heading context and a snippet. Every query word must match; the result reports its total and truncation. Local only. | | `docs_read` | One page by path, or one anchored section by heading id. Full-page Markdown is capped at 256 KiB with an explicit marker. This is the canonical explanation of millivolt's routes, configuration, protocol and operator behavior. Local only. | | `query` | One bounded `SELECT`, clamped on both rows and encoded size, truncating with an explicit marker. The only tool that can express a time range. | | `values` | The distinct values one dimension has in durable history, most frequent first, so a filter value is discovered rather than guessed. | | `explore` | Faceted breakdown over all matching history by one dimension, with at most 24 groups. No time window. | | `chart` | Server-authoritative time series, at most 31 clock-aligned buckets. Cannot group. | | `records` | One durable newest-first log page, with the paired `before_ms`/`before_id` cursor and the page's `model_canon` map. | | `snapshot` | The dashboard's live snapshot: sequence, feed, counters, storage signal, recent records. With durable storage enabled, a full snapshot is capped at `2 * dash_log_rows`; with storage off the ring is the only history and the snapshot is not capped. | | `prometheus` | The text exposition, which covers the ring rather than durable since-inception totals. | | `audit_status` | Capture sessions, their scope and live capture counts, the scope vocabularies, and the storage prerequisite. | | `audit_start` | Start or edit one capture session. Requires `confirm: "start capture"` and refuses when durable storage is off. | | `audit_stop` | Stop one session by id, or every session with `stop_all: "stop every capture session"`. | | `audit_captures_list` | Page the requests that have a stored capture, through a bounded `SELECT` over `requests WHERE debug = 1` whose `request_debug` sidecar exists and has not expired. Ids are canonical: empty and whitespace-padded record ids are excluded because `audit_capture_get` could not read them back. | | `audit_capture_get` | Read one capture document by request record id. | | `operator_state` | Pause, limits, quota/storm and restart status in one call. | | `set_pause` | Create, edit or resume pause holds. | | `set_throttle` | Set, merge or clear one provider's caps, including the optional anti-burst pair. | | `resume_quota` | Close one provider's open retry-mode quota gate. | | `set_config` | Revision-checked partial configuration patch. | | `config_get` | The live configuration document: values, running overrides (the keys the process pins outside the file, which `set_config` strips), the canonical usage and model metadata field names, defaults, per-key schema, restart-required set and the revision a patch must echo. Credential-class provider headers and last-reload error text are redacted; use the operator CLI with `--reveal` for the raw bytes. | | `reload_config` | Re-read the configuration file. | | `purge_preview` | Count what a filter would delete, without deleting. | | `purge` | Delete matching history permanently. | The routes are a closed set owned beside the HTTP client, so no tool can name an arbitrary path: nothing reaches the proxy's transparent inference catch-all, and no restore-adjacent route is exposed. The HTTP endpoint dispatches those same calls in process to the proxy's own gated handler, so they never leave the process and the gate re-validates the forwarded Bearer. `set_throttle` takes `provider` and any of `concurrency`, `requests`/`request_window`, `tokens`/`token_window`, `anti_burst` and `ramp_duration`; omitted dimensions keep their value. A count needs its own window only when the policy stores none for that dimension, a count of `0` clears the dimension with its window, and a window without a count is refused unless a count is already stored. `clear: true` drops the whole policy and cannot be combined with a dimension. The proxy owns the duration grammar and band and refuses an explicit on-ramp without an active limit. This tool addresses one provider and has no selector or schedule fields. Scoped selectors and recurring schedules are available through the HTTP API (`/admin/throttle`) and the dashboard console, not through MCP. `set_pause` refuses a resume that carries a scope, and refuses `all` combined with `new`, `clients` or `providers`. `config_get` and the `set_config` echo redact credential-class provider headers and last-reload error text; the CLI's `--reveal` flag is the explicit operator opt-in. A redacted document posted back through `set_config` keeps the loaded secret: the marker is restored at the write owner, so a read-then-repost round trip cannot erase a credential. ### Filters `explore`, `chart`, `records` and the explorer facets share one grammar, owned in one place. Filters are `dim:id`, repeated, split on the FIRST colon, AND-ed across dimensions and OR-ed within one dimension. `dim` is one of `client`, `provider`, `model`, `conversation`, `key`, `status`, `time`, `tool`, `error`. `key` is the SHA-256 hash of the client key. `status` is a single selector: an exact HTTP status code (`0` to `9999`) or a live in-flight class (`streaming`, `paused`, `throttled`, `pending`). It is not the explorer's status-class vocabulary, which travels as `status:2xx`. Two bounds apply to `f=` itself: at most 64 filters, and each `dim:id` value at most 2048 bytes. The list is refused whole when either is exceeded; no filter is silently dropped. An unknown dimension, malformed filter or unknown status is refused before the request is sent. `explore`'s `error_total` uses two units. With `dim: error` it is the union of distinct affected requests represented by the error cards, including recovered quality failures whose final status is 200; for every other dimension it is the canonical error count. A card share is `N / error_total` and cannot exceed 100%. Do not compare it across the two cases as one metric. ### Pagination `records` and `audit_captures_list` page newest-first with a paired `(before_ms, before_id)` cursor. Both fields or neither. The proxy scans a bounded number of rows per call, so a page can be short or empty while its `more` flag is still true. `exhausted` is true when a page that was given a cursor returns that same cursor, and also when `more` is false: treat it as "stop paging". A first page has no cursor to repeat, so any returned cursor counts as progress, including `(0, "")`. A non-advancing cursor means the scan budget cannot reach the next row, and treating it as "keep going" is an endless loop; an empty page with an advanced cursor is instead the normal shape of "the budget ran out before a match", so page again. Deep history needs this rather than `OFFSET`, which degrades on every page. ### Volume A model cannot read ten thousand rows. Every listing defaults to a small page, and every tool truncates with an explicit marker (`truncation.marker`) naming what was withheld and the way to continue for that tool, because only some of them have a cursor. What is *not* claimed: the explorer caps groups at 24 and the chart caps buckets at 31 SERVER-SIDE, so those drops happen inside the proxy and those tools' `truncation.truncated` is `false` when groups were dropped. A model that trusts that flag alone will read a short breakdown as a complete one. ### What each tool cannot do These are the limits that otherwise send a model down the wrong path. They are also in the tool descriptions, which is where a model reads them. - `describe` first. Without the live schema, the column meanings, the health predicates and the value vocabularies, correct SQL is guesswork. - **`explore` has no time window.** It always folds all history. It is also single-dimension: it cannot cross-tabulate provider by model. For either, use `query` with a `WHERE` on `started_at` and a `GROUP BY`. - **`chart` is the only windowed tool, and it cannot group.** A window broken down by a dimension is a `query`. - **`query` is the only tool that can express a time range.** There is no `unixnow()` in this SQLite build, so a relative window is `strftime('%s','now')*1000`, for example `WHERE started_at >= strftime('%s','now')*1000 - 3600000` for the last hour. Ready-to-run analysis SQL for exactly these time-ranged questions is collected in the [operator recipes](routes.md#operator-recipes). - An **unknown filter value is an empty result, not an error.** A misspelled client, provider, model, tool, conversation or key produces a confident zero. Call `values` before filtering on one of those. Model filters take the **canonical** name the proxy folds from `model_rules` (which is what the `values` tool returns), never a raw stored spelling. - **`values(model)` is paged and folded.** The tool reads raw spellings in bounded keyset pages, so a vocabulary larger than the proxy's `storage_query_max_rows` no longer fails with 413, then folds and merges the pages through `model_rules`. Past its documented raw-spelling, canonical-name, or total page-query bound it fails explicitly rather than returning a partial vocabulary. The ordinary `limit` still reports its own explicit truncation marker. - `time` is a **daypart bucket**, not a duration: `time:24h` matches nothing. The buckets are `night`, `work`, `evening` and `weekend`. - `status` takes the proxy's **status classes**, not HTTP shorthand: `2xx`, `cancel` (499), `4xx`, `5xx` and `err` (anything outside those, including 3xx and below 200). There is no `3xx` class of its own. ## Audit semantics Audit is the proxy's debug-capture subsystem. **Capture requires durable storage.** With `db_path` empty, `audit_start` is refused by the proxy with 503 `durable storage is disabled` and creates no session; every capture read-back is a 404 because no capture document can exist. `audit_start` and `audit_captures_list` therefore check the storage signal first and refuse with that explanation, and `audit_status` reports it. Starting a session that can never capture would be a silent no-op, which is worse than a refusal. **Scope is explicit.** An empty dimension matches anything; non-empty dimensions are AND-ed and names within one dimension are OR-ed. There is no unrestricted all-traffic scope, and there is no conversation dimension. Model names use the proxy's native model normalization, so use the `known_models` values rather than guessing spellings. A scope overlapping a live session is rejected (`409`) rather than silently replacing it. **Edits change only named dimensions.** With a session id, an omitted `clients`, `providers` or `models` field preserves that dimension and an explicitly empty array clears it. An omitted `duration` preserves the deadline and an explicit empty string clears it. A create response also carries `created_id`, the id its own request installed, so concurrent starts resolve their own sessions. `set_pause` follows the same presence rules for scope and duration when `hold_id` targets an edit; without `hold_id`, an omitted create scope is the documented global hold. **Starting requires a confirmation.** `audit_start` takes `confirm: "start capture"` because capture stores full request and response bodies. **Stopping does not delete, but a purge does.** Stopping a session, or all of them, ends future captures only: a stopped session's documents stay readable by record id until `debug_capture_ttl` expires. There is no capture-specific delete endpoint, but a `purge` deletes the stored capture document of every request it matches, in the same transaction as the request row. A filter that reaches a captured request takes its evidence with it. Only a backup taken beforehand restores it. **A scope name must be known when the vocabulary is.** `audit_start` checks every client, provider and model against the vocabulary `audit_status` reports. When that vocabulary is non-empty, an unknown name is refused with the known values offered, because a session scoped to a name that matches nothing would report `enabled: true` and capture nothing. An empty vocabulary is the exception: a proxy that has seen nothing knows no names, so refusing every name would make the tool unusable rather than safer and the check is skipped. A vocabulary that cannot be read is not a skip: the start is refused, because the guard cannot be applied at all. **Read-back is by record id.** `audit_capture_get` takes the REQUEST RECORD id (`requests.id`), not the session id. There is no listing endpoint, so `audit_captures_list` discovers those ids by querying `requests WHERE debug = 1` with an existing, unexpired `request_debug` sidecar and reports that it does so. The listing also excludes an id that is empty or padded with whitespace: the capture route trims the id before lookup while the sidecar key is stored verbatim, so such an id could be advertised but never fetched. Every listed id is therefore readable. Captured documents are sensitive: credentials are redacted from headers, but body content is not sanitized. **Sessions survive a restart.** The proxy restores them from its own state, so a start list may already contain sessions this server did not create. ## The destructive call `purge` permanently deletes matching request history. The proxy has NO server-side confirmation, NO confirm flag and NO two-step protocol. The operator credential and this tool's own guards are the entire gate. Three guards apply together, and each alone would be insufficient: 1. **The filter must constrain at least one field.** The bodyless request that deletes ALL history is never sent by this tool; an empty filter is refused before any request. 2. **The confirmation must be an exact phrase.** `confirmation: "permanently delete the matching millivolt history"`. It is also a required tool argument, so a call that omits it is rejected by the schema before the handler runs. 3. **`preview_token` must be the token `purge_preview` issued for THIS filter.** It is an opaque HMAC-bound token over the canonicalized filter, the count it authorized and an expiry, so it cannot be forged, cannot be reused for another filter, and expires **15 minutes after issue**. A bare count could not do this: it is only a number, so two different filters matching the same number of rows authorized each other, and a count obtained from anywhere - a `query`, a guess, a preview of a different filter - satisfied the comparison. That was a live bypass, not a theoretical one. Always run `purge_preview` first and pass its `preview_token` back verbatim. ### What the guard does not prevent Stated plainly, because an operator needs to know the size of the remaining window: - **The count compares a population, not row identities.** The count is re-checked immediately before deleting, and the reviewed count travels with the delete, where the proxy counts it again inside the delete transaction and refuses with 409 when it moved. Nothing un-previewed is deleted by a refused call, but two different populations with the same row count are indistinguishable, and a refusal means previewing again. - **A filter that reaches a captured request deletes its capture document too.** The preview and the deletion both say so in their output. Deleted history cannot be recovered from this API. The deletion preserves newer completions and in-flight work and does not reset live error-storm protection. `set_config` is the other consequential call, and it is a revision-checked partial patch: `values` carries only the keys to change, the whole result is validated before the file is written, CLI-overridden keys are stripped, and the response names the keys that need a restart. A supplied list or map replaces that whole field, so call `config_get` first for structured keys such as `providers` or `model_rules`, and echo the revision it returns. A failed reload is reported as a save, not as an error. The proxy writes the file and only then answers 500 with the reload failure in the body, so that response is decoded as the committed document it is: `saved` is `true` and `error` carries the reload message. Reporting it as a tool error would tell a model that a mutation that succeeded had failed, and its retry would then fail with 409. `reload_config` re-reads the configuration file. It is the one state-changing route the dashboard never calls. The operator token itself is process-bound and is not re-read. ## Errors Every failure is a tool result the model can read and correct, carrying the proxy's own `.error` text and the status: `401` (credential wrong or missing), `403` (the proxy's operator plane is unarmed, or the request carried only the dashboard session cookie), `404`, `409` (overlap or stale revision), `413` (a result limit), and `429` with its `Retry-After`. An unreachable proxy, a timeout, and a body that is not the expected document are reported the same way. The `.error` field is read regardless of `Content-Type`, because the operator routes are inconsistent and some answer `text/plain` with a JSON body. The returned text is scrubbed of the operator credential first, because a reflecting endpoint could otherwise put the `Authorization` value into its own failure message and hand it to the model. The scrub removes the literal token, any contiguous 8-byte fragment of it (so a credential split across two JSON string fields leaks no usable piece), percent escapes in any hex case (including full-byte nested forms), the query `+` form of a space, and JSON-escaped forms, including backslash-doubled text and surrogate pairs. Each form is decoded to a fixed point before matching: an encoding is either fully seen or the excerpt fails closed. Text that still decodes after three escape applications is not published partly decoded: the whole excerpt is replaced by `[redacted]` instead. That bound applies regardless of whether the credential appears in the body: a heavily re-encoded failure with no credential in it still loses its diagnostics, because a credential could be hidden past the bound. The scan is bounded to the region that can still reach the excerpt, and the raw bytes it reads are capped; a body whose whitespace collapse does not reach the excerpt target inside that cap fails closed to `[redacted]` too, for the same reason. The excerpt itself is whitespace-collapsed and truncated to 4 KiB before the ` [truncated]` marker is appended, so the model-visible text runs up to the 12-byte marker past 4 KiB. Redaction runs before the truncation, so a credential straddling the boundary cannot survive as a fragment. ## Try it against a disposable instance Run the endpoint locally against a disposable instance with the repository's lifecycle owner, never the main instance on `:8080`, and drive it with one stateless POST: ```sh ( set -e trap 'scripts/dev.sh stop' EXIT export MILLIVOLT_OPERATOR_TOKEN='dev-instance-credential' scripts/dev.sh curl -fsS -X POST http://127.0.0.1:8081/mcp \ -H "Authorization: Bearer $MILLIVOLT_OPERATOR_TOKEN" \ -H 'Accept: application/json, text/event-stream' \ -H 'Content-Type: application/json' \ -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"0"}}}' ) ``` The answer carries the server identity. A full client session is the client configuration above pointed at `http://127.0.0.1:8081/mcp`; a request without the Bearer credential answers 401 from the operator gate. ## Security The endpoint is driven with an operator credential, which is the whole dashboard plane: pause, limits, configuration, restart, purge. Treat the client configuration that carries it as sensitive and keep it out of shared repositories. See [Security](../SECURITY.md). ## Verification The unit suite uses a fake proxy that pins the exact method, path, query and `Authorization` header of every tool, maps canned responses onto typed output, and covers the error shapes, setup validation, truncation markers, the pagination rule and the purge guards. `tests/_go/internal/docs` checks the embedded copy against every source hash. `tests/python/site.py` checks the published Markdown twins, `llms.txt`, `llms-full.txt` and `docs.json` against the same page model. The paging rule has one owner shared with `audit_captures_list`, so the two listings cannot disagree. One of its tests runs outside that harness on purpose: - the redirect test stands up a second listener as the redirect target and asserts it received nothing, so "we refused the redirect" is an observation rather than an assertion about an error string. - the capture listing test executes the generated `SELECT` against a real SQLite schema with empty, space-padded, tab-padded and NBSP-padded sidecars, and asserts they are excluded while every returned id has an unexpired sidecar and is the exact spelling `audit_capture_get` can fetch. This page is mechanically pinned by `tests/python/mcp_docs.py`: the tool table must match the `addTool` registrations, every " tools" mention must match that count, every route the client owns must appear in the [routes and debugging route table](routes.md#operator-and-data-routes), and this page may name no operator path the client does not own. The live integration test starts a private instance through [scripts/dev.sh](../scripts/dev.sh) on a port and scratch database of its own, so concurrent runs do not share a port or database, drives the tools through a real MCP session, and runs a full capture cycle against neutral loopback traffic: ```sh MILLIVOLT_MCP_INTEGRATION=1 scripts/check.sh go test -count=1 ./internal/mcp/... ``` It runs in CI as part of the isolated browser-fixtures step, which already has a dev instance and an operator token; it is not part of the core source-and-unit job, because that job should not have to start a second proxy. It drives every registered tool against its own disposable instance. The state-changing tools are `set_pause`, `set_throttle`, `resume_quota`, `set_config`, `reload_config`, `audit_start`, `audit_stop` and `purge`. The run exercises `set_pause` (a hold scoped to the fixture client, resumed at once), `set_throttle` (a limit set and then cleared), `resume_quota`, `set_config` and `reload_config` against the scratch config copy, a full capture cycle (`audit_start` and `audit_stop`), and the purge path: the missing confirmation, the wrong phrase, an unauthorized token and a token issued for a different filter each refuse with the row count unchanged, and then an authorized purge deletes the fixture rows. The read-only tools are `describe`, `docs_list`, `docs_search`, `docs_read`, `query`, `explore`, `chart`, `records`, `snapshot`, `prometheus`, `audit_status`, `audit_captures_list`, `audit_capture_get`, `operator_state`, `config_get`, `values` and `purge_preview`; the run drives each of them. A successful purge and the config write are safe here because the instance, its database and its config copy all belong to the test and are discarded. The same run drives the proxy's own `/mcp` endpoint with a real streamable HTTP client: `initialize`, `tools/list` (25 tools), and `describe` and `records` returning the same documents the in-process session returns, plus the auth boundary (401 without a credential, 401 for a wrong Bearer from the shared gate, 403 for a cookie-only request or a wrong Bearer beside an admitted cookie) and the reserved `/mcp/` look-alike answering 404, never inference. --- # Operator CLI (mvctl) `mvctl` is the human command-line client for millivolt's operator plane. It calls the same documented `/admin/*` and `/metrics/*` routes as the dashboard and reads them through the shared MCP operator client, so the origin check, credential handling, redirect refusal, response bound and credential redaction have one owner. It adds no server route, no server setting and no second listener. `mvctl` is a sibling of `cmd/proxy`, `cmd/stress` and `cmd/release`. It is source-only: the container image still ships the proxy, and deployment lifecycle stays with Compose. ## Build and run ```sh go run ./cmd/mvctl status --url http://127.0.0.1:8081 go build -o mvctl ./cmd/mvctl ./mvctl --url http://127.0.0.1:8081 status ``` The origin is required: pass `--url` or set `MILLIVOLT_URL`. There is no production default, so a command cannot silently address `127.0.0.1:8080`. A trailing slash is normalized away. Only `http` and `https` origins are accepted; userinfo, a path, a query and a fragment are rejected. ## Authentication `mvctl` reads the operator credential from `MILLIVOLT_OPERATOR_TOKEN`, the one variable the proxy's boot gate reads. It never accepts the token on argv, because a command-line value is visible in `ps` and shell history. The shared client validates the token's length band before sending a request. ```sh export MILLIVOLT_OPERATOR_TOKEN='the-value-the-proxy-started-with' mvctl status ``` Every request presents `Authorization: Bearer`. No cookie jar is kept: a session cookie minted by one call is deliberately not reused, so a rotated token takes effect on the next command. A missing or wrong credential is reported with the operator gate's own status. A non-loopback origin should use `https`; the credential is otherwise sent in cleartext. ## Output and exit codes Human output is the default. `--json` (or `-j`) prints one JSON document, and command output is stable enough for a pipeline. Diagnostics go to stderr as `mvctl: ...`. After argument parsing, `--json` turns a command failure into `{"error": "..."}` on stdout so a caller can parse one stream. A usage error detected while parsing the arguments still goes to stderr and exits 2, because JSON mode is not established yet; a usage error a command reports after a successful parse follows `--json` like any other failure. | Exit | Meaning | | --- | --- | | 0 | success, including a preview that made no change | | 1 | transport, HTTP or proxy-reported failure | | 2 | usage error, unknown flag or invalid input | | 130 | interrupted during a request, stream or watch | `status` and `overview` print an allowlisted state projection with `--json`: request rows and the full effective config are omitted, so use `requests` and `config get` for those. Request rows in that projection would carry previews, response headers and retry metadata that the dedicated commands keep out. ANSI color is off when stdout is not a terminal. `--color`, `--no-color` and `NO_COLOR` override the default. Request labels, model ids, error types and proxy error text are sanitized before a terminal sees them, so a captured value cannot inject an escape sequence. ## Global flags | Flag | Meaning | | --- | --- | | `--url URL` | proxy origin; required unless `MILLIVOLT_URL` is set | | `--timeout DURATION` | bound one request, default the shipped `mcp_timeout`; 0 disables the deadline | | `--json`, `-j` | emit one JSON document | | `--yes` | confirm a state-changing command | | `--color` / `--no-color` | override terminal color detection | | `--help`, `-h` | command or global help | | `--version` | print the mvctl version and source revision | `--` ends flag parsing; when no command has been named yet, the next word may still name one. Boolean flags accept `--flag`, `--flag=true`, `--flag=1`, `--flag=yes`, `--flag=on`, and the false spellings `false`, `0`, `no`, `off`. ## Commands ### View | Command | Purpose | | --- | --- | | `mvctl status` | health, origin, dashboard build and headline metrics | | `mvctl overview` | KPI, storage, pause, throttle and debug state | | `mvctl requests [N]` | one page of the request log, default the shipped `dash_log_rows`; `N` must be inside that setting's schema band | | `mvctl query "SELECT ..."` | bounded SQL through `/metrics/query` | | `mvctl chart` | server-authoritative time series; `--window` is all history or whole minutes, `--filter` scopes it | | `mvctl prometheus` | the Prometheus text exposition | | `mvctl watch` | sample live counters and print a sparkline | | `mvctl health` | check `/healthz` | | `mvctl url` | print the normalized origin | `requests` accepts `--before-ms MS` and `--before-id ID` together to fetch the next page; the id is an opaque keyset position and may be empty, so it is sent exactly as given. The command prints the next cursor when the page has more rows. Its JSON rows intentionally omit captured content previews, response headers and retry attempts. Use the dashboard capture drawer when full bodies are needed. `query` accepts `--query-timeout DURATION`; the default is the shipped MCP query bound, and an explicit value must be greater than zero. The shared HTTP client carries that shipped bound on its query path too, which is the one `chart` takes. The statement must be a single `SELECT` without a semicolon, using the same pre-check as the MCP query tool. The SQL text is sent as one `q` parameter and is never logged. `chart` prints the server-authoritative series: bucket geometry, percentile and stat arrays, blended cost per million tokens and the buckets themselves. Its fold runs through the client's query path with `--query-timeout` as the one bound, so `--timeout` (the generic per-request deadline) never cuts it short. `--window` takes `all` or a whole number of minutes; `--filter` takes comma-separated `dim:id` pairs using the same dimensions as the log scope (`client`, `provider`, `model`, `conversation`, `key`, `status`, `time`, `tool`, `error`). `key` is the SHA-256 hash of the client key. A chart fold covers finalized requests only, so a live in-flight status class is refused. `watch` accepts `--samples N`, bounded by the command's settings surface, and `--interval DURATION` (default the shipped `dash_poll_interval`). Interrupting any request or stream exits 130. ### Config | Command | Purpose | | --- | --- | | `mvctl config get [KEY]` | show one setting or every saved setting | | `mvctl config keys [SUBSTRING]` | list schema keys | | `mvctl config diff` | compare saved file values with built-in defaults | | `mvctl config set KEY=VALUE ... [--yes]` | revision-checked save | | `mvctl config reload [--yes]` | re-read the config file | `config set` reads `GET /admin/config` for the current revision and the schema. The schema's `kind` decides coercion: `bool`, `int`, `duration`, `bytes` and `string` take their natural text form; `strings`, `providers`, `aliases`, `model_rules`, `request_overrides` and `sub_conversations` take a JSON value. A numeric-looking string stays a string. An empty value is never a boolean. An unknown key, an empty boolean or an undecodable composite value is a usage error before any write. The server writes the file and then answers 500 when the follow-up reload fails. `mvctl` treats that committed document as a warning, not a failure, and prints the reload error without inviting a retry that would fail with a stale revision. Credential-class provider header values are redacted from `config get`, `config diff`, the `config set` preview and its post-save document, in human output and `--json`, using the shared header-name vocabulary. `--reveal` prints them. A historical `last_reload` error is replaced by the redaction marker because a rejected value can be quoted there. Token counters, token windows and rate-limit values are measurements, not credentials, and are never redacted. ### Pause | Command | Purpose | | --- | --- | | `mvctl pause [show]` | list holds and queued requests | | `mvctl pause on --client X --provider P --yes` | create a scoped hold | | `mvctl pause on --model M --conversation C --yes` | narrow a hold to a model or conversation | | `mvctl pause on --new --yes` | hold clients first seen after the hold | | `mvctl pause on --all --yes` | hold every client and provider | | `mvctl pause on --id ID [flags] --yes` | edit one hold in place | | `mvctl pause off --id ID --yes` | remove one hold | | `mvctl pause off --all --yes` | remove every hold | `pause on` requires a scope: `--client`, `--provider`, `--model`, `--key-hash`, `--conversation`, `--new` or `--all`, unless `--id` names an existing hold to edit. Set dimensions intersect and an unset dimension matches any value, exactly as they do for a limit. `--duration` and `--max-queued` are optional. `pause off` deliberately rejects scope flags: the proxy identifies a hold by `id`, and a scope-looking command that cleared every hold would be unsafe. Use exactly one of `--id` or `--all`. A resume carries only the hold id because the proxy has no scope-selective resume. ### Throttle and quota | Command | Purpose | | --- | --- | | `mvctl throttle [show]` | list provider limits | | `mvctl throttle set --provider P [flags] [--yes]` | merge limit dimensions | | `mvctl throttle clear --provider P [--yes]` | drop a provider limit | | `mvctl quota [show]` | show storm and quota state | | `mvctl quota resume --provider P [--yes]` | close a retry-mode quota gate | `throttle set` flags are `--concurrency`, `--requests`, `--request-window`, `--tokens`, `--token-window`, `--anti-burst` and `--ramp-duration`. Windows and ramps take a duration such as `1m`. A count needs its window unless the policy already stores one for that dimension, and a count of `0` clears the dimension with its window; a window without a count is refused unless a count is stored. At least one dimension is required. The set and clear commands are provider-only. Scoped selectors and recurring schedules are available through the HTTP API (`/admin/throttle`) and the dashboard console, not through this CLI. ### Queue and capture | Command | Purpose | | --- | --- | | `mvctl drain --all [--yes]` | release every parked request | | `mvctl drain --provider P [--yes]` | release one provider's parked requests | | `mvctl drain --model M [--yes]` | release one model's parked requests | | `mvctl drain --client C [--yes]` | release one client's parked requests | | `mvctl debug [show]` | list capture sessions and their bounds | | `mvctl debug on --client C --yes` | start a scoped capture | | `mvctl debug off --id ID --yes` | stop one session | | `mvctl debug off --all --yes` | stop every session | Exactly one drain scope is required. `debug on` requires a scope unless `--id` names an existing session to edit, and it has no unrestricted all-traffic form, matching the MCP tool: a bare enable would capture every request body. Captures are sensitive even after header redaction, and a capture create needs durable storage; the proxy answers 503 without it. `debug off` identifies a session by `id` or `--all`, never by a scope flag, and the two are mutually exclusive. `debug on` refuses a scope name the proxy has not seen, because such a session reports enabled but captures nothing. ### History, backup and restart | Command | Purpose | | --- | --- | | `mvctl purge [filters] [--all] [--yes]` | count first, then delete | | `mvctl backup --out FILE` | download an archive | | `mvctl restore --file FILE [--yes]` | inspect by default, apply with `--yes` | | `mvctl restart --yes` | request a graceful restart | `purge` filters are `--client`, `--provider`, `--model`, `--conversation-id`, `--error-type`, `--status`, `--before-ms`, `--after-ms`, `--has-error` and `--debug`. Every value must parse, and a zero or unparsable numeric filter is a usage error rather than an omitted constraint. `--all` cannot be combined with a filter. The command always calls `/admin/purge/count` first and prints the count; only `--yes` calls `/admin/purge`, and an unfiltered purge also needs `--all`. Before deleting, the command re-counts and refuses when the total changed since the preview, then sends that count with the delete so the proxy checks it once more inside the delete transaction. That compares counts, not row identities, so the guarantee is a refused moved population rather than a row-identity binding. A missing, fractional, negative or non-finite count is a failure, never zero. A genuinely bodyless purge is the proxy's delete-everything form. `backup --out FILE` is required; there is no default path. `--config` and `--database` default to true (the command's settings surface) and are always sent explicitly, so "both off" cannot silently become the server's both-on default. The download streams to a private temporary file, checks the `MVB1` magic, syncs the file, renames, and syncs the directory entry only on success. An existing file is refused unless `--overwrite` is passed. A short HTTP body is a failed transfer, not a published archive. Backup and restore are bounded by cancellation, not by `--timeout`. `restore` defaults to `inspect=1`, which validates the archive without applying it; `--inspect` forces that mode even with `--yes`. `--yes` applies it. The preview decodes and preflights the whole archived database member on the server, so it costs about as much as the apply's decode; `--yes` alone skips the preview and applies directly. The member selection (`--config`, `--database`) and the mode (`--config-mode`, `--database-mode`; `merge` or `replace`) are always sent explicitly, and both the inspect and apply output name the selected members and modes. The default mode is `replace`, the proxy's documented default, so the preview names the broader action instead of hiding it. An inspect document's archived provider headers are redacted; `--reveal` prints them. The apply response's `restart_required` keys are called out in the output. ## Scope and non-goals `mvctl` does not install, start, stop or update containers. It does not read or write `proxy.yaml` directly, does not add a listener, and does not keep a second copy of the proxy's routes, config schema, credential rules or backup grammar. Those facts stay with `internal/mcp`, `internal/config` and `internal/backup`. The command uses only the Go standard library and those repository packages. Because the shared operator client lives in `internal/mcp`, the resulting binary links the MCP Go SDK and its OAuth2 and jsonschema dependencies. The backup grammar and archive magic live in the dependency-free `internal/backup/params` leaf, so `mvctl` does not link SQLite or zstd and never opens a database. --- # Operations > [!WARNING] > Read the [security boundary](../SECURITY.md) before exposing the listener. The whole embedded dashboard is protected: every dashboard, `/metrics/*`, `/admin/*` and `/mcp` request requires the `MILLIVOLT_OPERATOR_TOKEN` credential and is denied while it is not configured. At the application gate, only the unauthenticated `/healthz` liveness and `/readyz` readiness probes, the `POST /session` login handshake, origin-root brand/PWA files and transparent inference stay open; see the canonical [operator access](routes.md#operator-access) list. ## Overview This page is the operating overview. It covers health and readiness, durable storage and accounting, the runtime cost model, and the known limits. Each task-shaped page carries the detail. | If you want to | Start with | | --- | --- | | Change a setting, an override or a provider alias | [Configuration](configuration.md) | | Install, back up, upgrade or restart the deployment | [Containers and data](containers.md) | | Call an operator route, or debug traffic | [Routes and debugging](routes.md) | | Protect the listener | [Security](../SECURITY.md), [reverse proxy](reverse-proxy.md) | | Measure the deployment | [Performance and resource usage](performance.md) | ## Health and readiness Liveness (`/healthz`) answers "the process is up and serving HTTP". Readiness (`/readyz`) additionally answers "this instance can persist": it fails with `503` and a reason when the durable writer is wedged. Two arms answer that: the oldest record on the queue has waited, or no flush has succeeded, for longer than one flush interval plus a progress window. The window is four flush intervals clamped to at least 30 seconds and at most two minutes; the record's own interval is allowed on top, so a record inside its own cadence never convicts and a long configured interval cannot postpone the verdict by more than that one cadence. Every clock starts on arrival: the oldest record on the queue ages the first arm, and while no flush has succeeded since boot the first accepted record ages the second one, so a writer that never picks records up is convicted all the same. Readiness also fails when any durable flush has failed since boot. Readiness stays red after the first failed flush even once writes recover, because a lost batch is worth an operator's attention; restart or investigate. With durable storage disabled both routes answer `200`, since there is no pipeline to wedge. The Compose files use `-readycheck` as the service healthcheck, so `docker compose ps` reports `healthy` only for an instance that can persist. Point an external load balancer or orchestrator at `/readyz` for the same signal; both probe routes are unauthenticated and answer the exact body `{"ok":true}` on success. Storage state is also scrapeable at `/metrics/prometheus` (behind the operator credential) as `llm_proxy_storage_enabled`, `llm_proxy_storage_queue_depth`, `llm_proxy_storage_queue_capacity`, `llm_proxy_storage_dropped_total{cause}` (`overflow`, `after_close`, `panic`, `flush_failed`), `llm_proxy_storage_flush_failed_total`, `llm_proxy_storage_last_flush_timestamp_seconds` and `llm_proxy_storage_wedged`. The block is rendered on every scrape, outside the memoized request aggregate, so queue and drop movement shows up immediately. Alert on `llm_proxy_storage_wedged`, on a rising `flush_failed` counter, and on `overflow` drops that are not explained by a burst. ## Storage and accounting SQLite retains finalized records; the ring and pending registry support live observation. Full bootstrap windows are capped in durable mode and older log rows page from storage. Without storage, the ring is the available history. The durable log order owns one secondary index, `idx_requests_log(started_at, id)`. The log page pushes exact provider, client, key-hash, conversation and canonical numeric status predicates into SQL and rechecks them on that scan; model, status-class, tool, error and time-bucket filters stay Go-owned because they match canonical/derived values, not raw stored text. The export and purge filter surface is different: it accepts exact provider, model, client, conversation, error type, status code, `has_error`, `debug`, and time bounds, with no key-hash selector, and its equality-only preview count or purge scans the table instead of maintaining per-dimension composite indexes. `BenchmarkStoreDiskBytes` measures this on its deterministic synthetic fixture (10,000 and 100,000 seeded records). The current per-record and total bytes are published in [storage and retained projection](performance.md#storage-and-retained-projection); that is a reproducible fixture scope, not a prediction of a deployment's disk use or a capacity ceiling. Dropping a superseded request-log index only moves its pages to SQLite's freelist, so the first startup that finds and removes one also runs a one-time `VACUUM` after the migration. Fresh and already-migrated databases skip it. A `VACUUM` needs free temporary space up to roughly twice the database size; if it fails for space, locking or interruption, startup logs a warning and continues with every row intact, but the freed pages stay on the freelist. Reclaim them manually during a maintenance window: stop the proxy, run `sqlite3 /data/proxy.db 'VACUUM;'` (or any SQLite client) against `db_path`, and confirm `PRAGMA freelist_count` returns 0. The automatic pass is never retried on a later boot, so a failed attempt needs this manual step. Boot migrations also own a small set of one-shot data repairs for shapes older binaries could write but current writers never produce: an empty-string JSON document in `attempts` or `tool_names` is rewritten to the canonical `null`, and a positive `ttft_ms` with `final_attempt_at = 0` (the pre-0.3.34 start-anchored TTFT, recorded with upload, queueing and retry backoff included) is reset to 0 because the original per-attempt anchor was never stored and cannot be recovered. Each repair has its own `meta` marker (`attempts_repaired`, `tool_names_repaired`, `ttft_anchor_repaired`): the marker is written after a clean probe or after the rewrite, later boots read one row instead of rescanning the table, and deleting the marker (or restoring a database that predates it) re-arms the probe. A repair that cannot run fails boot closed instead of serving history its own consumers would reject. The first boot after an upgrade that finds rows to rewrite may pause while it scans and rewrites them, and the totals rebuild follows: the rewrite advances the projection epoch, which invalidates the persisted totals document. The rewrite can transiently grow the WAL, so keep free space on the database volume as for any other large write. Enqueue is bounded and nonblocking. Overflow or write failures can lose durable records even when HTTP succeeds. Bootstrap's `storage:{enabled,dropped,totals_degraded}` reports cumulative process-local loss and a failed startup history scan. Either appears in the status footer; degraded since-inception totals count only rows committed since process start. Increasing queue size absorbs bursts, not arbitrary sustained overload. Storage overload remains an unresolved capacity boundary, not a lossless mode. Measure it with the [isolated stress workflow](../CONTRIBUTING.md#performance-evidence): successful responses, exact durable accounting and process-local drops are separate acceptance criteria. A short successful burst does not establish sustainable writer throughput or an absolute request-capacity ceiling. `storage_query_timeout` bounds interactive dashboard reads (chart, explorer, log/bootstrap, capture reads) and the ad-hoc query route. A capture read waits first for the in-flight sidecar save of the same request, bounded by the drawer client's own request context, and only then starts the read budget, so a slow save cannot spend the budget the read needs. Startup projection preload, the boot totals rebuild, an administrative purge, and a backup merge's totals scan run under a separate internal ten-minute maintenance budget: they legitimately read or rewrite full history and must not inherit the interactive bound. The maintenance budget still honors an earlier caller cancellation or deadline, and its expiry fails the operation closed (a purge transaction rolls back). It is a fixed implementation guardrail, not a configuration field. One shared analytical projection retains compact data and sorted metric orders proportional to stored history. Initial preload costs startup CPU/RAM; fast dashboard reuse is not free startup or bounded total-history memory. Concurrent independent proxy writers sharing a database are not a supported coherent-KPI deployment. Expired debug sidecars are hidden from reads immediately by their TTL. Physical reclamation deletes bounded batches per writer tick and continues on a near-term tick, so a large expired set drains without holding the sole writer connection; the sidecar byte cap accounting is updated exactly for the rows removed. Scheduler groups and some observed-identity sets also retain historical labels. Bounded request queues do not bound all process memory; long-running workloads with many unique identities need their own memory/cardinality measurements. From a source checkout with Python installed, use the existing online backup into a new destination: ```sh python3 scripts/backup_db.py proxy.db /path/to/new-backup.db ``` The helper opens the source read-only and includes committed WAL state. Do not plain-copy an open database or omit its WAL. Protect backups, debug captures and exports as sensitive data. This helper backs up the database, not configuration. Operators can also download a self-checked config and/or database archive from [Settings backup and restore](containers.md#settings-backup-and-restore). For image-only deployments and both persistent volumes, use [container backup and restore](containers.md#image-only-backup-and-restore). ## Performance and footprint Separate deployment size, cold startup, idle footprint, request overhead and loaded throughput. They answer different questions; a small image is not a RAM limit, and a cached dashboard read is not a cold-start measurement. The runtime contains a stripped Go binary with embedded UI assets, dependency notices and the generated configuration, on a minimal non-root base. There is no frontend build/runtime dependency in deployment. Docker's reported image size describes an image artifact, not compressed network transfer, persistent volume growth or the process's resident memory. On the request path, one bounded body read/metadata decode feeds shared routing and accounting. Upstream connections are pooled; streamed observation and durable enqueue avoid a per-request history scan or synchronous SQLite write. On the dashboard path, the initial HTML embeds bootstrap state and shared analytical projections/cache slots avoid building a separate history copy for each view. These implementation choices reduce repeated work, not every cost of a request. RAM still includes active request/response buffers, connection pools, the live ring and pending state, and the history/cardinality costs documented under [storage](#storage-and-accounting). Raising queue/body/pool limits or concurrency changes that footprint. Retained history and observed identities have no universal process-memory cap; configuration limits must be evaluated together. The startup preload decodes every stored row once, so its transient allocations (SQL strings, attempts and tool-name JSON) exceed the retained projection several times over. The process returns those burst pages to the OS immediately after the preload completes; idle RSS after readiness reflects retained state, not the startup burst. The projection's per-record retained footprint is measured reproducibly by `BenchmarkProjectionFootprint` (retained and transient bytes per record over a seeded scratch database) and `BenchmarkProjectionProcessRSS` (process RSS after readiness), without a live instance; the [performance workflow](../CONTRIBUTING.md#performance-evidence) owns the commands. `go_memory_limit` is the operator's steady-state RSS lever: a soft cap on Go runtime managed memory (heap, stacks, runtime structures) applied once at startup. With the setting at 0, an operator `GOMEMLIMIT` wins and otherwise a detected finite cgroup memory ceiling of at least 80MiB (v2 `memory.max` or v1 `memory.limit_in_bytes`) becomes a soft limit at 80% of that ceiling so a container's own bound actually reaches the Go runtime; smaller ceilings are ignored and, with no `GOMEMLIMIT`, the runtime stays unbounded. A set value overrides `GOMEMLIMIT`, not `GOGC`, for the process lifetime. `GOGC` remains independent. The limit is soft - exceeding it makes the collector more aggressive (capped near 50% of CPU time) instead of failing requests - and bounds Go-managed memory only, never SQLite's page cache or OS overhead. The [performance and resource usage](performance.md) page is the single source for the current measured tables, hardware, method, reproduce commands and limitations. This section owns the implementation cost model above and the `go_memory_limit` behavior. Use the [isolated performance workflow](../CONTRIBUTING.md#performance-evidence) when adding evidence. Short local-upstream bursts and microbenchmarks are useful regressions, not sustained production capacity, provider-latency or lossless storage guarantees. ## Known limits - Recognized provider fields drive usage/cost; this is not independent billing. - Non-streaming usage/cost inspection keeps a bounded prefix, so larger valid responses can forward completely without complete accounting. - Optional non-streaming translation reads the complete upstream 200 body and the translated document under the configured `translated_response_max_bytes` each (default 32 MiB). Exceeding either fails closed with 502 `response_too_large` before any client bytes; discovery budgets do not bound the path. - General multi-line SSE accounting/native event assembly remains incomplete; some split terminal/content shapes can affect classification, not just metrics. - SQL result limits do not bound every intermediate SQLite allocation. - Automatically grouped sessions are not proof of agent ancestry. Explicit parent declarations and exact-leaf filtering are described in [protocol](protocol.md). - Existing values lost by older storage formats cannot be reconstructed. - Extreme numeric data can produce explicit accounting/encoding errors rather than invented saturated prices or token counts. --- # Configuration How the configuration surface is owned, found and edited: the generated example, the settings schema, request overrides and sub-conversation tracking. > [!WARNING] > `provider_aliases` is a durable history rename, and a `request_overrides` > header value is operator-supplied and stamped onto a request whose > destination the caller chooses with `X-Proxy-Base-URL`. Back up before an > intentional rename, and review both sections before enabling them. ## Configuration and CLI [proxy.example.yaml](../proxy.example.yaml) is generated from the canonical `config.Example()` and schema documentation. `Example()` starts with neutral `config.Default()` values and adds the enabled [Grok, OpenCode Zen and Cursor compatibility profiles](adapters.md#bundled-compatibility-profiles) and the opencode sub-conversation tracking exemplar (the `promptCacheKey` tracked param, strip on). It is not mutable repository state. Source deployments copy it to ignored local `proxy.yaml`; images bundle their own copy for fresh config volumes. Do not start Settings against the committed example. The proxy's flags are listed once, with their registration defaults and usage text, in the [settings reference](settings.md#proxy-cli-flags). The effective `-listen` value is validated after every override, and a handoff child rewrites `-pid-file` with its own pid on boot so a supervisor keeps tracking the live process. The three output-only flags are mutually exclusive and exit without loading local config, opening a database or starting runtime services. Precedence is built-in defaults, then the YAML file, then CLI overrides. A missing file loads defaults; verify the path rather than assuming a typo created a new empty overlay. On startup, unknown keys, unknown nested fields and extra YAML documents are dropped and the file is rewritten without them. An invalid recognized value fails startup without rewriting the file, and an unparseable file is moved to `PATH.invalid` and replaced with the documented defaults, except an ambiguous duplicate top-level key, which fails closed and stays in place. Because replacing it with defaults would silently widen the bind and drop the origin allowlist, an unparseable file first has `listen` and `allowed_base_urls` salvaged from it block by block and applied over the defaults; both `(unparseable yaml)` and ` (salvaged)` are reported in the boot log and `last_reload`. A named key with no usable value, or a salvaged value the config rejects, fails startup with the original file still in place. Reload rejects an invalid document without rewriting it. Settings and restore merge treat a missing file as defaults, while an invalid or unreadable document fails closed and is reported instead of substituting a default. Healthcheck is stricter: a named file that cannot be read fails the probe, and only an empty `-config` reaches the built-in defaults. Backup capture is stricter: a missing config file is an error because there is nothing to archive. Settings POST still rejects invalid inputs. CLI-overridden fields cannot be saved from Settings. Healthcheck loads whatever file it is given read-only and never rewrites it. To regenerate the public example and the settings reference after changing their canonical owners: ```sh go run ./cmd/proxy -print-example-config > proxy.example.yaml go run ./cmd/docgen -write ``` The shared checks verify generated-file consistency. Built-in defaults remain neutral; compatibility-profile values belong only in `config.Example()`, never consumer fallbacks or private credentials. Optional mappings are mechanisms, not provider registrations, for example: ```yaml providers: neutral.example: cost_keys: [billing.charge_usd] usage_keys: input_tokens: prompt_toks ``` Use the derived provider label as the map key. The example file and Settings schema document other maps, model rules and header templates; do not maintain a second complete list of defaults here. ### Finding a setting The generated [settings reference](settings.md) is the complete field list: every setting with its type, default, accepted values and hot-reload or restart behavior, grouped by the same categories as the Settings sheet. [proxy.example.yaml](../proxy.example.yaml) remains the commented deployment example. `GET /admin/config` exposes the same field/category metadata alongside saved values, neutral defaults, CLI overrides, the process-effective snapshot and the current revision. Its `last_reload` section reports the most recent configuration application: `ok`, `error` (empty when ok), `dropped_keys`, `restart_required` and the unix-millisecond `at` timestamp. Settings searches labels, keys and help across categories. Durations and byte sizes use a magnitude plus unit (`2s`, `32MiB`) in YAML and Settings; do not write a bare number and assume seconds. A bare integer is still accepted as a count of bytes. Integer counts (retries, percents, tokens) stay unitless numbers with the unit in the schema label when it is not obvious. | Settings categories | Surface | | --- | --- | | Server, Request, Upstream, Queue & retry, Error storm protection | The proxy's traffic surfaces, in that order. | | Quota pause, Conversations, Format translation | Provider reaction, grouping, and cross-protocol translation. | | Storage, Backup, MCP, Dashboard | Durable state, its archives, the LLM tool endpoint and the live view. | | Models, Providers, Request overrides | Grouping rules, upstream mappings and scoped rewrites. | The category labels, their help and their membership are owned by `schema.Categories()` and `Schema()` over `GET /admin/config`; the generated example carries the same sections with each field's type, range and default. The rows above are names only, not a second copy of either. The destructive controls sit in their category and each confirms first: a rebuild and restart is confirmed and single-flight, starting a capture through MCP needs its `confirm` phrase and stopping every session confirms separately, a database delete confirms in the console (and over MCP needs the exact phrase plus the preview token for that filter), and a restore inspects before its confirmed apply. ### Guardrails that are not settings Every numeric guard is classified when it is added. A value becomes a setting when an operator can reasonably need a different number for legitimate traffic. It stays an internal constant when changing it independently would break an invariant, encode a protocol maximum, or only tune an internal heuristic. | Family | Examples | Why it stays internal | | --- | --- | --- | | Protocol maxima | Chat `stop` list of 4 | The wire defines the practical maximum; a longer list is clamped to it with an approximation warning. | | Derived frame bounds | `NativeStreamFrameMax(eventMax)`, `ResponsesStreamFrameMaxBytes`, the proxy's translated frame buffer | Computed from the accepted source-event and retained-byte bounds plus the JSON escape expansion. A separate value could under-size a scanner, turning a valid frame into a translation error, or over-size it and defeat the retention bound. | | Recursion and canonicalization budgets | Anthropic nested block depth 16, tool-argument canonical depth 64, separator budget, canonical byte cap | They select a safe fallback (the raw-byte fingerprint) or cap decoder recursion. They are algorithm facts, not wire acceptance. | | Diagnostic echo bounds | 128-byte warning echo, 512-byte identity and provider-text retention, header echo caps | A request-sized diagnostic or identifier must never multiply across a stream or a stored row. Raising them cannot admit a request that is otherwise rejected; it only grows logs and frames. | | Record retention | Retained tool names (64), tool-call ids (1024), path/method/model/error caps, capture header caps | They bound the ring, pending snapshots and durable rows. Counts still count past the bound; only identities stop being retained. | | Metrics-only truncation | Non-streaming choice cap, SSE analyzer event buffer | The non-streaming choice cap never changes relayed bytes. The SSE analyzer cap can affect classification for events split across that bound, as the protocol limits note; raising it cannot admit a request that is otherwise rejected. | | Cursor run and store budgets | Parked run count, per-run pending call/byte caps, blob store caps, aggregate store budget | They bound per-run and process memory for a bridge whose counts come from live tool batches, not operator intent. The per-run and aggregate caps are coupled. | | Internal heuristic thresholds | Quality retry spool, error-body capture, discovery output allowance, storage sweep batch sizes, log scan and chart/explorer budgets | They decide whether an internal heuristic runs or how much work one pass may do. Changing them changes diagnostics or performance, not what a client may send or receive. | The operator-facing bounds are settings: `request_tool_calls_max`, `stream_tool_calls_max`, `translated_response_max_bytes`, `models_discovery_max_entries`, `non_stream_capture_max`, `stream_terminal_hold_max`, `stream_event_max_bytes`, `token_response_max` and `upstream_response_header_max`. `non_stream_capture_max` trades accounting accuracy for memory and never changes relayed bytes. A new guard must be classified in this table when it is added, and a guard that becomes operator-facing moves to the settings schema instead of keeping a parallel constant. ### Saving settings safely Settings collects a draft until Apply. Revert reloads the saved document, and closing a dirty sheet keeps it open until Apply or Revert; an open dirty draft also asks the browser to confirm a tab close or reload. The server validates the whole result and compares its revision before writing. A stale tab receives 409; reload/review its values instead of resubmitting an old full form. The sheet renders closed vocabularies as selects from the schema's `choices` and checks duration/byte text against the schema's numeric band before the POST; the server remains the authority. With no writable config the field controls are disabled, not only the Apply action. API saves overlay supplied top-level keys on the saved config. Omitted keys remain unchanged, but a supplied map/list replaces that entire field. For example, `providers`, `provider_aliases` and `model_rules` are not recursively patched. Read the current document and preserve wanted entries when editing a structured field. Explicitly empty collections clear those fields. The response separates saved values from the effective snapshot, reports restart-required keys and carries the same `last_reload` section as GET; the Settings sheet renders it as its last-apply row with apply time, failure, dropped-key count and restart notice. A successful file write can still be followed by a reload failure; `saved:true` means the file changed even when the response reports an error. Do not assume rollback. The `writable` indicator means a config path is configured, not that filesystem permissions or every future write are guaranteed. See [persistence](containers.md#persistent-state-and-configuration) and [reload behavior](containers.md#reload-restart-and-shutdown) before changing startup-bound fields. ### Provider aliases and model grouping `provider_aliases` merges provider labels in stored history at boot/reload and rekeys new requests. All aliases in one map commit as a single transaction, so a trigger or constraint failure on a later alias leaves every earlier label unchanged. It rejects self-maps and alias chains. This is a durable rename, not a temporary view: removing the alias does not reconstruct original labels. Existing in-memory ring records can retain their previous label until they age out or the process restarts. Back up before an intentional history rename. `model_rules` instead controls display/grouping. The list is capped at 64 rules. Its ordered exact, pattern and lowercase steps keep stored model spellings unchanged; an empty rule list groups by those raw spellings. The shipped pipeline folds case, strips one or more leading `vendor/` namespaces and trailing `:tag` deployments, strips a trailing run of serving decorations, and unifies `.` with `-` between digits. A decoration is what a provider adds around the same weights: | Decoration | Examples | | --- | --- | | Quantization or serving format | `-fp4`, `-nvfp4`, `-bf16`, `-int8`, `-q4_k_m`, `-gptq`, `-awq`, `-gguf` | | Service tier | `-fast`, `-speed`, `-value`, `-free` | | Release channel | `-preview`, `-latest`, `-beta`, `-alpha`, `-stable`, `-experimental` | | Reasoning or effort word | `-thinking`, `-reasoning`, `-xhigh`, `-high`, `-medium`, `-low`, `-minimal` | | Build stamp | `-1017`, `-20250929` | Model tiers, fine-tunes, specifications and parameter counts stay distinct: - Model tiers: `-flash`, `-pro`, `-mini`, `-ultra`, `-lite`, `-nano`, `-turbo`, `-max`. - Fine-tunes: `-abliterated`, `-rl`. - Specifications: `-coder`, `-base`, `-chat`, `-it`. - Parameter counts. Add an exact rule above the shipped pipeline to split a group, or one it cannot recognize to merge the spelling. The Settings editor supports reordering, parking and previewing rules. Raw export/delete predicates do not silently expand to a display group. Debug's native-model normalization is separate from these rules. ### Request overrides `request_overrides` is an ordered rule list that rewrites the upstream request before relay; the default empty list leaves every request untouched. Each rule has a scope and at least one action: `headers` sets or replaces upstream headers, `remove_headers` deletes them, and `body` sets the output-token ceilings `max_tokens` / `max_completion_tokens` (1..1000000) on a Chat Completions or Anthropic client body before translation. A Responses client body is not rewritten by the body action because its cap is `max_output_tokens`; header and removal rules still apply. A Chat client whose target is the Responses wire has the rewritten cap translated to `max_output_tokens`. Both override spellings address an Anthropic body's single `max_tokens` field, with `max_completion_tokens` winning when both are set. The list is capped at 64 rules and validated like every other setting: a rule needs at least one scope field and one action, duplicate scope triples are rejected, and one header name cannot be both set and removed in the same rule. Scope matching is exact leaf. A rule applies when every non-empty scope field equals the request's value exactly: - `client` against the classified client label (`X-Proxy-Client` when sent, except a label that only repeats the user agent's product name, which records the fuller user-agent identity; otherwise inferred from SDK/user-agent metadata). - `provider` against the canonical label after `provider_aliases`. - `model` against the recorded model id (for Cursor requests the base id the dashboard records, not the fused spelling). An empty scope field matches anything. There is no model-rule canonicalization, regex or wildcard broadening. All matching rules apply in list order, and for the same header or body field the later rule wins. The client label is a claim, not an authenticated identity. The caller chooses it, and the proxy never verifies it, so a request presenting another client's label is treated as that client here and by every other client-scoped feature (sub-conversations, Debug sessions, pause holds, the Cursor parked-run scope). See [client identity](../SECURITY.md#client-identity-is-a-claim-not-an-authentication) for what that means for a listener reachable by parties you do not trust. What it cannot do is move a credential: credential-class header names are refused at load and skipped at send, so the exposure is the policy attached to the label, not the secret behind it. Header rules are the operator's last word on the upstream wire: the resolved merge applies after client-forwarded values, configured provider headers and the client's `X-Proxy-Headers` map, so a matching override wins over all of them. Only the proxy-owned transport-mode names `accept-encoding` and `range` are stripped from the final upstream request before send, whichever source set them, so the transport keeps its own gzip negotiation and never sends a partial range; the client header maps (`X-Proxy-Headers` and an `X-Proxy-Auth-Header` naming one) also reject them with a 400 before any send. The request-override list refuses two further header classes at load. Protocol and framing names (`host`, `content-type`, `content-length`, `transfer-encoding`, `te`, `connection`, `keep-alive`, `proxy-authenticate`, `proxy-connection`, `trailer`, `upgrade`) and the whole `x-proxy-` control prefix belong to the proxy and the transport. Credential names belong to whoever issued the secret: `authorization`, `cookie`, `proxy-authorization`, and every name the shared credential vocabulary classifies. That vocabulary judges the header's *role*, not a list of vendor spellings: `auth`/`authorization`, `api-key`, `secret`, `password`, `credential`, `bearer`, `jwt`, `signature`, `cookie`, `key` and `token` are all credential-class wherever they appear, so `x-auth-key`, `x-amz-credential`, `x-functions-key` and `x-subscription-key` are refused along with `x-api-key`. Names that mention a key or a token but carry an identifier or a measurement stay admitted: `x-key-id`, `idempotency-key`, `cache-key`, `x-prompt-tokens`, `x-token-count`, and the `x-ratelimit-*` and `anthropic-ratelimit-*` families. That breadth is deliberate. An override value is operator-supplied and is stamped onto a request whose destination the caller chooses with `X-Proxy-Base-URL`, so a secret in any unclassified header name would be delivered to that caller's host; vendor spellings cannot be enumerated in advance, which is why the check keys on role rather than a list. The same vocabulary drives capture and debug redaction, so a stored capture never keeps a header the wire carried. At runtime the target's configured auth header is skipped as well, so a rule that predates this check, or a future grammar slip, still cannot move a credential. The client header maps reject `content-length` before any send so the transport keeps ownership of message framing. Header names are trimmed and canonicalized case-insensitively, and values must be non-empty single-line header values. Provider `headers` refuse neither class above. They are applied after the extracted credential is placed, so a credential name (including the target's own auth header) or an `x-proxy-` name is legal there and takes effect over the forwarded and extracted value. Configured provider values are still stripped for `accept-encoding` and `range` at the final request, exactly like override values. The body rewrite is fail closed. It applies to Chat Completions or Anthropic client bodies before translation, whatever the target (a Responses client body is not rewritten); when the body is not a JSON object, or the rewritten result would exceed `max_request_bytes`, nothing is written: the original bytes are used unchanged by the rewrite stage and are still translated when translation is active, with one process log line recording the reason. Only a passthrough relays them verbatim. A successful rewrite replaces the body for the request's whole life, so every relay attempt, absorbed retry and quality re-send reuses it, and the recorded request cap is re-stamped from the effective body. A rule sets only the body fields it names, and the re-read pair keeps the wire's documented precedence - `max_completion_tokens` wins over `max_tokens` - so a rule setting only `max_tokens` cannot raise a request whose client already sent `max_completion_tokens`: set both fields to the same value to override either spelling. Cursor targets carry no token ceilings on their wire, so the body section is skipped for them while header rules still apply at the native send. Overrides apply to inference sends only, never model discovery or token refresh. The list hot-reloads: changed rules apply to new requests. The Settings editor mirrors the server's checks and rejections live for each rule, marks empty scope fields as matching any request, and blocks Apply on the first invalid rule. To raise the output budget for one provider, either write: ```yaml request_overrides: - provider: nano-gpt.com body: max_completion_tokens: 65536 ``` or open Settings, select **Request overrides**, pick **raise the output budget for a provider** from the add-a-rule menu, and fill the provider scope; the other starter templates seed an empty rule and an upstream header for a client. ### Sub-conversation tracking `sub_conversations` gives requests from one client a conversation identity of their own, taken from a value the client already sends in its request body (exemplar: opencode's `promptCacheKey`). The default empty list leaves tracking fully off, and the feature is deny-by-default per client: an entry applies only when its `client` equals the request's classified client label exactly (the same leaf the request-overrides `client` scope matches; no wildcard, no case folding), and a request from any other client is never inspected, leaving its bytes and grouping untouched. Each entry lists 1..4 exact JSON field names, matched only at the request body's top level and checked in order; the first present field decides. A present value that is not a JSON string counts as absent and the next field is checked, while a present string that cannot be decoded or fails the identity rules drops the identity for the request - no later field is consulted. A value is kept only when it is non-empty after trimming, valid UTF-8, free of control bytes and at most 512 bytes, the same bound explicit session declarations satisfy. The UTF-8 judgment runs on the value's raw bytes before JSON decoding: the standard decoder coerces invalid UTF-8 inside strings to U+FFFD, so a raw invalid-UTF-8 sequence drops the identity while an escaped lone surrogate, whose bytes are ASCII, is kept and decodes to U+FFFD. The drop is conservative: the value's raw spelling is scanned inside a bounded window, so a value whose raw span exceeds the window (escape sequences spend more raw bytes than they decode to, and trimmable edges count before trimming) is dropped even when its trimmed form would pass. An absent or dropped identity is never a client error: on the passthrough wire the body is payload, and the request keeps automatic grouping. A translated target still requires the whole request body to be valid UTF-8, as [protocol](protocol.md#relay-retries-and-model-discovery) describes. The kept value groups the request under a `k:` conversation identity. Precedence is the explicit `X-Proxy-Session` header first, then the tracked value, then automatic client+key grouping; a request carrying both header and tracked value keeps the header identity. `X-Proxy-Parent-Session` still owns parent linkage, and declaring a parent still requires the explicit session pair, so a tracked identity never carries a parent. In the dashboard a `k:` id is an ordinary conversation id: lineage, filters, saved views and the request drawer read it like any other. `strip: true` removes every configured field of the entry that is present at the body's top level from the client body before relay or translation, including a Responses client body; the default `false` forwards them unchanged. Unlike the [request overrides](#request-overrides) body fields, strip is not limited to Chat Completions or Anthropic inbound. The removal rides the same body-rewrite engine: one document decode when both features match one request, the original bytes used unchanged by the rewrite stage when the body is not a JSON object or the rewrite would exceed `max_request_bytes`, and cursor targets skipping the body section. Without `strip`, the client's spelling reaches the upstream wire on a same-protocol passthrough; a translated body carries the value in `prompt_cache_key` or drops it. When a tracked value was extracted and the request translates to the Anthropic wire, the translated body gains a top-level `prompt_cache_key` carrying the value. This is a proxy tracking extension, not an Anthropic cache field: Anthropic caching uses `cache_control`, and an upstream that rejects unknown top-level fields will reject the injected request. Injection follows extraction alone: `strip` governs the tracked client-body fields, never the injected field. Validation runs at the load boundary for YAML and Settings saves alike: at most 16 entries, at most 4 params per entry, param names one JSON key segment each (trimmed, non-empty, at most 64 bytes, printable ASCII, no quote, backslash or control bytes), and duplicate clients or duplicate params within one entry are rejected. The list hot-reloads: changed entries apply to new requests. The Settings editor lives in the Conversations category and mirrors every check live per entry card: classified-client autocomplete from observed clients, ordered param rows with add/remove gated at the caps, the strip toggle, a count line against the entry cap, the server's exact rejection wording, and an Apply blocked on the first invalid card. To track opencode's prompt cache key and strip it from the upstream body, write: ```yaml sub_conversations: - client: "opencode" params: - "promptCacheKey" strip: true ``` --- # Settings reference Settings and the proxy CLI write the same fields. The dashboard Settings sheet saves them through `GET` and `POST /admin/config`, the YAML file holds them at rest, and the proxy flags override a few of them at launch. Every block below is generated from the code that owns it, so a setting's key, label, type, bound, reload behavior and help cannot describe one thing here and another in the dashboard. A `-` under accepted values means the schema declares no bound and no closed vocabulary for that key. Read [configuration](configuration.md) for precedence, the YAML file and the settings schema, [containers](containers.md) for image mounts, restore paths and the deployment lifecycle, and the [capture gallery](gallery.md) for the rendered sheet on both surfaces. ## Server Bind address, database, and ring depth restart to apply. shutdown_timeout applies at the next SIGINT/SIGTERM. #### `listen` Listen address · `string` Default `":8080"` · Accepted - · Restart required Host:port the HTTP server binds. Restart required. #### `db_path` Database path · `string` Default `"proxy.db"` · Accepted - · Restart required SQLite metrics store. Empty disables durability (ring only). Restart required. #### `history_size` History size · `int` Default `10000` · Accepted 1..1000000 · Restart required In-memory ring depth and startup backfill count. Durable history and paged request logs can extend beyond this window. Restart required. #### `shutdown_timeout` Shutdown timeout · `duration` Default `5s` · Accepted > 0s · Hot-reloads HTTP shutdown wait after active request contexts are canceled on SIGINT/SIGTERM. Storage draining follows separately. Must be > 0; applies at the next shutdown without restarting the listener. #### `restart_drain_timeout` Restart drain timeout · `duration` Default `10m` · Accepted >= 0s (0s = wait indefinitely) · Hot-reloads How long a dashboard-triggered restart waits for in-flight streams. Expiry aborts the restart and resumes accepting traffic; unfinished streams are not killed. New connections queue during a successful handoff. #### `read_header_timeout` Read header timeout · `duration` Default `10s` · Accepted >= 0s (0s = no timeout) · Restart required Slowloris guard on inbound headers. Restart required. #### `idle_timeout` Idle timeout · `duration` Default `2m` · Accepted >= 0s (0s = no timeout) · Restart required Keep-alive idle connections. No write timeout (it would kill SSE). Restart required. #### `request_body_idle_timeout` Request body idle timeout · `duration` Default `30s` · Accepted > 0s · Hot-reloads How long one inbound body read may wait for progress. Re-armed before every read and cleared at EOF or Close. Must not exceed request_body_read_timeout. Applies to requests started after reload. #### `request_body_read_timeout` Request body read timeout · `duration` Default `10m` · Accepted > 0s · Hot-reloads Absolute bound for reading one inbound request body after headers. Must be at least request_body_idle_timeout. The restore upload is exempt (idle bound only). Applies to requests started after reload. #### `response_write_timeout` Response write timeout · `duration` Default `10m` · Accepted > 0s · Hot-reloads Generic per-write bound for buffered responses. Streaming owners may arm a shorter bound. Must be at least sse_write_progress_timeout. Applies to requests started after reload. #### `go_memory_limit` Go memory limit · `bytes` Default `0B` · Accepted 32MiB..1024GiB (0B = auto cgroup, else unbounded) · Restart required Soft cap on Go runtime managed memory (heap, stacks, runtime), applied once at startup. With the setting at 0, an operator GOMEMLIMIT wins and otherwise a detected finite cgroup memory ceiling applies at 80% (ceilings below 80MiB are ignored); a set value overrides GOMEMLIMIT, not GOGC. Exceeding it turns the collector more aggressive (capped near 50% CPU), never fails requests. Does not bound SQLite's page cache or OS overhead. Restart required. ## Request Inbound body cap, tool-call history bound, SSRF allowlist, optional body-preview, token-refresh response cap, and debug-capture retention. #### `max_request_bytes` Max request bytes · `bytes` Default `32MiB` · Accepted 1KiB..1GiB · Hot-reloads Inbound body the proxy will buffer to read routing metadata. Larger bodies are rejected with 413. #### `request_tool_calls_max` Max request tool calls · `int` Default `1024` · Accepted 1..10000 · Hot-reloads Most Chat tool calls across the whole message history of one inbound request, including the deprecated per-message function_call shape. Anthropic tool_use and Responses function_call history are bounded by their protocol collection limits instead. Real agent turns replay a handful; raise it only for clients that resend unusually long tool loops, because every call is materialized in memory during translation. #### `allowed_base_urls` Allowed base URLs · `strings` Default `[]` · Accepted - · Hot-reloads SSRF allowlist of upstream base URLs. Empty allows any target (single trusted operator). An entry admits only the same scheme, host and port once the scheme's default port is filled in, plus that entry's path and its children, so a port-less entry does not admit other ports on the host. Each entry must be an absolute URL whose prefix is literally lowercase http:// or https://, with a lowercase hostname, no userinfo, query, fragment or trailing slash, and a port after a hostname in 1..65535. #### `capture_body_preview` Capture body preview · `bool` Default `false` · Accepted - · Hot-reloads Store a short bounded preview of the last user prompt / response in metrics. Off by default so content is not persisted. #### `debug_capture_ttl` Debug capture TTL · `duration` Default `24h` · Accepted 1h..168h · Hot-reloads How long a full debug capture (bodies and headers) is kept after the request finishes. Operator debug sessions choose which traffic is captured; this is blob retention, not the session timer. #### `debug_capture_max_bytes` Debug capture max bytes · `bytes` Default `1MiB` · Accepted 4KiB..16MiB · Hot-reloads Cap on one debug capture: retained headers are charged first and the remainder is split between the request body and the client-facing response bytes. Past the cap the rest is dropped and the capture is marked truncated. #### `debug_store_max_bytes` Debug store max bytes · `bytes` Default `512MiB` · Accepted 16MiB..8GiB (0B = unlimited) · Restart required Total durable debug-capture sidecar cap. New captures are skipped once the next write would exceed it; TTL expiry frees capacity. Bytes, not characters. 0 = unlimited. Restart required. #### `auto_token_refresh` Auto token refresh · `bool` Default `true` · Accepted - · Hot-reloads Exchange an expired or expiring JWT when the client sends X-Proxy-Refresh-Token. All supported providers use inference handback: HTTP 401 with token_expired and access_token in the error body, plus refresh_token only when the exchange returns a different replacement. No inference is sent; adopt the returned credentials and retry, retaining the old refresh token if no replacement is returned. Model discovery refreshes transparently without returning tokens. Stateless: no reusable credentials are retained. #### `token_response_max` Token response max bytes · `bytes` Default `1MiB` · Accepted 4KiB..16MiB · Hot-reloads Token-endpoint response read during transparent credential refresh. Past it the refresh fails open and the original request is proxied unchanged. ## Upstream Connection pool, header timeout, stream keepalives, model-discovery budgets and response-buffer bounds. #### `upstream_timeout` Upstream header timeout · `duration` Default `5m` · Accepted >= 0s (0s = no header cap) · Hot-reloads How long to wait for upstream response headers. Streaming bodies are not subject to this. Transient header timeouts are retried. #### `upstream_dial_timeout` Upstream dial timeout · `duration` Default `30s` · Accepted > 0s · Hot-reloads How long establishing a new upstream connection may take. Applies to new connections after reload; in-flight streams keep their transport. #### `upstream_dial_keepalive` Upstream dial keepalive · `duration` Default `30s` · Accepted > 0s · Hot-reloads TCP keepalive interval for upstream connections. Applies to new connections after reload. #### `upstream_tls_handshake_timeout` Upstream TLS handshake timeout · `duration` Default `10s` · Accepted > 0s · Hot-reloads How long the TLS handshake with an upstream may take. Applies to new connections after reload. #### `models_discovery_timeout` Model discovery timeout · `duration` Default `30s` · Accepted 1s..5m · Hot-reloads One upstream metadata budget for transparent key refresh, all model pages and optional enrichment. Never applies to LLM streams. New discoveries use reloaded limits. #### `models_discovery_max_bytes` Model discovery bytes · `bytes` Default `8MiB` · Accepted 4KiB..64MiB · Hot-reloads Aggregate response bytes read after transport decoding across model pages, unary RPCs and enrichment. Primary overflow fails; optional enrichment is discarded. This is not a strict heap limit. #### `models_discovery_max_pages` Model discovery pages · `int` Default `100` · Accepted 1..1000 · Hot-reloads Maximum upstream model responses per discovery, including a unary RPC and optional enrichment. Limits are captured once, never reset between pages. #### `models_discovery_max_entries` Model discovery entries · `int` Default `10000` · Accepted 100..1000000 · Hot-reloads Model entries one discovery request may retain across every page and enrichment source. The byte budget alone cannot bound the decoded map cost of a compact catalog, so a source past this count fails closed instead of returning a partial list. Raise it for providers that publish unusually large catalogs. #### `translated_response_max_bytes` Translated response max bytes · `bytes` Default `32MiB` · Accepted 4KiB..1GiB · Hot-reloads Whole buffered non-streaming upstream response a cross-protocol translation may read, and the translated document it may write back. Past it the request fails with 502 response_too_large rather than relaying a prefix that breaks the translated shape. #### `upstream_response_header_max` Upstream response header max bytes · `bytes` Default `1MiB` · Accepted 4KiB..64MiB · Hot-reloads Upstream response headers the proxy accepts. Past it the request fails as a transport error. The upstream client is rebuilt on reload, so new requests use the new value while in-flight streams keep the client they started on. #### `non_stream_capture_max` Non-streaming capture max bytes · `bytes` Default `1MiB` · Accepted 16KiB..64MiB · Hot-reloads Prefix of a non-streaming response body kept for usage and cost parsing. Larger documents still relay in full; their accounting may be incomplete. This is a capture cap, not a forwarding cap, and it is per in-flight non-streaming request outside max_inflight_response_bytes. #### `stream_terminal_hold_max` Stream terminal hold max bytes · `bytes` Default `64KiB` · Accepted 4KiB..1MiB · Hot-reloads Bytes withheld at the end of a relayed SSE stream while the relay decides between verbatim release and an in-band degenerate-error substitution. Past it the hold is abandoned and the bytes are released verbatim. A single complete event line can exceed it by up to stream_event_max_bytes. Raise it only if a legitimate terminal region is larger than the default. #### `stream_event_max_bytes` Stream event max bytes · `bytes` Default `2MiB` · Accepted 2MiB..16MiB · Hot-reloads Largest SSE event accepted on the passthrough relay and by every translated stream decoder: a single line, and the accumulated data of a multi-line event. The translated frame buffer is derived from this value, so both bounds move together. The floor is the built-in accepted size; raise it only for providers that emit larger events. #### `max_conns_per_host` Max conns per host · `int` Default `512` · Accepted >= 0 (0 = unlimited) · Hot-reloads Total connections per upstream host. Sized for concurrent SSE streams, not requests/sec. #### `max_idle_conns` Max idle conns · `int` Default `512` · Accepted >= 0 (0 = unlimited) · Hot-reloads Process-wide idle connection pool. #### `max_idle_conns_per_host` Max idle conns per host · `int` Default `512` · Accepted >= 1 · Hot-reloads Idle connections kept per upstream host. Cannot exceed max conns per host when that is set. net/http treats 0 as a default of 2, not unlimited, so this must be >= 1. #### `idle_conn_timeout` Idle conn timeout · `duration` Default `90s` · Accepted >= 0s (0s = never expire) · Hot-reloads How long an idle upstream connection is kept before closing. #### `upstream_stream_idle_timeout` Upstream stream idle timeout · `duration` Default `10m` · Accepted > 0s · Hot-reloads Cancel an upstream response body after this much silence. Releases the handler, upstream connection and scheduler slot. New response bodies use the reloaded value. #### `whole_document_read_timeout` Whole-document read timeout · `duration` Default `2m` · Accepted > 0s · Hot-reloads Fallback bound for one whole-document upstream read when the caller supplied no deadline. Buffered reads only; streaming SSE is never subject to it. #### `whole_document_read_max` Whole-document read maximum · `duration` Default `10m` · Accepted > 0s · Hot-reloads Ceiling for whole-document upstream reads. Must cover whole_document_read_timeout and models_discovery_timeout. #### `sse_keepalive_interval` SSE keepalive interval · `duration` Default `15s` · Accepted 1ns..10m · Hot-reloads How often a quiet streaming socket gets a `: keepalive` comment (queue wait, operator hold, mid-stream gaps). Range 1ns..10m. Applies to streams started after reload. #### `sse_write_progress_timeout` SSE write progress timeout · `duration` Default `1m` · Accepted > 0s · Hot-reloads Per-event and keepalive write bound for a client-facing SSE stream. A stalled reader fails that write instead of pinning shutdown. Must not exceed response_write_timeout. Applies to streams started after reload. ## Queue & retry Per-key concurrency, transparent 429/5xx retry and error classification, client error-class disclosure, and quality re-asks. #### `max_concurrent` Max concurrent · `int` Default `0` · Accepted >= 0 (0 = unlimited) · Hot-reloads In-flight sends per provider+key. 0 is unlimited. #### `max_queue_size` Max queue size · `int` Default `1024` · Accepted >= 0 (0 = unlimited) · Hot-reloads Queued waiters per group. 0 is unlimited. It also feeds the default queue cap of a new pause hold. #### `max_inflight_requests` Max in-flight requests · `int` Default `10000` · Accepted 0..1000000 · Hot-reloads Process-wide inference admission bound. Requests above it are refused with 503 and Retry-After instead of buffering their bodies. 0 = unlimited. Hot-reloads. #### `max_inflight_body_bytes` Max in-flight body bytes · `bytes` Default `256MiB` · Accepted 1MiB..64GiB (0B = unlimited) · Hot-reloads Process-wide budget for buffered inbound request bodies. Admission charges the bytes actually read and refuses with 503 when the next read would exceed the budget. A declared Content-Length reserves nothing, so an idle connection cannot hold the budget. Bytes past max_request_bytes are neither charged nor retained: that body is answered 413 by the cap owner. A refused request drains at most its remaining cap window so an over-cap body still answers 413 rather than a retryable 503. 0 = unlimited. Hot-reloads. #### `max_inflight_response_bytes` Max in-flight response bytes · `bytes` Default `256MiB` · Accepted 1MiB..64GiB (0B = unlimited) · Hot-reloads Process-wide budget for buffered upstream response bodies: whole-document translation of non-streaming responses, the non-streaming 200 quality spool (at most 1 MiB plus one 32 KiB chunk), and model discovery. Admission reserves the declared Content-Length when it fits the bounded read cap, otherwise that cap, and refuses with 503 and Retry-After before reading when the budget would be exceeded. The reservation is released once the buffer is written, translated, or abandoned. 0 = unlimited. Hot-reloads. #### `max_queue_wait` Max queue wait · `duration` Default `10m` · Accepted >= 0s (0s = unlimited) · Hot-reloads How long a request may sit in the queue before 429. 0s is unlimited. Operator-hold time is excluded. #### `max_retries` Max retries · `int` Default `5` · Accepted 0..100 (0 = no retries) · Hot-reloads Transparent retries of transient 429 / 5xx / transport failures. Durable quota 429s follow quota_pause_mode instead of this budget. #### `queue_retry_after` Queue Retry-After · `duration` Default `2s` · Accepted 0s..24h · Hot-reloads Retry-After hint sent when our own queue is full or the wait is exceeded. HTTP Retry-After is integer seconds, so the value must be a whole number of seconds. #### `base_backoff` Base backoff · `duration` Default `1s` · Accepted > 0s · Hot-reloads Initial adaptive delay. Doubles each attempt and each consecutive failed request, up to max backoff. A provider Retry-After/reset is a floor (never retry sooner) and does not reset this doubling. Cannot exceed max backoff. Must be > 0. #### `max_backoff` Max backoff · `duration` Default `2m` · Accepted > 0s · Hot-reloads Cap on adaptive backoff. Does not clamp a provider Retry-After / rate-limit-reset (daily limits are often 30–60m). Must be > 0. #### `quality_retries` Quality retries · `int` Default `1` · Accepted 0..3 (0 = disabled) · Hot-reloads Transparent re-attempts of a degenerate 200 (empty completion, empty tool_calls), a truncated stream that relayed no content yet, a retryable in-band provider error dropped before the wire, and Cursor's empty-resume re-ask. 0 disables. #### `thinking_retries` Thinking retries · `int` Default `1` · Accepted 0..3 (0 = disabled) · Hot-reloads Transparent rescue of requests that die or end without an answer during thinking: a stream truncated or reset after reasoning-only output, a cleanly finished reasoning-only completion, a retryable in-band provider error that arrived after reasoning-only output, and a non-streaming reasoning-only body. The fresh attempt appends after the relayed reasoning; answer content and tool calls are never rescued, and finish_reason length never is. Each rescue re-sends the full request (billed). 0 disables. #### `retryable_error_classes` Retryable error classes · `strings` Default `[]` · Accepted - · Hot-reloads Operator extensions to the built-in retryable in-band error vocabulary: exact type or code strings (case-insensitive) that a provider reports inside a 200 and that authorize a transparent re-send while no generation content was relayed. Built-ins already cover server_error, the 503 overloaded/unavailable family, api_error, overloaded_error, timeout_error and common gateway spellings; list custom gateway spellings here. Entries add to the built-ins and never move one; a durable quota/billing class is rejected at load, and in-band rate limits, unknown classes and errors after relayed content stay verbatim. Reload applies to new rescue decisions. #### `rate_limit_error_classes` Rate-limit error classes · `strings` Default `[]` · Accepted - · Hot-reloads Operator extensions to the built-in transient rate-limit vocabulary (rate_limit_error, rate_limit_exceeded, tier_limit_exceeded, resource_exhausted, rpm/tpm spellings and friends). A gateway that announces its limit under a custom type or code string (case-insensitive) is then retried transparently within the retry budget, counted in the rate-limit series, and classified rate_limit for clients. Unlike a bare 429, a non-429 status carrying the class still counts as a failed request on the health surface. Entries add to the built-ins and never move one; a durable quota/billing spelling is rejected at load. Reload applies to new classifications and re-reads stored history through the same vocabulary. #### `suppress_client_retries` Suppress client retries · `bool` Default `false` · Accepted - · Hot-reloads Make the proxy the clients' sole retry authority: stamp x-should-retry: false on every LLM-relay error response. The official OpenAI SDKs (python/node/go/ruby) honor it over their 408/409/429/5xx auto-retry and over Retry-After, including relayed provider hints; the proxy's own retry ladder has already run by then. Cannot suppress transport-level failures, and clients that ignore the header (the Vercel AI SDK / opencode stack) keep their own policy. #### `client_error_class` Client error class · `string` Default `"off"` · Accepted `off`, `header` · Hot-reloads Negotiate millivolt's canonical error class to clients on the LLM relay surface. off leaves relayed responses exactly as the provider sent them. header stamps X-Proxy-Error-Class (rate_limit, quota, unavailable, unreachable, timeout, content, client_error, proxy, canceled, upstream) and a class-derived X-Should-Retry: true|false on every error status before the status line commits, so a client can tell a rate limit announced by a 400 from a client fault and a durable quota block from transient backpressure without parsing provider envelopes. The body is never rewritten. suppress_client_retries still wins while it is on; mid-stream SSE errors cannot carry new headers. Reload applies to new responses. ## Error storm protection Bounded recovery queues for affected providers or exact model names. Hot-reloads; policy changes clear observations and wake waiters. Banner visibility changes do not reset protection. #### `storm_enabled` Enable protection · `bool` Default `false` · Accepted - · Hot-reloads Detect elevated upstream failure rates and queue affected sends for bounded recovery probes. Opt-in; requests can still fail on cancellation, exhausted retry budgets, queue limits or permanent errors. Retries can repeat upstream execution and billing. #### `storm_provider_enabled` Provider protection · `bool` Default `true` · Accepted - · Hot-reloads Gate every model on a provider only when all active models individually meet the sample and failure thresholds. Active means a request in the same detection window; idle models expire. #### `storm_model_enabled` Model protection · `bool` Default `true` · Accepted - · Hot-reloads Gate an affected exact recorded model name within its provider. Dashboard model grouping does not broaden this scope. #### `storm_banner_enabled` Show incident banner · `bool` Default `true` · Accepted - · Hot-reloads Show affected providers/models, safe error categories, failure percentages, queue state and recovery timing at the top of the dashboard. Changing visibility does not reset protection. #### `storm_window` Detection window · `duration` Default `1m` · Accepted 1s..1h · Hot-reloads Rolling time range for successful and eligible failed upstream attempts. Models with no requests in this window expire from provider-wide checks. #### `storm_min_samples` Minimum samples · `int` Default `20` · Accepted 1..1000000 · Hot-reloads Minimum successful or eligible failed upstream attempts in the detection window before a model qualifies. Provider-wide protection requires every active model to qualify. Retries count as separate attempts; this is not a distinct-request percentage. #### `storm_error_percent` Failure threshold · `int (percent)` Default `50` · Accepted 1..100 · Hot-reloads Minimum percentage of sampled upstream attempts with a selected failure. Applied to each exact model after the minimum sample count; all active models must qualify for provider-wide protection. #### `storm_initial_backoff` Initial recovery delay · `duration` Default `5s` · Accepted 1ms..1h · Hot-reloads Initial storm cooldown before a recovery probe. Cannot exceed maximum recovery delay. Valid upstream Retry-After/reset hints still apply independently. #### `storm_max_backoff` Maximum recovery delay · `duration` Default `2m` · Accepted 1ms..1h · Hot-reloads Cap on exponentially increased storm cooldowns. Must be at least initial recovery delay. Does not shorten upstream Retry-After/reset hints. #### `storm_backoff_multiplier` Recovery delay multiplier · `int` Default `2` · Accepted 2..10 · Hot-reloads Multiply the storm cooldown after a failed recovery probe, up to maximum recovery delay. #### `storm_jitter_percent` Recovery delay jitter · `int (percent)` Default `20` · Accepted 0..100 (0 = no jitter) · Hot-reloads Random spread as a percentage of the storm cooldown to separate recovery attempts. 0 disables jitter. #### `storm_recovery_successes` Recovery successes · `int` Default `2` · Accepted 1..100 · Hot-reloads Consecutive successful recovery probes required before queued traffic resumes normally. #### `storm_max_queue` Maximum queued requests · `int` Default `1000` · Accepted 1..100000 · Hot-reloads Maximum requests waiting for storm recovery. Excess work is rejected locally; requests and their bounded bodies stay in process memory while waiting. #### `storm_max_wait` Maximum queue wait · `duration` Default `5m` · Accepted 1ms..24h · Hot-reloads Maximum duration of each storm gate wait, not total request duration across all retries and other queues. Caller cancellation may end the wait earlier. #### `storm_max_scopes` Maximum tracked scopes · `int` Default `2048` · Accepted 2..100000 · Hot-reloads Bound on provider and exact-model detector scopes retained in process memory. Increasing it does not establish a total process memory ceiling. #### `storm_max_retries` Additional storm retries · `int` Default `20` · Accepted 0..1000 (0 = no additional retries) · Hot-reloads Additional retries beyond max_retries only while a relevant storm is active and the failure matches a selected eligible transient condition. Does not replay meaningful emitted upstream content. 0 keeps the ordinary retry budget. #### `storm_status_codes` HTTP failure statuses · `strings` Default `- "500" - "502" - "503" - "504"` · Accepted - · Hot-reloads Exact status strings selected for detection and additional storm retries: 429 or 500..599, without duplicates. Empty disables HTTP status triggers. 429 is opt-in because limits may be credential-specific; durable quota/billing 429s remain excluded from storm evidence (quota_pause_mode owns their reaction). #### `storm_transport_errors` Transient transport failures · `bool` Default `true` · Accepted - · Hot-reloads Include transport failures already eligible for the shared retry policy. Caller cancellation and permanent transport errors are excluded. #### `storm_stream_errors` Response and stream failures · `bool` Default `true` · Accepted - · Hot-reloads Include recorded in-band, stream and quality failures to protect future sends. Reports a generic response error without retaining error bodies for the banner. Never replays meaningful emitted upstream content. ## Quota pause Provider-wide reaction to a 429 carrying a durable quota/billing error (insufficient_quota/credits, spend limits): surface it immediately, park the provider behind a recovery gate, or hold it until an operator resumes. OpenAI-compatible wire only; Anthropic and Cursor upstreams keep the off behavior. #### `quota_error_classes` Quota error classes · `strings` Default `[]` · Accepted - · Hot-reloads Operator extensions to the built-in durable quota/billing vocabulary (insufficient_quota/credits, spend and hard limits, and the numeric business codes of some gateways). A matching type or code string (case-insensitive) is a durable account condition: the ordinary retry ladder never absorbs it, quota_pause_mode owns the reaction, and it counts as an error rather than flow control. Entries add to the built-ins and never move one; a spelling already built into another class is rejected at load. Reload applies to new classifications and re-reads stored history through the same vocabulary. #### `quota_pause_mode` Mode · `string` Default `"off"` · Accepted `off`, `retry`, `manual` · Hot-reloads Reaction to a 429 carrying a durable quota/billing error (insufficient_quota/credits, spend limits; in-band occurrences included): off surfaces it immediately; retry opens a provider-wide recovery gate - the triggering request absorbs the 429 and re-sends per the storm recovery delays, new requests park on the gate (bounded by storm_max_queue/storm_max_wait), and the gate closes after quota_pause_recovery_successes consecutive successful probes (the default 1 closes on the first 2xx); manual creates an indefinite provider pause that only an operator resumes. The built-in classification is fixed (quota_error_classes may add unknown spellings); only the reaction is configurable. OpenAI-compatible wire only; Anthropic and Cursor upstreams keep the off behavior. #### `quota_pause_recovery_successes` Recovery successes · `int` Default `1` · Accepted 1..100 · Hot-reloads Consecutive successful recovery probes that close a retry-mode quota gate. 1 releases the queue on the first 2xx. Meaningful only in retry mode. ## Conversations How the request log groups turns into conversations. #### `conversation_idle_gap` Idle gap · `duration` Default `30m` · Accepted > 0s · Hot-reloads A gap longer than this starts a new conversation for the same client+key. Generous so thinking pauses inside one task don't split it. Must be > 0. #### `conversation_max_open` Max open · `int` Default `64` · Accepted 1..100000 · Hot-reloads Cap on concurrently-open conversations per client+key. Oldest is evicted past it. #### `sub_conversations` Sub-conversation tracking · `sub_conversations` Default `[]` · Accepted - · Hot-reloads Per-client tracking of a request body field that carries a sub-conversation identity, exemplified by opencode's promptCacheKey. Each entry names one classified client exactly (no wildcard, no case folding) plus 1..4 exact JSON field names matched only at the request body's top level; the first present field decides (a duplicate top-level key drops the identity): a present value that is not a JSON string value counts as absent and the next field is checked, while a present string that cannot be decoded or fails the identity rules (non-empty after trimming, valid UTF-8, free of control bytes, at most 512 bytes) drops the identity for the request - no later field is consulted - and a dropped identity is never a client error. The drop is conservative: a value whose raw spelling exceeds the bounded scan window is dropped even when its trimmed form would pass. The value groups the request under a k: conversation identity, after the X-Proxy-Session header (s:) and before automatic client+key grouping; X-Proxy-Parent-Session still owns parent linkage. Requests translated to the Anthropic wire gain a top-level prompt_cache_key carrying the value. strip: true removes the entry's configured fields from the top level of the client body before relay or translation, including a Responses client body; the default false forwards the body unchanged. Unlike the request-overrides body fields, strip is not limited to Chat or Anthropic inbound. Param names are exact printable-ASCII JSON key segments up to 64 bytes, no quotes or backslashes; duplicate clients and duplicate params within an entry are rejected. Caps: 16 entries, 4 params per entry. Default: an empty list, tracking fully off. Hot-reloads. ## Format translation Cross-protocol translation policy, streamed tool-call bound, Anthropic defaults and Cursor agent.v1 bridging. #### `anthropic_default_max_tokens` Anthropic default max tokens · `int (tokens)` Default `4096` · Accepted 1..1000000 · Hot-reloads Injected into translated Anthropic requests when the client sent none of max_tokens, max_completion_tokens or max_output_tokens. #### `stream_tool_calls_max` Max streamed tool calls · `int` Default `1024` · Accepted 1..10000 · Hot-reloads Most tool calls one translated response stream may carry: concurrent open blocks for Anthropic and Responses, distinct calls for Chat (its stream has no per-call close event). A stream past the bound ends with a translation_error. Raise it only for providers that emit unusually large parallel tool batches. #### `translation_loss` Translation loss policy · `string` Default `"warn"` · Accepted `warn`, `reject` · Hot-reloads What a cross-protocol translation does when the target cannot represent a client field. warn (default) drops optional fields and reports them in X-Proxy-Warnings; reject fails the request with 400 naming the fields before any upstream send and refuses every recorded loss, including a documented approximation. Structural features are always rejected when they cannot be honored. A response-side loss is never fatal: a non-streaming response reports it in X-Proxy-Warnings, and a streaming response drops it without a header. Hot-reloads. #### `cursor_default_context_window` Cursor default context window · `int (tokens)` Default `0` · Accepted >= 0 (0 = omit (client default)) · Hot-reloads Optional context-window hint stamped onto every model in GET /v1/models when X-Proxy-Format is cursor. 0 omits the field. #### `cursor_park_ttl` Cursor park TTL · `duration` Default `3h` · Accepted > 0s · Hot-reloads How long a parked Cursor Run stream stays open waiting for tool results. Must be > 0. Expiry closes the stream; the next continuation cold-starts. #### `cursor_heartbeat_interval` Cursor heartbeat · `duration` Default `5s` · Accepted 1ns..10m · Hot-reloads How often the proxy pings a Cursor agent.v1 run (idle parked runs included). Range 1ns..10m. Applies to runs created after reload. ## Storage Durable SQLite write pipeline. Restart to apply. #### `storage_write_chan_cap` Write channel cap · `int` Default `8192` · Accepted 1..1000000 · Restart required Buffered records before overflow drops. Restart required. #### `storage_batch_cap` Batch cap · `int` Default `256` · Accepted >= 1 · Restart required Max records per SQLite transaction. Cannot exceed the write channel cap. Restart required. #### `storage_flush_interval` Flush interval · `duration` Default `500ms` · Accepted > 0s · Restart required Partial-batch flush cadence. Must be > 0 (a zero ticker panics). Readiness convicts a wedged writer after one interval plus a progress window of four intervals clamped to 30s..2m. Restart required. #### `storage_query_timeout` Query timeout · `duration` Default `10s` · Accepted > 0s · Restart required Bounds dashboard SELECT queries. Must be > 0. Open and AggAPI bind at start (restart required); pause/throttle persist reads the live snapshot. #### `storage_query_max_bytes` Query result bytes · `bytes` Default `8MiB` · Accepted 4KiB..64MiB · Restart required Maximum JSON bytes returned by /metrics/query; also bounds SQLite string/blob values. Oversized results fail with HTTP 413, never truncate. Does not cap all SQLite working memory. Restart required. #### `storage_query_max_rows` Query result rows · `int` Default `10000` · Accepted 1..100000 · Restart required Maximum rows returned by /metrics/query. Oversized results fail with HTTP 413; use SQL filters or LIMIT. Does not limit dashboard aggregates or exports. Restart required. ## Backup Download or restore a self-checked archive of the saved config, the SQLite history, or both. Restore inspects first and the apply is confirmed. Merge keeps live defaults and live request ids; replace loads and validates the backup, writes canonical YAML, and preserves CLI-overridden listen and db_path. #### `backup_max_bytes` Backup size cap · `bytes` Default `1GiB` · Accepted 1MiB..4GiB · Hot-reloads Maximum size of one Settings backup download or restore upload. Larger archives are refused. Downloads and restores stream through private staging files; this is the admit/reject ceiling, not a heap guarantee. ## MCP The operator-gated /mcp LLM tool endpoint: mcp_enabled registers the route or keeps it reserved (404, never inference), and the limits bound what its tools return to a model. Restart to apply; the route table and the endpoint's services are built once at boot. #### `mcp_enabled` Enable MCP endpoint · `bool` Default `true` · Accepted - · Restart required Register the streamable HTTP MCP endpoint at /mcp for LLM tool clients, operator-gated like /metrics and Bearer-only: a session cookie alone is refused. Off keeps the route reserved (404, never inference); the /mcp/ look-alike reservation and the operator gate stay either way. Restart required. #### `mcp_page_size` Page size · `int` Default `50` · Accepted 10..500 · Restart required Default page for the records, values and audit capture listings the endpoint serves to a model. Restart required. #### `mcp_query_max_rows` Query result rows · `int` Default `200` · Accepted 1..100000 · Restart required Default client-side row limit when a query call omits max_rows. An explicit max_rows may raise it up to this setting's maximum; the proxy's own storage_query_max_rows still answers 413 first. Restart required. #### `mcp_query_max_bytes` Query result bytes · `bytes` Default `128KiB` · Accepted 1B..64MiB · Restart required Client-side cap on the encoded size of one query tool result. Keeps whole rows, truncating with an explicit marker rather than silently. Restart required. #### `mcp_capture_max_bytes` Capture max bytes · `bytes` Default `256KiB` · Accepted 1B..64MiB · Restart required Size above which a stored capture document is withheld whole from the audit_capture_get tool; a document that large is more than a model should read in one call. Restart required. #### `mcp_query_timeout` Query timeout · `duration` Default `2m` · Accepted > 0s · Restart required Bounds one proxy read the endpoint makes that is not the proxy's own bounded SQL reader (a full-history chart or explorer fold can take seconds). Must be > 0. Restart required. #### `mcp_timeout` Request timeout · `duration` Default `30s` · Accepted > 0s · Restart required Bounds every in-process call the endpoint's tools make to the proxy, including mutations. Must be > 0. Restart required. ## Dashboard Live-view cadence and request-log window. Hot-reloads; open dashboards pick up the next tick. #### `dash_log_rows` Request log rows · `int` Default `60` · Accepted 10..500 · Hot-reloads Page size of the live request log (newest first). Scrolling near the bottom loads more, including older durable history. #### `dash_poll_interval` Poll interval · `duration` Default `5s` · Accepted > 0s · Hot-reloads KPI / pause / SSE-fallback poll cadence. Must be > 0. #### `dash_chart_refresh` Chart refresh · `duration` Default `15s` · Accepted > 0s · Hot-reloads How often the traffic chart re-fetches server-authoritative buckets and period totals. Must be > 0. #### `dash_explorer_stale` Explorer stale after · `duration` Default `15s` · Accepted > 0s · Hot-reloads How old a cached explorer breakdown may be before it is re-fetched. Must be > 0. #### `dash_background_refresh` Background refresh · `bool` Default `false` · Accepted - · Hot-reloads Keep the dashboard current while its tab is hidden: the poll runs at the browser's throttled background cadence (about one wake per minute on Chromium) and a Web Lock keeps the SSE feed from being frozen. Desktop Chromium only; Android and iOS still suspend background pages at the OS level. Off keeps the current pause-while-hidden behavior. ## Models Model grouping rules - an ordered rewrite pipeline merging spelling variants of the same model in every grouped surface. Records keep their exact spelling; hot-reloads. #### `model_rules` Model grouping rules · `model_rules` Default: ```yaml - {mode: lower, from: "", to: ""} - {mode: pattern, from: "^(?:[a-z0-9][a-z0-9._-]*/)+", to: ""} - {mode: pattern, from: "(?::[a-z0-9._-]+)+$", to: ""} - {mode: pattern, from: "(?:-(?:[a-z]{0,2}fp\\d+|bf\\d+|int\\d+|nf\\d+|[a-z]?q\\d+(?:_[0-9a-z]+)*|gptq|awq|gguf|exl2|mlx|onnx|ggml|fast|speed|value|free|preview|latest|beta|alpha|stable|experimental|thinking|reasoning|xhigh|high|medium|low|minimal|(?:\\d{4}|\\d{8})))+$", to: ""} - {mode: pattern, from: "(\\d)\\.(\\d)", to: "$1-$2"} ``` Accepted - · Hot-reloads Capped at 64 rules. Ordered rewrite pipeline that merges spelling variants of the same model in every grouped surface (explorer model dimension, scope filters, debug checklist, the Logs download and Settings delete-filter model optgroups, and the request-log leaf’s displayed name). Each rule is one step: exact (whole-string merge old → new), pattern (regex rewrite of all occurrences, $1 capture refs), lower (fold case). Rules apply once in order; the shipped default folds case, strips one or more leading `vendor/` namespaces, strips trailing `:tag` deployments, strips a trailing run of serving decorations (quantization or format such as `-fp4`, `-q4_k_m`, `-gptq`; service tier such as `-fast`, `-speed`, `-value`; release channel such as `-preview`, `-latest`; build stamp such as `-1017`), and unifies `.` with `-` between digits. Model tiers such as `-flash` or `-pro`, fine-tunes such as `-abliterated` or `-rl`, and parameter counts stay distinct. An explicitly empty list groups by the exact stored spelling. Stored values remain unchanged; purge/export match them exactly and request details show the original. Debug matching uses separate native base-model normalization, not these display rules. Hot-reloads. ## Providers Optional per-provider JSON field-name maps: usage/cost response keys, the models metadata endpoint merged into /v1/models, ensured tool names injected into relayed chat bodies, and optional upstream wire headers. Empty = auto-detect. #### `providers` Provider field maps · `providers` Default `{}` · Accepted - · Hot-reloads Per-provider cost/usage JSON field paths, optional models_path/models_keys enrichment, ensured tool names and headers (a string map). Field names only, never static prices. Custom cost_keys must report USD; leave cost_in_usd_ticks to automatic unit conversion. ensure_tools names function tools injected as inert stubs into relayed Chat Completions client bodies on a Chat outbound wire missing them. When the client sent no tools of its own, an absent, null, or auto tool_choice is forced to none; an explicit none stays; a required, named, or malformed choice skips injection entirely. A first-party client's tool-signature gate still recognizes the request. Header values expand {{uuid4}} to a fresh UUID, {{platform}} to the server's Rust-style os; arch pair, and {{opencode-msg-id}}/{{opencode-ses-id}} to fresh opencode-format identifiers; unknown templates stay literal. On HTTP relay requests, configured headers override forwarded/extracted-auth values and X-Proxy-Headers wins. accept-encoding and range are proxy-owned: the X-Proxy-Headers map rejects them with 400, and configured or override values are stripped from the final request. Model discovery and the Cursor bridge construct headers separately and do not apply X-Proxy-Headers. Headers can contain secrets; keep runtime configuration private. #### `provider_aliases` Provider aliases · `aliases` Default `{}` · Accepted - · Hot-reloads Map an old provider label to a canonical label. Rewrites existing stored history at boot and on reload; new requests use the canonical label. Existing in-memory records keep their prior label until they age out or the process restarts. Removing a mapping does not restore rewritten history. Chains and self-maps are rejected. ## Request overrides Scoped rewrites of the upstream request: set, replace or remove HTTP headers and set output-token ceilings per client, provider and/or model. Rules match exactly, all matching rules apply in list order, and the later rule wins. Default: an empty list, the feature fully off. #### `request_overrides` Request override rules · `request_overrides` Default `[]` · Accepted - · Hot-reloads Capped at 64 rules. Ordered rules that rewrite the upstream request before relay. A rule matches when every non-empty scope field equals the request's value exactly: client as the classified client, provider as the canonical label after provider_aliases, model as the recorded model id; an empty scope field matches anything. All matching rules apply in list order, and for the same header or body field the later rule wins. headers sets or replaces upstream headers (names are canonicalized, values must be non-empty single-line), remove_headers deletes them, and body sets max_tokens / max_completion_tokens (1..1000000) on a Chat Completions or Anthropic client body. Credential and protocol headers are rejected in an override at load (provider headers may use them; they apply after the credential). Credential class is judged by role, not by a list of vendor spellings: any name carrying authorization/auth, api-key, secret, password, credential, bearer, jwt, signature, cookie, key or token is refused, so x-auth-key, x-amz-credential, x-functions-key and x-subscription-key are covered. Identifier and accounting shapes stay admitted: key-id, cache-key, idempotency-key, and the token counters (x-prompt-tokens, x-token-count) plus the x-ratelimit-* and anthropic-ratelimit-* families. Protocol and framing names (host, content-type, content-length, transfer-encoding and the rest) and the whole x-proxy- control prefix are refused as well. The override applies after provider headers and the client's X-Proxy-Headers map and wins over both, with two enforced exceptions: the target's resolved auth header and Authorization are never set or removed by an override, and Accept-Encoding and Range are stripped after the override merge because the transport owns them. It covers inference sends only, never model discovery or token refresh. Body rewrites apply to Chat Completions or Anthropic client bodies before translation, whatever the target (a Responses client body is not rewritten), and re-stamp the recorded request cap; the Cursor bridge reads no body fields, so body is skipped for cursor targets while header rules still apply. An empty list leaves every request untouched. Hot-reloads. ## Proxy CLI flags The flag set in `cmd/proxy` owns these blocks. Every flag is parsed before any configuration is loaded, so the default below is the registration default, not the effective value after a config file or another override. #### `-config` `string` Default `"proxy.yaml"` path to config file (empty for built-in defaults) #### `-print-config` `bool` Default `false` print documented built-in defaults as YAML and exit without loading configuration #### `-print-example-config` `bool` Default `false` print the documented example with enabled provider profiles and exit without loading configuration #### `-version` `bool` Default `false` print application and source build metadata as JSON and exit without loading configuration #### `-listen` `string` Default `""` override listen address (e.g. 127.0.0.1:8081); keeps instances on distinct ports #### `-db-path` `string` Default `""` override storage DB path ("" = config value; "none" = disable durable storage) #### `-pid-file` `string` Default `""` write the process pid to this file after boot #### `-healthcheck` `bool` Default `false` probe GET /healthz on the configured listen address and exit nonzero on failure (Docker HEALTHCHECK); loads config read-only #### `-readycheck` `bool` Default `false` probe GET /readyz on the configured listen address and exit nonzero when storage is not ready (Compose readiness); loads config read-only --- # Containers and data This page covers the image, Compose operation, persistent volumes, backups, versions and the restart lifecycle. The settings surface is in [configuration](configuration.md). > [!WARNING] > This is an offline backup: stop callers first, allow active work to finish, > and keep the service stopped while copying. Never copy an open SQLite database > as plain files. Restore into a new project and fresh volumes, leaving the > original deployment and backups untouched. ## Containers The [Dockerfile](../Dockerfile) builds Linux `amd64` and `arm64` targets and runs as UID/GID `65532:65532`, with no source tree, Go toolchain or shell in the runtime image. JavaScript/CSS are embedded in the Go binary; Node and Python are test tools, not runtime dependencies. The binary is stripped of debug and symbol tables. The final stage copies only that binary, dependency notices, the generated example and prepared state directories onto the minimal distroless base. The [Compose quickstart](../README.md#docker-compose) downloads the image-only [compose.yaml](../compose.yaml). Normal deployment never builds from source. Local builds use the separate [development Compose](../CONTRIBUTING.md#container-development). A registry tag is usable only after its publishing workflow succeeds. ### Compose operation Keep the same Compose project name to reuse its named volumes. The normal file names the project `millivolt`; an explicit `-p` overrides it. The deployment inputs are: - `MILLIVOLT_IMAGE` (image reference) - `MILLIVOLT_PORT` (loopback host port) - `MILLIVOLT_OPERATOR_TOKEN` (the operator credential, forwarded into the container environment only) - `MILLIVOLT_STOP_GRACE_PERIOD` (container stop grace) - `MILLIVOLT_MEM_LIMIT` (container memory ceiling, default `1g`) - `MILLIVOLT_PIDS_LIMIT` (process/thread ceiling, default `512`) Their defaults live in [compose.yaml](../compose.yaml), not server configuration. Supply them consistently through the environment or an ignored local `.env` file. `docker compose config` shows the resolved deployment. The memory ceiling is also the source of the runtime's own soft limit: with `go_memory_limit` at its `0B` default, a detected finite cgroup ceiling at or above 80 MiB becomes a soft limit at 80% of itself, so the container bound reaches the Go runtime instead of leaving it unbounded. Raise the ceiling with `history_size`, request buffers, concurrency or observed identity cardinality; a ceiling under 80 MiB is ignored by the runtime and leaves only the hard OOM kill. `pids_limit` bounds process and thread amplification, not goroutines. The image carries a Dockerfile `HEALTHCHECK` that probes the unauthenticated `/healthz` through the binary's `-healthcheck` mode, so `docker compose ps` reports health without any operator credential. Both Compose files replace that probe with the [readiness check](operations.md#health-and-readiness), which is strictly stronger. ```sh # Start or adopt a newly pulled image while preserving volumes. docker compose pull docker compose up -d docker compose ps docker compose logs --tail 100 # Stop without removing containers or data. docker compose stop # Remove this project's containers and network, retaining named volumes. docker compose down ``` Do not add `--volumes` or `-v` to `down` unless deliberately deleting its configuration and history. Changing the Compose project name selects different volumes and can look like missing data. Back up before an upgrade; the existing image can be replaced, but a database migration is not necessarily reversible. See Docker's [Compose lifecycle](https://docs.docker.com/reference/cli/docker/compose/up/) and [volume removal](https://docs.docker.com/reference/cli/docker/compose/down/) documentation for the underlying behavior. Do not add a build section to the deployment file. The [development Compose](../CONTRIBUTING.md#container-development) owns source builds, its private port and its separate project identity. The Compose service drops Linux capabilities, disallows privilege escalation, uses a read-only root filesystem and a bounded temporary filesystem. These controls do not add application authentication or a tenant boundary. Access to a rootful Docker daemon through the `docker` group is effectively root access; see Docker's [Linux post-installation guidance](https://docs.docker.com/engine/install/linux-postinstall/). ### Persistent state and configuration | Container path | Persistence and permissions | | --- | --- | | `/data` | Writable database directory, including SQLite WAL/SHM sidecars. Mount the directory, not just `proxy.db`. | | `/config` | Writable private configuration directory. The image bundles `/config/proxy.yaml` from the generated example; if deliberately absent, the application uses neutral built-in defaults. | Fresh named config volumes receive the image's bundled example, owned by UID/GID `65532:65532` with mode `0600`. No separate configuration download is required. Replacing an image does not overwrite an existing saved configuration; review example-profile changes deliberately when upgrading. These bundle semantics apply to images built from this revision. Existing release images are not changed by a main-branch update; consult the documentation at the selected release tag for older images. Existing volumes and host bind mounts must be owned by the runtime UID, or the database file must already be mode `0600`. Read/write permission alone is not sufficient: startup narrows a broader database mode to `0600` and fails closed when it cannot, naming the file path, owner and current mode. A foreign-owned `0600` database is left unchanged; a broader foreign-owned file must be pre-set to `0600` or chown'd to `65532:65532`. Bind mounts hide bundled files: supply a config yourself when using one, or intentionally accept the application's missing-file defaults. Use private directory permissions and mode `0600` for a supplied config; with rootless Docker or user namespaces, account for the host UID mapping. Do not make sensitive directories world-writable to work around ownership failures. A first launch cannot prepare its own bind-mount host directories. When a bind-mount target does not exist, Docker creates it as `root:root` mode `0755`, which the runtime UID cannot write, and startup fails with SQLite's `unable to open database file` error. The log names the database path, the directory state and the runtime UID. Pre-create the host `data/` and `config/` directories owned by `65532:65532` before the first start, or `chown -R 65532:65532` them after a failed start, then start the container again. Named volumes avoid this entirely: Docker seeds them from the image with correct ownership. Settings saves by creating a temporary file beside the resolved config and atomically renaming it; a symlinked config path is followed to its final target, so the link stays in place and the target receives the document. The restore rollback publishes through the same writer, so a failed restore of a symlinked config keeps the link too. A read-only mount, or a bind mount of only the config file, cannot support that save contract. Mount the whole writable directory, or deliberately use read-only configuration and update it outside the dashboard. Do not mount the committed example as a mutable runtime config. The container listens on `:8080`; restrict publication on the host with `-p 127.0.0.1:8080:8080`. Binding to loopback *inside* the container prevents ordinary published-port access. Docker port publication does not add operator authentication. Do not share one database among independent proxy containers. Upstream URLs are reached from the proxy container, so `localhost` in `X-Proxy-Base-URL` means that container, not the Docker host. Use an upstream hostname reachable on its network, such as another Compose service name. Host-service access needs an explicitly configured host-gateway mapping or another reachable host address; it is not included in the supplied deployment. Arguments after the image replace its complete default command. If overriding a server flag, also keep `-config /config/proxy.yaml -db-path /data/proxy.db` unless intentionally selecting other paths or disabling storage. The `-print-config` and `-print-example-config` commands print their respective documents without starting a server. Replace the container to upgrade or adopt startup-bound settings; the dashboard cannot rebuild an immutable image. Keep the same volumes and back them up before upgrades. If only a saved config file changed, ordinary `up -d` need not replace the container. Apply startup-bound changes explicitly: ```sh docker compose up -d --force-recreate millivolt ``` Container stop sends SIGTERM, which cancels unfinished requests before draining storage. Allow enough stop grace time for the configured shutdown and queued writes; forced termination can prevent that drain. No stop/flush can recover records already dropped under storage overload. ### Image-only backup and restore The supplied deployment can back up both named volumes without a source checkout, Python, a shell inside the image or a helper image. This is an **offline** backup: stop callers first, allow active work to finish, and keep the service stopped while copying. Run from the deployment directory with the same Compose project and environment used to start it. If you use `-p`, repeat that selection on every Compose command. Do not concurrently replace or restart the source container. The following saves the original Compose file, the pulled image's registry digest, and both volume archives in a new private directory. It expects the supplied single-service layout, `/data` and `/config`, with a saved `proxy.yaml` present. Privately retain any local `.env` or other deployment inputs as well. ```sh ( set -eu umask 077 backup_dir="$(mktemp -d ./millivolt-backup.XXXXXXXX)" printf 'Backup directory: %s\n' "$backup_dir" container_id="$(docker compose ps -q millivolt)" test -n "$container_id" image_id="$(docker inspect --format '{{.Image}}' "$container_id")" docker image inspect --format '{{index .RepoDigests 0}}' "$image_id" > "$backup_dir/image.txt" cp compose.yaml "$backup_dir/compose.yaml" docker compose stop millivolt test "$(docker inspect --format '{{.State.Status}} {{.State.ExitCode}}' "$container_id")" = 'exited 0' docker compose cp millivolt:/data/. - > "$backup_dir/data.tar" docker compose cp millivolt:/config/. - > "$backup_dir/config.tar" docker compose start millivolt ) ``` A failed stop or copy exits without automatically restarting the service; inspect the failure before resuming. Copy the completed backup to protected independent storage. The data archive includes any WAL/SHM files, not just `proxy.db`. Never use this file-copy procedure while SQLite is open. An archive cannot recover records already dropped or writes lost through forced termination. Restore into a **new project and fresh volumes**, leaving the original deployment and backups untouched. Enter the completed backup directory and an unused loopback host port when prompted. The original supplied Compose file is required: do not substitute resolved `docker compose config` output, custom external-volume names or bind mounts, which can point the new project at existing data. ```sh ( set -eu umask 077 printf 'Backup directory to restore: ' IFS= read -r backup_dir test -s "$backup_dir/data.tar" test -s "$backup_dir/config.tar" MILLIVOLT_IMAGE="$(head -n 1 "$backup_dir/image.txt")" case "$MILLIVOLT_IMAGE" in *@sha256:*) ;; *) exit 1 ;; esac printf 'Unused loopback host port: ' IFS= read -r MILLIVOLT_PORT case "$MILLIVOLT_PORT" in ''|*[!0-9]*) exit 1 ;; esac test "$MILLIVOLT_PORT" -ge 1 test "$MILLIVOLT_PORT" -le 65535 export MILLIVOLT_IMAGE MILLIVOLT_PORT restore_dir="$(mktemp -d ./millivolt-restore-XXXXXXXX)" restore_project="$(basename "$restore_dir" | tr '[:upper:]' '[:lower:]')" cp "$backup_dir/compose.yaml" "$restore_dir/compose.yaml" restore_compose() { docker compose --env-file /dev/null -p "$restore_project" -f "$restore_dir/compose.yaml" "$@" } existing_containers="$(docker ps -aq --filter "label=com.docker.compose.project=$restore_project")" test -z "$existing_containers" existing_volumes="$(docker volume ls -q --filter "name=^${restore_project}_")" test -z "$existing_volumes" restore_compose create millivolt restore_compose cp --archive - millivolt:/data < "$backup_dir/data.tar" restore_compose cp --archive - millivolt:/config < "$backup_dir/config.tar" restore_compose up -d millivolt restore_compose logs --tail 100 millivolt printf 'Restored project: %s\nCompose directory: %s\nHost port: %s\n' "$restore_project" "$restore_dir" "$MILLIVOLT_PORT" ) ``` Keep the tar streams intact and retain `--archive` on restore: the private config must remain owned by runtime UID/GID `65532:65532` with mode `0600`; the restored database is subject to the same ownership and mode check at boot. Check Settings and retained history on the new private port before redirecting clients. Save the printed project name and deployment inputs for later commands; the example bypasses automatic `.env` loading during recovery. Docker documents [tar-stream copying](https://docs.docker.com/reference/cli/docker/container/cp/) and [Compose archive mode](https://docs.docker.com/reference/cli/docker/compose/cp/). The repository's isolated container check exercises stopped-volume backup and restore into fresh volumes, including settings, history and file permissions. Source deployments can instead use the live SQLite [online-backup helper](operations.md#storage-and-accounting) or the Settings archive described next. ### Settings backup and restore The dashboard Settings Backup category downloads and restores a self-checked `.mvb` archive. Choose config, database, or both. The archive is a packed SQLite snapshot (`VACUUM INTO`, so free pages are not copied) and/or the schema YAML, then zstd-compressed with a content checksum and an outer SHA-256 of the uncompressed payload. Encoding streams each member into the compressor while hashing and counting it, writes the archive once into a staging file, and self-checks that file before returning it; an archive that fails decode or semantic checks is never emitted, so a produced backup is never a silent corrupt file. The database member is read from the packed snapshot file in bounded chunks instead of being copied whole into memory, so the download cost does not grow with a heap copy of the database. A source file that changes size or cannot be read fails the encode before an archive is emitted. Restore streams the upload through the `backup_max_bytes` ceiling into one private staging file and decodes it with the same header, zstd, SHA-256 and member-framing checks. When a live store can stage it, the database member is written straight into a private file on the live database's volume instead of being materialized on the heap; a restore that selects the database member without a store fails closed before staging, and a member the request did not select is stream-discarded after its framing checks. Config bytes stay buffered within the same 16 MiB config-document bound `Config.Validate` enforces on the canonical rendered YAML, so a config that validates can never exceed what an archive may retain. A config member that declares a larger size is rejected before its bytes are copied, and encoding refuses to emit one, so an older archive whose config member exceeds that bound fails restore. The bound covers only the config member; the database member stays streamed under the payload ceiling. Restore validates that hash, the zstd frame, member framing and config `Validate()` before applying anything. A selected database member is decoded into its private staging file first, then must pass `PRAGMA integrity_check`; only that checked file is published as the pending snapshot, which restore then preflights for admission before the config member is written. Decode holds the uncompressed payload to the same `backup_max_bytes` ceiling as the upload (and to the Schema maximum, whichever is lower), so a small compressed file cannot expand past what the operator configured; an over-cap payload answers 413 with the upload's wording and stages no pending snapshot. Download lists how many live settings differ from default and how many stored requests would be packed. Picking a file first inspects the archive: created time, size, member sizes, each config key as default vs modified vs the live file, and the database request span when a live store can stage the member. Inspect loads the live file through the same strict, bounded reader used by config reload and restore, not the boot repair path: an unknown nested field, an extra YAML document, a non-regular file, or a file above the 16 MiB document bound fails the inspect instead of comparing against a repaired or partial view. Inspect decodes and counts the database member in place and computes its overlap with the live store without writing another copy. Without a live store it reports `present:true`, the decoded `bytes`, `inspectable:false` and the reason, and produces no request span. Config **replace** loads and validates the backup, writes canonical YAML, and preserves CLI-overridden `listen` and `db_path`. Config **merge** overlays only keys that differ from `Default()` onto the live file, so live custom values stay unless the backup also customized them. An omitted `config_mode` or `database_mode` defaults to replace; an explicitly empty or whitespace value is rejected as malformed rather than treated as absent. Either way the restored config hot-applies through the same owner as a Settings save, is reflected in `last_reload`, and reports restart-required keys. Database **replace** streams the decoded file into a pending snapshot next to `db_path`, syncs it, and atomically renames it into place; Open admits the pending file only after it checks again, and the restore waits for process restart. A `db_path` that is a symlink is resolved to its real target first, so staging, admission, the rollback set and the state marker all belong to the file SQLite actually opened; the link stays intact and points at the replacement. A combined merge compares the same resolved identities rather than the two spellings. Database **merge** attaches the decoded file read-only and inserts request ids that are not already in the live store immediately (live rows win on conflict); the attached snapshot rejects writes, while the live database stays writable for the merge, and no restart is required. Its request rows, debug sidecars and since-inception totals commit as one transaction, so a failed totals write leaves no partial merge behind. Truncated or bit-flipped files are rejected. The `backup_max_bytes` setting is the encoded-archive and the decoded-payload ceiling alike; encoding enforces it while the stream is written, so an over-cap archive is removed instead of partially served, an over-cap upload answers 413 before decoding, and a decoded payload over it answers 413 before staging a snapshot. Restore phases are ordered, not atomic. The database member runs first: a replace stages the decoded file and preflights it with the same schema, writer-INSERT and request-scan value checks boot runs before touching the live database, bounded by the same maintenance budget as boot maintenance, and a merge commits rows into the running store. One restore owns the whole transaction from the first uploaded byte, so two restores cannot stage and decode two full members on the database volume at once; an inspect shares that serialization because it stages the same members. Only then is the config written and hot-applied. A failed stage, preflight, or merge leaves the config file and live snapshot unchanged, and a failed config write or reload rolls the config back and restores a replace's previous pending slot. A successful live merge has no rollback: if the later config phase fails, the merged rows stay applied. A crash between the database and config phases is likewise not atomic; the pending snapshot or completed merge can be durable while the config file still has its previous bytes. Preserved `.incoming`, `.incoming.invalid-`, `.incoming.invalid-interrupted-`, `.previous`, `.snapshot-state`, and `.millivolt-restore-pending-*` files next to `db_path` are boot-recovery evidence and may need operator inspection after such a crash; do not remove them while a restore may have been in flight. Quarantine names carry a unique token and keep their sidecars beside them, so repeated rejections accumulate instead of overwriting the previous one. The `.millivolt-restore-pending-*` copy is a hard link to the snapshot that was queued before the restore; it is created before the successor replaces the slot, so the pending queue is never empty, and a crash on either side leaves a recoverable snapshot instead of an orphan that the 24 hour staging sweep eventually deletes. A volume without hard links (exFAT, vfat, and several FUSE or SMB mounts) rejects `link(2)`; there the slot is moved aside instead, still without copying the snapshot bytes, and a crash leaves only the preserved sibling, which boot re-admits. When the stage fails before the successor replaces the slot the restore drops the copy again, and when a boot finds a rejected successor or an empty slot it renames the preserved copy back onto the slot and admits once, so the queued snapshot is neither hidden nor swept. A combined config and database merge answers 400 when the restored config's effective `db_path` differs from the running store's file, because the rows would be written to a database the next boot does not open; merge the database separately or use database replace. At boot, a pending snapshot is admitted only after the same schema, trigger, writer-insert, request-scan value and capture-id preflight `Open` runs. The trigger fence rejects every unowned trigger in the snapshot, not only those attached to `requests`: a trigger body is arbitrary SQL, so one attached to `meta` or `request_debug` can delete or rewrite request rows. The capture-id fence rejects a `request_debug` id that is NULL, non-text or blank: such a row can never be fetched by id or reached by the expiry sweep, yet still counts against the debug byte budget. Restore merge applies the same fence before it imports any row. Recovery consults the durable `.snapshot-state` marker rather than inferring a transition from the files present. A markerless rollback set from a pre-marker release cannot be classified: when a live main exists, recovery keeps it and removes the `.previous` debris. Discarding a non-empty `.previous-wal` in that markerless state logs a startup warning, because its committed frames cannot be distinguished from debris and may be lost. Marked pending and committed recovery behavior is unchanged. Both directions stage database-sized transient files in the live database's own directory, so capacity follows the data volume rather than a hardened deployment's small `/tmp` tmpfs. Budget roughly one compressed upload plus one packed database copy during a restore, and for a replace also the pending copy at `db_path`. The download's `VACUUM INTO` snapshot directory is removed after the archive is encoded, including on failure or a client disconnect. A restore removes its private upload file and decoded database directory after inspect or apply on every path: a normal response, an invalid or over-cap archive, a validation or apply failure, or a client disconnect. A config-only upload may use the process temp directory because it carries no database-sized member. A packed file already on the staging volume (VACUUM dest, decoded member, pending `.incoming`) is integrity-checked in place, and inspect, overlap, replace and merge consume it by path, so none writes a second database-sized copy. A container's `/tmp` is a small tmpfs by design and never caps backup size. ## Versions and images [VERSION](../VERSION) is the application release source. The dashboard asset fingerprint is separate: it detects changed embedded frontend bytes, not a semantic release or a complete backend change. `main` is promotion-only. Full source, browser and live MCP integration, and native container checks run on all `testing` pull requests, code-bearing `testing` pushes, manually dispatched `testing` gates, and manually pushed `v*` tags; docs and site-only pushes use `docs.yml`. They also run on Dependabot's `main` security PRs. An ordinary promotion pull request from `testing` requires its head to be the current `testing` tip, `main` to be already merged back into `testing`, and a successful full-gate record for that exact commit: the canonical `check.yml` push run's own check suite, with `unit`, `browser` and both native container legs. When a final docs-only commit follows the closest gated `testing` commit, it reuses that gate if every path since it is docs/site-only; the fallback selects the closest gated testing commit from canonical `testing` push runs, with no commit-count bound. The proof polls pending exact check runs a bounded number of times and fails closed if they are still pending. For a code-bearing head, let the `testing` `check.yml` push gate finish before opening or re-running a promotion PR. A docs/site-only head has no canonical `check.yml` push run of its own, so wait for `docs.yml` or only the `required` docs validation. It does not rerun the suite. A code-bearing promotion must also have a `VERSION` newer than every existing release tag. Docs/site-only promotions skip the version and full-gate proofs, validate docs, workflow policy and the site in `required` within its ten-minute budget, and do not rebuild images or cut a release. Merge with a merge commit. The merged-PR release workflow tags that reviewed merge commit and publishes the image; a delivered code-bearing main push is an idempotent fallback. It never pushes `main`. Dependabot security pull requests are the only other exception and normally do not cut a release by themselves; an untagged pending `VERSION` can publish. A manually pushed `v*` tag ships exactly as tagged. If the version tag already points at the released commit but its GitHub release is missing, the publish job fails closed with a repair instruction instead of reporting a successful re-run. Repair the existing tag's release after verifying the image; the job never moves a tag or republishes images. The full maintainer checklist lives in [Contributing](../CONTRIBUTING.md#branch-workflow). `go run ./cmd/proxy -version` (or `/millivolt -version` in the image) prints JSON with `version`, `revision`, `modified`, and `go_version`. Unavailable VCS metadata is reported honestly as revision `unknown` or `modified:null`. The registry target is `ghcr.io/llm-4-people/millivolt`. Check the [container package](https://github.com/LLM-4-People/millivolt/pkgs/container/millivolt) and [image workflow](https://github.com/LLM-4-People/millivolt/actions/workflows/check.yml) for what has actually been published; a version in a checkout does not prove an image exists. Use a published digest to pin exact bytes. Version tags identify releases but are not intrinsically immutable; moving branch and stable aliases can change. The [workflow](../.github/workflows/check.yml) runs source, race and browser checks, then native container smoke checks on Linux `amd64` and `arm64` for all `testing` pull requests, code-bearing `testing` pushes, manually dispatched `testing` gates, and manually pushed `v*` tags, and for Dependabot's `main` security PRs. The `unit`, `browser` and `container` jobs start together. Docs and site-only pushes use `docs.yml` instead. A normal promotion PR proves the current testing tip, merged-back main and the full-gate record for that commit, or for the closest gated testing commit with only docs changes after it, and requires a successful [escalation](../.github/workflows/escalation.yml) run (every Go test under the race detector plus a five-run clock-sensitive flake sweep) for the reviewed head; docs-only promotions validate the docs, workflow policy and site directly. The merged promotion PR carries the reviewed merge commit and releases it; a delivered code-bearing `main` push only reuses the same reviewed merge and is idempotent. Publication is limited to this repository's merged promotion pull requests, delivered pushes to `main`, matching release tags, and `testing`-branch pushes (which publish only the moving `testing` tag). The release `publish` job waits for `required`. The `publish-testing` job waits for `unit`, `browser` and `container` instead, because a testing push has no promotion proof to wait for. A release run whose `VERSION` is already tagged, such as a Dependabot security merge, normally does not publish; if main carries an untagged pending `VERSION`, that run can publish it. Open or unmerged pull requests do not publish images; a merged promotion PR does. | Image tag | Meaning | | --- | --- | | `main` | Moving production image for the released current `main` revision; updated by a non-prerelease versioned release cut from a merged promotion PR or a delivered main push. A prerelease does not move it, and a manually pushed release tag does not either. | | `testing` | Moving development image after a successful testing-branch push; never a release. Its OCI version label reports the embedded `VERSION`, which can lag the latest release. | | `sha-` | Release tag for the exact cut revision built and published; the version bump is part of the reviewed promotion merge, so this is a main commit. Testing pushes do not publish it, while a manually pushed tag publishes it for the tagged revision. | | Version from `VERSION` | Created by the matching `v`-prefixed release tag on the promotion merge commit, or by a manually pushed tag. | | `latest` | Moving stable alias updated by every non-prerelease release, including a release cut by a merged promotion PR, a main push or a manually pushed tag. Prereleases and testing pushes do not update it. A tag push moves the version, `sha-` and `latest` tags only; `main` stays where the last main-branch release put it. | Release builds request an SBOM and max-mode provenance. The image and its attestations are published before the tag and release are updated; the workflow never pushes `main`. A failed image build does not leave a public release pointing at a missing image. Verify the published manifest and attestations with `docker buildx imagetools inspect`, verify the repository-associated artifact attestation with `gh attestation verify oci://ghcr.io/llm-4-people/millivolt: -R LLM-4-People/millivolt`, and pin the digest for deployment. A tag name alone is not an integrity check. Set `MILLIVOLT_IMAGE` to the desired full image reference or published digest before the Compose pull/up commands. The `0.x` version series is initial development, not a promise of stable APIs across releases. GitHub initially publishes GHCR packages privately, even when their repository is public. A package administrator must make the package public separately and verify an anonymous pull. Repository linkage grants workflow access, not public visibility; see GitHub's [package access documentation](https://docs.github.com/en/packages/learn-github-packages/configuring-a-packages-access-control-and-visibility). ## Reload, restart and shutdown `SIGHUP`, `POST /admin/reload`, a successful Settings save, and a backup config restore use the same hot-reload owner. Invalid reloads preserve the running config. Reloadable transport/scheduler settings affect new work without closing active transports. Startup-bound consumers keep their boot settings until restart. Settings reports `restart_required`; its effective configuration snapshot can contain new values that those consumers have not adopted yet. The schema/example identify which fields require restart. Rebuild & restart requires Go on PATH and the original source tree available at its compiled-in path. Moved/trimpath builds may report it unavailable. It builds the working tree exactly as it is: uncommitted, unstaged and untracked files are compiled, so a restart is not a review gate and a dirty tree ships as-is. The rebuilt binary is written into the process temp directory, which must be writable (`TMPDIR`, or `/tmp` when unset); an unusable directory is refused with that directory named rather than as a build fault. The choreography builds first, stops acceptance while retaining the listening socket, drains active handlers, flushes storage, then hands the socket to a ready child. A flush failure aborts and resumes the old process. A drain timeout still flushes and hands off: the in-flight requests are cut, but everything finalized before them is durable before the child opens the database. The flush has its own bound rather than sharing the drain window. The handoff also takes the config transaction lock for its whole flush fence, so a start while a reload, Settings save or backup restore is running is refused with `409` naming the operation instead of queueing behind it; the fence itself is bounded by `restart_drain_timeout` plus the flush bound. A restore holds that lock across its streaming upload, so `POST /admin/reload` and a Settings save answer `409` while one is in flight rather than waiting for a client-paced upload to finish. Parked native-adapter runs are process-local and do not survive a successful restart. Ordinary SIGINT/SIGTERM shutdown is different: it cancels active stream contexts, then shuts down HTTP and drains/closes storage. Do not describe it as preserving unfinished streams. Already-dropped records cannot be recovered by any flush. Starting another matching proxy executable on an occupied port can trigger same-binary port reclamation (SIGTERM, then escalation). This is not the graceful rebuild workflow; choose distinct ports and databases for separate instances. --- # Routes and debugging This page covers the operator and data routes, access control, pause and limits, error storm and quota protection, debug and preview capture, export and deletion, restart, and the dashboard data and request inspector. > [!WARNING] > A genuinely bodyless `POST /admin/purge` deletes the whole history, and a > restore applies data before the config phase; both are irreversible without a > backup. Exports, debug captures and retained headers can carry sensitive > content. ## Operator and data routes The routes below belong to millivolt. They are operator-gated unless the row is marked open; the canonical open-path list is in [operator access](#operator-access). Read handlers enforce their supported methods; wrong-method requests must not fall through to inference. The namespace is standardized: `/metrics/*` and dashboard routes are the protected read plane, and `/admin/*` is the operator action plane; the credential gates both. | Route | Action / important contract | | --- | --- | | `GET /healthz` | Unauthenticated liveness probe for Docker HEALTHCHECK and load balancers. Reports process/HTTP liveness only, no storage depth. Trailing-slash look-alikes (`/healthz/...`) are reserved and answer 404, never inference. | | `GET /readyz` | Unauthenticated readiness probe. `200` when the durable pipeline can persist; `503` with a reason when the writer is wedged or a flush failed since boot. Compose uses it through `-readycheck`. Trailing-slash look-alikes are reserved and answer 404, never inference. | | `GET /favicon.ico` | Unauthenticated ICO brand mark. Browsers fetch this without Authorization; it is registered on the mux so it never reaches the inference catch-all. | | `GET /favicon.svg`, `/apple-touch-icon.png`, `/icon-*.png` | Ungated SVG/PNG icons for tabs, home screens and the web app manifest. | | `GET /manifest.webmanifest` | Ungated web app manifest (name, standalone display, 192/512 icons). | | `GET /sw.js` | Ungated service worker. Precaches brand icons/manifest; cache-first only for a `/dash/` URL carrying exactly the stamped version, network-first with cache fallback for unversioned or stale `/dash/` URLs and brand paths. Navigations to `/` are fetched live (never cached: login vs bootstrap) with a static offline fallback. Its never-cache list is generated at load from the same canonical namespace table that drives the gate and the reserved routes, so the three cannot drift; it never intercepts `/`, `/index.html`, `/healthz`, `/readyz`, `/models`, `/sw.js`, `/admin/*`, `/session/*`, `/metrics/*`, `/mcp/*` or `/v1`. | | `POST /session` | The one open operator route: exchanges the credential for the session cookie the dashboard's live feed needs. Throttled like every gated route. Lives outside `/admin` so an external authentication layer can front the admin plane without gating millivolt's own login. | | `GET/POST /admin/config` | Schema/file/effective state; save `{revision,values}`. Stale revision returns 409, and so does a save while a restore holds the config transaction (its upload is client-paced). Saved-but-reload-failed is explicitly reported. GET and save responses carry a `last_reload` section: `ok`, `error`, `dropped_keys`, `restart_required`, `at`. GET returns provider header values verbatim, because the settings sheet must round-trip them on save; redaction applies to the model-visible `config_get` tool and the operator CLI instead (see [cli](cli.md) and [mcp](mcp.md)), where `--reveal` is the opt-in. Posting a redacted document back is safe: a credential header whose value is the marker keeps the loaded secret, while a posted real value replaces it. The same applies to `set_config`, so a model can read and re-post without erasing credentials. | | `POST /admin/reload` | Re-read config and report restart-required keys. Answers 409 while a restore holds the config transaction. | | `GET/POST /admin/restart` | Status / rebuild. `GET ?watch=1` streams progress; concurrent starts are rejected, and so is a start while a config transaction (reload, Settings save, backup restore) holds the reload lock: `409` names the operation. | | `GET/POST /admin/pause` | Inspect/add/edit/resume holds. POST requires `paused`; optional ID targets one hold. GET reports `queued` (admission and send-gate waiters, the populations an operator drain can release) and `storm_waiters` (requests parked by the storm/quota gate before admission, which only the gate opening or a quota resume frees). | | `GET/POST /admin/quota` | Inspect storm state (shared with the banner) / `POST {"provider","resume":true}` closes a retry-mode quota gate; parked sends resume immediately. | | `GET/POST /admin/debug` | Inspect/add/edit/stop capture sessions. POST requires `enabled`; starting a session requires durable storage (503 `durable storage is disabled` otherwise); optional ID targets one session. | | `GET/POST /admin/throttle` | Inspect and edit limits. A policy is named by its opaque `id` or by its selector. Scope dimensions `providers` (the singular `provider` field is still accepted), `models`, `clients`, `key_hashes` and `conversations` intersect; an empty dimension matches any value. An optional schedule uses `days` (0 = Sunday), `start_minute` and `end_minute` in server-local time; equal minutes mean all day and a start after the end spans midnight. POST accepts `concurrency`, `requests`/`request_window`, `tokens`/`token_window`, `anti_burst` and `ramp_duration`; supplied limits merge. A count requires its own window only when the policy has no stored window for that dimension: a fresh count without one is refused, while a count may raise an already windowed dimension alone. A count of `0` clears that dimension and its window. A window without a count is a refused no-op unless a count is already stored. `clear:true` drops the whole policy. Anti-burst and its ramp require a single-provider policy without a schedule. GET also returns `known_providers`, `known_clients` and `known_models` for selector pickers. | | `POST /admin/queue/drain` | Release parked requests without waiting for their limit, pause hold or retry window, covering both the admission queue and the send gate. Requires exactly one scope: `all:true`, `provider`, `model` or `client`. Released requests answer with a retryable 503, never a provider error; a request already exchanging bytes with the provider finishes normally. | | `GET /admin/debug/capture?id=` | Load an unexpired durable debug sidecar; absent/no-store returns 404. `&download=1` serves the same document as a gzip artifact named `millivolt-debug-.json.gz`, falling back to the record id when `captured_at` does not decode. | | `POST /admin/purge/count` | Preview the same deletion/export predicate; traffic can change the count afterward. A genuinely bodyless POST counts the whole set, mirroring the bodyless delete; otherwise the body is `{"filter":{...}}`. Filters are body-only, so any query key is rejected with 400 before the body is read. | | `POST /admin/purge` | Delete matching finalized records. A genuinely bodyless POST deletes everything: that is the deliberate full-history command, it answers `200` rather than `404`, and any query key must be moved into a body instead. Otherwise the body is `{"filter":{...},"reviewed_count":N}`: an empty/invalid filter is rejected, and a supplied reviewed count is counted again inside the delete transaction, refusing with 409 when the row set moved. `status_code=0`, `debug=0` and `has_error=0` mean "no constraint" on that dimension, not a match for zero or false. The response is `{"deleted":N,"buffer_removed":N}`. Filters are body-only: any query key is rejected with 400 before the body is read, so `?provider=...` can never broaden the no-body full delete. | | `GET /admin/backup` | Download the self-checked `.mvb` Settings backup archive of config and/or database members. `?config=1` and `?database=1` select members, defaulting to both when no key is given; an explicit empty selection is an error. The query grammar is strict: parse errors, repeated or unknown keys, and malformed values all return 400. | | `POST /admin/restore` | Restore an uploaded `.mvb` Settings backup archive, validated before anything is applied. `?inspect=1` previews the archive (created time, member sizes, the config values plus their changed-key lists against defaults and the live config, and the database request span when a live store stages the member) without applying anything. A store-less preview still reports the member's presence and decoded size with `inspectable:false` and the reason. `?config=1` and `?database=1` select members as on backup; `?config_mode=` and `?database_mode=` take merge or replace and default to replace when omitted; an explicit empty or whitespace value is rejected as malformed. Success reports the `restart_required` keys. The upload is exempt from `request_body_read_timeout` (the idle bound still applies) and an expired read deadline answers 408. The query grammar is strict: parse errors, repeated or unknown keys, and malformed values all return 400. | | `GET /metrics/export` | Download all or exactly filtered finalized records as a gzip artifact named `millivolt-logs-.json.gz`; gunzip it for the JSON array. `status_code=0` and `debug=0` mean "no constraint" on that dimension. A debug capture rides as a sibling `debug_capture` member only for a row that has a stored sidecar, so debug bodies appear only while durable storage is live; a ring-only export never carries them, and debug bodies stay sensitive either way. | | `GET /metrics/query?q=` | Restricted SELECT with timeout/output limits; 503 when durable storage is disabled. Not a hostile-query sandbox. | | `GET /metrics/bootstrap` | Dashboard state and full/incremental recent-record snapshot. | | `GET /metrics/live/stream` | Replayable finalized SSE feed plus ephemeral pending lifecycle/reset events. | | `GET /metrics/agg/chart`, `/explorer`, `/log` | Scoped history chart, faceted explorer, and durable log paging. The Requests-card `s=` selector accepts an exact code or a live in-flight class on all three, but only the explorer folds pending rows: the chart answers a live class with `400 invalid scope` (its buckets are finalized-only, so an accepted class could only read as "no traffic"), and the log keeps its standing empty page so the archive pager does not stop. | | `GET /metrics/prometheus` | Prometheus exposition over the in-memory ring, not durable since-inception dashboard totals. | | `GET /metrics/pprof/` | Standard Go profiling handlers behind the same credential as every `/metrics` route: the index listing plus the `profile` (CPU), `heap`, `allocs`, `goroutine`, `block`, `mutex` and `threadcreate` profiles at `/metrics/pprof/`. Unregistered subpaths answer the reserved namespace 404, never inference. See the capture workflow below the route table. | | `POST /mcp` | Streamable HTTP MCP endpoint, operator-gated like `/metrics` and enabled by default. `mcp_enabled: false` removes it from the process; the route stays reserved, answers 404 and never reaches inference. Serves the 25 MCP tools, Bearer-only and stateless. See [MCP transport](#mcp-transport) below. | Pause/Debug/Limits successes may include a persistence warning: runtime state was applied, but saving it failed. Do not retry as though the mutation rolled back. With storage disabled, Pause and Limits keep their memory-only state intentionally. Debug is the exception: starting a session answers 503 because with no store every matched request would retain bodies for a sidecar that could never be fetched. ### MCP transport The gate admits the session cookie on gated paths, but this endpoint refuses a cookie-only request with `403`. It forwards the caller's own Bearer on its in-process calls to the routes above, so no session cookie or per-request state is retained between requests. The admitted credential keys an in-memory cache of its SDK server object. The cache is never logged or persisted, and it lets repeated calls avoid rebuilding the 25 registrations. `GET` and `DELETE` answer `405`. `/mcp/...` look-alikes are reserved and answer `404`, never inference. A reserved-namespace spelling whose separator is single-encoded (`%2f` or `%5c`, any hex case) is refused the same way. A double-encoded separator (`%252f`) is not a reserved-namespace spelling: it decodes to a literal `%2f` inside one segment, so it can reach neither the MCP handler nor a reserved namespace and is ordinary inference traffic. Three tools, `docs_list`, `docs_search` and `docs_read`, answer from the embedded documentation index and make no proxy request. The profiling subtree is operator-gated like every `/metrics` route: an unarmed plane answers 403, a wrong credential 401, and a reserved-namespace spelling with an encoded separator answers the reserved 404. `go tool pprof` cannot send an `Authorization` header, so fetch the profile with curl and analyze the file against a locally built binary. Against the dev instance ([`scripts/dev.sh`](../scripts/dev.sh), loopback `127.0.0.1:8081`): ```sh go build -o /tmp/millivolt-profiling ./cmd/proxy curl -fsS -H "Authorization: Bearer $MILLIVOLT_OPERATOR_TOKEN" \ -o /tmp/cpu.pprof 'http://127.0.0.1:8081/metrics/pprof/profile?seconds=30' go tool pprof /tmp/millivolt-profiling /tmp/cpu.pprof ``` The `seconds=N` parameter on `/metrics/pprof/profile` is the capture window: that one request stays open for the whole window while samples accumulate. The valid range is 1 to `response_write_timeout` (600 by default); an absent or unparseable value uses 30 and a larger one is lowered to the bound, so a capture never outlives the delivery contract unnoticed. A process runs one CPU profile at a time, so a second concurrent `profile` request fails with 500 until the first finishes. On `/metrics/pprof/heap`, `gc=1` forces a garbage collection cycle before the sample. The proxy's own `/mcp` endpoint serves this whole surface to an LLM, using the same credential and the same routes. It adds no route of its own and dispatches each tool call in process through the same gate, so its forwarded Bearer is re-validated exactly like an external request. For its irreversible call, the operator credential and the tool's own guards are the entire gate: an empty purge filter, a purge without the exact confirmation phrase, and a purge without the `purge_preview` token issued for that same filter are each refused. The preview token expires 15 minutes after it is issued, so a purge more than 15 minutes after its preview needs a fresh one. It also refuses a capture session on a proxy without durable storage, and an unknown name in a capture scope or a throttle provider is offered the known values instead. The throttle provider check fails open when the proxy's vocabulary is empty or cannot be read; the capture scope check accepts any name when the vocabulary is empty but refuses when the vocabulary cannot be read. It never follows a redirect, and it presents the credential only in the `Authorization` header. See the [MCP server guide](mcp.md). ### Operator access `MILLIVOLT_OPERATOR_TOKEN` is the single operator credential for the whole dashboard. It is read once at boot from the process environment, never from proxy.yaml, so it cannot leak through Settings, `-print-config`, `/admin/config` output, snapshots or logs. The gate is deny by default and lives in one chokepoint in front of every route. This is the canonical open-path list; [Security](../SECURITY.md) and the [reverse-proxy guide](reverse-proxy.md) summarize it instead of maintaining a second list. - Open: `GET /healthz`, `GET /readyz`, the `POST /session` login handshake (the one open operator route), origin-root brand/PWA files (`/favicon.ico`, icons, `/manifest.webmanifest`, `/sw.js`) and transparent inference. Provider credentials ride the same header name and are never inspected by the gate. - Gated: the dashboard HTML and `/dash/*` assets, every `/metrics/*` surface and every `/admin/*` route, including unclassified future admin paths and unsupported methods, plus the `/mcp` endpoint and its reserved `/mcp/*` look-alikes. A gated MCP request also needs its credential as a Bearer header: the session cookie alone is refused there (`403`), because the caller's credential is forwarded on the endpoint's internal calls. Present the credential as `Authorization: Bearer `. A valid Bearer request on a dashboard, admin, metrics or dash route also mints a session cookie, because the dashboard's live feed uses EventSource, which cannot send Authorization headers. The `/mcp` endpoint deliberately returns no cookie: its caller forwards the credential on the endpoint's internal calls. The cookie is HttpOnly (script never reads it), SameSite Strict, scoped to 12 hours, and signed with a key derived from the operator token, so a container restart with the same credential keeps the session. Rotating the token invalidates outstanding cookies. The cookie carries `Secure` only when the request arrived over TLS or any `X-Forwarded-Proto` token names a secure scheme (`https` or `wss`). The header is a list: the canonical NGINX sample overwrites it with this server's `$scheme`, and an ingress that appends its own scheme after the client's `http` must not downgrade the cookie - hence any token, not the first. A loopback plain-HTTP dashboard keeps working while a TLS presentation gets the flag. Every response to a secure presentation also carries `Strict-Transport-Security: max-age=15552000` (180 days), set once at the outermost server wrapper from the same secure-presentation test. There is no `includeSubDomains`, because millivolt cannot know whether sibling or parent hostnames are TLS-capable. A plain-HTTP response carries no header at all (browsers ignore it there), so the dev instance is unaffected. The lifespan is deliberately under a quarter: moving the dashboard off TLS becomes reachable again within that window instead of pinning a browser for a year. Do not expose the listener over plaintext networks anyway. The login page served for unauthenticated dashboard visits exchanges the entered value for that cookie through `POST /session` and works without JavaScript; API clients can call the same endpoint or simply send the Bearer header on every request. An expired cookie re-prompts in the dashboard or reappears as the login page on navigation. Unregistered `/admin/*`, `/session/*`, `/metrics/*` and `/mcp/*` paths are reserved: they answer 404 and are never forwarded upstream. The gate also refuses 404 when an owned-namespace path is written with a percent-encoded separator (`%2f` or `%5c`, any hex case), a duplicated separator (leading or interior) or a dot segment: the mux cannot route it to the reserved subtree unchanged, so it would otherwise fall through to inference. A double-encoded separator (`%252f`) is not a reserved-namespace spelling: it decodes to a literal `%2f` inside one segment, matches no reserved route, and is ordinary inference traffic. Failed authentications are throttled per source IP. The immediate hop is the only trusted one, so the throttle key is the rightmost `X-Forwarded-For` element (the address that hop appended) and `RemoteAddr` when the header is absent; a client can prepend anything, and an unparseable element is skipped. The shipped [NGINX example](../deploy/nginx.conf) resolves its own peer with `real_ip` first, so the rightmost element is the real client. A directly exposed instance lets a client choose its own key, which only ever relaxes the throttle, never the credential decision: after five rejected credentials the IP is locked out with a window that doubles from 10 seconds to a 15 minute ceiling. The throttle counts only presented-but-wrong credentials, so an expired-cookie dashboard polling in a background tab cannot lock its own operator out, and a verified credential is always admission: presenting the correct Bearer during a lockout clears the budget, because the lockout exists to rate-limit guessing, not to lock the operator out of their own dashboard. The map is hard-capped at 4096 source IPs with idle expiry, so spoofed-address floods cannot grow it; reaching the cap is logged once per episode, because from then on a new source evicts a tracked one. Crossing the five-failure threshold is logged once per source. A lockout response is `429` with a coarse rounded `Retry-After` and a generic JSON body, except `GET /` (and `/index.html`): the sign-in page is still served, read-only, with the same `Retry-After`, the form intact and the lockout named in its notice slot, because the form is the only route back to a credential and `POST /session` keeps working through the lockout. With the variable unset or empty the deny-all state holds: the whole dashboard returns 403, the open `POST /session` handshake cannot mint a cookie, and only `/healthz` and `/readyz`, brand/PWA files and inference respond. A nonempty value shorter than 16 or longer than 512 bytes fails the boot instead of silently weakening the gate. The credential is process bound: config reload does not re-read it, so rotating the value requires a real process restart (a supervisor restart or a fresh deployment; the dashboard's restart handoff inherits the parent environment, so it keeps the old credential by design). The dashboard asks for the value once per browser tab, keeps it in sessionStorage, attaches it to gated calls only and drops it as soon as the server rejects it. Non-browser callers send the header directly, for example `curl -H "Authorization: Bearer $MILLIVOLT_OPERATOR_TOKEN" -XPOST .../admin/purge`. Environment variables are readable by same-user processes through `/proc`, so run the proxy under a dedicated account. ### Pause and Limits Pause creates holds; it does not disconnect already-running sends. Matching new sends and retries wait until the hold expires or is resumed. Choose all traffic, any combination of the five scope dimensions, or clients that were unseen when a new-clients hold was created. "New" does not mean new requests or new conversations. A hold scopes on the same five dimensions a limit does: providers, models, clients, upstream key hashes and conversations. Values inside one dimension are alternatives, an unset dimension matches any value, and set dimensions intersect - the same rule limits use, because the scheduler matches both against one request identity. When named clients and new are combined, either client condition may match. A hold has its own duration and queue cap; the cap defaults to `max_queue_size` when the hold does not set one. Excess waiting work can be refused; an indefinite hold needs an explicit resume. Existing holds can be edited in place or resumed individually, while Resume all clears all holds. Overlapping scopes are rejected instead of silently replacing a different hold. The HTTP shape is closed. A resume carries only `paused:false` and an optional `id`; supplying a scope, duration or queue cap beside it is a 400, because the proxy has no scope-selective resume and accepting one would clear more holds than the caller named. `all:true` cannot be combined with `new` or any scope dimension; the global hold already covers them. Named clients plus `new` remains a valid union. An edit restates the dimensions it supplies and preserves the ones it omits, so supplying an explicit empty array is how a dimension is cleared. These are enforced by the handler, not only by the MCP and CLI wrappers. The known client/provider/model vocabularies (the pause and capture pickers, and the snapshot a new-clients hold compares against) are bounded at 4096 names per dimension. Client labels and model ids come from the request, so they are evictable: past the cap the least recently observed name makes room for the newest, so a client that keeps sending requests outlives a burst of one-shot labels that arrived after it. Names an operator documents - a hold scope, a debug session, a limit policy (including one installed from a request's `X-Limit-*` headers or restored at boot) - are pinned and are never evicted by later traffic, so a flood of distinct labels cannot hide the client you scoped a hold to. Pins are derived from the documents that exist, not accumulated: resuming a hold or ending a debug session releases its names back to ordinary eviction. When a full vocabulary is entirely pinned, a new observed name is dropped rather than displacing operator intent, and a scope that cannot be listed is named in the log instead of disappearing silently. A hold scopes on the caller's claimed client label, so it separates your own tools from each other, not trusted callers from untrusted ones; a caller that sends another client's label is parked, or not parked, as that client. See [client identity](../SECURITY.md#client-identity-is-a-claim-not-an-authentication). Limits applies to an intersection of conditions: providers, models, clients, key hashes and conversations. Values inside one condition are alternatives; an empty condition matches any value, so a rule with no condition covers every request. A request must satisfy every non-empty condition, and every matching policy charges its own lease, so the tightest cap wins. A provider-only rule keeps the per-key concurrency and fairness behavior below; adding any other condition narrows the same machinery without changing it. Conditions select the request's recorded identity, so models match the request spelling, keys match the SHA-256 digest, and conversations match the selected leaf. A rule may also carry a recurring schedule: `days` (0 = Sunday), `start_minute` and `end_minute` in the server's local timezone. Equal minutes mean all day, and a start after the end spans midnight, with the after-midnight portion belonging to the previous day. An empty schedule is always active. Outside a rule's window it does not charge, settle or queue, and a request that matches no other rule proceeds normally. A policy's identity is the digest of its normalized conditions and schedule. Editing either one therefore creates a new identity, and two rules cannot silently share counters. Provider-only policies keep the version 1 persisted shape and still accept the singular `provider` field; a rule with any other condition persists its selector arrays. `clear:true` removes the whole policy and cannot be combined with a limit dimension: the handler refuses the ambiguity instead of dropping the supplied value and reporting success. Choose concurrency, requests per window, tokens per window and the optional anti-burst ramp. The console shows whether a policy came from the console or a request header, its schedule, and available budget and in-flight/queued state. Queued counts every request the scheduler parks for that policy - admission waits and send-gate waits behind a 429/5xx pacing window alike - not only work blocked by the configured cap. Send-gate waits are paced by the provider's retry hint and the retry budget; `max_queue_size` and `max_queue_wait` bound only the admission FIFO. Provider admission is fair across keys: a release or refill rotates across the groups actually waiting on the shared provider gate (concurrency or the request/token bucket), granting one request per group per pass. A group parked only on its own pacing window, hold, anti-burst ramp or per-key cap is outside that fair set and does not delay another key's fast path. A release settles the gate and drains once, and the drain stops as soon as a grant refills a saturated provider concurrency cap. Header semantics are in [protocol](protocol.md#routing-headers). Persisted policy does not mean the live scheduler's counters and in-flight requests survive a restart. **Drain queue** releases every request the scheduler has parked, or only those whose recorded provider, model or client matches the one named in the console, without waiting for a limit window, pause hold or retry pacing to open. The selector reads each parked request's own identity, so it also releases work parked by a hold, by pacing, or by a provider without a limit row. The overview shows the total parked count. Each released request answers with a retryable 503 and is recorded as `queue_drained`, never as a provider error; a request already exchanging bytes with the provider finishes normally. The API also accepts the same three scopes for scripted drains. Anti-burst is opt-in and scoped to each provider+key group. It is available only on a single-provider rule without a schedule, because a ramp is defined over the provider-plus-key groups the scheduler owns; the server rejects the pair on any other rule. With `anti_burst` enabled and a concurrency target of at least 2, a group starts at one concurrent request and gains allowance linearly until it reaches the rule's `concurrency` target after `ramp_duration`. It is inert unless all three are set: the flag, a target of at least 2, and a duration above zero. `ramp_duration` accepts the same 1s..24h window grammar as request and token windows, including `1d`; empty, `0` or `0s` turns it off. Requests beyond the current allowance stay in the existing per-key FIFO. A fully idle group whose idle period reaches `ramp_duration` starts a new epoch on its next grant, while a shorter idle gap continues the existing one. Enabling the ramp while a group already has in-flight work anchors its epoch at the current occupancy, so running streams are not stalled. Changing the flag, duration or concurrency target resets every group's epoch. Header updates change only the three cap dimensions and preserve the stored anti-burst pair. Clearing the last active cap drops the whole policy, ramp included. The limit composer carries the checkbox and the ramp chips; applying sends both, and a form whose caps are all zero sends the pair off rather than an invalid on-ramp. Operator limit edits persist synchronously and report a warning when the store rejects the write. Header-driven limit changes apply in memory immediately and are persisted by one coalescing background writer: a burst of changes becomes the latest whole-policy document, with bounded retry and backoff after a store failure. A closed store is not retried - the generation stays dirty and the writer stops - while a busy one keeps the ladder. That document also carries the anti-burst pair, and one written before those fields restores them off. A crash can therefore lose the most recent header-driven changes, and a failed save never rejects inference. Graceful shutdown and restart flush the latest policy before storage closes or the restarted process opens the database. Provider labels are capped at 255 bytes and the persisted throttle document at 8 MiB; normal provider labels derived from base URLs remain below both bounds. Admission is bounded end to end. - `max_inflight_requests` is the process-wide inference ceiling (default 10000). - `max_inflight_body_bytes` is its buffered body budget (default 256 MiB; admission charges the allocation hint, the declared length bounded by `max_request_bytes` and the read hint, and reconciles it against the bytes actually read, so a short body refunds the difference and a chunked one is charged as it arrives). - `max_inflight_response_bytes` is its budget for buffered upstream response bodies (default 256 MiB): whole-document translation of non-streaming responses, the non-streaming 200 quality spool (at most 1 MiB plus one 32 KiB chunk), and model discovery. A request-slot refusal is answered with `503` and `Retry-After` before the body is read. The body budget charges the allocation hint at admission and reconciles it against the bytes actually read; when the hint cannot be covered the slot is still taken, and the first read decides without retaining the upload. The per-request cap still owns the status: a refused request drains at most the remaining `max_request_bytes` window so an over-cap body answers the definitive `413` instead of a retryable `503`, while a body that fits the cap answers `503`. Request-body reads must make progress within `request_body_idle_timeout` and finish within the absolute `request_body_read_timeout`, so a stalled peer releases its reservation without cutting off a slow but progressing upload. The `POST /admin/restore` upload is exempt from the absolute bound only: a full database archive can legitimately outlast it on a slow link, while the idle bound still cuts a client that stops sending, and an expired deadline answers `408` with its own message instead of the archive-cap `413`. Buffered whole-document response reads use `whole_document_read_timeout` when the send has no deadline and `whole_document_read_max` as the ceiling even when `X-Proxy-Timeout-Ms` is longer, so a slow or stalled body releases its response-buffer reservation; a shorter caller deadline wins and configuration validation preserves the `models_discovery_timeout` budget below the ceiling. The three admission budgets accept `0` = unlimited and hot-reload. `max_queue_size` (default 1024 per group) and `max_queue_wait` (default 10m) bound a held or saturated provider's parked work; both accept explicit `0` = unlimited. Independent per-key/header concurrency remains unlimited by default. The scheduler retains at most 100000 distinct provider+key groups, evicting the least-recently-idle group (no in-flight or queued work) to admit a new key at the bound; a new key is refused only when every retained group has active work, so a table full of active groups still fails closed. Recent pacing/backoff state for evicted groups is kept in a separate 4096-entry bounded LRU. The scheduler also retains at most 4096 header-created provider limits, refusing new ones past that bound rather than growing process memory without limit. A new request body has the `request_body_read_timeout` budget, SSE writes have the `sse_write_progress_timeout` per-event progress bound, and an upstream body that stops sending for `upstream_stream_idle_timeout` is canceled. Request budgets are strict sliding windows: at most the configured count is admitted in any trailing window of the configured length, so an idle provider banks nothing and a busy one cannot spend future allowance as a present burst. Charging the window is what makes the limit match a provider that counts requests per rolling or fixed minute; a refilling budget could emit the whole count in one instant and then refill through the same span, which is how a `100 per minute` setting reached a provider's `101/100`. Each admitted request is charged once, at proxy admission, not again for each absorbed upstream retry. A count above 65536 is enforced by even spacing at the same ceiling instead of one timestamp per slot, so the memory a window holds stays bounded. Token budgets remain refilling buckets: prompt tokens are reserved from the captured characters divided by four and then settled against final reported usage, and a reservation larger than the whole capacity may proceed while credit is positive, leaving debt that refill or settlement repays. The client's output ceiling (`max_tokens` and its spellings) is deliberately not reserved: it is a ceiling, not expected usage, and reserving it would turn the token budget into a worst-case concurrency cap. A token budget therefore also sets the sustained average rate: 500K tokens per 6h refills at about 23 tokens/s, so once the initial budget is spent, later requests wait for refill unless actual usage settles lower than the reservation. ### Error storm protection Open Settings, select **Error storm protection**, and turn on **Enable protection**. It is opt-in in both neutral defaults and the deployment example. Choose the rolling window, minimum samples and failure threshold, then select HTTP failure statuses, transient transport failures and response/stream failures. Provider protection requires all models receiving requests in that same window to meet the thresholds individually. A single affected model is held on its own while other active models remain healthy. If only one model is active, that model can establish a provider-wide incident. Provider and model protection can also be enabled independently. Configure initial/maximum recovery delays, multiplier, jitter, recovery successes, additional retries, queue capacity, per-wait timeout and retained scope capacity in that category. The generated [example](../proxy.example.yaml) supplies every type, range and default. Existing Queue & retry limits and provider Limits still apply; neither `storm_max_wait` nor a per-send deadline is a total request timeout. Use a client timeout appropriate to the desired end-to-end wait. Each clickable banner identifies its current scope and safe failure category (HTTP status, transport error or response error), percentage/count of failed upstream attempts, detection window, queued callers and next probe or recovery progress. Open an incident to inspect the distinct requests that encountered a selected failure, failed attempts, total sampled attempts, error category, queue count and next retry time. Provider entries also show affected versus active models. Distinct request counts follow each request's latest selected failure in the rolling window, so repeated retries do not inflate that count. The percentage still describes upstream attempts. These are process-local observations, refreshed through the normal dashboard bootstrap/poll cadence; they are not chart history or the Errors chart's distinct-request percentage. Scope counts can overlap where a caller waits on both provider and model protection. Hide the banner independently without disabling protection. Changing a storm policy resets its samples/incidents and wakes waiters to re-evaluate the new policy. Banner visibility alone preserves detector state. A reset does not cancel work already sent or erase request history. Restart starts detection afresh, and queued requests do not persist. History deletion does not reset live outage protection. Storage stays outside the detector: fixed 64 time buckets retain counters per scope, with oldest-edge precision within approximately `storm_window / 64`. `storm_max_scopes` includes provider aggregates and exact provider/model scopes; stale unreferenced healthy scopes can be evicted, while active or referenced scopes remain protected. Scope identifiers over the internal 512-byte retention bound and exhausted scope capacity reject admission rather than silently bypass protection. These bounds are not a total heap ceiling: waiting callers retain bounded request bodies, and existing ordinary queues remain separately sized. The [protocol guide](protocol.md#error-storm-protection) defines retry safety, quota exclusions, native handshakes and client-facing failures. ### Quota pause Open Settings, select **Quota pause**, and choose the reaction to a 429 whose error envelope is a durable account condition - `insufficient_quota`, `insufficient_credits`, spend/hard limits, the recognized numeric billing codes (the built-in classification is fixed and `quota_error_classes` may add spellings; only the reaction is configurable): - **off** (default): the 429 surfaces to the client immediately. Nothing is parked or retried. - **retry**: a provider-wide recovery gate opens on the first such 429 from an OpenAI-wire provider. The triggering request absorbs the 429 and re-sends per the error storm recovery delays; every new request to the provider parks on the gate (bounded by `storm_max_queue` and `storm_max_wait`); the gate closes after `quota_pause_recovery_successes` consecutive successful probes (the default 1 closes on the first 2xx) and the queue drains. A provider `Retry-After` on the quota 429 paces the next probe like any other 429. - **manual**: an indefinite provider-scoped pause (an operator hold labeled with the quota class) is created automatically; every request, triggering one included, queues until an operator resumes it on the pause surface. When a pre-existing hold already covers the provider, no duplicate is created: the triggering request queues under that hold only if it parks the triggering client too, and a hold that parks other clients only lets the triggering request surface the 429 immediately. The retry-mode gate shares the storm scheduler's parking, probes and pacing, so storm recovery settings configure it without storm protection being enabled. It surfaces in the storm banner as a quota incident with a **Resume now** action (`POST /admin/quota`); manual-mode holds appear on the pause surface with their reason and resume like any hold, and they derive their provider pin through the same commit owner as a manual pause so the provider stays offered in the scope pickers. A quota 429 never arms the rate-limit pacing trip (the rate-limit health series still counts the 429 itself), and quota re-sends never burn the transient `max_retries` budget. They are bounded by `storm_max_wait` instead, measured from the request's first park: when it expires the client receives the canonical retryable `error storm protection: ...` 429 rather than waiting on a gate that never opens, and the provider stays parked for the next request. The retry gate is in-memory - a restart drops it and the provider's next durable quota 429 re-arms it - while manual-mode holds persist like every operator pause. In-band occurrences (the same envelope inside a 200 stream) arm the pause the same way; that request already answered, so it is never parked. Translated Anthropic-format and Cursor upstreams keep the off behavior in every mode. The built-in vocabulary is fixed; `quota_error_classes` may add a gateway-specific spelling as a durable quota class (it may never move a built-in spelling between classes, and it re-reads stored history through the same vocabulary). ### Debug and preview capture Debug selects clients, providers and/or models and starts a capture session for a chosen duration or until stopped. At least one dimension is required; selected dimensions intersect, with alternatives inside each dimension. It does not have an unrestricted "all" switch. Sessions can be edited or stopped individually; Stop all ends every active session. A stop carries only `enabled:false` and an optional session `id`; a scope or duration beside a stop is a 400 rather than a silent stop-everything. Conflicting scopes are rejected. Debug model matching uses the native-model normalization in [debug.go](../internal/proxy/debug.go), separate from display-only model rules. Do not assume that every model spelling is an independent exact scope. The session timer selects future captures. `debug_capture_ttl` separately controls retained capture lifetime after completion, and `debug_capture_max_bytes` bounds one capture document: retained headers are charged first and the remainder is split between the request and the client-facing response body. Oversized captures are marked truncated. `debug_store_max_bytes` (default 512 MiB) caps the whole capture sidecar; once the next write would exceed it, new captures are skipped and TTL expiry frees capacity again. Capture writes are counted in bytes, not characters. The request drawer loads an available sidecar only for a captured request; expired/absent captures are not recreated from ordinary metrics. Completion is published before the sidecar is encoded and saved, so a slow sidecar write delays neither the live end nor durable accounting. A drawer fetch that follows the live end waits for that request's in-flight save instead of answering a transient 404; a disconnected client stops waiting. Starting a capture requires durable storage. With `db_path` empty, `enabled:true` without an ID answers 503 `durable storage is disabled` and creates no session. A store-less process also has no persisted session document to restore, so it never starts with an enabled session; if one were present in memory, the request path still refuses to build a capture tap, so no headers or bodies are retained for an unfetchable sidecar. Editing or stopping a named existing session remains available. Retained request headers use the same policy as recorded response headers: - names are visited in sorted order for the budget; - each value is capped at 4 KiB and marked with `…` when cut; - at most 16 values are kept per name; - ordinary headers share a 16 KiB total budget per header map; - a header name is capped at 512 bytes; - at most 256 names are retained per map. A longer name or an extra name is omitted without a marker, exactly like a value beyond the budget. Credential headers (the configured auth header, `Authorization`, cookies, and token-, API-key- or secret-bearing names) become `[REDACTED]` before any truncation, so a cut value is never a credential prefix. Headers or values beyond the budget are omitted without a marker; the capture schema has no header-truncation field. The capture JSON is encoded without HTML escaping, so `<`, `>` and `&` appear literally while JSON control escapes remain. The drawer's Debug section also carries a Download button. It fetches the same capture with `download=1` and saves a gzip artifact named `millivolt-debug-.json.gz`, using the record id in the name when the capture does not carry a readable `captured_at`. Captured request and response bodies render through the same bounded code surface as record previews, with wrap, copy and expand controls; the Download button remains the whole-capture path. The capture cards also surface its schema and captured-at time, conversation and key correlation, first-answer timing, usage, cost and attempt count. `capture_body_preview` is a different opt-in feature: it retains short prompt/ response previews in the normal record without starting a Debug session. The two features also retain different request bytes: on the passthrough wire the record's prompt preview is the client's original body, captured at the metadata decode before `request_overrides` or the sub-conversations strip rewrites it, while a debug capture's request body is the rewritten upstream body sent after any rewrite. A translated Chat Completions or Anthropic target re-decodes the translated body for its prompt preview; a translated Responses target keeps the client-body prompt preview, because the generated item list is not re-decoded for preview. The response preview follows the client-facing response content on every target. Both features can retain sensitive content even when credential headers are redacted. Exports/backups can outlive the configured retention period. See [Security](../SECURITY.md#upstream-destinations-and-credentials). ### Export and deletion The operator console's export section downloads finalized request records as JSON, not the process's stderr log and not a complete administrative audit trail. With durable storage it reads the database; otherwise it uses available ring records. Open export, choose either all history or matching records, and review the previewed count before downloading. Provider, model, client and conversation merge the server's known vocabularies with whatever the live record ring holds, so a quiet proxy can still filter by what has run since it started, and each of them also accepts a typed value. Status is the one list-only dimension, because a status code is not a name anyone can type reliably. A debug-only export can include retained capture sidecars. A debug capture is written beside its record as a `debug_capture` member, and only when the row still has a stored sidecar. The artifact therefore carries debug bodies only while durable storage is live: a ring-record export, or a sidecar already reclaimed by the capture retention sweep, has the record alone. The bodies remain sensitive either way. The export predicate has no constraint for `status_code=0` and `debug=0`; both are off rather than a match for the zero status or the false flag. A genuinely bodyless `POST /admin/purge` deletes everything in one call, which is why a mistyped query key is refused instead of silently ignored. The download is always a gzip artifact named `millivolt-logs-.json.gz`; gunzip it to read the JSON array of records. Compression is applied at the export writer itself, so every export shape (filtered, all or debug-only, from the ring or durable storage) downloads compressed the same way. This is a breaking change for scripts that consumed the plain JSON download: there is no uncompressed fallback, so scripted consumers must decompress the artifact first. A durable export holds one read snapshot for the whole stream, so the server bounds it with an internal total budget. If the budget expires after bytes were written, the download is aborted mid-artifact instead of completing as a successful gzip: the saved file is partial and must be discarded. A caller cancellation or shorter caller deadline is reported separately from this server budget. In the dashboard, both database deletions live in Settings, in the storage category, and nowhere else. Delete matching uses the same filter/count owner as the export, and its scope is the chosen filter; Delete everything wipes the whole database. Deletion requires confirmation in the dashboard; it does not happen when the sheet opens. A selected filter makes a read-only `POST /admin/purge/count` preview; the count first commits the writer's queued batch into the read view, so a just-finalized request cannot read as empty. That barrier is bounded by the query timeout: a wedged writer lets the preview answer from the committed snapshot instead of spinning the sheet forever, and the delete's transactional re-check stays the guard. Neither the caller's deadline nor a client cancel fails the preview, even when it expires before the barrier does: each step runs detached on its own query budget, so a short dashboard deadline answers the same as an unbounded one. A ring-only deployment removes through the boundary the count saw, so a record that finalizes during the delete stays behind instead of disappearing uncounted. Current traffic can still change the count before the action, but the UI retains the previewed age cutoff rather than silently moving it, and sends the rendered count with the delete so the proxy re-checks it inside the delete transaction and refuses a moved row set with 409 instead of deleting unpredicted rows. A 409 drops the sheet's preview and re-counts, so the operator confirms the new set instead of repeating the refusal. The filter selections reset every time the sheet refills, so a stale invisible filter never arms a destructive default. Both actions are irreversible without a backup. Every purge, backup and restore writes one audit line to the process log with its terminal outcome, before the response is written: the action, the rendered selection (`filter="all"` for a bodyless full delete, `filter="unparsed"` while the body or query has not been accepted, and `members="unparsed"` likewise for a backup/restore), the previewed count when one was supplied, and the committed `deleted` / `buffer_removed` counts, or the error. Even a rejection at the method gate or the query grammar, before the selection exists, leaves a line. Each value is quoted with Go string escaping, so a label carrying a newline or a quote stays one token on one physical line. A lost or truncated response therefore stays diagnosable, and a retry can be compared against what was actually attempted. These lines are the destructive-action trail; they are not a general request log. Export/delete filters are exact raw record fields, not display-name model canonicalization. The API also supports conversation/error-type/time filters beyond the console's common choices. See the shared [filter decoder](../cmd/proxy/log.go) and [PurgeFilter](../internal/storage/purge.go). Empty-body full deletion is deliberate and destructive; never use malformed or empty JSON as an all-records command. The count preview applies the same empty-filter rule, so `{}` answers 400 on both routes rather than counting the full set for an ambiguous body; a genuinely bodyless count previews the whole set, exactly like the bodyless delete. The delete fence preserves newer completions and in-flight work. A completed purge, and an expired debug-capture sweep, checkpoint-truncates the WAL after the committed delete: `secure_delete` zeroes freed cells in the pages SQLite writes, and the checkpoint removes the pre-delete frames that would otherwise keep the old page images readable in the `-wal` sidecar. A reader holding an older snapshot can defer the checkpoint; the writer's near-term tick retries it while the reader is gone, and a clean shutdown makes the final attempt after the read pool is closed. ### Settings and Restart Settings only reads configuration when opened. Its search, category rail and structured editors work on a local draft; Apply uses the [revision-checked save contract](configuration.md#saving-settings-safely). The console's system section reads restart availability and status when opened. The restart action is disabled while a restart is in progress or rebuilding is unavailable. Progress reports actual build/drain/flush/handoff phases; it is not a simulated timer. After a successful handoff, reconnect/bootstrap refreshes history, aggregates, operator state and config. Changed embedded frontend assets trigger a page reload through that same refresh owner. The [lifecycle section](containers.md#reload-restart-and-shutdown) distinguishes source rebuilds from container replacement and ordinary shutdown. ## Dashboard data and request inspection The KPI band is global. Explorer selections and the request status filter scope the timeline and request log; selecting a card adds a filter and entity links pivot within that shared route. Browser back/forward restores hash-based scopes. Chart preset, timeframe, percentile and series visibility persist in browser storage. These saved views are local browser state, not shared server settings. The In-flight tile separates work that is sending from work the scheduler is parking. Its summary names the two totals that fit the narrowest tile - active and queued - and the composition rail under the value splits every live state (sending, pending, operator-paused, throttled) by color; hovering the rail, or focusing it with assistive technology, reads the exact per-state counts. A queued request is one the proxy has not sent upstream yet: it can be waiting in the admission queue (per-key concurrency, pacing, a provider cap or an operator hold) or in the retry send gate behind a 429/5xx pacing window. The request table marks the same state with the `throttled` pill, and the status selector can filter on it. Explorer dimensions are Providers, Models, Client, Conversations, Tools, Time, Status, Errors and Keys. Their meanings are server-owned: model grouping and time-bucket membership are not browser guesses. Conversations can show declared parents and main/sub/unresolved counts; read the [lineage contract](protocol.md#main-and-sub-conversations) before interpreting them. Tools and error signatures can each have multiple memberships per request. Tool cards count affected requests for N and share, with request-level card metrics counted once per affected request; their calls KPI separately sums request ToolCalls, so repeated same-name calls do not inflate the request count. A request using several tools contributes to each tool card. Error/429 health badges and every gallery share use distinct affected requests. The [visual guide](dashboard.md) covers every timeline and its controls. - Overview is the summary: it carries no plot, and its tiles take the whole card (requests over the blended token volume, tokens in/out/cached with their shares, spend with the blended per-Mtok price, the errors / 429 pair, and average latency / speed over every measured request with each metric's low-high sample range). - Traffic compares requests/errors. - Tokens separates available input/output/reasoning/cache usage and tracks the cache-hit share of input on a percentage axis. - Speed + latency plots throughput on its own axis with TTFT and the decode window (first token to last, the span decode speed measures) sharing a duration axis. - Errors compares error count/rate. - Cost places reported USD spend beside request volume. One percentile selection controls the timing series (the Overview keeps only the timeframe control). Period readouts are computed from period samples, not averages of bucket percentiles. Insufficient samples remain unavailable. All time means the available retained history, not deleted, dropped or never-observed requests, and the timeframe control never offers a window wider than it. The chart endpoint takes either a minute `window` (or `all`) or an explicit `from_ms`/`to_ms` pair for calendar ranges; each payload reports the resolved right edge `to_ms` and the unscoped `data_from_ms` extent the control gates on. ### Request rows and details The log starts with recent rows and pages older durable records as you scroll. Live rows show in-flight/queue/retry progress; final rows retain their outcomes. Retry disclosure opens absorbed attempts under the original request, not another inference record. The attempts log keeps the newest `max_retries` plus `storm_max_retries` entries (at least one): a durable quota wait re-sends without spending either budget, so the tail is what explains a long parked request. Click the detail arrow or a non-link row cell to open its drawer; previous/next controls navigate requests without changing the scope. The inspector presents available facts in four classified groups. - Request covers identity/status/times, time bucket, wire sizes, client/runtime, provider metadata, model and captured parameters. - Content covers conversation and parent ids, turn and size composition, and prompt/response previews. - Usage covers token splits, generated tokens, timing, throughput, cost, transport decomposition and tools. - Diagnostics covers queue/rate limits, retries, in-band upstream responses, retained headers and Debug. A summary readout carries the status, route, identity and first numbers, and a final upstream failure renders as an error alert above the groups. Only the selected group is shown; card headers collapse, and the active group and collapse choices survive live updates. A typed 429 is flow control, so its reason, code and message stay in the rate-limit card instead of the error alert; a rate limit carried by a non-429 status is a final failure and renders the alert like any other. An in-band upstream response that arrived after a committed success is disclosed in Diagnostics rather than counted as a retry. Live pause and throttle state appear as explicit rows while a request is in flight. Copy actions cover the values that matter (identifiers, paths, models, headers, error detail and captured bodies), field-list cards and the header's JSON action. One clipboard path reports success or denial in a polite status line. Long captured bodies use a bounded code surface with line count, wrap, copy and expand controls; a long single line scrolls inside that surface instead of widening the inspector. Narrow viewports get the full-width inspector with keys stacked above values and touch-sized copy and close targets. Recognized parameters include sampling and output limits, stop/logprob controls, reasoning/verbosity, response format, tool choice and streaming flags. Explicit zero/false parameters remain distinct from absent ones. Ordinary HTTP relay metadata does not infer missing parameter defaults. Adapters differ: Anthropic metadata comes from the translated body, including its configured output-limit default, and Cursor stores the base model ID rather than its fused spelling. See [adapters](adapters.md) before treating these fields as an exact representation of the caller's original request. Opening ordinary details does not fetch private request content. Prompt/response previews appear only if captured, and Debug sidecars use their separate opt-in and retention contract. Even ordinary labels, error text and headers can be sensitive; do not publish request drawers from private history without review. ### Reading timing, cost and health Request duration starts after upload and includes metadata work, queue/hold/retry waits and response handling. TTFT is measured from the upstream attempt's own send start to the first token (reasoning or answer), so every retry has its own TTFT. The value excludes proxy scheduler queueing, request parsing and retry backoff, but it does span the upstream transport window from request handoff (dial, TLS, connection-pool wait, header wait). A request that produced no token, one that never sent an upstream attempt of its own (a parked cursor resume), and any non-streamed response have no TTFT: the log shows a placeholder, not 0 ms, and aggregates skip it. Rows written before 0.3.34 stored `ttft_ms` anchored at request start (upload, queueing and retry backoff included) and were zeroed once by the boot repair, so they report TTFT as absent; their `started_at` and `first_token_at` timestamps remain, keeping `pre_token_ms` and decode windows usable. TPS is the single generation-speed measure: (generation tokens - 1) over the first-token to last-token window. Average TTFT and average TPS are the arithmetic means of those per-request samples. TPS is absent for fewer than two tokens, a non-positive window, or a response that did not stream (no token timestamps). The compact surfaces (the KPI band and the Overview timing tile) abbreviate this unit as `t/s`; roomier readouts spell it `tok/s`. The numerator assumes the first observed content chunk carries one token; a provider that bundles several tokens into that chunk can still overstate the rate. These are provider-attempt measurements, not claims about total user wait or system-wide throughput. Money stays USD in storage and APIs. The shared UI formatter uses cents below $1, including fractional cents, and dollars for larger magnitudes. Usage/cost comes from recognized upstream fields or configured field mappings; unavailable reporting and a displayed zero do not establish a zero bill. Reasoning/cache splits depend on what the upstream actually reports. When a stream carries no usage object, or carries one whose output count is zero, the recorded output token count is restored from the streamed chunk count, which includes tool-argument fragments; the count is an estimate, not a provider report. Chat tool calls are deduplicated by delta index while the retained index set holds up to 1,024 distinct indexes. Once the set is full a further index cannot be retained, so each of its fragments counts again; a hostile stream of repeated indexes can overstate the tool-call count without growing memory past the bound. Errors count affected requests with a genuine failure, including recovered upstream failures. Rate limits count requests that saw a rate-limit class on the final, a retried or an in-band attempt: an HTTP 429, a 429 whose status line a committed 200 could not relay, or a provider spelling the classifier recognizes (such as `tier_limit_exceeded` on a 400); each request counts once. The two signals can overlap: a bare 429 is not an error, but a non-429 status that carries a rate-limit class failed the request and counts in both. Client-local cancellation is likewise distinct from an upstream failure - except a 499 whose record carries a structured error type: that observed a failure before the client left, either the provider's in-band stream failure (typically an error event the client received and aborted around) or a truncated stream whose completion marker never arrived, independent of which relay write failed. It counts as the upstream error it is. Zero explorer health badges are hidden independently. Error-dimension card headlines count failure events (final plus absorbed), while card N counts distinct affected requests. The gallery share denominator `error_total` is the union of affected requests represented by the error cards, so a card's share is N / `error_total` and can never exceed 100%. This denominator includes recovered quality failures whose final status is not an error (for example an absorbed empty-completion attempt), counting each affected request once even when no request ends in an error status. For every other dimension `error_total` remains the canonical affected-request error count. Do not treat every displayed error number as the same metric. The explorer gallery renders at most 24 cards. Its fold tracks exact per-value accumulators for at most 8192 distinct values in the active dimension; later values are combined into a synthetic `other` card with aggregate count, cost, token, tool and health fields, competing for the 24 cards by that count. The synthetic card carries `synthetic: true` on the wire, renders as `other (grouped)`, and is not filterable. A real value named `other` keeps its own exact card and filter; `other` is a display bucket only when the card is synthetic. Rail distinct-value counts, footer scope totals and conversation summaries still cover the whole filtered history, so card counts need not add up to them. ### Snapshots, live feeds and monitoring The HTML includes the canonical bootstrap state so the first paint need not wait for a second state request. Live SSE carries pending progress and finalization; KPI/operator polling and configured aggregate refresh keep derived surfaces current. Reconnects use sequence/feed identity and fall back to full snapshots when replay cannot be trusted. The footer distinguishes live/offline, active holds/Debug, process-local storage drops, totals degraded by a failed startup history scan, and a degraded operator-state restore. The last one is the difference between "nothing is held" and "a saved hold or limit document could not be read": the served `pause` and `throttle` snapshots carry `restore_degraded`, so a truncated document does not boot as a healthy, unpaused proxy with its saved rules silently gone. The flag stays true for the life of the process; a restart that reads a document which decodes is what clears it. Each key holds one value, so the next accepted operator write replaces the unreadable document rather than preserving it: capture the bytes first if they are needed as evidence. An offline indicator alone does not prove inference is down. None of these mechanisms makes initial history preload instantaneous. For integrations, use the documented read routes rather than scraping the DOM. Bootstrap/live data is a recent observation window; scoped aggregate routes query available history; JSON export provides finalized records; restricted SQL queries durable data within configured time/output limits. The API is in initial development, so pin and verify the selected revision for integrations. Prometheus is a bounded-ring view. Its request/token/error totals are gauges that can shrink after eviction, purge or restart, not monotonic lifetime counters; do not apply counter-only `rate()`/`increase()` assumptions to them. Its TTFT quantiles use the ring's sample contract, which differs from dashboard R7 history percentiles. Metric names and units are exposed with HELP/TYPE metadata at `/metrics/prometheus`; [prometheus.go](../internal/metrics/prometheus.go) owns them. An unchanged finalized ring revision serves the exact last exposition without another ring copy, aggregate or TTFT sort; any ring content transition rebuilds. ### Operator recipes The gated query route (`GET /metrics/query?q=`, above) and the [MCP](mcp.md) `query` tool both take exactly one `SELECT` with no statement separator, and results are bounded by `storage_query_max_rows` and `storage_query_max_bytes`; an over-budget result is an error on the route, so keep wide scans behind a `started_at` window or a `LIMIT`. The statement gate also denies the read-only capabilities SQLite exposes outside ordinary tables and columns: `ATTACH`, `DETACH`, `PRAGMA`, `LOAD_EXTENSION`, the `sqlite_dbpage` and `dbstat` table-valued functions, the whole result-bearing `pragma_*` family and the `readfile`/`writefile` functions. Those would read raw pages, page statistics or the absolute database path, which is not part of this console's surface. The denial is lexical over the raw query text, string literals included, so a keyword spelled inside a literal is refused too: `SELECT 'attach' AS x` and `SELECT 'pragma_table_info'` both fail. A name only has to contain the bare word at an ASCII boundary; `attachment` and `attach_x` pass. The driver executes a multi-statement string wholesale, returning the rows of the last statement, so both rules are load-bearing: without the `;` refusal a second `ATTACH` statement would run behind a `SELECT` prefix the prefix check accepts, and the keyword scan alone would not see it. Column meanings come from the MCP `describe` tool. `started_at` is Unix milliseconds: a relative window is `started_at >= strftime('%s','now')*1000 - 3600000`, and calendar math divides by 1000 and applies SQLite's `unixepoch` modifier, with `localtime` for the host's zone. The recipes below earn their keep in the derived shapes no dashboard surface computes: the outlier ratio, the write/read ratio, the latency decompositions. Their ingredient aggregates are conceded up front, the way the decode window already is: the explorer sums per-group counts and cache reads (its `N` and `Cache` fields) and a per-group blended $/Mtok (its `cost_per_mtok` KPI, the same cost-reporting in+out rule), and the chart's dec series samples the shared decode window (`last_token_at - first_token_at`) that recipe 1 decomposes latency around. Several of the columns a recipe reads do overlap the aggregate scans: `ttft_ms`, `reasoning_tokens`, `status_code`, `attempts`, `output_tokens` and `cache_read_tokens` all appear in the dashboard's scan lists. The recipes' private reading set, which no aggregate scan reads, is `queue_wait_ms`, `first_answer_at`, `retry_after_ms`, `cache_write_tokens`, the `chars_*` columns, `total_tokens`, `req_max_tokens`, `duration_ms` and `retries`. Each recipe is a question, the SQL that answers it, and how to read the result. **Where did the time go on a slow request?** ```sql SELECT started_at, duration_ms, queue_wait_ms, retries, first_token_at - started_at AS pre_token_ms, ttft_ms, last_token_at - first_token_at AS decode_ms, started_at + duration_ms - last_token_at AS tail_ms FROM requests WHERE started_at >= strftime('%s','now')*1000 - 3600000 AND first_token_at > 0 ORDER BY duration_ms DESC LIMIT 20 ``` `pre_token_ms` is the wait the caller felt, including queue and retry waits; `ttft_ms` is measured from the adopted upstream attempt's send start, so a retried request shows a small ttft beside a large `pre_token_ms`. Rows written before 0.3.34 carry a start-anchored `ttft_ms` and were zeroed once by the boot repair, so their `ttft_ms` reads 0 (absent) while `pre_token_ms` and `decode_ms` stay usable from the stored timestamps. `decode_ms` is the streaming window and `tail_ms` what remains after the last token, where a millisecond of rounding noise is normal. A request that produced no token, or was not streamed, has nothing to decompose, hence the `first_token_at > 0` gate. **How long did a model think before answering?** ```sql SELECT started_at, model, reasoning_tokens, first_answer_at - first_token_at AS thinking_ms FROM requests WHERE started_at >= strftime('%s','now')*1000 - 3600000 AND first_answer_at > 0 ORDER BY thinking_ms DESC LIMIT 20 ``` `thinking_ms` spans the first token of any kind, reasoning included, to the first answer token; 0 means the first token already was the answer. `first_answer_at` is streaming-only on the generic relay; cursor-format non-streaming requests set it (the bridge synthesizes each delta and tracks it via cursorTrackDelta), and such rows legitimately enter the gate because `first_token_at` is stamped by the same helper. Tool-call-only streams keep 0 and stay outside the gate. **How long does an agent idle between conversation turns?** ```sql SELECT conversation_id, started_at, started_at - LAG(started_at + duration_ms) OVER (PARTITION BY conversation_id ORDER BY started_at) AS idle_ms FROM requests WHERE started_at >= strftime('%s','now')*1000 - 86400000 AND conversation_id != '' ORDER BY conversation_id, started_at ``` `idle_ms` is the gap between one request's end and the next request's start inside the same conversation; the first turn has no predecessor and stays null. Order by `idle_ms DESC` instead to surface the biggest stalls first. **What retried before the client saw a response?** ```sql SELECT r.started_at, r.id, r.retries, r.status_code AS final_status, j.value->>'status_code' AS attempt_status, j.value->>'at' AS attempt_at, j.value->>'retry_after_ms' AS retry_after_ms FROM requests r, json_each(r.attempts) j WHERE r.started_at >= strftime('%s','now')*1000 - 86400000 AND json_array_length(r.attempts) > 0 AND COALESCE(json_extract(j.value,'$.in_band'),0)=0 ORDER BY r.started_at, j.key ``` One row per absorbed retry, oldest first. The `in_band` filter excludes a 429 recorded behind a committed 200: that entry is health evidence, not a retry. The `json_array_length` gate is required: `attempts` defaults to the JSON literal `null`, and `json_each` over that literal still yields a row. `retry_after_ms` appears (nonzero) when the provider sent a `Retry-After` header or a ratelimit-reset header (`x-ratelimit-reset-requests`/`-tokens`, `anthropic-ratelimit-requests-reset`, `anthropic-ratelimit-tokens-reset`), bounded by the proxy's hint ceiling. **Did a conversation's cache writes pay for themselves?** ```sql SELECT conversation_id, COUNT(*) AS requests, SUM(cache_write_tokens) AS cache_written, SUM(cache_read_tokens) AS cache_read, CASE WHEN SUM(cache_read_tokens) > 0 THEN ROUND(1.0 * SUM(cache_write_tokens) / SUM(cache_read_tokens), 2) END AS write_per_read FROM requests WHERE conversation_id != '' GROUP BY conversation_id ORDER BY cache_written DESC, cache_read DESC LIMIT 20 ``` `write_per_read` at or below 1 means the reads repaid the writes; reads with no writes ride a cache somebody else warmed. A conversation with writes and no reads paid for a prompt that never came back. **When does traffic arrive?** ```sql SELECT strftime('%w', started_at/1000, 'unixepoch', 'localtime') AS weekday, strftime('%H', started_at/1000, 'unixepoch', 'localtime') AS hour, COUNT(*) AS requests, SUM(cost) AS cost, ROUND(AVG(duration_ms), 1) AS avg_ms FROM requests GROUP BY weekday, hour ORDER BY weekday, hour ``` `weekday` is `0` (Sunday) through `6` and `hour` is two digits in the host's local zone; drop `localtime` for UTC. Cost is USD, summed as stored. **How far does the token estimate drift?** ```sql SELECT model, COUNT(*) AS requests, ROUND(AVG((chars_system + chars_user + chars_assistant + chars_tool) / 4), 1) AS avg_chars_est, ROUND(AVG(input_tokens), 1) AS avg_input, ROUND(AVG(1.0 * input_tokens / ((chars_system + chars_user + chars_assistant + chars_tool) / 4)), 3) AS input_vs_chars4, ROUND(AVG(1.0 * total_tokens / ((chars_system + chars_user + chars_assistant + chars_tool) / 4 + req_max_tokens)), 3) AS total_vs_est FROM requests WHERE input_tokens > 0 AND chars_system + chars_user + chars_assistant + chars_tool > 0 GROUP BY model ORDER BY input_vs_chars4 DESC ``` The scheduler reserves prompt chars/4 plus `req_max_tokens` when the client sent one. `input_vs_chars4` compares real input tokens with the chars/4 share, and `total_vs_est` compares settled total tokens with the full reservation. A ratio far from 1 on a well-represented model means the reservation systematically over- or under-reserves for it. **Which requests paid far above their model's blended rate?** ```sql SELECT o.model, o.id, o.started_at, ROUND(1e6 * o.cost / (o.input_tokens + o.output_tokens), 2) AS per_mtok, ROUND(1e6 * a.cost_sum / a.tok_sum, 2) AS model_avg_per_mtok, ROUND((1e6 * o.cost / (o.input_tokens + o.output_tokens)) / (1e6 * a.cost_sum / a.tok_sum), 2) AS ratio FROM requests o JOIN (SELECT model, SUM(cost) AS cost_sum, SUM(input_tokens + output_tokens) AS tok_sum FROM requests WHERE cost > 0 GROUP BY model) a ON a.model = o.model WHERE o.cost > 0 AND o.input_tokens + o.output_tokens > 0 ORDER BY ratio DESC LIMIT 10 ``` `per_mtok` is the request's own rate over its in+out tokens, and `ratio` scales it against the model's blended average, computed over cost-reporting requests only so unpriced traffic cannot dilute it. A ratio far above 1 marks expensive outliers such as cache misses or long-tail pricing; flip the sort to find the bargains. SQLite has no `median()`, which is why the blended average plus a sorted ratio stands in for it. --- # Reverse proxy and remote access Use a protected ingress when accessing millivolt beyond a trusted local machine. The whole dashboard is gated by the shared `MILLIVOLT_OPERATOR_TOKEN` credential; there is no tenant isolation and no per-user identity. At the application gate, only the unauthenticated `/healthz` and `/readyz` probes, the `POST /session` login handshake, origin-root brand/PWA files and transparent inference stay open, so a load balancer can health-check the service without holding the credential and a browser can reach the login handshake. The canonical open-path list is in [operator access](routes.md#operator-access); this guide only adds the ingress boundary. Read [Security](../SECURITY.md) first. ![Reverse proxy topology: the API client and dashboard browser reach NGINX over TLS, NGINX forwards over loopback to millivolt, and millivolt sends requests to the selected upstream](images/diagrams/reverse-proxy.svg) The diagram is a text-equivalent summary of the example below: NGINX terminates TLS and enforces access; millivolt owns routing, the operator session, and the history; the upstream receives the selected credentials. ## NGINX setup [deploy/nginx.conf](../deploy/nginx.conf) is the canonical example. It serves millivolt at the origin root, with NGINX on the same host as the loopback-published Compose service. Use a maintained NGINX release with the `http2 on` directive (introduced in 1.25.1), not the deprecated `listen ... http2` syntax. The sample splits two prefix locations because they do not share an access model: - `location /v1` is the OpenAI-compatible inference base: preserve provider `Authorization` and routing headers, and allowlist API clients. - `location /` covers everything else, including the dashboard and operator plane (`/dash/`, `/metrics/`, `/admin/`) plus the application-open `POST /session`, `/healthz`, `/readyz` and brand/PWA paths: allowlist operators, and keep `Host` plus the session cookie so Settings origin checks and the live feed work; the comparison is case-insensitive, so a case-preserving ingress is fine. The `/session` handshake is open at millivolt so login works, but this location still allowlists it at the edge. A URL-capable MCP client uses this location with `/mcp` appended and the operator Bearer header: the application's MCP endpoint is Bearer-only and refuses the session cookie. Shared streaming proxy settings live on the `server` so both SSE paths stay unbuffered. Prefix `/v1` wins over `/` for `/v1/chat/completions` and `/v1/models`; `GET /models` without that prefix follows `location /` unless you add an exact `/models` block with the inference rules. 1. Start the normal [Compose deployment](../README.md#docker-compose), retaining its loopback-only host port. 2. Copy the example into your NGINX HTTP configuration. Replace its example hostname, certificate paths and the two TEST-NET allowlists. The supplied address is a placeholder; unmatched clients are denied. The lists may differ when operators and API clients are not the same sources. Set `proxy_pass` to the chosen host port if `MILLIVOLT_PORT` is not 8080. 3. Validate with `nginx -t` before reloading your existing NGINX service. Confirm a trusted client and a denied client on both `/` and `/v1`. 4. Use `https://your-hostname/` for the dashboard and `https://your-hostname/v1` as the client API base. Keep sending the actual upstream base/key as described in the [client setup](../README.md#connect-a-client). The sample is a `server` block, included under NGINX's existing `http` context, not a replacement for the entire installation configuration. It does not issue certificates or configure your firewall. Do not expose the original proxy port alongside the protected HTTPS endpoint. ### Streaming and request handling The example disables response/request buffering and proxy retries so the edge does not batch SSE events or independently replay inference. It leaves HTTP statuses intact, disables edge compression/caching, and ignores special upstream `X-Accel-*` controls for buffering, redirects, rate limits, caching and charset. Millivolt uses SSE, not a WebSocket endpoint; no `Upgrade` block is needed. It preserves the external `Host`, including a nondefault port, so browser-origin checks on Settings and other mutations keep working. It also overwrites `X-Forwarded-Proto` with this server's `$scheme`, so a TLS presentation reaches millivolt as `https` and the session cookie carries `Secure`; a client-supplied value cannot spoof it. Overwriting is the recommended form. If your ingress appends its scheme instead (`$http_x_forwarded_proto,$scheme`), the application still marks the request secure when any token does, so a client-supplied `http` in first position cannot downgrade the cookie. The sample also resolves the client address with `real_ip`: ```nginx set_real_ip_from 127.0.0.1; real_ip_header X-Forwarded-For; ``` Without it, millivolt sees this hop's address as `RemoteAddr` for every request and the per-source authentication throttle gets one bucket, so a single wrong guess locks out every operator. `set_real_ip_from` lists the addresses allowed to send a replacement address: this host's own loopback. If another load balancer, tunnel or proxy fronts NGINX, add its address range there and set `real_ip_recursive on` once every hop in the chain is listed. `real_ip` walks `X-Forwarded-For` from the right and keeps the last untrusted address, so a client that prepends a forged entry cannot displace the real one; NGINX's default `$proxy_add_x_forwarded_for` then appends the resolved address on the way inbound, and the per-source authentication throttle reads that last element. The record field is separate: `client_ip` records the first `X-Forwarded-For` element when that header is present and the socket peer otherwise; the value is diagnostic, never verified, and a directly reachable caller can set it. The resolved address is a throttle key, not an identity: keep the location allowlists in force. Those lists evaluate the resolved address, so a caller that already reaches this host's loopback chooses what they see: they separate remote networks from each other and from loopback callers, not loopback callers from each other. NGINX's body-size gate is disabled in the sample: millivolt's existing `max_request_bytes` remains the application limit, rather than an accidentally smaller duplicate at ingress. Choose edge idle timeouts for your deployment. `proxy_read_timeout` limits the gap between upstream reads, not the total lifetime of a stream. These behaviors follow the official [proxy module](https://nginx.org/en/docs/http/ngx_http_proxy_module.html), [HTTP core module](https://nginx.org/en/docs/http/ngx_http_core_module.html), [HTTP/2 module](https://nginx.org/en/docs/http/ngx_http_v2_module.html) and [realip module](https://nginx.org/en/docs/http/ngx_http_realip_module.html). ## Credentials and access control The sample uses a trusted IP/VPN allowlist on each location, preserving provider `Authorization` on `/v1` and the operator Bearer or session cookie on `/`. Replace each placeholder with the addresses you intend to trust for that plane. If another load balancer is in front of NGINX, list its addresses in `set_real_ip_from` and switch `real_ip_recursive` on, so the trusted chain matches reality; a hop that appends to `X-Forwarded-For` without being listed is ignored on the left of the real client. TLS encrypts traffic but does not authorize callers. A secure presentation also makes millivolt answer `Strict-Transport-Security: max-age=15552000` (180 days, no `includeSubDomains`) so a browser pins the scheme; NGINX's own `add_header` is not needed for it, and adding a second HSTS header at the edge would send two values. See NGINX's [access module](https://nginx.org/en/docs/http/ngx_http_access_module.html) and [TLS configuration](https://nginx.org/en/docs/http/ngx_http_ssl_module.html). If you instead add HTTP Basic authentication on `/v1`, `Authorization` contains the ingress credential and cannot simultaneously contain the provider bearer key. Supply the provider key through `X-Proxy-Key`, and clear the consumed ingress header before forwarding with `proxy_set_header Authorization "";`. The same conflict applies on `/`: a browser behind Basic auth cannot also present the dashboard's Bearer credential in the same header, so gated actions fail. Prefer an address/VPN allowlist as in the sample, or an ingress that adds its own credential through a different mechanism and forwards `Authorization` untouched. Do not put Basic auth on `/v1` to "protect inference" while leaving `/` on Bearer: that still overwrites the provider key. The same separation applies to other ingress-only credentials: do not forward them to an LLM provider. Clients must support your ingress authentication and the proxy's custom routing headers. The session handshake lives at `/session`, outside the admin plane, which supports one specific pattern: an external authentication layer may front `/admin/*` alone, because millivolt's own login stays reachable and the mutation and settings routes become double-gated, the external layer plus millivolt's credential. Operators fronting `/admin` this way must still pass the session cookie and `Host` through to the proxy, since the dashboard's fetch-based mutations rely on both. ## Container networking and other ingress servers When NGINX itself is a container, `127.0.0.1` points back to NGINX, not millivolt. Attach it to the same private Docker network and use `http://millivolt:8080` as its upstream instead. Only the edge needs a publicly reachable port. Do not bind millivolt to loopback inside its container, which would prevent peer access. `MILLIVOLT_PORT` changes host publication, not this internal service port. For another ingress implementation, preserve these same boundaries: protect the origin, keep inference and the operator plane as separate access rules, forward the original authority and required custom headers, allow streaming without response buffering, avoid automatic inference retries, and choose upload/idle limits deliberately. Subpath hosting is not the supplied deployment: dashboard assets and routes expect the origin root. --- # Performance and resource usage This page is the single source for the published performance and resource numbers. It records what was measured, how it was measured, and what the result does not prove. The [performance and footprint](operations.md#performance-and-footprint) section in operations owns the implementation cost model and `go_memory_limit` behavior. These are scoped local measurements of the native proxy process, not deployment guarantees. The page was regenerated from the result bundle on 2026-11-16 UTC. ## Artifact and idle footprint | Artifact or state | Bytes | Display | Notes | | --- | ---: | ---: | --- | | Static amd64 binary | 21,663,904 | 20.7 MiB | Stripped, no CGO | | amd64 runtime image | 38,022,232 | 36.3 MiB | Docker-reported content size | | Compressed image archive | 9,887,262 | 9.4 MiB | `docker save` piped through `gzip -9` | | Idle process, empty SQLite | 30,204 KiB | 29.5 MiB | After a 15-second quiet interval | | Idle process CPU | 2 scheduler ticks | 0.067% | 2 ticks over 30 seconds | The image size includes the distroless base, dependency notices, and the embedded UI. It is not a compressed download size, a registry digest, or a persistent-volume estimate. Idle RSS and CPU describe one process on this host. A short sample cannot prove that a longer idle period uses exactly the same resources. ## Method | Item | Setting | | --- | --- | | Date | 2026-11-16 | | Source revision | [`22c0b6b73655`](https://github.com/LLM-4-People/millivolt/commit/22c0b6b73655a349e086c345bc80907efbe16822) | | Go toolchain | go1.27.4, `CGO_ENABLED=0` | | Platform | Linux/amd64 under WSL2 | | CPU | Intel(R) Core(TM) Ultra 9 285H, 12 visible logical CPUs | | Visible RAM | 62 GiB | | Configuration | Public example with private listen and database paths | | Debug captures | Off | | Body previews | Off | | Browser and live observer | None during load stages | | Upstream | Local synthetic server on loopback | | Load shape | Closed-loop HTTP stages, then drain | | RSS sampling | 100 ms | | CPU unit | 100% equals one logical CPU | | Accounting check | Exact stages reconcile rows, tokens and cost; a dropping stage reconciles durable plus dropped | | Host load (1-minute) | 23.36 at start, 13.38 at end | The public example configuration was used, with private listen and database paths supplied by `scripts/dev.sh`. The method table records the harness declarations for captures, previews, the observer and the upstream. CPU and RSS describe the native Go process, not Docker's container-wide accounting. Percentiles are nearest-rank. ## Fast-response throughput Each stage used a fresh disposable database and the same synthetic response: 10 content frames, usage `100/10`, and fixture cost `0.001`. The upstream added no intentional delay. Each stage started requests for 5 seconds, then let active requests drain. The final row is a separate 10-second run used to reach about 129,442 durable rows without a storage drop. | Client workers | HTTP requests | Requests/s | p50 | p95 | p99 | Proxy CPU | Peak proxy RSS | Durable rows | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | 1 | 6,879 | 1,375.7 | 0.66 ms | 1.09 ms | 1.66 ms | 52.6% | 61.5 MiB | Exact, 6,879 | | 8 | 30,264 | 6,050.9 | 1.17 ms | 2.28 ms | 4.06 ms | 239.3% | 88.3 MiB | Exact, 30,264 | | 32 | 53,081 | 10,567.2 | 2.55 ms | 6.01 ms | 8.63 ms | 374.3% | 93.0 MiB | Exact, 53,081 | | 128 | 66,813 | 13,346.4 | 8.40 ms | 20.19 ms | 28.29 ms | 465.2% | 119.0 MiB | Exact, 66,813 | | 128, 10 s | 129,442 | 12,931.5 | 8.50 ms | 21.30 ms | 31.58 ms | 459.7% | 124.5 MiB | Exact, 129,442 | The exact fast-response point is 13,346.4 requests/s at 128 workers, with p95 20.19 ms and p99 28.29 ms. The 128-worker 10-second stage used 227.3% CPU and 33.7 MiB peak RSS in the separate load generator and synthetic upstream process. All HTTP responses succeeded in these stages, including the 128-worker stage. That is not a promise that every configuration and history size stays exact under overload. The durable queue is bounded and best-effort; the deliberate overload fixture below shows what a drop result looks like. ## Sustained capacity The sustained capacity point is the highest tested closed-loop level that completed a 300-second run with zero durable drops and exact accounting. Each level started from a fresh disposable database. The upstream response and client shape were the same as the fast-response table: 10 content frames, usage `100/10`, and fixture cost `0.001`. | Workers | Duration | HTTP requests | Requests/s | p50 | p95 | p99 | Proxy CPU | Peak proxy RSS | Durable rows | Drops | | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | 32 | 300.00 s | 2,584,965 | 8,616.5 | 2.54 ms | 9.93 ms | 19.71 ms | 289.1% | 107.0 MiB | Exact, 2,584,965 | 0 | | 48 | 300.01 s | 3,683,159 | 12,276.8 | 3.12 ms | 8.92 ms | 15.68 ms | 421.1% | 121.9 MiB | Exact, 3,683,159 | 0 | | 64 | 300.00 s | 4,649,811 | 15,499.2 | 3.62 ms | 8.18 ms | 12.03 ms | 522.6% | 125.8 MiB | 4,602,957 kept | 46,854 | | 128 | 300.03 s | 4,169,782 | 13,897.8 | 7.77 ms | 20.25 ms | 32.11 ms | 451.4% | 141.1 MiB | 3,775,709 kept | 394,073 | The exact 300-second point is 12,276.8 requests/s at 48 workers, with p95 8.92 ms and p99 15.68 ms. The 64-worker point dropped 46,854 rows (1.0% of its requests). A level that drops is an overload observation, not a capacity result; the bounded durable queue is not lossless at every load. ## Streaming concurrency The synthetic upstream emitted 10-frame response across approximately 1-second stream, followed by usage and a terminator. Each stage started its fixed number of requests and then drained. Latency includes the intentional upstream delay, so these numbers are not proxy-only overhead. | Workers | HTTP requests | Requests/s | SSE events/s | Output tokens/s | Response KiB/s | p50 | p95 | p99 | First-byte p95 | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | 1 | 5 | 1.0 | 12 | 10 | 0.6 | 1,009.61 ms | 1,010.49 ms | 1,010.49 ms | 2.22 ms | | 8 | 40 | 7.9 | 95 | 79 | 4.7 | 1,007.21 ms | 1,009.82 ms | 1,010.22 ms | 3.59 ms | | 32 | 160 | 31.4 | 377 | 314 | 18.8 | 1,007.57 ms | 1,016.44 ms | 1,059.19 ms | 8.23 ms | | 64 | 320 | 62.9 | 755 | 629 | 37.6 | 1,008.33 ms | 1,037.34 ms | 1,054.23 ms | 31.17 ms | | 128 | 640 | 124.5 | 1,494 | 1,245 | 74.4 | 1,009.33 ms | 1,074.57 ms | 1,107.40 ms | 69.27 ms | | 256 | 1,280 | 248.0 | 2,976 | 2,480 | 148.2 | 1,010.96 ms | 1,093.08 ms | 1,119.54 ms | 89.49 ms | | 512 | 2,560 | 490.0 | 5,880 | 4,900 | 292.9 | 1,008.64 ms | 1,150.71 ms | 1,153.58 ms | 145.75 ms | | 1,024 | 3,072 | 494.8 | 5,938 | 4,948 | 295.7 | 2,016.27 ms | 2,144.02 ms | 2,157.64 ms | 1,133.32 ms | | Workers | Proxy CPU | Peak proxy RSS | Upstream peak | Durable rows | Drops | | --- | ---: | ---: | ---: | ---: | ---: | | 1 | 0.2% | 31.7 MiB | 1 | Exact, 5 | 0 | | 8 | 1.0% | 32.7 MiB | 8 | Exact, 40 | 0 | | 32 | 3.7% | 36.7 MiB | 32 | Exact, 160 | 0 | | 64 | 6.9% | 40.2 MiB | 64 | Exact, 320 | 0 | | 128 | 13.2% | 49.8 MiB | 128 | Exact, 640 | 0 | | 256 | 24.6% | 67.4 MiB | 256 | Exact, 1,280 | 0 | | 512 | 48.6% | 103.7 MiB | 512 | Exact, 2,560 | 0 | | 1,024 | 48.6% | 137.9 MiB | 512 | Exact, 3,072 | 0 | These stages measure concurrency and end-to-end latency for that fixed fixture. They do not measure SSE event rate, bytes per second, token throughput, TTFT, chunk-size or hold-duration effects, sustained streaming, or the point where streaming saturates. First-byte p95 is the first response byte, not the first generated token. The 1,024-worker stage reached 512 upstream connections and remained exact. This is a configured concurrency result on one host, not a universal request-capacity ceiling. The largest level also matches the example configuration's `max_conns_per_host` of 512. No higher level, sustained streaming stage, or automatic saturation sweep was run; see [Limitations](#limitations). ## Streaming throughput `scripts/perf.sh streaming` runs the longer end-to-end throughput ramp. Each level starts a fresh private `scripts/dev.sh` instance and sends requests through the real proxy to a local synthetic SSE upstream. Each level ran for 60 seconds with 100 content frames over a roughly 1-second stream. | Workers | HTTP requests | Requests/s | SSE events/s | Output tokens/s | Response KiB/s | p50 | p95 | p99 | First-byte p95 | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | 1 | 57 | 0.9 | 96 | 94 | 4.8 | 1,067.07 ms | 1,079.25 ms | 1,085.80 ms | 3.02 ms | | 8 | 448 | 7.5 | 760 | 746 | 37.9 | 1,066.92 ms | 1,125.67 ms | 1,146.53 ms | 6.08 ms | | 32 | 1,824 | 30.3 | 3,086 | 3,026 | 153.7 | 1,055.96 ms | 1,067.91 ms | 1,073.94 ms | 3.17 ms | | 64 | 3,648 | 60.2 | 6,140 | 6,020 | 305.9 | 1,061.67 ms | 1,072.37 ms | 1,088.48 ms | 3.88 ms | | 128 | 7,296 | 120.5 | 12,291 | 12,050 | 612.3 | 1,060.13 ms | 1,069.44 ms | 1,113.02 ms | 5.44 ms | | 256 | 14,592 | 239.2 | 24,395 | 23,916 | 1,215.2 | 1,061.58 ms | 1,138.38 ms | 1,174.13 ms | 11.75 ms | | 512 | 28,789 | 471.5 | 48,094 | 47,151 | 2,395.8 | 1,063.37 ms | 1,141.20 ms | 1,300.62 ms | 15.32 ms | | 1,024 | 29,696 | 478.6 | 48,819 | 47,861 | 2,431.9 | 2,128.84 ms | 2,162.96 ms | 2,285.10 ms | 1,091.76 ms | | Workers | Proxy CPU | Peak proxy RSS | Upstream peak | Durable rows | Drops | | --- | ---: | ---: | ---: | ---: | ---: | | 1 | 1.2% | 31.5 MiB | 1 | Exact, 57 | 0 | | 8 | 5.5% | 37.1 MiB | 8 | Exact, 448 | 0 | | 32 | 14.9% | 47.8 MiB | 32 | Exact, 1,824 | 0 | | 64 | 28.2% | 59.0 MiB | 64 | Exact, 3,648 | 0 | | 128 | 57.9% | 81.3 MiB | 128 | Exact, 7,296 | 0 | | 256 | 115.1% | 114.9 MiB | 256 | Exact, 14,592 | 0 | | 512 | 233.1% | 152.2 MiB | 513 | Exact, 28,789 | 0 | | 1,024 | 239.9% | 189.4 MiB | 512 | Exact, 29,696 | 0 | Throughput peaked at 478.6 requests/s at 1,024 workers. At 1,024 workers the upstream peak stayed at 512, throughput was 478.6 requests/s against 471.5 at 512 workers, and p50 latency rose from 1.06 s to 2.13 s. The extra clients queued behind the configured 512 upstream connections. Every level completed with exact durable accounting unless its row reports drops. This run started under the host load the harness recorded, so absolute rates are comparable only within this page and this host. ## Durable writer and overload These are isolated Go benchmarks from the test overlay, not HTTP runs. Every row and drop count is verified against successful requests. The overload benchmark deliberately fills the bounded queue, so its drop count is a fixture result, not a recommended operating point. | Path | Durable rows/s | Allocations | | --- | ---: | ---: | | Direct batch insert at the configured batch capacity | 62,921 | 425,396 B/op, 7,279 allocs/op | | Queued `Record` plus `Flush` at the queue capacity | 56,055 | 13,594,117 B/op, 232,920 allocs/op | | Overload fixture | Wire requests/s | Durable rows/s | Dropped rows | | --- | ---: | ---: | ---: | | 32 workers | 72,619 | 19,994 | 101,507 | | 128 workers | 64,161 | 14,605 | 113,242 | The overload fixture confirms the accounting invariant `durable + dropped = successful requests`. Increasing queue capacity alone cannot make sustained overload lossless. A higher HTTP success rate is not a complete-accounting result unless the durable row count and drop delta agree. ## Storage and retained projection The storage fixture stores complete records, indexes, projection triggers, and marshaled response headers in SQLite. Disk bytes include the database, WAL, and shared-memory sidecar after a final flush. | Measure | 10,000 records | 100,000 records | | --- | ---: | ---: | | SQLite bytes total | 6.84 MiB | 49.20 MiB | | SQLite bytes per record | 717.3 | 515.9 | | Projection retained, built | 449.6 B/record | 470.8 B/record | | Projection interned dimensions | 442.0 B/record | 429.1 B/record | | Projection scanned decode | 580.3 B/record | 576.0 B/record | | Projection transient allocation | 3,125 B/record | 3,434 B/record | The projection process benchmark is a relative measure inside the test binary, not the deployed proxy's RSS. At 100,000 records it measured 94.6 MiB process RSS and a 66.0 MiB projection delta; at 10,000 records it measured 40.6 MiB process RSS and a 12.0 MiB delta. The test binary and its linked packages are part of the absolute figure. A fast-response run reached a 119.0 MiB peak with 66,813 durable rows while no dashboard projection was preloaded. Use the projection delta above for the retained analytical cost, not the loaded-process peak as a substitute. ## Query and metadata paths The query benchmarks seed a 100,000-row scratch database and decode a 200-row page. These are single-iteration measurements on a shared host, so small differences are not meaningful. | Query | Before cursor | With cursor | Allocations before cursor | | --- | ---: | ---: | --- | | Provider equality | 4.24 ms | 3.85 ms | 1.38 MB, 5,455 allocs | | Model equality | 2.81 ms | 2.93 ms | 1.38 MB, 5,445 allocs | | Client equality | 36.52 ms | 18.73 ms | 1.38 MB, 5,442 allocs | | Key-hash equality | 33.68 ms | 18.28 ms | 23.5 KB, 265 allocs | | Conversation equality | 40.16 ms | 21.73 ms | 23.2 KB, 264 allocs | | Status equality | 38.98 ms | 20.99 ms | 698.1 KB, 2,942 allocs | The request metadata path performs one bounded metadata decode; the allocation column is measured per call. | Metadata body | Time per request | Throughput | Allocations | | --- | ---: | ---: | ---: | | 1 KiB | 428 ns | 2,580.8 MB/s | 16 B, 1 alloc | | 100 KiB | 4,808 ns | 21,314.0 MB/s | 16 B, 1 alloc | | 1 MiB | 75,397 ns | 13,908.4 MB/s | 16 B, 1 alloc | ## Implementation benchmarks These are the isolated benchmark results the design notes cite. They are current measurements from the same run as the tables above, not a historical before and after across revisions. | Benchmark | Time | Bytes per op | Allocs per op | | --- | --- | --- | --- | | `BenchmarkAggregateFold/chart` | 2.74 ms | 136,736 B | 28 | | `BenchmarkAggregateFold/explorer` | 4.38 ms | 209,696 B | 37 | | `BenchmarkAggregateFold/explorer_tool` | 3.73 ms | 807,840 B | 38 | | `BenchmarkAggregateFold/chart_window` | 413,833 ns | 17,616 B | 28 | | `BenchmarkHandleLogPage/empty/limit_60` | 1.70 ms | 987,376 B | 3,533 | | `BenchmarkHandleLogPage/sql_provider/limit_60` | 1.95 ms | 550,080 B | 2,645 | | `BenchmarkHandleLogPage/go_status_class/limit_60` | 1.89 ms | 903,504 B | 2,686 | | `BenchmarkHandleLogPage/go_status_class_5xx/limit_60` | 4.88 ms | 1,819,616 B | 21,125 | | `BenchmarkHandleLogPage/go_tool_missing/limit_60` | 5.00 ms | 1,883,032 B | 29,360 | | `BenchmarkHandleLogPage/empty/limit_200` | 4.26 ms | 2,889,608 B | 7,999 | | `BenchmarkHandleLogPage/sql_provider/limit_200` | 4.92 ms | 2,888,720 B | 7,993 | | `BenchmarkHandleLogPage/go_status_class/limit_200` | 4.14 ms | 2,890,912 B | 8,015 | | `BenchmarkHandleLogPage/go_status_class_5xx/limit_200` | 3.88 ms | 1,889,744 B | 21,029 | | `BenchmarkHandleLogPage/go_tool_missing/limit_200` | 4.72 ms | 1,957,512 B | 29,258 | | `BenchmarkProjectionMetricAppend/history_100000` | 11,393 ns | 960 B | 5 | | `BenchmarkProjectionMetricAppend/history_1000000` | 11,716 ns | 960 B | 5 | | `BenchmarkProjectionMetricAppend/append_cross_100000` | 1.58 ms | 3,956,976 B | 14 | | `BenchmarkProjectionMetricAppend/append_cross_1000000` | 1.54 ms | 3,956,976 B | 14 | | `BenchmarkAggregateUnchanged/identity_marshal` | 33,009 ns | 6,456 B | 6 | | `BenchmarkAggregateUnchanged/unchanged` | 1,338 ns | 0 B | 0 | | `BenchmarkKPI/records_10000/miss` | 403,272 ns | 83,632 B | 6 | | `BenchmarkKPI/records_10000/hit` | 1,024 ns | 32 B | 3 | | `BenchmarkKPI/records_100000/miss` | 7.47 ms | 804,528 B | 6 | | `BenchmarkKPI/records_100000/hit` | 1,986 ns | 32 B | 3 | | `BenchmarkKPI/records_1000000/miss` | 91.84 ms | 8,005,296 B | 6 | | `BenchmarkKPI/records_1000000/hit` | 2,153 ns | 32 B | 3 | | `BenchmarkObserveModelNames4096` | 7.59 ms | 3,613,216 B | 70,975 | | `BenchmarkObserveModelNameEvent/rules0` | 29,568 ns | 3,136 B | 19 | | `BenchmarkObserveModelNameEvent/rules64` | 99,814 ns | 26,992 B | 29 | | `BenchmarkObserveModelNameEventWired/rules0` | 15,688 ns | 3,136 B | 19 | | `BenchmarkObserveModelNameEventWired/rules64` | 76,644 ns | 31,088 B | 30 | | `BenchmarkSkipJSONString/plain/102400` | 9,647 ns | 0 B | 0 | | `BenchmarkSkipJSONString/escaped/102400` | 8,821 ns | 0 B | 0 | | `BenchmarkSkipJSONString/tail/102400` | 6,882 ns | 0 B | 0 | | `BenchmarkSkipJSONString/plain/1048576` | 84,535 ns | 0 B | 0 | | `BenchmarkSkipJSONString/escaped/1048576` | 86,790 ns | 0 B | 0 | | `BenchmarkSkipJSONString/tail/1048576` | 67,966 ns | 0 B | 0 | | `BenchmarkRequestMetadataShapes/escaped` | 629,819 ns | 35,552 B | 370 | | `BenchmarkRequestMetadataShapes/many_messages` | 254,854 ns | 32,280 B | 30 | | `BenchmarkRequestMetadataShapes/multimodal` | 168,469 ns | 16,240 B | 73 | | `BenchmarkRequestMetadataShapes/tool_schema` | 85,121 ns | 3,480 B | 22 | | `BenchmarkRequestMetadataShapes/digit_description` | 73,440 ns | 2,648 B | 6 | | `BenchmarkAnalyzerFeed/content` | 3,503 ns | 0 B | 0 | | `BenchmarkAnalyzerFeed/reasoning` | 2,881 ns | 0 B | 0 | | `BenchmarkAnalyzerFeed/tool_call` | 6,972 ns | 240 B | 1 | | `BenchmarkAnalyzerFeed/usage` | 3,131 ns | 0 B | 0 | | `BenchmarkAnalyzerFeed/negative` | 1,798 ns | 0 B | 0 | | `BenchmarkAnalyzerToolNameDedupe/10000` | 4.88 ms | 192,096 B | 10,121 | | `BenchmarkAnalyzerToolNameDedupe/100000` | 48.39 ms | 694,264 B | 100,220 | ## Reproduce `scripts/perf.sh` owns the repeatable local run: `bench`, `stress`, `streaming`, `sustained`, `image`, or `all`. `scripts/perf.sh docs` runs the full set, writes `perf/results.json`, and regenerates this page. It refuses a dirty tree unless `PERF_ALLOW_DIRTY=1` is set, because published numbers must name a commit. The full run uses the documented levels and durations: 5 seconds for each fast stage, 300 seconds for each sustained level, and 60 seconds for each streaming throughput level. Override the level lists and durations with `PERF_FAST_LEVELS`, `PERF_SUSTAINED_LEVELS`, `PERF_STREAMING_LEVELS`, `PERF_FAST_DURATION`, `PERF_SUSTAINED_DURATION`, `PERF_STREAMING_DURATION`, `PERF_STREAMING_HOLD` and `PERF_STREAMING_CHUNKS`. The bundle records what ran; a changed run changes the page. The individual commands below are the same evidence path. Use a fresh `DEV_COPY_DB=0` instance for each workload. Never target the main instance on port 8080. The stress harness verifies that the target is the private `scripts/dev.sh` instance before sending traffic. ```sh ( set -e trap 'scripts/dev.sh stop' EXIT export MILLIVOLT_OPERATOR_TOKEN='dev-instance-credential' DEV_COPY_DB=0 scripts/dev.sh go run ./cmd/stress -target http://127.0.0.1:8081 \ -concurrency 1,8,32,128 -duration 5s -stream-duration 0 ) ``` The isolated benchmarks use the test overlay: ```sh scripts/check.sh go test -run='^$' -bench='BenchmarkProjectionFootprint|BenchmarkStoreDiskBytes' \ -benchtime=1x -count=1 ./internal/web scripts/check.sh go test -run='^$' -bench='BenchmarkProjectionProcessRSS' \ -benchtime=1x -count=1 ./internal/web scripts/check.sh go test -run='^$' -bench='BenchmarkStorageWriter' \ -benchtime=2s -benchmem -count=1 ./internal/storage scripts/check.sh go test -run='^$' -bench='BenchmarkQueryBeforeEquality' \ -benchtime=1x -benchmem -count=1 ./internal/storage scripts/check.sh go test -run='^$' -bench='Benchmark(RequestMetadata|ProxyDurableContention)' \ -benchtime=2s -benchmem -count=1 ./internal/proxy ``` The image measurements use a disposable local image: ```sh docker build --platform linux/amd64 -t millivolt-perf . docker image inspect millivolt-perf --format '{{.Size}}' docker save millivolt-perf | gzip -9 | wc -c ``` ## Limitations - The host was shared. Other workloads can change absolute rates and latency. - The upstream was a local synthetic server. Provider latency, TLS, retries, payload shape, and real identity cardinality are not represented. - These are short closed-loop bursts, not sustained production capacity. No result here is a capacity ceiling or a guarantee. - The streaming concurrency table is a short ramp on a 10-frame, roughly 1-second fixture. The streaming throughput table is a 60-second per-level ramp on a 100-frame, roughly 1-second fixture. Both are synthetic-fixture measurements, not universal maxima. The throughput table's 1,024-client level is above the example `max_conns_per_host` of 512 and measures queueing; it is a fixed level list, not an automatic saturation search. Its token rate is the fixture's usage count, not provider-token throughput. - The 300-second sustained sweep uses the fast-response fixture with no intentional stream hold (`-stream-duration 0`). A level with durable drops is an overload observation, not a maximum streaming rate. The streaming throughput ramp is the only sustained held-stream run; it stops at its fixed levels and does not search beyond the largest listed level. - Performance checks are manual (`scripts/perf.sh docs`), not CI. Each level is one run per invocation on one host; there is no repetition or variance study. - Durable accounting is best-effort. The fast stages happened to remain exact; the overload benchmark intentionally does not. - Container-wide CPU/RSS, cold-start time, browser rendering, arm64 image size, disk throughput, and a full dashboard preload over a large authorized history were not measured for this page. - The image archive is a local artifact, not a published registry digest. - Do not compare these numbers directly across differing hosts, workloads, or revisions. --- # Security ## Deployment boundary millivolt is a trusted-operator tool, not a multi-user security boundary. It has no authorization roles or tenant isolation. The whole dashboard is gated by the shared `MILLIVOLT_OPERATOR_TOKEN` environment credential and denied while it is unset (see [operator access](docs/routes.md#operator-access)): there is one operator identity, no per-user accounts and no audit trail of who acted. Only the `GET /healthz` liveness and `GET /readyz` readiness probes, the `POST /session` login handshake, origin-root brand/PWA assets and transparent inference stay open at the application gate; the canonical open-path list is [operator access](docs/routes.md#operator-access). The upstream API key is forwarded for inference; it does not secure the dashboard or administration. The supplied listen address binds all interfaces. Prefer an explicit loopback listen for local use. Before network exposure, restrict access with network controls and an appropriately authenticated ingress. Protect the entire operator surface, including dashboard/bootstrap/live records, SQL, Debug, exports, Settings, Pause, Limits, purge and restart. Inference clients can also change provider-wide limits through request headers. For Docker, publish on host loopback, for example `-p 127.0.0.1:8080:8080`. A non-root container and a public image do not authenticate callers or make a publicly reachable operator listener safe. Protect persistent config/data volumes as sensitive state, including backups and debug captures. Browser same-origin mutation checks and framing denial are defense in depth behind the operator credential, not a substitute for it. The live metrics feed does not opt into cross-origin browser sharing; the dashboard uses a same-origin EventSource authenticated by the HttpOnly session cookie. This does not prevent direct non-browser reads or secure a publicly reachable listener. Explicit non-browser calls remain supported; gated endpoints need the `Authorization: Bearer` credential or a session cookie minted through `POST /session`. The `/mcp` endpoint is the exception: it requires the operator credential as a Bearer header and refuses a session cookie alone with `403`, because the caller's credential is forwarded on its internal calls. ## Upstream destinations and credentials `X-Proxy-Base-URL` selects an HTTP(S) destination. An unrestricted caller can therefore make the proxy contact destinations reachable from its host. `allowed_base_urls` constrains accepted upstream targets: an entry admits the same scheme, host and port (with the scheme's default port filled in) and that entry's path and its children. A port-less entry therefore admits the default port only, not every port on that host. Use trusted destinations and appropriate egress controls. This is not a DNS/IP isolation sandbox. The proxy does not follow upstream redirects. Ordinary metrics group credentials by SHA256 rather than retaining the raw key. Known secret and configured authentication headers are redacted in retained metadata/captures without changing forwarded headers. This does not sanitize arbitrary text: request/response bodies, URLs, error messages, model names or user-supplied labels can contain sensitive data. Preview capture and Debug capture are opt-in, but enabling them can persist that data. ## Client identity is a claim, not an authentication `X-Proxy-Client` (and the classified client derived from SDK and user-agent headers when it is absent) is chosen by the caller. Nothing verifies it: a request that sends another client's label is treated as that client by every feature that scopes on client - `request_overrides` (so it inherits that client's header and token-ceiling rules), `sub_conversations` (so it gets that client's field mapping and strip list), Debug capture sessions, pause holds including the "new clients" comparison, and the Cursor parked-run scope. The recorded client, and therefore dashboard grouping and the `client` filter, also follow the claim. The recorded client address is likewise a hint. `client_ip` records the first `X-Forwarded-For` element when that header is present and the socket peer otherwise; the value is diagnostic, never verified, and a directly reachable caller can set it. Do not treat `client_ip` as a verified source for rate limiting, allowlisting or attribution outside the proxy. The operator gate's brute-force limiter is a separate key: it reads the address the immediate hop appended (the last `X-Forwarded-For` element) and falls back to the socket address, so it distinguishes clients behind a correct ingress but a directly reachable caller can still pick its own bundle. The example ingress trusts exactly one loopback hop (`set_real_ip_from 127.0.0.1`, [deploy/nginx.conf](deploy/nginx.conf)) and its `allow` lists evaluate that rewritten address. A caller that can already reach the loopback listener therefore chooses the address the lists see: they separate remote networks from each other and from local callers, not local callers from each other. Treat a location allowlist as reachability control for a network, not as identity. Treat client scoping as a way to separate your own tools and traffic, not as a security boundary between mutually untrusted parties. A proxy reachable by someone who should not act as another client needs that separation enforced outside these labels: distinct `allowed_base_urls`, distinct upstream credentials per caller, or a separate proxy instance. Credential-class header names are refused in `request_overrides` and redacted in retained data, so a claimed label cannot by itself move an upstream secret; the exposure it does create is the policy attached to the label, not the credential behind it. Token-refresh handback responses contain credentials in the error body for the client to adopt. Do not publish them, browser exports, databases, backups, local config, login-script output or screenshots containing private history. Provider favicons can make browser requests to Google's favicon service, exposing the provider hostname and the browser's network address to that service. `referrerpolicy="no-referrer"` omits the dashboard referrer, not the requested hostname. Hostname syntax checks do not establish that every qualified provider label is public. The documentation capture workflow blocks all external browser requests; that isolation is not the ordinary dashboard's network policy. ## Administrative data and resource limits SQL is restricted to read-only queries with execution/output limits, not a sandbox for adversarial resource consumption. Debug has bounded capture sizes and retention, but captured bodies remain sensitive. Exports and backups can outlive those retention settings. Metrics storage is asynchronous and best-effort. Queue overflow/write failures can lose records; this is not a tamper-proof, complete compliance audit log. Do not rely on the dashboard as the sole security or billing record. See [operations and limitations](docs/operations.md#storage-and-accounting). ## Code scanning GitHub code scanning (CodeQL) reports on pull requests and the default branch. Alerts are triaged against the [deployment boundary](#deployment-boundary), not dismissed for convenience. True positives are fixed and covered by regression tests; accepted false positives record their rationale in the alert instead of being hidden by a broad path exclusion. The current categories and their trust-boundary evidence: - Operator-only read-only SQL (`internal/storage/query_admin.go`). The query console sits behind the operator gate, accepts one SELECT, and enforces execution time, row, column and result-byte limits. The operator is already trusted with the database, so this is not an unauthenticated SQL surface. - JSON, SSE and gzip response writers. The operator surfaces behind the gate encode the proxy's own output, and framing and compression state are bounded by the response-size guard. Two writers are intentionally reachable without the operator gate: the `/healthz` and `/readyz` probe JSON and the transparent inference SSE relay, which streams the upstream response to a client that supplied its own provider credentials. Neither exposes operator data (the readiness reason names queue depth and flush failures only), so this category is not an authentication bypass. - Bounded allocations (`cmd/proxy/mcp.go`, `internal/sse/frames.go`). The MCP recorder cap is the fixed `mcp.MaxResponseBytes` read budget (64 MiB) plus one byte, not a configurable limit. The SSE frame's requested size is the marshaled payload length plus the fixed framing overhead; `slices.Concat` may round the capacity up to the allocator's size class. The payload is bounded by the relay's scanner and encoder frame caps and the response-size guard. Input is rejected at those bounds before the sized allocation, so neither is an attacker-controlled unbounded size. The operator-cookie and integer-conversion findings were true positives and are fixed. The operator session cookie now sets `Secure` when the request arrived over TLS or through an https-forwarding ingress. The jsonnum, config and throttle paths now parse or bound at the target width, with regression tests. The allocation findings are closed by construction, using overflow-safe capacity owners. The operator SQL console is an accepted boundary, and the transparent response-writer wrappers are false positives from interface fan-out; their alerts are dismissed with a recorded reason. Dismissing or resolving a code-scanning alert is a maintainer action in the GitHub security UI; this note records triage rationale and does not dismiss any alert automatically. ## Scope In scope: the proxy, dashboard, MCP server, container image, and repository automation. Out of scope: findings that require a compromised host, an operator who already holds the credential, a misconfigured ingress, or a third-party provider. A report that depends on the documented trusted-operator boundary must show how that boundary is crossed. ## Safe harbor The maintainers will not pursue or support legal action for good-faith research that stays within this scope, avoids privacy violations, service disruption, and data destruction, and allows reasonable time to fix the issue before disclosure. ## Reporting a vulnerability Do not publish credentials, private request content, or actionable exploit details in a public issue. Use the repository's enabled [private vulnerability reporting](https://github.com/LLM-4-People/millivolt/security/advisories/new) channel. Include the affected revision, deployment assumptions, impact, and a minimal reproduction using neutral local fixtures. Test only systems you own or are explicitly authorized to assess. ## Supported versions Security fixes target the latest published release. The `0.x` series is in active development; older releases are not backported. | Version | Supported | | --- | --- | | Latest published 0.x release | Yes | | Older releases | No | ## Disclosure process Reports are acknowledged on a best-effort basis; no response-time SLA is promised. Maintainers will validate the report, prepare a fix, publish a GitHub security advisory, and credit the reporter unless they prefer otherwise. Please allow time for a coordinated fix before public disclosure. Dependency review and a scheduled Go vulnerability scan run in CI. A clean scan is not a security guarantee, and neither replaces a private report. GitHub documents [private vulnerability reporting](https://docs.github.com/en/code-security/how-tos/report-and-fix-vulnerabilities/configure-vulnerability-reporting/configure-for-a-repository). --- # Architecture and ownership millivolt is one Go HTTP service with an embedded, no-build dashboard. The client-facing protocol and upstream relay/translation are separate boundaries; the [compatibility matrix](adapters.md#client-and-upstream-compatibility) defines their supported combinations. [Contributing](../CONTRIBUTING.md) covers change and verification rules; [protocol](protocol.md) and the [operations overview](operations.md) describe public behavior. ## Source map | Owner | Responsibility | | --- | --- | | [cmd/proxy](../cmd/proxy/main.go) | Config/CLI, dependency wiring, routes, process lifecycle. Restart, log, backup and operator-gate handlers have separate existing files. | | [internal/config](../internal/config/config.go) | Config/defaults/validation, split by subject into `config_types.go`, `config_defaults.go`, `config_clone.go`, `config_load.go`, `config_validate.go` and `config_validate_overrides.go`; schema, YAML generation, revision-checked Settings, model-rule execution, and the credential-class header vocabulary and redactor in `credentials.go`. | | [internal/settingsdoc](../internal/settingsdoc/settingsdoc.go) | Renders the generated [settings reference](settings.md) from `Schema()`, the defaults passed through the YAML writer and the proxy flag registration; [cmd/docgen](../cmd/docgen/main.go) exposes it as `-write` and `-check`, and the config suite fails when the page drifts. | | [internal/backup](../internal/backup/archive.go) | Streamed self-checked operator archive format (VACUUM INTO snapshot + zstd + SHA-256), with byte- and file-backed database members. One decoder core serves byte and file callers; restore stages its upload and decoded database as private files instead of heap copies. `params/params.go` owns the backup/restore query keys, boolean spelling, restore modes and archive magic shared by the HTTP handlers and `mvctl`; the leaf imports no encoder, SQLite or zstd. | | [internal/proxy](../internal/proxy/proxy.go) | Routing, admission integration, retry/relay, request metadata, scoped request overrides, native-run ownership, operator state/capture; inbound/outbound protocol selection and canonical paths in [protocol.go](../internal/proxy/protocol.go); the quota-pause reaction owner [quotapause.go](../internal/proxy/quotapause.go) (settleQuotaFailure/addQuotaHold) and GET/POST /admin/quota. | | [internal/scheduler](../internal/scheduler/scheduler.go) | Provider/key admission, retry pacing, scoped holds, and limit policies selected by provider, model, client, key hash and conversation with optional recurring schedules; the provider-wide durable quota/billing recovery gate (quota facet, probe-paced recovery). | | [internal/metrics](../internal/metrics/metrics.go) | Records, numeric/usage/outcome semantics, pending/ring lifecycle, observers and Prometheus. | | [internal/sse](../internal/sse/analyzer.go) | Streaming content/usage/TTFT inspection for Chat, Responses and Anthropic SSE ([analyzer_anthropic.go](../internal/sse/analyzer_anthropic.go)), split into decode, feed, usage, terminal, error, fill and report modules; client-facing OpenAI SSE frames and completion envelopes. | | [internal/format](../internal/format) | Explicit wire codecs (Chat, Responses, Anthropic) over the canonical [IR](../internal/format/ir/ir.go), Connect/protobuf framing, and the same-protocol translation facade. | | [internal/storage](../internal/storage/store_open.go) | Asynchronous SQLite writing, schema/read fidelity, totals, purge fence, packed snapshots and bounded queries. | | [internal/web](../internal/web/aggregate.go) | Embedded shell/assets, bootstrap, canonical history projection and chart/explorer/log aggregates; `aggregate.go` owns the core row model and each surface has its own `aggregate_*.go` module. The site build republishes `css/themes.css`, `js/theme_boot.js` and `js/theme_state.js` byte-for-byte, so both surfaces share one palette registry and one apply path. | | [cmd/mvctl](../cmd/mvctl/main.go) | the human operator client: command dispatch, per-command flags, human and JSON rendering, and destructive-action guards. It reuses `internal/mcp` for the operator HTTP client, `internal/config` for setting kinds and bands, and `internal/backup` for archive and query grammar. | | [internal/mcp](../internal/mcp) | The MCP tool server for the operator plane: one HTTP client owner (credential, request building, error extraction), one scope-filter owner, one tool registry served by the proxy's streamable HTTP `/mcp` endpoint, the destructive-call guards, and the streamable HTTP handler the proxy mounts at `/mcp`. It owns no analytical projection of its own; the three docs tools read `internal/docs` locally. | | [internal/docs](../internal/docs/docs.go) | Embedded generated documentation index: strict decode of `docs.json`, ordered list, section read and ranked search for `docs_list`, `docs_search` and `docs_read`. The checked-in file is produced by `scripts/build_site.py --write-manifest`, and a source-hash test rejects a stale copy. | | [scripts/gallery](../scripts/gallery) | The documentation capture registry, scene composition, capture manifest and generated gallery page. `tests/docs_capture.py` executes the scenes against a private redacted instance; `scripts/site_plates.py` upgrades registered images into the published plate chrome. | ## Logging [internal/logging](../internal/logging/logging.go) is the one owner of process log lines. It wraps the standard logger's sink and gives a request-scoped line the record id minted at the read boundary, so the line joins its stored row, capture and neighbouring lines. Process and operator-state lines pass no id. `log.Fatal` exits and the earliest flag errors that precede the process logger stay direct. ## Release identity and promotion [version.go](../version.go) owns VERSION validation and SemVer precedence. The [workflow](../.github/workflows/check.yml) owns the promotion gate and release publication: `unit`, `browser` and `container` run the full gate on all `testing` pull requests, code-bearing `testing` pushes, and manually pushed `v*` tags, plus Dependabot's `main` PRs; docs and site-only pushes use `docs.yml`. The three gate jobs start independently and `required` aggregates them. `required` reuses that gate for ordinary promotions by requiring the head to be the current `testing` tip, `main` to be already merged back into it, a successful full-gate record for that exact commit or for the closest gated `testing` commit when every path since it is docs/site-only, and a VERSION newer than every tag for non-Dependabot code-bearing promotions. A non-Dependabot promotion also requires a successful [escalation](../.github/workflows/escalation.yml) run (forced full-race sweep plus the repeated clock-sensitive flake sweep) for the reviewed head; the proof fails closed while that run is missing, pending, failed, from another workflow, branch or commit. Pending exact check runs get a short bounded wait and the proof fails closed if they are still pending; let the `testing` push gate finish before opening or re-running a promotion PR. Docs/site-only promotions skip the VERSION and full-gate proofs and validate docs, workflow policy and the site in `required`. The `required` job has a ten-minute budget for that path. `publish` tags the reviewed promotion merge, publishes the image and creates the release without pushing `main`; a manually pushed `v*` tag ships exactly as tagged. `publish-testing` instead waits for `unit`, `browser` and `container` directly, because a testing push has no promotion proof to wait for. The `main-testing-first` ruleset adds the pull-request and merge-commit requirements with no bypass actors. [Contributing](../CONTRIBUTING.md#branch-workflow) owns the maintainer checklist; [operations](containers.md#versions-and-images) owns the published tag contract. ## Request and scheduling path `readRequest` retains bounded original bytes and calls the single metadata decoder. Ordinary content is measured without retaining decoded prompt/schema trees; opt-in previews stay bounded. Exceptional decoder compatibility belongs at that owner, not a second routing parser. Before the typed decode, `parseLLMRequest` bounds decoded JSON-string expansion: valid UTF-8 and JSON escapes decode to no more bytes than their spelling, so only raw invalid UTF-8 can consume the small allowance (the standard decoder writes a three-byte U+FFFD per invalid byte). A larger expansion is rejected before materializing; malformed JSON keeps its passthrough contract, and small invalid sequences still pass through. `parseLLMRequestFast` is a narrow lexical pre-pass for a minimal top-level object: `model`, `stream`, `messages` with printable-ASCII role and content strings, and `system`. It extracts the same routing and composition fields without the typed decode and delegates every other shape, including all parameter collections, escapes, duplicates, content part arrays, and preview capture, to the existing decoder and its bounds. Differential tests pin byte-for-byte record and rejection equivalence. `SkipJSONString` finds each candidate quote with one `IndexByte` and reads its escape parity from the backslash run before it, so escaped content is one forward pass instead of a restart per escape. The metadata preflight counts message elements, content parts and tool calls in one walk instead of three. Current scanner and metadata-shape costs are published under [implementation benchmarks](performance.md#implementation-benchmarks) (`BenchmarkSkipJSONString`, `BenchmarkRequestMetadataShapes`). The streaming analyzer scans each frame once for its classifier keys and classifies lexically wherever it can. Chat tool-call frames are scanned once for the element index, id and `function.name`, with the typed decode kept only as the fallback for a shape the lexical grammar rejects. Current frame and dedupe costs are published under [implementation benchmarks](performance.md#implementation-benchmarks) (`BenchmarkAnalyzerFeed/tool_call`, `BenchmarkAnalyzerToolNameDedupe`). A differential corpus and a fuzz target compare both paths over every analyzer field they touch. The Anthropic translator independently requires valid UTF-8 input, so it cannot materialize the replacement expansion either. The Start stamp follows body upload and precedes metadata/translation CPU. Upstream whole-document buffers share one process-wide `max_inflight_response_bytes` reservation: translated non-streaming responses, the non-streaming 200 quality spool, and model-discovery pages. Admission reserves the declared Content-Length when it fits the bounded read cap, otherwise that cap, and refuses with a retryable `503` and `Retry-After` before reading the upstream body. `resolveRequestOverrides` in [request_overrides.go](../internal/proxy/request_overrides.go) merges the matching `request_overrides` rules once per request, after routing metadata is known, and its merge has exactly two application stages: the final header stage of `buildUpstreamRequest`, and the cursor identity headers at the native send. Its body stamp is fail closed (an undecodable or oversize result relays the original bytes with one log line, prefixed by the request id minted at the read boundary so the line joins the stored row), and a nil resolution - the feature off or nothing matched - leaves the request path byte-identical. The sub-conversation strip is the single body-rewrite engine's second consumer: it fires independently of override resolution, after the tracked value is extracted from the buffered body and before translation, with [conversation.go](../internal/proxy/conversation.go) owning the extraction and the `k:` identity. A provider's ensured tool names are the same engine's third consumer: they are restricted to the OpenAI wire, because the injected stub tools are chat/completions-shaped and must never reach a translated-anthropic or cursor body, and they fire with any override and strip actions through one document decode. Immutable config/client snapshots hot-swap for new work; old transports finish active requests. Admission is provider+key scoped, with persistent limit policies layered at the same grant owner. A policy intersects its non-empty selector dimensions and an empty dimension matches any value, so a provider-only rule is the provider-wide cap while a narrower rule reuses the same machinery. Every matching policy charges its own lease, and a scheduled policy outside its window neither charges nor queues. Acquisition release settles exactly once. The provider gate and token-bucket epoch captured at admission keep clearing/replacing a cap from charging another generation's reservation. A provider-to-groups index in `Scheduler` keeps each group's slot in a fair-waiting set: a group joins when its queued head was refused by the shared provider gate (concurrency or the request/token bucket), not when it is parked on its own pacing window, hold, anti-burst ramp or per-key cap. A release settles the gate, then drains that set once in rotating one-grant-per-group passes, starting after the last group served, so one key cannot take the whole provider allowance while peers wait; the drain stops as soon as a grant refills a saturated provider concurrency cap. The lock-free fair count is the fast path's precise witness: an unrelated key admits immediately while peers wait on their own predicates, and a group that stops on one of those predicates leaves the set. An anti-burst epoch starts a group at allowance 1 and raises it by `floor((concurrency-1) * elapsed / ramp_duration)`, clamped to the target; a fully idle period of at least the ramp duration resets the epoch on the next grant, a shorter gap continues it, and changing the flag, duration or target resets every epoch. [tokenrefresh.go](../internal/proxy/tokenrefresh.go) separates provider exchanges from response policy. Exchange adapters return credential data only; the shared `resolveKey` ends successful inference refreshes with handback before admission, live publication or recording. `resolveKeyTransparent` owns the model-discovery exception. The token-refresh tests cover both mechanisms, adoption/retry, absent replacement tokens, applicability/failure gates and bounded exchange responses. Retries, provider hints, hold waits and per-send deadlines have distinct owners. Capture context error before canceling it for cleanup. Never retry a canceled caller or replay meaningful emitted stream content. Quality failure handling is explicitly exceptional; see [adapters](adapters.md) and the relay tests. Queue visibility has one owner across both park points: the bounded admission FIFO and `WaitSend`'s send gate. Every parked request publishes the live queued edge (`Throttled`) for the whole wait and clears it on the grant. `Stats` reports send-gate waiters separately (`SendWaiters`) and the operator snapshots add them to the global and per-provider queue totals, so a 429 pacing window reads as queued work rather than idle streams; a retrying owner waiting out its own Retry-After hint is queued for the same reason. Send-gate waits stay separate from the admission queue bound so `max_queue_size` keeps its own meaning. [scheduler/storm.go](../internal/scheduler/storm.go) owns bounded rolling attempt counters, active-model evidence, cross-key provider/model gates and exclusive recovery permits. Provider activation requires all models with activity inside the same window to independently qualify. Old permits cannot recover newer incidents. Initial admission waits for readiness without reserving a probe; only the actual send boundary claims a probe after key admission. Storm policy changes reset observations and wake waiters without mutating retained history. Each request carries an observation for its provider and model, moving distinct failed-request membership between fixed buckets across retries without retaining a request-ID map. No storm observation scans metrics history or waits for SQLite. [proxy/storm.go](../internal/proxy/storm.go) maps config once into scheduler options and bridges existing relay/native send outcomes. A 2xx permit settles after relay or before an existing quality re-ask or thinking rescue re-send, preventing double samples and stream replay. Safe protocol-level reason labels carry no provider error bodies. The banner uses the canonical bootstrap builder and browser apply/render gates, with no client-side event arithmetic. Storm scheduler/proxy/config regressions and the shared UI/browser suite cover these boundaries. Important coverage includes request metadata/parser fuzz tests, `BenchmarkRequestMetadata`, `BenchmarkProxyRequest`, `TestConcurrentStreamingAccounting`, transport lifecycle tests and scheduler efficiency/race tests. Proxy requests must not perform history scans or wait for SQLite accounting. ## Finalized records, pending lifecycle and replay `PublishLive(begin/update)` snapshots pending state without adding durable rows. The proxy registers its deferred `Record` before publishing `begin`, so a subscriber that fails while handling the begin event cannot leave a permanent in-flight row. `Buffer.Record` atomically finalizes the row, retires pending state, updates its gauge/revision and emits end. Exactly one finalized record per request is the accounting unit; proxy-owned IDs cannot collide with reused client request IDs. A debug capture registers its pending sidecar marker in `debugRuntime` before that publish and encodes/saves after it, so the sidecar write delays neither the live end nor durable accounting. A drawer fetch for the id waits on the marker, then reads the authoritative store; a resolved marker without a row keeps the normal 404. `Buffer.Record`, `PublishLive` and `Backfill` share the one record-retention pass ([internal/metrics/retain.go](../internal/metrics/retain.go)) at that boundary. It bounds client-controlled paths/methods and provider-controlled model/request/finish/error text, tool names and attempt metadata; normal values stay byte-identical. The bounds and the UTF-8 cut marker are documented in the [protocol](protocol.md). Existing lock order keeps sequence assignment and finalized publication ordered. A feed epoch changes on restart/deletion; queued fanout retains its mutation epoch. Subscribers attach before snapshot capture and overflow retires that subscriber so reconnect can replay, rather than silently dropping permanent rows. When more than one dashboard subscriber is attached, `Record` and `PublishLive` attach one lazy observed-JSON cache to that published event ([internal/metrics/event_cache.go](../internal/metrics/event_cache.go)). The first consuming stream encodes the observer result and every subscriber of that same event reuses the retained bytes; an encode error is retained too, so all of them close and resync. Zero or one subscriber keeps the direct `json.Marshal` path with no cache allocation, and no cache outlives its event or crosses a feed epoch. `SnapshotRequest` is the shared atomic cursor/feed gate for bootstrap and SSE. A foreign/stale cursor requires a full snapshot; deltas are never truncated. Snapshots include current pending rows and their revision. Full durable-mode snapshots cap the recent window, with archives fetched separately. The browser's one upsert owner deduplicates finalized replays. Pending upserts cannot overwrite finalized rows or a newer pending revision of the same ID. The authoritative gauge revision is independent: an older event may still add a missing different row without rolling back the gauge. Full bootstrap closes the stream before capture; only the winning refresh reopens it. Stale same-feed full SSE snapshots recapture through that barrier by sequence/pending revision. Tests live under [tests/](../tests/README.md), with Go package mirrors in `tests/_go/`. Use the overlay-aware `scripts/check.sh go test` entry point. See buffer snapshot/stream tests and the harness sections under `tests/ui_harness/sections/` for the lifecycle and bootstrap races. Do not collapse these row, gauge, epoch and request-ownership gates into a whole-event drop rule. ## Durable writes and deletion `Store.Recorder(buffer)` publishes the live final record and nonblocking enqueue under one short producer fence. SQLite is the durable authority; enqueue overflow and failed batches remain visible best-effort losses. `internal/storage/health.go` is the one owner of that loss accounting and of the pipeline's liveness. Every discard is counted under exactly one cause (`overflow`, `after_close`, `panic`, `flush_failed`); `Dropped()` sums them for the dashboard counter, and the exposition renders each cause so alerting can separate overload from shutdown, a panic and a failing database. The same owner publishes the oldest queued record's arrival and stamps every flush attempt, so `/readyz` can tell a stalled writer from a merely idle one: a non-empty queue whose oldest record, or last successful flush, has passed one flush interval plus the wedge window (four intervals clamped to 30 seconds and 2 minutes) reads wedged. The interval on top is the record's own cadence, so a record inside it never convicts and a long cadence cannot hide a stuck writer for longer than that. The arrival stamp comes from the producer when the queue was empty as well as from the writer, so a writer that dies before its first pickup still ages the record, and with no flush since boot the first accepted record ages the last-success arm too, so a zero flush stamp is not a blind spot. A failed flush stays sticky. `/healthz` stays the liveness contract; readiness is the stronger answer a scheduler needs. `tests/_go/internal/storage/health.go` pins the cause split, the wedge/ready transition and the disabled-store answer, and `tests/_go/internal/metrics/prometheus_extra.go` pins that the storage block is rendered per scrape rather than from the ring-revision memo. The one writer owns transactions and incremental totals. Its insertion plan binds arbitrary values, using exact zero/empty literals only for absent scalar work, and preserves existing conflict/rollback semantics. Record columns, migration, scanner and request-parameter presence bits must evolve together; explicit zero/false is not absence. Boot migrations also own one-shot data repairs for shapes current writers cannot produce. The empty-string `attempts`/`tool_names` document becomes the canonical `null`. The pre-0.3.34 start-anchored `ttft_ms` (`ttft_ms > 0` with `final_attempt_at = 0`) is zeroed because its request-start anchor was never stored and cannot be re-derived; a snapshot merge applies the same TTFT canonicalization so a pre-repair backup cannot reintroduce the shape. Each repair is guarded by a `meta` marker and re-armed when that marker is absent (a restored backup predating it). A repair that rewrites rows fires the projection update trigger, advancing the epoch and invalidating the persisted totals document so boot rebuilds it. Provider-alias repair applies every pair in one transaction, in source-key order, so a later trigger or constraint failure rolls earlier label rewrites back instead of leaving history half-merged. The durable log order is `(started_at, id)`, owned by the sole request-log index `idx_requests_log`. The web log page pushes exact provider, client, key hash, conversation id and canonical numeric status predicates into SQL; a filtered page rechecks them while scanning the ordered index. Model filters remain wholly Go-owned: canonical matching trims every stored spelling and applies the configured rules, so raw SQL equality would silently drop a padded or rewritten stored model that still matches. Status classes, tools, errors and time buckets also stay in Go. Within one dimension, repeated filters are OR alternatives and are pushed as one SQL `OR` group when that dimension is exact; a group that cannot be pushed stays Go-owned so the SQL candidate set never narrows below the group's union. The ordered export and purge use the separate `PurgeFilter` surface (provider, model, client, conversation, error type, status code, `has_error`, `debug`, and time bounds; no key hash), whose equality-only preview count or purge scans the table. Results, order and pagination stay exact without six per-dimension composite indexes; a time bound becomes an index range. An upgrade that actually finds and drops one of those superseded indexes repacks the database once with `VACUUM`, because SQLite otherwise only moves the freed index pages to its freelist. Fresh and already-migrated databases skip it, and a failed reclaim logs a warning without failing startup: every row stays intact and manual reclamation remains possible. A successful reclaim resets the projection witness, because `VACUUM` may renumber implicit rowids. The page handler classifies active filters: when every predicate is SQL-exact it requests exactly the remaining rows, and Go-only predicates grow geometrically from the page size to the 512-row batch cap within the 4,096-row scan budget. Both preserve the exclusive `(started_at, id)` cursor and `more`. The candidate statement selects only the columns the residual groups read, and the derived error signature and tool labels are filled only when the plan declares them. Current filtered and unfiltered page costs are published under [implementation benchmarks](performance.md#implementation-benchmarks) (`BenchmarkHandleLogPage`). `Store.Clear` serializes administrative purges, captures an accepted-record boundary and publishes one command. The writer flushes earlier records, deletes matching records/sidecars and rebuilds totals transactionally, then removes only matching ring rows through that boundary. New completions survive. Failed SQL does not mutate the ring; postcommit memory effects finish even after caller cancellation. The atomic command check must not acquire the producer lock for every ordinary consumed record. A non-empty purge, and a drained expired-capture sweep, checkpoint-truncates the WAL after the commit: `secure_delete` cleans the pages SQLite writes, and truncation removes the pre-delete frames that would otherwise keep the old page images in the `-wal` sidecar. The checkpoint is bounded and best effort; a busy reader raises a pending flag that the writer's near-term tick retries, and `Store.Close` makes the final attempt after the read pool is closed, without failing the committed deletion. The restricted SQL console (`GET /metrics/query` and the MCP `query` tool) is gated in `validateQuery`: one `SELECT`, no `;`, no `ATTACH`/`DETACH`/`PRAGMA`/`LOAD_EXTENSION` word, and no `sqlite_dbpage`/`dbstat`/`pragma_*`/`readfile`/`writefile`. The scan is lexical over the raw text, string literals included, and the read pool's `mode=ro` pins only the main database; SQLite still opens an `ATTACH`ed file writable. Follow-up gap: the pinned modernc.org/sqlite v1.61.0 exposes an authorizer per physical connection (`AuthorizerRegisterer`, installed through `RegisterConnectionHook`) and a `_defensive=1` DSN mode, and this build wires in neither. An authorizer policy at read-pool open plus defensive mode would turn the keyword list into defense in depth instead of the only owner. Operator backup and restore stage database-sized work as private files. `backup.EncodeFile` streams the packed snapshot, encoded archive and self-check in bounded chunks. On restore, the handler streams the capped upload into one private file, and `backup.DecodeFile` shares the byte decoder's header, zstd, SHA-256 and member-framing owner while writing the database member into a unique staging directory on the live database's volume. Config bytes remain buffered; the database member does not. Inspect and overlap integrity-check or attach that file in place, replace streams it into the pending snapshot and atomically renames it, and merge attaches it to the held writer transaction. Decode and validation finish before an apply path takes the writer connection. The upload and decoded database files are removed on every handler path, including over-cap, invalid archive, apply failure and client disconnect. A boot-time sweep reclaims only known staging names older than a safety age, examining the directory in bounded batches up to a per-boot entry budget; the live database, pending snapshot, rollback set, state marker and quarantined admission evidence are outside that registry. A rejected or interrupted pending file is preserved under a unique `.incoming.invalid-` or `.incoming.invalid-interrupted-` name with its sidecars, so a later rejection cannot overwrite earlier evidence. A pre-existing pending snapshot is hard-linked aside before the successor replaces the slot: the restore drops that link again on its own rollback, and a boot that rejects the successor renames the link back onto the slot and admits it once, so the queue entry is never orphaned into the sweep. The attached inspect/merge and decoded source use the shared `SQLiteURI` with `mode=ro`, so a snapshot path with URI metacharacters stays literal and writes through the attached schema fail. Relevant coverage: storage writer/purge/request-parameter tests, `BenchmarkStorageWriter` and `BenchmarkProxyDurableContention`. The latter deliberately overloads storage and separates durable throughput from drops. Non-stream responses decode once into one analysis: quality outcome, usage, model/id, finish reason, tool names, bounded preview and the top-level error envelope share that pass, and the relay applies the same value at commit. Cost-bearing bodies add the existing provider-keyed cost pass; streaming analyzer semantics are unchanged. ## History projection and aggregate math One durable analytical projection retains compact contributions, request IDs, interned dimensions and immutable sorted metric row orders. It is not a second database or a per-view history cache. Startup preload and incremental catch-up share its builder. SQLite transactions capture rowid high-water/destructive epoch/schema authority; destructive changes rebuild. Failed or canceled reads never publish stale rows or advance authority. Folds check the caller's context once per 4,096-row block, and the ring visitors use the same cadence, so an abandoned request stops without publishing a partial result. The `has429` flag (and any future vocabulary-derived classification) comes from the installed vocabulary, not from a stored column, so the projection stamps the metrics classifier revision and forces a full reclassification rebuild when a reload changes it. The chart and explorer memos include the same revision in their key, so neither a stale projection nor a pre-reload memo can answer a post-reload request. Materialization runs off the projection lock with one single-flight builder; concurrent readers keep serving the previous complete snapshot and wait for that builder instead of repeating its scan. The builder's token is released by a defer, so a panic in the scan, decode or interning step frees the waiters instead of wedging them until restart. A destructive change builds a complete replacement off-lock and swaps it under a short write lock. The durable since-inception totals are one meta row written in the same transaction as every insert, purge and merge on the held writer connection; its recorded projection epoch is the mutation witness. A merge writes its request rows, debug sidecars and totals document in one transaction, so a failed totals write rolls the whole merge back instead of advancing the epoch over partial rows. An unsupported external writer that advances the epoch leaves the pair mismatched, so the next boot rebuilds rather than trusting it. A boot with a changed epoch rebuilds once from the rows instead of trusting a stale cache, and a full purge has no surviving rows to scan. Restart readiness therefore no longer scales with the sum of a totals scan plus a separate preload scan. Chart and explorer each retain one immutable last-result slot. Keys include validated view/scope, canonical model rules, finalized/database revisions and, for explorer, pending revisions. Clock-aligned geometry also gates chart reuse. One in-flight fold per memo is shared by concurrent same-key callers: a different key never waits, and a waiter that finds no published entry after the leader ends recomputes as the next leader instead of inheriting its error. The stability recheck hands over the fold's compiled rules identity instead of re-reading the config and re-marshalling up to 64 rules. The current 64-rule recheck cost is published under [implementation benchmarks](performance.md#implementation-benchmarks) (`BenchmarkAggregateUnchanged`). The bootstrap KPI fold retains one exact totals slot keyed by the ring's finalized revision and the store's durable accounting revision; live counters are filled at serve time and the ring snapshot itself is never retained. The unwritten set stays ring-derived. A record can be missing from the durable store while never having been queued: an enqueue that overflowed the writer channel and a batch whose insert failed are both dropped by design and counted by cause in the storage health owner; `Dropped()` sums the causes for the dashboard counter. The writer queue is therefore not a superset of "possibly not yet durable", and only the written-id set can say which ring rows are already durable, so the O(ring) probe under `aggMu` is the exact dedupe boundary. Current cold-fold costs at the default ring and the one-million-record ceiling are published under [implementation benchmarks](performance.md#implementation-benchmarks) (`BenchmarkKPI/records_1000000/miss`); the ceiling is a non-default `history_size`, and a correct-but-slow cold fold is the accepted cost of never double counting. SQLite data_version is read on one retained connection; unsupported external writers are not made safe merely by cache invalidation. Log pages push exact provider/client/key/conversation/status equalities into SQLite before decoding; model filters are always Go-owned. The Go scope matcher remains authoritative for canonical models, status classes, errors, tools, time buckets and live-only classes. Ad-hoc SELECT results are encoded once during the bounded scan and the endpoint writes that array directly; row/byte cap failure still precedes every response byte. Explorer uses dictionary IDs, scoped predicates and bitsets. The projection also owns two derived tables keyed by its immutable build: the interned conversation topology (identities, parent declarations, resolved issues and public-ID groups) and the canonical model IDs/dictionary for the compiled model-rule revision. Appends extend both by the new rows; destructive replacement publishes fresh tables, and a rules change rebuilds only the model table. Requests mark their own rail/gallery flags on the shared topology, resolve lineage metadata only for the returned conversation cards, and overlay ring/pending declarations without mutating the published build. Projected metric samples stay in shared sorted orders: single-valued dimensions retain one group id per row, while tool/error membership spans preserve multi-valued occurrences. An error's identity is `type|code|msg`; a backslash or the `|` separator inside a part is escaped, so arbitrary error text can never merge two distinct errors into one dictionary id. Ordinary text keeps the historic spelling, which preserves saved error-filter ids. Exact group accumulators are capped at `explorerGroupBudget` (8192) distinct values per fold. Values beyond the budget fold into one synthetic `other` card carrying aggregate count, cost, token, tool and health fields; no per-value accumulator and no per-card request-ID set is allocated for either an exact card or them. An error card's affected count is a counter plus the previous request id: one row's visits are contiguous, so that guard dedupes exactly what the old ID set did while staying bounded regardless of history. The card carries an explicit `synthetic` wire marker and no filter key, so a real value sharing its display label keeps its own exact, selectable card. The card ranking remains exact among tracked values, with `other` competing by its aggregate count, and overflow memberships map to the reserved other slot so their percentile samples still select correctly. Rows store the group's 16-bit membership slot, not its dimension-wide dictionary id, so the per-row table is two bytes and the selected-card lookup is a bounded probe instead of a max-id-sized array; the current 100,000-row provider fold cost is published under [implementation benchmarks](performance.md#implementation-benchmarks) (`BenchmarkAggregateFold/explorer`). Rail distinct-value counts, scope totals and conversation summaries remain whole-history values and are not derived from the truncated group set. Selected groups use the same exact rank reader as chart buckets, and only top cards derive percentile payloads. Pending dedupe retains only relevant pending IDs, not another history-sized seen set. Chart's rank reader also produces the occurrence-weighted avg/min/max in the same traversal, so a metric order is no longer walked separately for stats. The all-history order is a compacted immutable base plus a bounded value-sorted pending delta (16,384 samples): a catch-up append folds only its new samples, so it stays O(delta log delta) regardless of history, and reaching the bound arms one background fold that runs off the request and publishes the merged base under a one-assignment write. Readers keep the current base and pending until then, so an interrupted fold only leaves the published order in place. The fold owns its armed flag behind a deferred release: an unwind there (recovered and logged) clears the arm with the pending delta intact, so the next append folds again instead of the flag staying set and pending keeping the whole delta for the process lifetime. An append that crosses the bound at one million rows is published under [implementation benchmarks](performance.md#implementation-benchmarks), now off the request (`BenchmarkProjectionMetricAppend`). Chart and explorer walk the two runs through one merge iterator, so a fresh view allocates no history-sized copy; a materialized merge stays cached per published snapshot only for consumers that need a slice. A full rebuild replaces the base, clears pending and discards a fold that has not published yet. A retained last-65,536-row order serves bounded chart windows whose earliest in-window durable row is inside that tail. It uses the same compacted base plus bounded pending delta design and its own per-snapshot view holder, so a sub-threshold catch-up merges only its new batch instead of copying the tail; a rebuild of the newest 65,536 rows replaces its base, clears the pending delta and releases samples that fell out of the tail. Older or unscoped windows fall back to the all-history order, so the fast path never changes rank or stat results. Chart membership is indexed from the first row a bounded window accepts and sized only to that window's remaining span. The current 15-minute view over 100,000 rows is published under [implementation benchmarks](performance.md#implementation-benchmarks); the All path keeps its historical allocation (`BenchmarkAggregateFold/chart_window`). Both consumers share R7 rank/interpolation and the minimum-sample gate. Period speed/TTFT percentiles use period samples, never means of bucket percentiles. Prometheus and the stress harness instead share `metrics.Percentile`'s nearest-rank contract; do not conflate these two intentionally distinct APIs. Checked counts/sums/ratios prevent wrapped tokens or non-finite JSON. Cost stays USD in storage/API; missing/unreportable data must not become fabricated prices. `Record.IsError` and `HasRateLimit` deliberately differ. Final/retried 429 counts once per affected request, not via the broad pacing flag. A real failure and 429 may overlap. A 429 whose status line a committed keepalive 200 hid is retained as one in-band attempt entry, so the rate-limit count still sees the upstream status while the same request counts as an error through its final `error_type`; that entry is not a retry and adds no second error entry. `metrics.IsNonRetryableQuotaErr` is the fixed durable quota/billing classification vocabulary: the retry ladder, storm evidence and in-band precedence all consult it, so the account condition never counts as a rate limit. A plain 499 (the local client's own abort) is a flow event; a 499 carrying a structured error observed the upstream's in-band failure before the client left and counts like its 200 twin. The shared first-membership gate keeps duplicate tool/error occurrences from duplicating these health counts without changing event/sample multiplicity. Tests cover raw/projected/durable/ring/pending parity. `internal/metrics/errorclass.go` owns the canonical classifier: one `Classify(facts)` answers "what is this", and the retry ladder, the storm observer, the Cursor transport, the health predicates, the client-facing class and the published SQL all read it. `metrics.OutcomeIsError` (composed by `Record.IsError`) owns the health arms, and the MCP describe document and the durable purge filter render `metrics.IsErrorSQL`/`HasRateLimitSQL` per call instead of carrying their own predicates, so an operator extension applied by a reload reaches the published SQL too. Classification reads the status AND the provider's own type/code, never message text. The two questions are separate and answer differently in the two documented families: a 429 whose body names a durable quota/billing condition is ClassQuota for the retry ladder and quota_pause_mode while the health surface counts the 429 itself as flow control (not an error), and a rate-limit class on any non-429 status is counted in the rate-limit series while the health surface also counts the failed request as an error. The health predicate reads the status and the error type only, so a spelling carried solely in `error_code` (a shape the envelope parser does not produce - it pairs every code with a type) can move the class but not the health answer. The operator's `rate_limit_error_classes`/`quota_error_classes`/`retryable_error_classes` extensions are installed as one value (`metrics.InstallClassExtensions`, installed by `proxy.New`/`proxy.Reload`) and every consumer reads the effective vocabulary, including the browser mirror, which receives it in the bootstrap payload when extensions are installed. Extensions add only: config validation rejects a spelling that names a built-in of another class; the one documented exception is `retryable_error_classes` naming a built-in client-fault spelling, the explicit override for a gateway that mislabels transient failures. With `client_error_class: header` the relay's outer writer stamps `X-Proxy-Error-Class` and a class-derived `X-Should-Retry` on every error status before the line commits (`suppress_client_retries` still forces false); relayed bodies stay verbatim. See the files under `tests/_go/internal/web/`: `aggregate_projection.go`, `aggregate_dimensions.go`, `aggregate_percentiles.go`, `aggregate_explorer_fold.go`, `aggregate_rate_limit.go` and `conversation_lineage.go`. ## Canonical observer meaning and conversation lineage Go's compiled model-rule executor owns canonical grouping. Observer payloads carry exact raw-to-canonical names and a rules-content revision; only full authoritative snapshots install a new revision. Deltas, archive and Debug names merge under that revision, with recapture on conflicts. Browser dictionaries prune to loaded rows/selector needs. RE2 patterns do not execute in JavaScript. The installed observer provider is an immutable atomic holder, so SSE record/end serialization never takes the ring lock. Canonical names are memoized across observer calls under the compiled rules revision, and each known-name vocabulary caches its sorted view under a set revision. Both caches are bounded and invalidate on content change, so bootstrap polls do not re-run the rules or re-sort thousands of names. The single-name memo owns the whole payload under one mutex keyed by the rules revision, so a repeat at event rate costs one lock and one lookup. Current event and vocabulary memo costs at the 64-rule maximum are published under [implementation benchmarks](performance.md#implementation-benchmarks) (`BenchmarkObserveModelNameEvent`, `BenchmarkObserveModelNames4096`). The canon the observer reads is memoized per live config pointer, so consecutive events hand the compiled-rule cache the same backing array and it skips the whole-set compare; the wired shape's current cost is published there too (`BenchmarkObserveModelNameEventWired`). Pinned names are derived from the live holds, debug sessions and limit policies, never accumulated, so a removed document releases its scope. The derivation reads and applies the whole roster under the owning document locks: every document commit happens under the same lock and re-derives right after, so a stale derivation can never overwrite a newer roster and unpin a live scope. Observer records carry server-local `time_bucket`; browser-local dates are presentation only. Raw names remain raw for exact export/delete semantics. The explorer's one iterative lineage resolver validates declarations by client/key/conversation before applying rail/gallery scopes. The resolved graph is a projection-owned table, interned by dimension slot; a request only marks flags and overlays unprojected ring/pending declarations. Parent-only dictionary names do not become observed memberships. Conflicts, cycles and mixed namespaces remain unknown; missing parents are not invented roots. Summary counts follow full-filter rail scope; conversation cards follow the gallery cross-filter. Payloads retain small resolved fields, not a per-request graph. Parent pivots remain exact namespace-qualified leaves. ## Operator plane clients The HTTP client in [internal/mcp](../internal/mcp) owns the operator credential, request building, the bounded response read and the `.error` extraction that reads the flat error body regardless of the declared Content-Type. The credential is redacted from both the transport error and the failure body. Redirects are refused rather than followed: `net/http` strips `Authorization` only when the hostname changes, so a same-host different-port `307` would replay the credential and the request body. Its route paths are one closed set beside it, so no tool can name a path outside the operator plane: nothing reaches the transparent inference catch-all, and no restore-adjacent route is exposed. Three registry tools are local-only: `docs_list`, `docs_search` and `docs_read` answer from the `internal/docs` index instead of constructing an HTTP request. They share the generated page model with the published site and never touch the operator credential. The explorer's `f=dim:id` / `s=` grammar is owned once and shared by every scoped tool, and the tools never loop: a non-advancing cursor is reported as exhaustion because the aggregate scan budget is bounded per call. An empty page is deliberately not exhaustion, because the proxy advances its cursor on every row it scanned, including rows the scope excluded, so an empty page with an advanced cursor is a page that must be continued. The same client backs the whole tool registry. The proxy's own `/mcp` endpoint injects a transport that dispatches each synthesized request straight to the proxy's gated handler, with the caller's Bearer attached so the gate re-validates it, and a named non-routable origin (`http://millivolt.internal`) that is never dialed and never derived from the request Host. The endpoint is stateless and Bearer-only: every POST gets a temporary server session bound to that request's credential, nothing survives it, and the dashboard session cookie is refused there because a cookie must not create an internal call. The SDK's own localhost DNS-rebinding `Host` check is deliberately off: the documented reverse-proxied placement preserves an external `Host` while dialing the proxy over loopback, so the check would reject that placement. Protection therefore rests entirely on the Bearer, which the same gate re-validates on every request; a missing or wrong credential is denied before any tool runs. The consequence is explicit: the endpoint admits any `Host` on the listener it is reachable from, with a valid credential, and any `Host`-based allow rule has to sit at the ingress rather than here. The destructive surface is guarded where the risk lives. A purge needs a filter that constrains something (the bodyless delete-everything command is never sent), an exact confirmation phrase, and the opaque `preview_token` that `purge_preview` issued for that same filter, its count and an expiry. A count cannot bind the two: it is only a number, so two filters matching the same number would authorize each other. The token is stateless and HMAC-bound under the operator credential, so nothing is persisted, no server-side cleanup is needed, and a rotated credential invalidates every outstanding preview. It carries its own expiry, enforced when the token is verified, so a preview authorizes a deletion only shortly after it was taken. A refusal discloses no count, because a refusal that reports the live count is a count oracle a model can retry its way through. The count is compared a second time inside the delete transaction in `Store.purgeTransaction`, so a population that moved refuses with 409 before any row is removed. The remaining limit is stated rather than claimed away: a count compares a population, not row identities, so a same-size different row set passes and a refusal needs a fresh preview. Capture refuses to start without durable storage, and checks a scope name against the vocabulary the proxy reports, because either way the session would report success and store nothing. Stopping a capture session deletes nothing, and a purge deletes the stored capture document of every request it matches; the tools say which is which rather than implying otherwise. When that vocabulary is readable and non-empty, an unknown capture scope or throttle provider is refused and the known names are offered; both checks accept any name when it is empty, and the throttle provider check also accepts one when the debug read fails, while the capture tool refuses when the same read fails. See the [MCP server guide](mcp.md). ## Dashboard render owners HTML embeds the canonical bootstrap builder's JSON, safely escaped in an inert data block. Transit gzip is negotiated from `Accept-Encoding` with only the `q` parameter read: unknown parameters are ignored, a bare `*` accepts gzip, and an explicit gzip entry outranks the wildcard; an unparseable or out-of-range qvalue refuses. Deferred external assets download alongside HTML; the Settings sheet is the one exception: its ordered `settings*.js` set is injected by the Settings trigger on first open, one module at a time under the same versioned URL, and single-flighted across concurrent opens. Saved hash/chart choices apply before rendering; invalid/missing seed falls back to the endpoint. Dynamic HTML/state is private/no-store. Every local `/dash/` CSS/JS URL in the shell carries the content version as `?v=`. On a hit, the service worker answers a `/dash/` request carrying exactly that version from its stamped CacheStorage before the network; on a miss it fetches once, stores a successful response, and returns it. This is the one cache-first path. A request with no `v`, a stale value, or more than one `v` stays network-first with cache fallback, as does every brand/PWA path, so an old index cannot pin new bytes under an old URL. The server serves the exact-version response `private, max-age=31536000, immutable`, while a missing or stale version revalidates with `no-cache`. A Bearer-authenticated response that also mints a session cookie keeps the gate's stricter `no-store`. Immutable asset representations and revalidation identities are precomputed. Content version changes use the same winning bootstrap gate to reload once. The cache-first path is asset caching only, never an offline app shell: navigation to `/` stays live with the static offline fallback, and live/operator routes are never cached. Vanilla modules under [static/js](../internal/web/static/js) have distinct owners: - `core`: formatters and shared state. - `feedback`: the one clipboard and toast path. - `entities`: identities and record labels. - `explorer`: routing and cards. - `chart`: uPlot plans. - `menu`: the one open/close, placement, dismissal and focus contract for menu panels. - `chrome_menus`: the header menu registry and the remaining theme tray, nav and dimension class menus. - `operator_console`: the header readout, console shell, section rail, shared wording and bounded session change ledger. - `operator_console`'s four policy modules `operator_limits`, `operator_holds`, `operator_export` and `operator_overview`, each owning its own section. - `chrome_operator`: the mutation gate and request shape. - `chrome_lists`: the operator vocabularies, model grouping, capture storage gate and footer words. - `chrome_restart`: the restart choreography. - `chrome_settings`: the Settings shell and trigger, the accepted debug snapshot, hold liveness and the download paths. - `timeframe`: the chart time vocabulary, stored-extent availability and the timeframe control. - `log`: the live log frame. - `log_rows`: the rows, day dividers and column registry. - `log_archive`: the durable archive window. - `log_drawer`: the detail drawer lifecycle and retry-attempt navigation. - `log_detail`: the detail primitives. - `log_debug`: the drawer's Debug capture section. - `chrome_storm`: the storm banner, incident identity and details dialog. - `theme_boot`: the palette ids, labels, order, defaults and the pre-paint guard. - `theme_state`: normalization, the stored pair, the one apply path and the browser chrome color, shared by the dashboard and the site. - `theme`: the dashboard preview tray, comparator state and keyboard lifecycle. - `settings`: the lazily loaded Settings sheet. - `live`: bootstrap/SSE/derived state. `theme_boot.js` is the one registry: palette ids, labels, order and defaults, plus the blocking guard that applies a stored pair before first paint. `theme_state.js` owns normalization, the single stored shape, the one apply path, the browser chrome color and the optional chart/menu callbacks. A commit changes the live document pair; normalization also repairs storage at load when the stored value is invalid. `theme.js` owns only the dashboard tray: preview, comparator state and the keyboard lifecycle. The site's `site_theme.js` renders the same registry into its own header tray and reuses the same state module. Hover or keyboard focus previews inside the tray only, and the polite live region follows the keyboard cursor: pointer-induced focus never announces a candidate. Arrow keys, Home and End move the cursor and its one tab stop; Enter/Space or a click commits. Compact touch, short and narrow layouts hide the comparator, preview line and hint while keeping the current caption, surface control and all eight rows. `css/themes.css` owns the complete surface, accent and material families every palette resolves in both modes and the Black/Clear mode material fallbacks. The site publishes these exact bytes and loads them after its generated tokens and before `site.css`; its extra contrast rungs derive from the active mode tokens with `color-mix`, so there is no second palette table. `dashboard.css` owns the default Instrument Black base, the header chrome and the theme tray, including the comparator/caption-bar layout. The dashboard sheet is split by surface, linked in this order: `dashboard.css` (palette, base, chrome and KPI band), `explorer.css`, `overlays.css` (drawer and settings sheet), `charts.css`, `requests.css` (including the shared semantic status fills) and `inspector.css`, with `themes.css` next, then `console.css` for the readout and operator console only, and the vendored uPlot sheet last. The link order is the cascade; `dashboard.css` remains the `:root` palette and type owner: its `@font-face` rules load the embedded Instrument Sans and JetBrains Mono faces from `static/fonts`, and its `--sans`/`--mono` tokens name the families every surface consumes. The docs build publishes the same files and adds Instrument Serif as the site-only display face for marketing and documentation titles, so neither surface makes a third-party font request. Each palette owns `--shadow-rgb` and `--veil-rgb`, and the mode veil tokens compose that hue, so cool Clear palettes get cool elevation and scrims. `chrome_settings.js` owns the Settings trigger, loader, open-state probe, shared status line and the single close entry point; the lazy `settings*.js` modules own the sheet's state, editors and apply/revert/backup/purge behavior. A clean open at the same document revision and writable state still fetches freshness but keeps the retained rows and baseline; revision/writable changes, drafts and Revert refill. The model-rule preview reuses one traffic-example cache keyed by the live revision, and its test spelling is preview-only: it never revalidates the draft or marks the sheet dirty. One memoized derivation serves scoped rows and shared readouts. O(1) ID upserts, pure-replay suppression and rAF-coalesced live rendering avoid repeated scans. Incremental log groups preserve scroll/selection; archive paging uses paired timestamp/ID cursors and stays outside the live arrival-ordered ring. One single-entry memo reuses the merged ring+archive window within a live revision, keyed by the live data and its revision plus the archive array identity/length and the scoped-ring (or prior merged) array identity; archive replacement or trim, reset, a live revision bump and a data replacement each rebuild it. The explorer memoizes its filter-set identity on the canonical, order-independent scope (filter set plus status, excluding the `by/` breakdown). A by-only hash change repaints the rail/gallery and fetches that one breakdown while the scoped log window, chart and drawer keep their state. Unchanged explorer renders write nothing beyond card string construction: the rail and gallery HTML is compared before assignment, the shared dimension trigger is rewritten only when its dimension/count presentation key changes, and gallery layout sizing runs only after the gallery content actually changes. Server chart responses alone own buckets/totals at configured cadence. Presets, series, money/unit formatters and visible dimension order each have one registry. Derived series (error rate, cache hit, blended totals) recompute from those server bucket sums; the browser never folds events into chart data. Hidden lines keep stable columns; percentile/range changes reuse the plot. Isolated samples use filled/ring points without coordinate jitter. Shared axis formatting avoids duplicate labels. The percentile dropdown is the only visible pXX label owner; conditional error/429 badges omit zero independently. Operators use shared mutation/revision gates, lossless Settings collection, stable event-path outside-click handling and a focus/inert dialog lifecycle. CSS owns responsive breakpoints. Preserve accessibility, saved state and cheap boot; jsdom tests do not replace real desktop/mobile canvas checks. ## Documentation captures `scripts/gallery/data.py` is the one registry for every published application capture: path, title, description, scene, viewport and setup. `tests/docs_capture.py` owns the scene strategies, and `scripts/gallery/compose.py` composes the two surfaces of each plate. A new view is one registry entry plus a scene, never a second list in the capture harness or the site build. The capture writes `docs/images/capture.json`: per-plate bytes, dimensions and SHA-256, the verified record count and a digest of `internal/web/static`. The digest is the freshness gate; a dashboard change fails `tests/python/gallery.py` until the plates are recaptured. The same run regenerates `docs/gallery.md`. `scripts/site_plates.py` is the one upgrade from a registered image to a plate. It serves both a plain Markdown image and the generated page's semantic figure, so the published gallery, the prose guide and the Markdown source agree about number, station, route and the Black/Clear split legend. `scripts/gallery/__init__.py` loads and validates the registry at import, so a missing alt, duplicate path, unknown scene or bad viewport fails before a browser starts. CI runs the same module through `tests/python/gallery.py`. --- # Contributing Start with the [architecture map](docs/architecture.md). Keep changes focused on existing shared owners, include regressions, and use neutral fixtures rather than real providers. Setup, safety and verification rules live in this guide. Use the [issue tracker](https://github.com/LLM-4-People/millivolt/issues) for non-sensitive bugs and proposals, and [pull requests](https://github.com/LLM-4-People/millivolt/pulls) for changes. Security reports belong in the private channel in [SECURITY.md](SECURITY.md). ## Support and first change [SUPPORT.md](SUPPORT.md) routes usage questions, reproducible defects, concrete proposals, and security reports. For a first change, fork the repository, branch from `origin/testing`, update the existing shared owner, and add a targeted regression. Run `scripts/check.sh` and report what ran. Open the pull request against `testing`, not `main`. Use the `good first issue` label only for documentation, tests, or tooling with a clear scope and expected result. Do not label credential handling, authentication, storage, protocol-core, or release work. Ask in the issue before starting. ## Branch workflow New features branch from and land on `testing` first, with their documentation and relevant checks in the same change. Fetch the current remote branch before starting follow-up work. Use feature branches targeting `testing` for review, or push a completed, checked feature directly there when that is authorized. `main` is promotion-only. Do not develop on it, push it directly, or bypass its rules. The `main-testing-first` ruleset requires a pull request and the `required` check, allows merge commits only, and has no bypass actors. The `long-lived-branch-integrity` ruleset also blocks deletion and non-fast-forward updates. Administrators have no bypass either. When release automation needs new behavior, change the workflow in a pull request; never add a bypass actor or use an administrator override. Inspect the active ruleset with: ```sh ruleset_id="$(gh api repos/LLM-4-People/millivolt/rulesets \ --jq '.[] | select(.name == "main-testing-first") | .id')" gh api "repos/LLM-4-People/millivolt/rulesets/$ruleset_id" \ --jq '{name,enforcement,bypass_actors,current_user_can_bypass}' ``` It must report `enforcement: active`, an empty `bypass_actors`, and `current_user_can_bypass: never`. ### Commit as you go Commit each completed fix locally as soon as its targeted check passes. This includes every individual finding from a review, audit, or multi-agent round: do not hold those edits in the working tree until the end, and do not squash several findings into one final commit. If a fix depends on an earlier uncommitted change, commit that coherent change first, then commit the dependent fix. Keep unrelated user changes out of the commit. A push still waits for the full push gate on the exact commit being pushed. A commit subject also feeds the generated release notes: every subject whose type is not one of `build`, `brand`, `checks`, `chore`, `ci`, `deps`, `doc`, `docs`, `merge`, `refactor`, `release`, `style`, `test`, `tests`, `tooling` or `version` becomes a Highlight on the release that first contains it, and internal types are folded under "Other changes". Write the subject for an operator reading the release page. Completed feature pushes to `testing` publish only the moving development image. All pull requests to `testing`, all code-bearing pushes to `testing`, manually dispatched `testing` gates, and manually pushed `v*` tags run the full gate. Docs and site-only pushes to `testing` run the smaller documentation gate in `docs.yml`. An ordinary promotion pull request to `main` runs only `check.yml`'s lightweight `required` promotion gate. It verifies the head is the current `testing` tip, `main` is already merged back into `testing`, and the full gate already succeeded for that commit: the canonical `check.yml` push run's own check suite, with `unit`, `browser` and both native container legs. A final docs-only commit after the closest gated `testing` commit reuses that gate when every path since it is docs/site-only. Every non-Dependabot promotion also starts [`escalation.yml`](.github/workflows/escalation.yml) on its head and must carry a successful `race` job for that exact commit. The proof waits under a bounded budget and fails closed. The fallback selects the closest gated testing commit from canonical `testing` push runs, with no commit-count bound. For a code-bearing head, let the `testing` `check.yml` push gate finish before opening or re-running a promotion PR. Pending exact check runs get a short bounded wait and the proof fails closed while they remain pending. A docs/site-only head has no canonical `check.yml` push run of its own, so wait for `docs.yml` or only the `required` docs validation. A Dependabot security pull request targets `main` directly and therefore still runs the full gate. ### Promote testing to main 1. Fetch and align the branches. `testing` must contain the feature work and `main` must be an ancestor: ```sh git fetch origin git switch testing git merge --ff-only origin/testing ``` If `main` is not an ancestor of `testing`, merge `origin/main` into `testing` first and resolve conflicts before continuing. 2. For a code-bearing promotion, bump `VERSION` on `testing` to a version newer than every existing `v*` tag. Normal SemVer precedence applies; a prerelease is lower than the same stable version. Commit the bump separately: ```sh printf '\n' > VERSION git add VERSION git commit -m "bump version to " git push origin testing ``` A docs/site-only promotion may skip this step. `required` detects that every changed path is covered by the docs/site filter (for example Markdown, `docs/`, `site-src/`, the docs and Pages workflows, and `LICENSE`), and the merged main push stays docs-only. 3. Open a pull request from `testing` to `main`. For a code-bearing head, wait for the `testing` `check.yml` push gate to finish first. A docs/site-only head has no canonical `check.yml` push run of its own, so wait for `docs.yml` or only the `required` docs validation. Then wait for `required`. The proof polls pending exact runs a bounded number of times and fails closed while they remain pending, so opening or re-running the PR early only produces a red check. `required` does not rerun the full `testing` gate. For every non-Dependabot promotion it verifies that the head is the current `testing` tip and that `main` is already merged back into `testing`; code-bearing promotions must also have a successful full-gate record for that exact commit, or for the closest gated `testing` commit when every path since it is docs/site-only, and `VERSION` newer than every release tag. A non-Dependabot promotion also needs the escalation workflow's `race` job to succeed for this head; it starts with the PR, and `required` waits for it under a bounded budget. If it fails with "Promote the current testing tip", push the final commit to `testing` and reopen or update the PR. If it fails with "Merge main back into testing", merge `origin/main` into `testing` first. If it fails with "changes non-docs paths after the gated commit", push the final code commit and let its full gate finish. If it fails with "VERSION must be newer", bump `VERSION` above every tag and push the commit. If GitHub does not create the `testing` push run, dispatch the full gate on the current tip and wait for it to succeed; the proof accepts that run as an equivalent testing gate: ```sh gh workflow run check.yml --ref testing ``` 4. Merge with a merge commit: ```sh gh pr merge --merge ``` Squash and rebase are disabled on `main`; they would replace the reviewed testing history instead of preserving it. 5. The merged-PR release run starts as soon as the promotion merge lands; wait for its publish job (and the Pages run, when applicable) to finish, then merge `main` back into `testing` so the next promotion is up to date: ```sh git fetch origin main git switch testing git merge --ff-only origin/main git push origin testing ``` If `testing` moved after the promotion merge, merge `origin/main` normally and resolve conflicts before the next promotion. For a code-bearing promotion, the merged-PR release run creates the annotated release tag on the reviewed merge commit and publishes the image. It never pushes `main`. A delivered `main` push is an idempotent fallback; a manually pushed `v*` tag remains the explicit recovery path. Do not push a `v*` tag as part of feature work; the manual tag path is only for an explicitly verified release and must match `VERSION`. Dependabot version updates target `testing`. Dependabot security updates target the default branch and are the only exception to the testing-ancestry check; they also skip the version check. They still run the full gate and the `required` check. A security-only merge normally publishes no image and creates no tag because `VERSION` is already tagged; if `VERSION` is still untagged, that merge can publish the pending version. Merge `main` back into `testing` and include the fix in the next versioned promotion. Docs and site-only pushes to `testing` skip image work. A docs/site-only promotion to `main` skips the `VERSION` and full-gate proofs, validates the Markdown, generated site, workflow policy and docs contracts in `required`, and the merged-PR release path records the change as docs-only, so it does not rebuild images or cut a release. A Pages-relevant change deploys the site from the `main` push, or from the Pages `workflow_dispatch` fallback when that push workflow is not delivered. `AGENTS.md` is intentionally local and ignored by the publication guard. Keep its working instructions aligned with this shared guide; do not force-add private instruction files to publish workflow policy. ## Development dependencies Linux is supported. Version sources are: - Go: [go.mod](go.mod). - Node: [.node-version](.node-version). - Python: [.python-version](.python-version). - `jq`: the workflow contract suite executes the promotion gate's shell predicates, so the core checks need the `jq` binary on `PATH`. - Browser tooling: [requirements-dev.txt](requirements-dev.txt). - Documentation site: [requirements-docs.txt](requirements-docs.txt). The proxy embeds its vanilla JavaScript/CSS and needs no frontend build or Node runtime in deployment. Node/jsdom, Python/Playwright, Python-Markdown and PyYAML are development and documentation tools. ```sh npm ci --ignore-scripts python3 -m venv .venv . .venv/bin/activate python -m pip install --require-hashes -r requirements-dev.txt python -m playwright install chromium ``` On fresh Linux installations, Chromium may also need system libraries; use `python -m playwright install --with-deps chromium` with appropriate installation permissions. Go race checks require `CGO_ENABLED=1` and a working C compiler; the regular proxy build does not require a native SQLite driver. ## Checks ```sh scripts/check.sh ``` The shared script owns the core verification commands. They include offline documentation-link, publication-ignore and typography regressions (Git is required), the generated-site contract, the documentation readability check and the workflow/automation policy. That policy suite runs the workflow files of the ref under test. A `pull_request` run evaluates its own head revision rather than the base branch's, so a workflow change is judged as this pull request ships it. Review the workflow diff itself, since the run that validates it was configured by the same commit; only the post-merge `main` state proves what landed. `npm test` runs the jsdom suites and the request-log merge regression probe separately. All tests and disposable harnesses live in [tests/](tests/README.md), with suffix-free physical filenames. Go's temporary overlay adds its required virtual `_test.go` names; Python unit modules live in `tests/python/`. Use `scripts/check.sh go test` for targeted Go checks and `scripts/check.sh go vet` or `scripts/check.sh go mod tidy` to include test code and dependencies. Raw `go test ./...` does not discover the relocated tests and can report success without running them. Re-run checks after your last edit. Concurrency changes need race coverage. JavaScript unit checks do not establish real canvas rendering, layout, focus or viewport behavior. ### Documentation site The public site is generated from the existing Markdown, not maintained as a second documentation tree. `site-src/` owns the layout, styles, and brand assets; `scripts/build_site.py` renders the pages, rewrites repository links, and emits the search index, sitemap, 404 page and the machine-readable docs outputs described below. The generated `site/` directory is ignored and never committed. ```sh python3 -m pip install --require-hashes -r requirements-docs.txt python3 -B -O -m tests.python.site python3 scripts/build_site.py --output site ``` Both requirements files are hash-pinned and every install command passes `--require-hashes`, so resolution is reproducible and a substituted file fails the install. After changing a pin, regenerate the hash block for that version from the PyPI JSON document (`https://pypi.org/pypi///json`, `digests.sha256` of every non-yanked file, one `--hash=sha256:` line each), keep the pins sorted as they are, and run the install above once to prove the file. For a local preview, serve `site/` from a separate terminal with `python3 -m http.server --directory site 8082`; do not point it at the proxy or use the main `:8080` instance. The `docs.yml` workflow builds the site for documentation changes, and `pages.yml` deploys from `main` through the official GitHub Pages Actions flow. Repository-level Pages enablement and the `github-pages` environment are one-time administrator settings. The shell every page loads before content renders is owned by `SHELL_ASSETS` in `scripts/build_site.py`: `tokens.css`, `themes.css`, `site.css`, `theme_boot.js`, `theme_state.js`, `site_theme.js` and `site.js`. Each is published at a content-hashed name such as `site.11c6502e41.css`. `tokens.css` is derived from the dashboard palette. `themes.css`, `theme_boot.js` and `theme_state.js` are published byte-for-byte from `internal/web/static`, so the dashboard and the site share one palette owner, one registry and one state module. `tests/python/site.py` checks that identity, the load order and the registered palette ids. GitHub Pages sets its own `Cache-Control`, so the URL is the only cache key this repository controls. A page cached from the previous deployment names the previous hash, and it must find those bytes. The build republishes the last `SHELL_RETENTION` generations of each shell file, regenerated from the revisions that produced them, which is why the Pages workflow checks out full history. A checkout without history still builds the current page correctly but cannot retain earlier generations. Do not add a hand-written reference to an unhashed name; `tests/python/site.py` rejects one. ### Machine-readable documentation `scripts/build_site.py` drives `scripts/site_llm.py`, which writes a Markdown twin beside every rendered page, plus `llms.txt`, `llms-full.txt` and `docs.json`. The MCP server embeds a checked-in copy at `internal/docs/docs.json`; never edit that file by hand. Regenerate it after any Markdown, title, navigation or description change: ```sh python3 scripts/build_site.py --output site --write-manifest ``` `tests/python/site.py` fails when the embedded copy is stale, when a page does not advertise its Markdown twin, or when a generated URL names a missing file. The MCP tools `docs_list`, `docs_search` and `docs_read` read that index and make no proxy request. ### Documentation prose Write for a reader who is scanning. Keep one idea per paragraph and roughly 110 words or fewer. Use a list or table when a sentence carries three or more parallel items, and split a paragraph when it changes subject. `tests/python/docs_readability.py` enforces the word limit on hand-written Markdown. `docs/settings.md` is generated from the schema and is excluded. The site is published at `https://millivolt.llm4people.dev/`. That origin has two owners, and neither is a file in this repository: the Pages custom domain in the repository settings, which is what GitHub serves and certifies, and the DNS record, which is a `CNAME` from `millivolt` to `llm-4-people.github.io` with the proxy disabled. `scripts/build_site.py` warns when a build for `SITE_URL` names a different host than `DEFAULT_BASE_URL`, because that is what a half-finished migration looks like. GitHub ignores a `CNAME` file in an Actions deployment, so this repository does not ship one. ### Brand plates The README covers, specification strips, Open Graph card and GitHub social preview are generated, not hand-edited. `scripts/brand/facts.py` owns the headline, the four specification facts and the sample-history values. `scripts/brand/manifest.json` owns each target's dimensions and hash. Regenerate every plate after a copy or fact change: ```sh python3 -m pip install --require-hashes -r requirements-docs.txt python3 -m scripts.brand.build ``` The PNG renders need Playwright's Chromium; the SVG strips do not. `python3 -m scripts.brand.build --check` verifies every committed asset against the manifest and re-renders both SVG strips from source. CI runs that check. The tool count in the specification strip comes from `scripts/mcp_registry.py`, the same parser the MCP docs test uses, so adding a tool cannot leave the art behind. Upload `.github/social-preview.png` under the repository's Settings, Social preview after it changes. That upload is a GitHub setting, not a repository file. ### Push gate Never push `main` directly: open a promotion pull request and merge it with a merge commit. Do not push `testing` until the exact commit you will push has passed the same CI gate jobs. A request to push is not a waiver. Package-scoped `go test` and `npm test` during development are not sufficient. CI's source-and-unit step is the default core suite: ```sh scripts/check.sh ``` A run may not start while a code-scanning finding is unresolved. `scripts/security_gate.py` reads `code-scanning/alerts?state=open` from the repository API in the `unit` job and again in `publish` before the version tag is created. Fix the finding, or record a dismissal with a reason that explains why the code is correct; a dismissed alert is not open, and the security page keeps that reason beside the location. An API the gate cannot read fails closed rather than passing. CI sets `GOTOOLCHAIN=local`, so the host Go must match the `go` version in `go.mod`. On a host with an older base Go, install Go 1.27.2 or select the matching toolchain before running the gate; `GOTOOLCHAIN=local` alone cannot upgrade an older installation. Full-gate CI events also run the isolated browser fixtures (desktop and mobile) in the `browser` job. Use `scripts/dev.sh` plus `scripts/check.sh browser` as below, with `MILLIVOLT_OPERATOR_TOKEN` set. Re-run both after the last commit. A red GitHub check after a push means the gate was skipped, stale, or unmonitored: fix, re-run, and push only a passing tree, then watch the new run to completion. That same `browser` job also runs the live MCP integration test, which `scripts/check.sh browser` alone does not run. It is mandatory on the same commit: ```sh MILLIVOLT_MCP_INTEGRATION=1 scripts/check.sh go test -count=1 ./internal/mcp/... ``` The test starts a private dev instance through `scripts/dev.sh` on its own loopback port and scratch database, drives every registered tool, and stops the instance again, so it does not touch the instance the browser check uses. Before every push, check open pull requests and report any that are awaiting the user's review or approval. Do not approve or merge them without explicit authorization. Use: ```sh gh pr list --state open --json number,title,author,isDraft,reviewDecision,mergeable,statusCheckRollup,url ``` After pushing, monitor the GitHub Actions run for the pushed commit until every required job completes successfully. The local gate cannot see the browser, MCP integration, container, code-scanning, or dependency-review jobs. Find the run with `gh run list --branch --limit 1`, then wait for it with `gh run watch --exit-status`. A queued or in-progress run is not green. If any job fails, treat the push as incomplete: fix the root cause, rerun this gate, and push the fix. Do not leave `testing` or `main` red or start unrelated work on top of a failing run. For changes to configuration, update the Config/default/validation/schema owners and regenerate the example as described in [configuration](docs/configuration.md). Do not edit `proxy.yaml`: it may be a running operator's configuration. ## Isolated development Never use an existing main instance on port 8080 for tests. Use the lifecycle script, with cleanup even if verification fails: ```sh ( set -e trap 'scripts/dev.sh stop' EXIT export MILLIVOLT_OPERATOR_TOKEN='dev-instance-credential' scripts/dev.sh scripts/check.sh browser --target http://127.0.0.1:8081 ) ``` The dev instance shares your shell environment, so a local `MILLIVOLT_OPERATOR_TOKEN` arms its operator gate; browser and stress harnesses need the same value for gated fixture cleanup. An instance started without it serves reads only and denies every mutation. `scripts/dev.sh status` reports the private instance. The script builds first, then replaces only its tracked dev process; a failed build leaves it running. The tracked identity is the pid file. `status`/`stop` fall back to the listener on the instance's port when the file names a dead pid. A rebuild child rewrites the file only once it is fully booted, so a child that dies before readiness would otherwise strand the instance. The fallback still requires the process to carry this instance's exact `-pid-file` flag. Starting/restarting development resets its disposable database and private config, seeded from the public example by default. `DEV_CONFIG=proxy.yaml` explicitly opts into copying an operator config; it is never selected implicitly. UI-triggered rebuilds keep that private config and update the tracked PID. Overrides are deliberately restricted: `DEV_PORT` is a non-production port, `DEV_HOST` is loopback, and `DEV_DB` is a permitted scratch filename or `none`. `DEV_COPY_DB=1` seeds a consistent online backup of local history: use it only when that history is authorized for the task, and treat the scratch copy as sensitive. `DEV_COPY_DB=redacted` instead derives the separately validated documentation copy described below; `0` keeps fresh fixture state. Do not use broad process kills or start an ad-hoc second main instance. The browser fixtures share private-target/config/database and redirect guards. They disable SSE and test bootstrap/poll rendering, not uninterrupted live-feed delivery. They remove only their synthetic client records. Their assertions must remain enabled under Python `-O`; the guard suite verifies this. ### Live Chrome DevTools verification Frontend changes are additionally verified in a visible Chrome window through Chrome DevTools - headful, never headless. Sign in with the dev operator token, drive the real fixture traffic and the actual controls, and check console output, element geometry, the accessibility tree and screen flow in the rendered page. Narrow fixture data passes the harnesses but can hide distribution problems (wide values, wrapping, density); the live session is where those surface. Keep the dev instance and the window open while you look; stop the instance with `scripts/dev.sh stop` when done. Fixture traffic for these sessions has its own lifecycle owner, never a paid provider and never the main instance on `:8080`: ```sh scripts/dev-traffic.sh # bursts of neutral loopback traffic at the dev instance scripts/dev-traffic.sh status scripts/dev-traffic.sh stop ``` Bursts land about a minute apart so one-minute chart buckets gain a bucket per burst; each run includes recovered 429s (rate-limited, not errors) and one final 400 so both health surfaces carry data. ## Container development Normal [compose.yaml](compose.yaml) only runs a published image. For a local source build, use [compose.dev.yaml](compose.dev.yaml) from the checkout root. It is a complete Compose file, not an overlay: project and service are both `millivolt-dev`, and the container takes the project-scoped name Compose derives from them. ```sh docker compose -f compose.dev.yaml up -d --build docker compose -f compose.dev.yaml logs --tail 50 docker compose -f compose.dev.yaml down ``` The development project keeps its containers and named volumes separate from a normal deployment, including when `COMPOSE_PROJECT_NAME` is set outside the checkout. Its `MILLIVOLT_DEV_PORT` and `MILLIVOLT_DEV_IMAGE` inputs are defined only there, and it shares the deployment Compose's `MILLIVOLT_MEM_LIMIT` and `MILLIVOLT_PIDS_LIMIT` bounds. Choose an unused development port if `scripts/dev.sh` is already running. Rerun the same `up --build` command after source changes. The resulting runtime image has no compiler or source tree, so dashboard rebuild is unavailable. `down` preserves its named volumes unless you explicitly request deletion. This source-image workflow does not replace `scripts/dev.sh`: the host browser and stress fixtures require that script's private process/database identity. ## Documentation screenshots The published gallery uses recorded metrics from an explicitly authorized, redacted local-history snapshot, not fabricated timing or cost values. Never point capture tools at a main instance or serve a raw history copy for screenshots. The source is the checkout's local `proxy.db`, opened read-only through the existing online-backup helper. This workflow needs explicit authorization for that history. From a checkout with browser dependencies installed and that authorized source: ```sh ( set -e export DEV_PORT=18081 DEV_HOST=127.0.0.1 export DEV_DB=/tmp/millivolt/millivolt-dev-docs.db export DEV_CONFIG=proxy.example.yaml DEV_COPY_DB=redacted trap 'scripts/dev.sh stop' EXIT scripts/dev.sh up python3 -B -O -m tests.explorer_check \ --target "http://${DEV_HOST}:${DEV_PORT}" --docs-history docs/images ) ``` Choose an unused permitted dev port. Before startup, `dev.sh` derives a new metrics-only database: identifiers are replaced, content/headers/debug data are removed, and recorded timing, token and cost values are retained. The raw online snapshot is temporary and never served. Treat even the derived database as sensitive; retained usage and timestamps can reveal activity patterns. The view list is a registry, not a capture script. `scripts/gallery/data.py` declares every plate: its published path, title, description, scene, named viewport and setup. `tests/docs_capture.py` owns the scene strategies (page, element, scoped explorer, absorbed retries, drawer, console, settings, chart, window picker, theme tray). `scripts/gallery/compose.py` turns each scene into one plate: the same document is photographed on Instrument Black and Instrument Clear, then split along the diagonal from bottom-left to top-right. `applyState` is the one theme mutation path, so an open drawer, a selected filter or an open tray survives the second half. The run writes `docs/images/capture.json`, which records every plate's bytes, dimensions and SHA-256, the verified record count, and a digest of `internal/web/static`. It also writes the generated `docs/gallery.md`. Commit both with the images. `tests/python/gallery.py` fails when a registered plate has no capture, a capture is not registered, a byte or dimension drifts, the digest no longer matches the dashboard source, or the gallery page is stale. A UI change therefore requires a recapture in the same branch, not a later one. Capture last. The gallery is a picture of the shipped dashboard, so a markup, CSS or layout change dates every image that shows the affected surface. Recapture in the same branch once the last layout edit is in, never before it, and recapture again if a later edit moves what an image shows. A recapture diff is the review set: inspect every changed plate, and expect geometry changes (for example the request table's 8px cell padding and retry sub-rows) to come with them. The capture mode revalidates every row and its provenance, plus exact public configuration bytes, before and after browser capture. It permits only read-only same-origin requests: no inference, POST, purge or Settings save. Images are staged before publication, and the exit trap stops the private process. Review every plate before committing. SSE is deliberately disabled, so the images' offline indicator describes a static capture session, not current service health. Captures keep the same category folders under `docs/images/`: `overview`, `charts`, `explorer`, `requests`, `console`, `settings` and `theme`. The site fails on any image under `docs/images/` without a Markdown consumer, so a recapture that drops a view cannot leave an orphan behind. The independent `--docs-images` option remains a strict synthetic regression. Run it only with `DEV_COPY_DB=0` and direct those two images to a scratch directory, not `docs/images`. It creates and cleans up its own six-request fixture and rejects copied history. Neither capture mode is a performance or compatibility benchmark; do not weaken one mode's admission checks to accommodate the other. ## Performance evidence The published tables, method and limitations live in [performance and resource usage](docs/performance.md). That page is the single source for measured numbers; do not copy them into other documentation and do not edit the page by hand. It is rendered from `perf/results.json`, and `tests/python/site.py` fails when the two disagree. `scripts/perf.sh` owns a repeatable local measurement run without touching the main instance. It starts fresh private instances on a disposable port and stops them through `scripts/dev.sh`: ```sh scripts/perf.sh docs # every published measurement, bundle and page scripts/perf.sh bench # isolated Go benchmarks scripts/perf.sh stress # private fast and streaming ramps scripts/perf.sh streaming # sustained end-to-end SSE throughput ramp scripts/perf.sh sustained # fixed-duration fast-response capacity levels scripts/perf.sh image # disposable image content and compressed size scripts/perf.sh all # image, bench, then stress ``` `docs` refuses a dirty tree, runs the complete documented set, writes `perf/results.json`, and regenerates `docs/performance.md`. Commit the bundle and the page together. Never hand-edit either one; a missing stage, failed response or unbalanced durable result aborts the run instead of publishing a partial page. The bundle is also the source of the facts the page describes: the commit, date and platform, each stage's response shape (frame count, usage and unit cost), the configured durations, and the harness declarations for captures, previews and the observer. `cmd/stress` records the fixture shape in its workload line and the collector validates it per stage, so a table and the sentence beside it cannot describe different runs. The landing headline reads the same bundle; `tests/python/performance.py` renders two bundles that differ only in those fields and fails if any published sentence keeps its old text. For request, SSE-event, response-byte and output-token rates, use the `streaming` mode. It derives those rates from responses the client accepted byte-for-byte, and it is still a synthetic-fixture measurement, not a universal capacity claim. For an individual stress run with the private dev instance already running: ```sh go run ./cmd/stress -target http://127.0.0.1:8081 -concurrency 1,8,32,128 -duration 5s ``` The harness does not manage or reconfigure the proxy. It uses a local synthetic upstream and checks exact bytes, durable rows/tokens/cost and the same-process storage-drop delta. A drop is a failed stage even if every HTTP request succeeded, unless the run asks for an overload observation with `-allow-drops`. That mode still requires `durable + dropped = successful requests` and still aborts on a response failure. Read `go run ./cmd/stress -h` for workload/safety controls. The ramp retains its cumulative fixture history across measured stages, then removes only its unique client records after measurements finish. Cleanup has its own bounded wait even after cancellation and revalidates private process, config and database ownership before every deletion. A cleanup failure is reported, never silently treated as success. Report workload, duration, concurrency, configuration, contention and whether storage/observers were active. Separate request throughput from durable throughput. Closed-loop bursts, microbenchmarks and local upstreams do not establish sustained production capacity. The current storage contract and capacity caveats live in [operations](docs/operations.md#storage-and-accounting). Use the existing `BenchmarkStorageWriter` and `BenchmarkProxyDurableContention` for isolated writer/contention comparisons; neither replaces a real HTTP run. For the analytical projection's memory, use the self-seeded `BenchmarkProjectionFootprint` (retained and transient bytes per stored record) and `BenchmarkProjectionProcessRSS` (process RSS after readiness) from the test overlay; they need no running instance: ```sh scripts/check.sh go test -run='^$' -bench='BenchmarkProjection' -count=1 ./internal/web ``` ## Versioning and publication [VERSION](VERSION) is the only semantic version source; do not duplicate it in package manifests, build arguments or image metadata. Normal and prerelease SemVer are supported without build metadata. A release tag must be exactly `v` followed by that file's version. The release CLI validates the same contract: ```sh millivolt_version="$(tr -d '\r\n' < VERSION)" go run ./cmd/release -revision "$(git rev-parse HEAD)" -tag "v$millivolt_version" ``` Publication is automatic. A code-bearing versioned promotion releases when the promotion PR merges, after the full gate has passed for the promoted commit, or after the closest gated `testing` commit plus a docs-only delta. A non-Dependabot promotion also requires a successful [escalation](.github/workflows/escalation.yml) run for the reviewed head. The promotion PR starts it automatically, and `required` waits for it under a bounded, fail-closed proof. The merged-PR event carries the exact merge commit; a delivered `main` push is an idempotent fallback, so a dropped push event cannot leave the release untagged. A Dependabot security PR targets `main` and runs the full gate itself rather than reusing a `testing` result. The release `publish` job waits for `required`. The `publish-testing` job waits for `unit`, `browser` and `container` instead, because a testing push has no promotion proof to wait for: 1. A code-bearing promotion PR owns the version bump. `required` compares VERSION against every existing `v*` tag with SemVer precedence and rejects a non-Dependabot main PR unless VERSION is newer. Docs/site-only promotions skip that comparison, validate docs, workflow policy and the site in `required`, and stay release-free. If VERSION is already tagged, no release is cut: this is the Dependabot security-merge case, and the next versioned promotion releases the change. 2. The workflow creates the annotated `v` tag on the reviewed promotion merge commit and builds and publishes the multi-arch image with `main`, `sha-*`, `version` and (for non-prereleases) `latest` tags, including an SBOM and max-mode provenance. The `sha-*` tag and OCI revision label name the promotion merge commit that was built. 3. Only after the image exists does it push the tag and create the GitHub release. It never pushes `main`. Release notes filter merge commits, dedupe subjects, use the lowercase image path, include the digest, and link the compare range. A failed image build leaves no public release pointing at a missing image. `main` and `latest` exclude prereleases; the `testing` alias never moves them. Docs and site-only pushes to `testing` skip image work. They validate Markdown, the generated site, workflow policy and docs contracts. A docs/site-only main promotion also skips the VERSION and full-gate proofs but runs the same Markdown, generated-site, workflow-policy and repository checks in `required`. A Pages-relevant merged main push stays docs-only, so Pages deploys without rebuilding images or cutting a release. The merged-PR release path classifies the change and skips publication. A code-bearing promotion must carry a VERSION newer than every existing tag and cuts a release. A manually pushed `v*` tag still ships exactly as tagged through the same pipeline, with the image published before the release is created. A manual tag must match VERSION. Code-bearing pushes to `testing` publish the moving `ghcr.io/llm-4-people/millivolt:testing` image after the same checks and container builds, without cutting a release or any version tag. Its OCI version label reports the embedded VERSION, which can lag the latest release. Manual validation of a candidate remains available: ```sh scripts/check.sh release -revision "$(git rev-parse HEAD)" -tag "v$millivolt_version" ``` Run the shared checks before relying on a release. Confirm the hosted container build and smoke tests succeed, then check the package visibility and anonymous pull. Do not report a release/image as available merely because validation or a local build passed. Image usage and persistence limits live in [operations](docs/containers.md#versions-and-images). The gate clears inherited Git routing/configuration variables before checking source identity, so another worktree or external ignore policy cannot satisfy the release checks accidentally. Only push the intended release tag after that check succeeds. Main changes go through the protected pull request path. The workflow publishes images automatically and creates the release with notes generated from the commit log. Native container checks run before the multi-platform publish job. After the promotion run, verify the published tag and image: ```sh gh release view "v$millivolt_version" docker buildx imagetools inspect "ghcr.io/llm-4-people/millivolt:$millivolt_version" ``` ### If a release fails - Before the tag is pushed, no public release exists. Fix the failure and rerun the main workflow or promote a new version. - After the tag is pushed, keep the tag in place. If the GitHub release is missing, create it from that tag or publish a new patch version. An automatic tag push does not start a second workflow run. - Do not move or delete a published version tag. For local container verification, build with Docker, resolve the image's immutable ID, then use the same fixture as CI: ```sh docker build -t millivolt:check . millivolt_image_id="$(docker image inspect --format '{{.Id}}' millivolt:check)" scripts/check.sh container --image "$millivolt_image_id" ``` The shared container mode validates Compose defaults and owns only its disposable, network-isolated Docker fixture, selected by immutable image ID. It checks startup, HTTP, Settings and durable state across stop/start, then backs up the stopped volumes and restores into another fresh container. Recovery must retain settings, history and private file ownership/modes. It never publishes a host port or reuses an operator volume. It complements, rather than replaces, the host lifecycle in `scripts/dev.sh`. ## Change checklist Use sentence case or lowercase in prose, headings and labels. Avoid em dashes; preserve exact protocol names, identifiers, user data and license wording. - Add a targeted regression for a reproduced defect; preserve existing coverage. - Keep defaults, units, schemas and state transitions at their canonical owner. - Measure allocations/latency when changing the hot path or analytical folds. - Check saved views, mobile layout and accessibility for visible changes. - Update the relevant existing documentation, not duplicate specifications. - Exclude local configs, databases, tokens, captures and generated test artifacts. - Preserve dependency licenses. Report vulnerabilities using [SECURITY.md](SECURITY.md), not a public issue containing sensitive reproduction data. Dependabot checks Go, npm, Python, Docker and GitHub Actions dependencies weekly with grouped minor and patch updates. Review dependency changes and re-run checks; runtime version files still require deliberate updates and verification. The Docker `golang` base is deliberately left manual because its tag must move with go.mod; distroless updates remain automated. Security updates target `main` as described in the branch workflow. Dependency review runs on pull requests, and a scheduled `govulncheck` workflow covers the Go module graph. The scanner is pinned as a tool dependency in `tools.go`, so it resolves from this module and `go.sum` covers it instead of a version being fetched fresh on each run. These scans complement tests; a clean scan is not a security guarantee. --- # Support millivolt is pre-1.0 and maintained on a best-effort basis. There is no response-time SLA. ## Read first - [Documentation index](docs/README.md) - [Quickstart](README.md#quickstart) - [Operations](docs/operations.md) - [Reverse proxy](docs/reverse-proxy.md) - [Security](SECURITY.md) ## Where to ask - **Usage, configuration, or deployment question:** open a [usage question](https://github.com/LLM-4-People/millivolt/issues/new?template=question.yml). - **Reproducible defect:** open a [bug report](https://github.com/LLM-4-People/millivolt/issues/new?template=bug_report.yml). - **Concrete proposal:** open a [feature request](https://github.com/LLM-4-People/millivolt/issues/new?template=feature_request.yml). - **Security vulnerability:** use the [private advisory channel](https://github.com/LLM-4-People/millivolt/security/advisories/new). Do not open a public issue. Search existing issues before opening a new one. Include the millivolt version or revision and a sanitized configuration shape. Remove credentials, private prompts, databases, and host-specific paths. --- MIT License Copyright (c) 2026 millivolt contributors Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. --- # Third-party notices millivolt's [MIT license](LICENSE) covers the project's own work, not a relicensing of dependencies. ## Embedded dashboard library [uPlot 1.6.32](https://github.com/leeoniya/uPlot/tree/1.6.32) is vendored as `internal/web/static/vendor/uplot.js` and `uplot.css`. Copyright (c) 2022 Leon Sorokin. Its complete [MIT notice](internal/web/static/vendor/uplot.LICENSE) is kept beside the assets and included in the embedded filesystem. Preserve that notice when redistributing the library or built dashboard. The notice was checked against the [upstream versioned license](https://github.com/leeoniya/uPlot/blob/1.6.32/LICENSE). Update the version, assets and notice together when changing this dependency. ## Embedded typefaces The dashboard and the documentation site ship four WOFF2 faces in `internal/web/static/fonts`: Instrument Sans (roman and italic), Instrument Serif, and JetBrains Mono. They are Copyright 2022 The Instrument Sans Project Authors, Copyright 2022 The Instrument Serif Project Authors, and Copyright 2020 The JetBrains Mono Project Authors. All three families are licensed under the [SIL Open Font License 1.1](internal/web/static/fonts/LICENSE); the complete notice and copyright lines are kept beside the files and included in the embedded filesystem. The dashboard, the docs site and the generated brand plates all consume these exact faces, so no third-party font request is made at runtime. Preserve the notice when redistributing the fonts or a built dashboard. ## Module and development dependencies [go.mod](go.mod) and [go.sum](go.sum) identify Go dependencies; their upstream module distributions carry their own licenses. [package-lock.json](package-lock.json) pins development-only jsdom dependencies. [requirements-dev.txt](requirements-dev.txt) pins Python browser tooling; Playwright selects its corresponding Chromium build. [requirements-docs.txt](requirements-docs.txt) pins the Python-Markdown renderer (BSD-3-Clause) used to build the static documentation site; PyYAML (MIT) is a development dependency used by the workflow policy checks. Node, Python and these build/test dependencies are not required to run the proxy. ## Container distribution The [Dockerfile](Dockerfile) runs [the notice collector](scripts/licenses/main.go) against the exact target binary's Go dependency graph. It preserves module, package-ancestor and Go toolchain license, copyright, notice and patent files, plus the embedded uPlot notice, under `/usr/share/licenses/millivolt`. The deterministic `manifest.json` records versions, relative paths and SHA-256 content hashes. Missing notices and module replacements fail the build for review; this collector does not classify licenses or grant additional rights. The pinned distroless runtime retains its own files and dependency notices. Its [upstream project](https://github.com/GoogleContainerTools/distroless) documents the base-image contents. Node, Python and development dependencies are not copied into the runtime image. When distributing a binary by another route, generate and include the same bundle with `go run ./scripts/licenses -target linux/amd64 -out NEW_DIRECTORY` (or `linux/arm64`) and review the actual distribution. This source notice alone is not a complete binary-release license bundle. ---