# millivolt documentation
> Self-hosted, cross-protocol inference proxy with a live dashboard, SQLite history, and an operator plane. One Go binary; no hosted dependency.
# Documentation
millivolt is one self-hosted service for routing model traffic across OpenAI
Chat Completions, OpenAI Responses and Anthropic Messages, seeing what happened,
and controlling the operator surface. Start with the path
that matches what you need to do.
## First ten minutes
1. [Install and first run](install.md) from Compose or a source checkout.
2. Send one request on a client protocol and find it in the
[dashboard](dashboard.md); the [capture gallery](gallery.md) shows every
surface in both themes.
3. Read [configuration](configuration.md) before changing the example or a
setting.
4. Read [security](../SECURITY.md) before exposing the listener.
5. Keep [operations](operations.md) nearby for health, storage and the known
limits.
## If you want to ...
| Task | Start with |
| --- | --- |
| Install and send a first request | [Install and first run](install.md) |
| Configure a setting, request override or provider alias | [Configuration](configuration.md) |
| Look up a setting's default or a CLI flag | [Settings reference](settings.md) |
| See every dashboard surface on both themes | [Capture gallery](gallery.md) |
| Expose the listener behind TLS or a network boundary | [Reverse proxy](reverse-proxy.md), [security](../SECURITY.md) |
| Debug a 429 or a 5xx | [Routes and debugging](routes.md#error-storm-protection), [pause and limits](routes.md#pause-and-limits) |
| Back up or restore | [Settings backup and restore](containers.md#settings-backup-and-restore), [image-only backup](containers.md#image-only-backup-and-restore), [online database backup](operations.md#storage-and-accounting) |
| Upgrade or restart | [Versions and images](containers.md#versions-and-images), [reload, restart and shutdown](containers.md#reload-restart-and-shutdown) |
| Integrate a client or translate between protocols | [Protocol](protocol.md), [adapters](adapters.md) |
| Drive the operator plane from an LLM | [MCP server](mcp.md) |
| Read or search the docs from an LLM | [llms.txt](https://millivolt.llm4people.dev/llms.txt), [docs.json](https://millivolt.llm4people.dev/docs.json), [MCP server](mcp.md#tools) |
| Drive it from a terminal | [Operator CLI](cli.md) |
| Contribute a change | [Contributing](../CONTRIBUTING.md), [architecture](architecture.md), [support](../SUPPORT.md) |
The [operations overview](operations.md) owns health, storage and the runtime
cost model. Measurements live in [performance](performance.md); ownership and
invariants live in [architecture](architecture.md). Project pages are
[support](../SUPPORT.md), [releases](https://github.com/LLM-4-People/millivolt/releases),
[license](../LICENSE) and [third-party notices](../THIRD_PARTY_NOTICES.md).
## Verification and limitations
The generated [settings reference](settings.md) lists every setting and proxy
flag with its type, default, accepted values and reload behavior. Every page
also has a Markdown twin, and `llms.txt`, `llms-full.txt` and `docs.json` index
the same sources for an LLM or the `docs_list`, `docs_search` and `docs_read`
MCP tools.
[proxy.example.yaml](../proxy.example.yaml) is the commented deployment example.
`internal/config` owns defaults, validation and Settings metadata; local
`proxy.yaml` is not a public reference or a test fixture.
Current commands live in [Contributing](../CONTRIBUTING.md#checks). Runtime
limits live in [operations](operations.md#known-limits); measurement caveats
live in [performance](performance.md#limitations). The
[architecture map](architecture.md) connects invariants to their shared
implementation and regression coverage. Past development notes are not an
operating guide or a capacity guarantee.
---
# Install and first run
Install millivolt from the published image or a source checkout, set the
operator credential, and send a first request on each client protocol. The
[quickstart](../README.md#quickstart) is the short form of the same procedure;
this page carries the full path.
## Before you start
The container path needs Linux with Docker Engine and the Compose plugin. No Go,
Node or source checkout is needed for it. The source path needs the Go toolchain
required by [go.mod](../go.mod).
> [!IMPORTANT]
> One shared operator credential protects the dashboard and operator plane only.
> Inference is transparent: it forwards your upstream key and does not use the
> operator token. There are no user accounts. Keep the listener on loopback, or
> put the network or ingress boundary in front. With the token unset, the
> dashboard denies every request. Read [Security](../SECURITY.md) before network
> exposure.
## Install with Docker Compose
```sh
curl -fsSL https://raw.githubusercontent.com/LLM-4-People/millivolt/main/compose.yaml -o compose.yaml
export MILLIVOLT_OPERATOR_TOKEN='replace-with-a-long-random-value'
docker compose up -d
```
Open [http://127.0.0.1:8080/](http://127.0.0.1:8080/) and sign in with that
token. If the container exits, run `docker compose logs --tail 50`.
The Compose file publishes on loopback only. It keeps history and settings in
named volumes, runs non-root with a read-only root filesystem, and accepts
`MILLIVOLT_PORT` for an unused host port. For bind mounts, pre-create
directories writable by UID/GID `65532:65532`; see
[persistent state](containers.md#persistent-state-and-configuration). For image
selection, backups, updates, shutdown, or protected ingress, see
[container operation](containers.md#containers) and the
[reverse-proxy guide](reverse-proxy.md).
The published image is `ghcr.io/llm-4-people/millivolt:main` for amd64 and
arm64. Version tags identify releases; pin a digest when exact bytes matter. See
[versions and images](containers.md#versions-and-images).
## Install from a source checkout
Install the Go toolchain required by [go.mod](../go.mod), then:
```sh
git clone https://github.com/LLM-4-People/millivolt.git
cd millivolt
test -e proxy.yaml || cp proxy.example.yaml proxy.yaml
export MILLIVOLT_OPERATOR_TOKEN='replace-with-a-long-random-value'
go run ./cmd/proxy -config proxy.yaml -listen 127.0.0.1:8080
```
For isolated development and checks, see [Contributing](../CONTRIBUTING.md).
## Environment and defaults
The Compose inputs are environment variables; their defaults live in
[compose.yaml](../compose.yaml), not server configuration. Supply them
consistently through the environment or an ignored local `.env` file.
| Variable | Meaning |
| --- | --- |
| `MILLIVOLT_OPERATOR_TOKEN` | The single operator credential. It is read once at boot from the process environment and never from `proxy.yaml`, so rotating it requires a real process restart. A nonempty value shorter than 16 or longer than 512 bytes fails the boot. Unset or empty leaves the dashboard denied. |
| `MILLIVOLT_IMAGE` | Image reference; defaults to `ghcr.io/llm-4-people/millivolt:main`. |
| `MILLIVOLT_PORT` | Loopback host port; default `8080`. |
| `MILLIVOLT_STOP_GRACE_PERIOD` | Container stop grace; default `30s`. |
| `MILLIVOLT_MEM_LIMIT` | Container memory ceiling; default `1g`. |
| `MILLIVOLT_PIDS_LIMIT` | Process and thread ceiling; default `512`. |
`docker compose config` shows the resolved deployment. A source deployment
configures the same service through `-config PATH` (default local `proxy.yaml`),
`-listen ADDRESS` and `-db-path PATH`; the generated example and the Settings
schema are the field reference. See [configuration](configuration.md).
## Send a first request
Point a client at the proxy and supply the upstream base URL and key on each
request. The proxy registers no providers or keys. `X-Proxy-Format` selects a
different upstream protocol; absent, the request passes through. See
[protocol](protocol.md) and [adapters](adapters.md).
OpenAI Chat Completions:
```sh
curl -sS http://127.0.0.1:8080/v1/chat/completions \
-H "X-Proxy-Base-URL: $UPSTREAM_BASE_URL" \
-H "Authorization: Bearer $UPSTREAM_API_KEY" \
-H 'Content-Type: application/json' \
-d "{\"model\":\"$UPSTREAM_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"hello\"}]}"
```
OpenAI Responses:
```sh
curl -sS http://127.0.0.1:8080/v1/responses \
-H "X-Proxy-Base-URL: $UPSTREAM_BASE_URL" \
-H "Authorization: Bearer $UPSTREAM_API_KEY" \
-H 'Content-Type: application/json' \
-d "{\"model\":\"$UPSTREAM_MODEL\",\"input\":\"hello\"}"
```
Anthropic Messages, using the native upstream's own auth header, prefix and API
version:
```sh
curl -sS http://127.0.0.1:8080/v1/messages \
-H "X-Proxy-Base-URL: $UPSTREAM_BASE_URL" \
-H "X-Proxy-Key: $UPSTREAM_API_KEY" \
-H 'X-Proxy-Auth-Header: x-api-key' \
-H 'X-Proxy-Auth-Prefix;' \
-H "anthropic-version: $ANTHROPIC_VERSION" \
-H 'Content-Type: application/json' \
-d "{\"model\":\"$UPSTREAM_MODEL\",\"max_tokens\":1024,\"messages\":[{\"role\":\"user\",\"content\":\"hello\"}]}"
```
The semicolon in `-H 'X-Proxy-Auth-Prefix;'` is important: curl sends a present
empty header. `-H 'X-Proxy-Auth-Prefix:'` instead omits it, leaving the proxy's
`Bearer ` default. A response means the proxy answered; check the status code
for the upstream outcome. The request appears in the dashboard.
On a shared Compose network, use `http://millivolt:8080/v1` instead of loopback.
`/v1/chat/completions`, `/v1/responses` and `/v1/messages` are all available.
Each can reach Chat, Responses or Anthropic upstreams; only a Chat client can
target the Cursor bridge. Read
[upstream compatibility](adapters.md#client-and-upstream-compatibility) and
[cross-protocol translation](adapters.md#cross-protocol-translation).
## The dashboard
The dashboard is the site root, [http://127.0.0.1:8080/](http://127.0.0.1:8080/).
It asks for the operator token once per browser tab, keeps it in sessionStorage,
attaches it to gated calls only and drops it as soon as the server rejects it.
The same credential gates `/metrics/*`, `/admin/*` and `/mcp`; transparent
inference does not use it. See [operator access](routes.md#operator-access) for
the canonical open-path list.
## What to read next
[Documentation](README.md) routes by task. For the next step, read
[configuration](configuration.md) before changing settings, or
[the dashboard tour](dashboard.md) to find the request you just sent.
---
# Dashboard guide
The plate below is the complete dashboard. Use this guide to follow its
charts, explorer, request drawer, operator console and Settings without
hunting through the interface.
These are real application captures from explicitly authorized local history.
Recorded timing, token and cost values are retained; identifiers are replaced,
and request content, retained headers, debug captures and operator state are
removed. Settings and the theme tray show the public example, not private
configuration.
No requests, holds, limits or debug sessions were created to make the
pictures. The capture session blocks external requests and disables live SSE,
so its **offline** indicator describes a static screenshot session, not
service health. The images are not latency guarantees, capacity measurements
or independent bills.
Every capture in this guide and in the [capture gallery](gallery.md) is one
state photographed on both primary surfaces and split along the diagonal. The
upper-left half is Instrument Black, the lower-right half is Instrument Clear.
Data, scroll and any open surface are identical across the cut.

## Error storm banner
When enabled in Settings, error storm protection adds a live banner below the
header. Click an incident to inspect the number of requests with selected
errors, failed upstream attempts, error category, queued requests, next retry
time and recovery progress.
It names the affected provider or exact provider/model, shows its safe error
category, failed upstream-attempt percentage and window, queue count and
recovery probe state. Provider incidents require all models active in the same
window to have elevated errors.
The banner shows that active-model evidence; its attempt percentage differs
from the Errors chart's affected-request rate. It clears when protection
recovers and can be hidden independently in Settings. See [configuration and
behavior](routes.md#error-storm-protection).
The static captures below show no banner because the capture session has no
active incident; the banner stays hidden until protection fires.
## Timelines
The KPI band remains global. Explorer filters scope the timeline and request
log together. The timeframe control in the card head changes chart history,
not the global KPI band. It offers a zoom ladder from one minute through one
year plus **All time**, and the calendar presets today, yesterday, this week
and this month.
Its panel draws the stored coverage as a rail and disables any window wider
than the stored history, so a selection can never promise more history than
exists; **All time** means exactly the retained history, and a calendar preset
clamps its start to the oldest stored request. The selection persists per
browser.
Legend controls show or hide individual series; hover a bucket for its precise
values and time interval. Counts, sums and percentiles come from the server. A
window with fewer than four plotted points paints a waiting message instead of
the chart: one to three buckets are too few to read a trend, and the totals
strip still tells the story until a fourth point arrives. A span with no
stored history says so; a span inside history with no requests says that
instead.

A failed chart request never blanks a good payload: one transient error should
not hide the history you were reading. The context row states which of the two
situations applies. With nothing drawn it reads
`chart unavailable - retrying`; with a last-good payload still on screen it
reads the window and `stale, retrying`, so old buckets are never presented as
a current reading. Either way the next successful request clears it.
The **In-flight** KPI tile separates sending work from parked work. Its
summary names two totals that fit the narrowest tile - active and queued - and
the composition rail under the value splits every live state by color:
sending, pending, operator-paused and throttled. Hover the rail, or focus it
with assistive technology, for the exact per-state counts.
Queued covers both the admission queue (per-key concurrency, pacing, provider
caps, operator holds) and the retry send gate, so a provider's 429 pacing
window shows its parked requests as queued instead of as idle streams. The
request table's `throttled` pill marks the same state per row.
### Overview
**Overview** is the summary: it carries no plot at all. The tiles are the
whole surface and they take the card. Every headline metric reads its period
value first, then its measured companion fact, then its evolution sparklines:
- requests / tokens: the request count over the blended token volume.
- tokens in/out/cached: one in / out / cached triple, with the input share of
all tokens and the cache hit share of input underneath. Each share line
names what it compares.
- spend with the server's blended per-Mtok price (the same blended-price rule
as the KPI band, scoped to the viewed period and cost-reporting requests
only).
- the health pair (errors in the error red / rate-limited requests in the 429
tone, over the request denominator).
- the merged timing tile: average latency / average speed over every measured
request in the period (server-computed from the same samples the percentiles
rank, never an average of bucket percentiles), each metric's low-high sample
range underneath.
Each tile carries one sparkline; combined tiles draw one self-scaled line per
metric half in exactly the half's color. Sparks span only traffic-bearing
buckets - the same empty-bucket removal the plots apply - so window padding
never squeezes the data into a corner.
The metrics picker in the card head chooses what shows: one menu of checkable
rows, checked metrics render and unchecked ones leave the band (it reflows),
and the selection persists per browser like every chart view.
Every selected tile always renders in both layouts: the band wraps into as
many whole rows of 170px tracks as the card needs, and the page scrolls past
them in the base layout; the picker's count reports the selection itself, so
it can never disagree with the band. The wide layout is the 1200x750
one-screen lock: the page locks to the viewport with no scrolling, the
explorer compresses to one gallery row, and the timeline column floors at
572px so the 5-metric selection wraps into at most two rows.
Every tile keeps one fixed size whatever it carries - spark or not, one share
line or two, and the no-data skeleton swaps in without moving the card - and
the band sits at its natural height at the top of the card. Only the timeframe
control applies.

### Traffic
**Traffic** compares request and error counts. Bars represent bucket totals,
not smoothed rates. Empty intervals may be omitted in this preset; the chart
explains that below its legend.

### Errors
**Errors** compares the count of affected requests with an error-rate line on
a separate percentage axis. A request that only encountered HTTP 429 is not an
error. A recovered failure can still contribute an error signal.

### Token usage
**Tokens** compares input, output, reasoning and cached usage. The upstream's
reported fields determine which splits are available. The compressed vertical
scale keeps large prompt/cache volumes and small outputs readable together.
A cache-hit line on a separate percentage axis tracks cached prompt tokens as
a share of input per bucket. The totals strip adds each side's share of the
blended total, the cache share of input, and the in:out tile: the raw pair
(one line, halves in the token bars' colors) with the input share riding
underneath - a percentage reads the same in every locale, where a small
comma-decimal ratio looked like a count.

### Speed and latency
**Speed + latency** compares generation speed (first token to last token) with
time to first token on separate axes. TTFT is measured from each request's
upstream attempt send, so a retry has its own TTFT. One percentile dropdown
controls both series; the legend does not repeat it. Choose p50, p95 or p99.
Each statistic needs at least four valid samples; smaller sets and missing
measurements remain unavailable. A bucket plots only when both measurements
exist at the selected percentile, and those unmeasured buckets compact out of
the timeline instead of stranding gap points in dead space - the context row
below the plot says when that happened.
Period readouts use the period's samples, not an average of bucket
percentiles.

### Provider-reported cost
**Cost** shows reported spend alongside request counts. Values below $1 use
cents, including tooltips and totals. Missing provider cost does not establish
a zero bill, and there is no hardcoded per-model price table.

## Explorer
Changing the dimension in the rail changes the grouping, not the active
filters. Selecting a card adds or replaces that dimension's exact filter;
selecting it again removes it. Filters from other dimensions remain active.
The breadcrumb chips remove individual selections or return to all traffic.
The gallery ignores its own dimension's filter so alternatives remain visible;
the timeline and request log still use the complete selected scope.
Multi-valued dimensions, such as tools and errors, can overlap rather than
partition requests.
| Dimension, in rail order | What it groups |
| --- | --- |
| Providers | Server-derived upstream identities after configured aliases. |
| Models | Model groups produced by the configured canonicalization rules. |
| Clients | Explicit client labels or inferred SDK/User-Agent identities, retention-bounded before the ring and store. |
| Conversations | Conversation IDs from explicit sessions, tracked body params (`k:` identities) or automatic grouping, with declared ancestry when resolvable. Selecting a conversation selects that leaf, not its descendants. |
| Tools | Observed tool names; one request can belong to several tools. |
| Time | Request start in the server's local timezone: weekday night 00:00-08:00, work 08:00-16:00, evening 16:00-24:00, or a single weekend group. This is separate from chart history range. |
| Status | Coarse outcome classes and live states. The Requests dropdown instead selects exact HTTP codes or named live states, grouped into live states and HTTP status; it stays a native select so the operating system picker is the touch affordance. |
| Errors | Distinct observed error identities, including absorbed failures; one request can have several. HTTP 429 alone is not an error. |
| Keys | SHA-256 hashes of upstream credentials, never raw keys. Labels show only an eight-hex-character prefix; identical short labels do not establish identical full hashes. |
### Providers
Provider cards compare request share, reported cost, tokens, cache use, timing
and health. Error and HTTP 429 badges count distinct affected requests and
omit zero values independently. A request can contribute to both badges.

### Models
Model cards group configured model variants for comparison. The shipped rules
fold provider decorations (vendor namespaces, tags, quantization, service
tiers, release channels and build stamps) into the base model, so one model
served by several providers is one card. Model tiers, fine-tunes and parameter
counts stay distinct.
Selecting a card scopes the timeline and request log together. Model grouping
is configuration-owned; stored model spellings remain available for exact
export and deletion filters. Adapters can normalize those spellings before
storage; see [request metadata](routes.md#request-rows-and-details).


The [capture gallery](gallery.md#nine-ways-into-the-traffic) carries all nine
dimensions. This guide shows the two that move the most.
Conversation ancestry is explicit, not guessed from timing or names. The
authorized snapshot has no parent declarations, so these screenshots do not
fabricate an agent tree. See the [client protocol](protocol.md) for
relationship headers and the limits of missing, conflicting or ambiguous
declarations.
## Request inspection
### Request table
The request table combines recent finalized records with current in-flight
state during ordinary live use. Rows show status, retry information, latency,
throughput, token/cache usage, tool calls, duration, queue time and reported
cost.
Columns size to their content and keep a per-column floor: a value wider than
every value seen before grows its column, and a later shorter value never
shrinks it back. A live row finalizing from `streaming` to its recorded status
therefore cannot pull the status column back; a final value wider than its
in-flight placeholder can still grow its own column once. When the card is
narrower than the learned columns, the table scrolls inside its own region.
The region is keyboard-scrollable and labelled, the identity cell carries the
full `client · provider · model` value in a hover title, and compact status
pills and the actions button abbreviate by design. Its status selector and
older-history paging operate on the current scope.


### Request details
Click a row or its detail button to open the inspector. The header keeps the
request identity and the previous/next controls. The summary below it shows
the live or final status, method and path, client, provider and model, plus
duration, TTFT, tokens, speed, cost and retries when captured. The full field
inventory is classified into four sticky groups, each with the number of cards
it contains:
- Request: identity, time bucket, request and response sizes, client/runtime,
provider metadata, and captured model and parameters.
- Content: conversation and parent ids, turn and size composition, prompt and
response previews.
- Usage: input, output, reasoning, answer, cache and generated tokens; timing,
throughput, cost, transport decomposition, and tools.
- Diagnostics: queue and rate limits, absorbed retries, in-band upstream
responses, response headers, and the Debug capture when one exists.
Only the selected group is shown; its cards behave like the previous single
page. Each card header can collapse or expand its section, and both the active
group and collapse choices survive live updates until the inspector closes.
Copy actions sit on the values that matter (identifiers, conversation ids,
paths, models, headers, error detail and captured bodies), on field-list
cards, and on the header's JSON action for the whole record or the selected
attempt. One polite status line confirms each copy or reports a denied
clipboard write.
Long values wrap. Captured prompt, response and Debug bodies open in a bounded
code surface with a line count, wrap toggle, copy action and expand/collapse
control; a long single line scrolls inside that surface instead of widening
the inspector. A final upstream failure appears as a bordered alert above the
cards with its type, code, message and copy action, plus the HTTP status when
the failure was an HTTP response.
A rate limit the provider announced in its own type or code counts in the
rate-limit card even when the status was not 429; a request that ended on one
still appears as an alert, and a bare 429 stays the flow-control handshake it
is. On narrow viewports the inspector fills the width, stacks each key above
its value, keeps touch-sized copy and close targets, and respects the bottom
safe area.
Previous/next controls follow the scoped table order. Explicit zero and false
parameter values are retained instead of treated as missing. While a request
is in flight, pause and throttle state appear as explicit rows.

### Tokens and performance
Open Usage for token splits, cache hit, generated-token composition,
timestamps, the adopted attempt's send anchor, throughput, reported cost,
finish state and transport timing.
Absorbed retry attempts appear under Diagnostics when the selected request has
them, each with its status, provider error detail, the attempt's own provider
request id, and captured upstream server, model, processing and rate-limit
facts. An in-band upstream response that arrived after a committed success is
disclosed separately rather than presented as a retry. Clicking an absorbed
attempt, a retry sub-row in the request table or an entry in the drawer's
retry list opens that attempt's own detail view.

These captures deliberately omit request content and Debug payloads; ordinary
opt-in captures remain sensitive.

## Operator console
The header's operator readout is the one control surface. It stays in the
header rail at every width, including the phone header; the theme tray and
Settings sit behind the actions flyout there.
It reads `steady` when no hold or capture is active, no limit policy is
enforcing, no request is parked by the storm/quota gate and no restart is
running, then one segment per active hold (`paused 1`), capture session
(`captures 1`), enforcing limit policy (`limits 2`) and gated request
(`gated 3`).
A stored policy whose schedule window is closed enforces nothing, so it is
named separately as `off schedule N` instead of being counted as a cap; the
server owns that verdict and reports it per policy, so the header cannot
re-derive a different answer from the stored minutes. The wording names the
behaviour, because `scheduled N` read as "the cap is on a schedule and
working" when it meant the opposite.
While a rebuild runs, `restarting` comes first, with ` · step N` when the
server reports a real step. The counts are live state: a hold that expires, a
stopped capture or a cleared limit leaves on the next state repaint. A hold
takes the warning tone because it is the one state that stops live traffic; a
restart takes it too, with a pulsing dot.
If the live stream was open and then drops, the line keeps its last known
words and appends ` · offline`; if the operator poll instead stops answering,
it appends ` · stale`. Both mark the words as last known rather than current.
The header readout is not a live region; it updates when operator state,
connectivity or restart status changes, and the trigger's accessible label
carries the same words plus the action the trigger performs in its current
state. The open console's head line repeats those words without a second total
and is a polite live region.
Activating the readout opens the console. On a wide viewport it is an anchored
popover under the trigger: 520px wide, and only as tall as its content up to a
560px cap, scrolling inside it.
At 800px or narrower it becomes a bottom sheet with a definite height of
`min(88dvh, 100dvh - 16px)`, leaving a strip of page above it for
outside-press dismissal. The height is definite rather than a cap: the body
and the pane grow into the remainder, so a tall composer scrolls inside the
pane instead of being clipped by the panel.
On a landscape phone (800px or narrower and 500px or shorter) the header,
rail, pane and ledger tighten and the sheet gives up its top gap, sitting
flush with the top edge so only the 8px bottom dismissal strip remains; a very
short viewport turns every fixed band into space the form loses.
The section rail is one horizontally scrollable strip and fades its trailing
edge while more sections remain; at 560px or narrower, where six chips
genuinely cannot share one row, it becomes a two-row grid. That threshold is
the content, not the device class: a landscape phone is 640-844 wide and keeps
the single-row strip.
Escape, an outside press, the close button, or opening the theme tray or
Settings closes it; opening the console closes the theme tray and Settings
first. The session remembers the last section, so reopening returns where the
operator left. Opening the console itself performs no action.
The rail names six sections: overview, limits, pause, capture, export and
system. Limits, pause and capture carry a count badge: for limits the number
of rules enforcing right now, for pause the active holds, for capture the
running capture sessions. A rule outside its schedule window is installed but
enforcing nothing, so it does not raise the limits badge.
The overview is the rule register: one row per hold, capture and every
installed limit, each jumping to the section that owns it, with an
off-schedule limit marked `off schedule` rather than reading as a live cap,
plus the queue drain control when requests are actually parked.
A request parked by the storm/quota gate is named in its own block instead:
that gate runs before admission, so a drain cannot release it, and the block
points at the controls that can. The counts are not repeated there, because
the head line and the section badges already state them.
Limits, pause and capture hold rule rows and one composer; export holds the
download builder, and system holds the restart and Settings cards.
### Rule rows
Every rule is one calibration line: a kind tag in its state color, the human
scope, the caps or schedule, then a live column for occupancy and actions. The
scope sentence is generated from the selector the server enforces, so an empty
dimension reads inside the sentence instead of as a separate checklist.
Every rule kind offers the dimensions its own matcher honours: a limit and a
hold both scope on providers, models, clients, key hashes and conversations,
because the scheduler matches both against one request identity, while a
capture scopes on clients, providers and models.
- A limit row names its budgets and schedule plus any remaining request and
token budget; who set it rides the row title rather than a third line.
- A hold row names its remaining time, its queue cap or `queue unlimited`, and
its queued count.
- A capture row names its retained count, its run time, and its duration or
`until stopped`.
A row is the one edit target: activating it opens its composer in edit mode.
The action column offers copy, which starts a draft from the same scope and
schedule with no budgets, and the terminating action for that kind: remove,
resume or stop. A copied limit that still matches its source selector is
refused and named, because a limit's identity is its selector: applying it
would edit the original rather than add a second rule. The edit action is not
repeated there, because the row already performs it.
On a narrow phone the row takes two lines so the scope can be read in full
instead of being clipped; the budget wraps beneath the scope rather than being
dropped, because a limit's scope identifies it while its budget is what it
does.
A rule list that outgrows the pane offers a filter once it holds enough rules
to need one, so a console with two rules never spends a row on an empty box.
The filter matches the text each row already renders, which gives limits,
pause and capture one search rule and makes a new row field searchable without
a second list of what to match.
A filter that hides everything says how many rules are in force and offers to
clear itself, so an empty result never reads as an empty policy.
### Composer anatomy
Limits, pause and capture share one composer: a titled bench, one scope strip,
budget or duration controls, a live preview sentence, and a footer with cancel
and the one commit action. The composer opens above the rule list, so its form
and its commit are in view the moment the operator asks for one. A phone pins
that action row to the pane's bottom edge.
The scope is one wrapping strip of dimension chips rather than five stacked
rows. An unset dimension is a dim chip you click; a set one carries its values
inside the same chip. One picker opens at a time, below the whole strip, with
a filter only once the list is long enough to scan, the known values as
checkable rows, and a free-value field except for key hashes.
Provider, model, client and conversation scopes accept a typed value; the key
scope lists only SHA-256 hashes already present in the live record ring. The
preview states what the commit will enforce.
Limits add concurrency, request and token budgets with their windows, an
optional anti-burst ramp, and an optional day-of-week schedule with
server-local from and to times. Burst control and schedule are peers on one
row, each opening its own body below. A window is a native select: the list is
long and the track is one number wide, so the current choice is stated in one
line and the rest is the platform's own picker.
The ramp is offered only on a single-provider rule without a schedule. Equal
from and to times mean all day, and a window that starts after its end spans
midnight. Pause adds everything, new-clients and named clients/providers
modes, a duration or until-resumed choice, and an optional queue cap.
Its head also carries a one-click bounded hold (`hold all 15m`), which writes
the same all-traffic document through the same gate, ledger and undo path. It
is bounded by construction: an indefinite hold is still available, but only
through the composer where the operator chooses it.
Capture starts a scoped Debug session and adds client, provider and model
scopes, a duration or until-stopped choice, and the retention and byte facts
for the current configuration.
### Confirmation and the change ledger
Removing a limit, resuming every hold, stopping every capture, draining the
queue and restarting open the shared confirmation dialog. It states the
consequence, focuses cancel first, and names the action on its confirm button.
Individual stop, and a resume with nothing queued, commit directly; resuming a
hold that has parked requests opens the shared confirmation first, because it
releases that backlog to the provider. An edit commits when its composer is
applied. All are recorded.
The console footer is a bounded session ledger. It shows the newest change and
offers Undo while that change has a defined inverse. Undo applies it once and
appends an `undid: ...` receipt; the next change becomes the new undo target.
The ledger has one owner: the console footer. Up to eight changes are retained
for the page session.
Changes without an inverse, such as a download or a restart, are recorded
without an undo action. The ledger is browser memory, clears on reload and is
not an audit log. A rejected write is named in the open pane, not hidden
behind a closed panel.
### Export
The export section owns record exports. Choose all history or matching
records. Matching records offers provider, model, client, conversation and
status in the same one-strip selector the composer uses, then errors-only,
debug-only, newer-than and older-than, plus an exact error type field. The
receipt and its download stay in the same view as the filters that shape them.
Provider, model and client merge the server's known vocabularies with whatever
the live record ring holds, so a quiet proxy or a fresh restart still offers
them; all four accept a typed value as well, so a dimension the server has
never seen is not a dead end. Status is the one list-only dimension, because a
status code is not a name anyone can type reliably.
Both modes preview through the same server count: all history is the same
predicate with an empty filter. The receipt always names the number of records
the download will carry, the download stays disabled until a nonzero count is
ready, and the ledger repeats that number. A mode switch invalidates the count
rather than quoting the previous mode's total, and filter mode with nothing
selected asks for a filter instead of counting the unfiltered set.
Downloads are gzip JSON artifacts. Configuration and database backups stay in
Settings, and the console links there.
### Request detail
A request's detail drawer carries the two console prefills that record can
seed: `hold this scope` and `capture this scope`. Both close the drawer and
open a draft scoped to that request, so holding the client of a slow response
does not require reading the value, closing the drawer and retyping it into a
picker.
They are prefills, not commits: the composer still gates every write, so no
confirmation is added. The open is deferred one task, because the press that
triggered it would otherwise reach the header's outside-press closer and
dismiss the console it had just opened.
### System
The system section owns restart and the console's link to Settings. It
explains the rebuild and handoff, lists the four steps as they actually start
(`rebuild from source`, `finish in-flight streams`, `flush metrics to disk`,
`start the new process`), and disables restart while a run is active or the
build cannot restart in place.
A restart asks for confirmation and is recorded in the ledger without an undo
action. Queue bounds, retry budgets, storage, retention and dashboard cadence
are Settings fields, not console policies; the section links there instead of
duplicating them.
Static captures of the operator console publish to `docs/images/console/` from
the same authorized history workflow as the rest of the gallery. The retired
menu screenshots were removed with the menus they pictured. Every section is
recorded in its neutral starting state: no draft is committed, no rule is
selected and no filter is armed.







See [routes and debugging](routes.md) for exact admission, filtering and
failure behavior, [containers and data](containers.md) for restart, and
[operations](operations.md) for persistence, before using these controls.
## Theme

Theme has two surfaces and eight palettes. Each palette owns a complete
surface family (base, panels, rail and borders), its accent family, and its
shadow and veil hue, so a switch changes the shell rather than only a few
accent edges. Clear is the light family; Black is the near-black family.
The palette ids, labels, order and defaults live once in `js/theme_boot.js`.
`js/theme_state.js` owns normalization, the stored pair, the one apply path and
the browser chrome color; `theme.js` owns only the dashboard tray. The public
documentation site publishes `css/themes.css` and both scripts byte-for-byte
from the dashboard and renders the same eight palettes and two surfaces in its
own header tray. Each origin keeps its own choice under the `millivolt.theme`
key, so the site and dashboard do not share browser storage.
Instrument (the default), Slate, Midnight, Nord, Forest, Rose, Ember and
Contrast are available in both modes. Contrast is the maximum-separation
palette: black with signal yellow in Black, white with black in Clear.
The theme tray stays open while choosing and shows a two-pane comparator: the
committed pair on the left and the candidate on the right. Both panes use the
same token-faithful miniature with a three-color series legend; the captions
outside the decorative panes name the committed and candidate pairs. The
preview line stays muted while it reads "Preview: same" and takes primary ink
only for a real candidate.
Hovering or focusing a palette or surface updates only the preview pane and
captions. Clicking, Enter or Space commits immediately and persists the pair;
there is no apply step. Arrow keys, Home and End move focus and the preview
cursor without committing, and the focused row keeps the group's single tab
stop so Tab leaves and returns to it. The current surface's Clear/Black radios
sit under a visible Surface label in the caption bar.
Pointer activation commits without writing the polite candidate status; only
keyboard focus announces a preview. The trigger's arrow keys open the tray on
the saved palette; while open they move the preview cursor. Reset restores
Instrument Black; Close only dismisses the tray.
On touch or coarse pointers the comparator, preview line and hover hint are
hidden, leaving the current caption, the surface control, all eight palette
rows, Reset and Close.
A window 800px wide or narrower uses the same compact picker inside the header
actions flyout even with a fine pointer, and the flyout rather than the tray
or palette list is the one vertical scroller. At a viewport height of 760px or
below the comparator, preview line and hint are hidden.
In windows wider than 800px the tray scrolls so every action remains
reachable; narrower windows keep the header flyout as the single scroller. At
360px or below the caption bar stacks into one column with a full-width
surface control, and the palette hints are hidden while names and swatches
remain.
Reopening the tray or flyout brings the saved palette row into view instead of
restoring an arbitrary previous scroll offset. The choice is stored in this
browser only and is applied before the first paint, so a reload does not flash
the default. Charts follow the selected palette; status colors follow the
selected surface, and the browser's theme color follows the selected palette's
rail. The setting is presentation only: it does not change proxy behavior,
stored history or the configuration revision.
## Settings
Settings uses the same schema, validation and defaults as configuration files.
Search finds fields across categories. Changed values are revision-checked;
every row whose field is restart-bound carries its own `restart` marker
instead of only a category banner.
Closed vocabularies render as selects from the schema's `choices`, and
duration/byte values are parsed and checked against the schema's numeric band
before Apply. A read-only config disables its field controls, not only Apply.
The category rail right-aligns its row counts beside the label; on a phone it
becomes one horizontally scrollable chip strip so the field pane keeps the
height. Switching categories resets the field scroller, and the footer status
is a live region with a distinct error tone.
Structured fields replace their whole map/list value rather than recursively
merging nested entries. The generated [configuration
example](../proxy.example.yaml) documents every field, including categories
not pictured here. The [capture gallery](gallery.md#the-patch-panel) records
five categories, including the backup and restore sheet.
### Dashboard cadence
Dashboard settings control log-window size and polling/aggregate refresh
cadence. Live rows and the in-flight gauge remain distinct from periodic chart
and explorer aggregation.

### Queue and retry
Queue and retry settings bound admission and retry behavior. Their help text
distinguishes queue waiting, provider retry hints, retry budgets and capacity.
Tune them for the upstream and workload; the picture uses the public example.

### Storage
Storage settings control the SQLite writer and its bounded queue/batches.
Metrics persistence is best-effort under sustained overload; a successful
inference response does not prove its record was stored. The dashboard reports
process-local storage drops.
The category also carries both database deletion controls, which live here and
nowhere else: Delete matching removes only the records matching its provider,
model, client, status, errors, debug and age filters after a live count
preview and a confirmation, and Delete everything permanently deletes all
metrics history, the live ring and the entire durable database. Neither action
can be undone.

### Provider compatibility
The example enables Grok, OpenCode Zen and Cursor compatibility profiles
without credentials or upstream registrations. Their headers and metadata
mappings are editable operator-selected snapshots, not guaranteed current
provider contracts. See [adapters](adapters.md#bundled-compatibility-profiles)
before relying on them.

See [Contributing](../CONTRIBUTING.md#documentation-screenshots) before
refreshing the gallery. Never publish private configuration, bodies,
credentials or raw history copies.
---
# Capture gallery
37 plates across 7 stations, captured 16 November 2026 UTC from 100,158 recorded requests with identifiers replaced and content removed.
Every plate is one state photographed on both primary surfaces and split along the diagonal from the bottom-left corner to the top-right. The upper-left half is Instrument Black, the lower-right half is Instrument Clear. Nothing else changes between the halves: the same data, the same scroll, the same open surface.
Values are a static sample from recorded history, not a benchmark. The offline indicator describes the capture session.
## The whole bench
The default view at 1920 px and at phone width; every plate carries the Black and Clear surfaces across one diagonal.
01 · The whole bench Global totals stay fixed while the explorer scopes the timeline and log; upper-left Black, lower-right Clear.
02 · 390 pixels wide Below 1200 pixels the locked bench becomes a scrolling page and the nine-item rail becomes one picker.
## Six readings of one ruler
The plotted presets, the summary tiles, and the window picker that bounds them.
03 · Summary tiles The overview preset answers what the window contains before any series is drawn.
04 · Traffic Requests and errors share one time axis; the totals strip carries the window rate.
05 · Errors A 429 is a rate limit, not an error; the two series stay separate.
06 · Tokens Cached reads are drawn beside billed input, so a high cache share is visible rather than inferred.
07 · Speed and latency The percentile control applies to every measured rate in the preset.
08 · Provider-reported cost Spend is whatever the upstream reported, plotted beside the request volume that produced it.
09 · The window picker Windows wider than the stored history are disabled, so the picker cannot promise history that is not there.
## Nine ways into the traffic
The dimension rail, its cards, and one scoped reading.
10 · Providers The rail counts distinct values; each card shows its share and the signals that matter for that dimension.
11 · Models The model dimension applies the configured grouping rules before it counts.
12 · Clients The client dimension answers which caller produced a share of the window.
13 · Conversations Conversation filters select exact leaves; the rail reports main and sub conversations separately.
14 · Tools The tool dimension counts calls the relay observed, not calls a client claimed.
15 · Time Time is a grouping like any other, so a bucket can be filtered and compared with the rest of the window.
16 · Status The status dimension separates what the client received from what the upstream first returned.
17 · Error class The error dimension groups by the proxy's canonical class, not by provider wording.
18 · API keys Keys are hashed in the copy, so a group is stable without exposing a credential.
19 · A scoped reading A card toggles an exact filter; the chip removes it, and the timeline and log follow the same scope.
## Every crossing
The request log, an absorbed retry, and the drawer that explains a single crossing.
20 · The request log Each row is one crossing; the columns are the facts a later review needs first.
21 · An absorbed retry A retry the client never saw is still a recorded upstream attempt.
22 · The request drawer The drawer opens on the record, not on a reconstruction of it.
23 · Tokens and performance Usage is grouped by the question it answers, so timing and tokens do not compete in one list.
24 · Diagnostics The diagnostics group is where a transparent retry stops being invisible.
## The operator plane
The five policy sections and the composer behind them.
25 · Overview The overview is the register of what is in force, not a second settings form.
26 · Limits An empty policy surface states the state instead of hiding it.
27 · The limit composer A draft is local until it is committed; the composer previews the effect before a write.
28 · Pause A hold parks new work for a scope without changing a route.
29 · Capture A capture session retains bodies for a bounded scope and time; the panel states both.
30 · Export The receipt counts what the chosen mode will write before the file is produced.
31 · System A restart is a state machine with named steps, not a button with a spinner.
## The patch panel
Five categories of the public example configuration.
32 · Dashboard Every field is generated from the schema, including its help and the restart badge.
33 · Queue and retry Retry policy, error classes and pacing sit together because they act on the same wait.
34 · Storage Deletion is scoped and previewed; the storage facts come from the running store.
35 · Providers Provider data is a field map in configuration, not a branch in the router.
36 · Backup and restore A restore is inspected before it is applied; the sheet says what will be replaced.
## One bench, two surfaces
The shared theme tray on Black and on Clear.
37 · The theme tray Palette, surface and order come from the same registry the site publishes; the seam is the surface switch made visible.
---
# Protocol and client integration
The client-facing interface accepts OpenAI Chat Completions, OpenAI Responses
and Anthropic Messages; a Chat request can also target the Cursor Connect
bridge. The upstream-facing protocol can differ from any of
them. Per-request headers select the destination, credentials and optional
native adapter.
The [compatibility matrix](adapters.md#client-and-upstream-compatibility)
maps the supported combinations; [cross-protocol translation](adapters.md#cross-protocol-translation)
documents the canonical model and its loss policy. No endpoint/API-key registry
is required. Read
[Security](../SECURITY.md): routing credentials do not authorize operator access,
and the whole dashboard plane is gated by `MILLIVOLT_OPERATOR_TOKEN`
([operator access](routes.md#operator-access)); only `/healthz` and
`/readyz`, `/favicon.ico` (and the other origin-root brand/PWA files), the
`/session` login handshake and the relay stay open.
## Endpoint behavior
The dashboard, embedded assets, `/admin/*` actions, `/healthz`, `/readyz`, the
`/session` login handshake, the `/mcp` MCP endpoint and registered `/metrics/*`
routes belong to millivolt. Their methods and meanings are listed in
[routes and debugging](routes.md#operator-and-data-routes). `GET /v1/models` and
`GET /models` use the discovery path described below. `POST /v1/chat/completions`,
`POST /v1/responses` and `POST /v1/messages` select the client wire protocol.
Other paths reach the relay catch-all and require valid routing headers; the
client chooses the upstream endpoint and model.
A known client path is translated only when `X-Proxy-Format` selects a
different upstream protocol. An absent `X-Proxy-Format` selects passthrough, as
does an explicit format equal to the client protocol. Without a configured
request rewrite, a passthrough request's body bytes are relayed unchanged;
configured request overrides, a sub-conversation strip or ensured tools can
still rewrite the upstream body (see
[request overrides](configuration.md#request-overrides)).
`X-Proxy-In-Format` overrides the protocol named by the path, so an otherwise unknown path
is translated when that header names the inbound wire; without the override,
paths outside the three known endpoints are never translated.
A translated request addresses the outbound protocol's canonical
endpoint (`/v1/chat/completions`, `/v1/responses` or `/v1/messages`) instead of
forwarding the client's path, unless `X-Proxy-Path` overrides that endpoint. A same-protocol request on one of those three
endpoints normalizes to the same canonical spelling, while one on any other path
keeps the path the client chose.
The path is matched in its escaped spelling, so
an encoded spelling of a known route (for example `/v1%2Fchat%2Fcompletions` or
`/v1%2Fmodels`) is an unknown path: on a same-protocol passthrough, or when no
inbound override is present, it is relayed unchanged and is never normalized,
translated or treated as model discovery.
When `X-Proxy-In-Format` selects a
wire that differs from `X-Proxy-Format`, the request is a translation like any
other and addresses the outbound protocol's canonical endpoint. A base URL with
a path joins by comparing the first client segment's decoded spelling with the
base's last segment, so an encoded client spelling of that segment can be
dropped from the joined upstream path.
Forwarding a body is not the same as understanding every field in it. The normal
relay does not rewrite an unfamiliar JSON shape just to observe it, but usage,
first-token timing and quality classification depend on recognized formats.
Native translation is explicit and narrower than arbitrary HTTP forwarding.
There is no automatic provider failover or model-selection registry.
## Routing headers
Only `X-Proxy-Base-URL` is required by the proxy. The upstream may require a key.
Routing validation and request construction live in
[request_target.go](../internal/proxy/request_target.go) and
[request_upstream.go](../internal/proxy/request_upstream.go).
| Request header | Meaning |
| --- | --- |
| `X-Proxy-Base-URL` | Absolute HTTP(S) upstream base; no userinfo, query or fragment, including a bare trailing `?` or `#` (a percent-encoded `%23` remains path data). |
| `X-Proxy-Key` | Upstream key; takes precedence over `Authorization`. Without it, a Bearer credential or the supplied Authorization value is used. No credential is also allowed. |
| `X-Proxy-Auth-Header` | Key's upstream header; defaults to `Authorization`. The chosen name joins the credential class for that request even when its own spelling looks harmless (`X-My-Gate`): it is redacted from retained headers and skipped by request overrides like a real credential. |
| `X-Proxy-Auth-Prefix` | Prefix before the key; defaults to `Bearer `. A present empty header selects no prefix. |
| `X-Proxy-Path` | HTTP relay: replace the incoming path before joining it to the base URL. Cursor uses its [native path rule](adapters.md#cursor-connect-bridge); model discovery replaces its default models endpoint. |
| `X-Proxy-Query` | HTTP relay: nonempty query override; otherwise preserve the incoming query. Model discovery uses it for the upstream discovery call; Cursor ignores it. |
| `X-Proxy-Headers` | HTTP-relay header overrides as a JSON object of string arrays, e.g. `{"x-example-version":["2026-01"]}`. `accept-encoding` and `range` are proxy-owned and rejected with 400. A JSON `null` member is treated as an omitted header. Values are JSON-decoded, so a raw obs-text byte (0x80..0xFF) that is not valid UTF-8 reaches the wire as U+FFFD instead of its original byte; send the UTF-8 spelling if the byte matters. Not applied to model discovery or the Cursor bridge. |
| `X-Proxy-Timeout-Ms` | Positive milliseconds. HTTP relay: a fresh headers-and-body deadline per attempt, excluding queue/hold/backoff waits. Cursor applies it only to send header waits; model discovery uses its separate configured budget. |
| `X-Proxy-Format` | Upstream format: `openai` (Chat Completions), `openai-responses` (Responses), `anthropic` (Messages), or `cursor` (agent.v1). Absent means passthrough; configured request rewrites can still change the body. See [supported combinations](adapters.md#client-and-upstream-compatibility). |
| `X-Proxy-In-Format` | Optional client-protocol override: `openai`, `openai-chat` or `chat` (Chat Completions), `openai-responses` or `responses` (Responses), or `anthropic` (Messages). Aliases normalize to the canonical wire. Normally unnecessary because the request path selects the protocol. |
| `X-Proxy-Client` | Observability/client label; one that only repeats the user agent's product name records the fuller user-agent identity instead. Otherwise inferred from client SDK/User-Agent metadata. |
| `X-Proxy-Session` | Explicit conversation ID; otherwise a configured tracked body param or automatic grouping applies. |
| `X-Proxy-Parent-Session` | Direct declared parent; requires an explicit own session on the same request. |
| `X-Proxy-Max-Concurrency` | Positive provider-plus-key scheduler-group cap. Each admission updates the shared group's policy; later admissions, including headerless ones using config defaults, can replace it. Not the persistent operator limit policy, and not applied to model discovery, which returns before inference admission. |
| `X-Proxy-Limit-Concurrency` | Sticky provider-wide in-flight cap. |
| `X-Proxy-Limit-Requests` | Sticky provider-wide request rate as `COUNT/WINDOW`. |
| `X-Proxy-Limit-Tokens` | Sticky provider-wide token rate as `COUNT/WINDOW`. |
| `X-Proxy-Refresh-Token` | Opt-in stateless expired-token refresh for supported mechanisms. See [refresh](adapters.md#token-refresh). |
Every `X-Proxy-*` routing, limit and format header is read with first-value
semantics: when a request carries the header more than once, the first value is
authoritative and later values are ignored. The rule is deterministic across
the whole control prefix, including `X-Proxy-Query` and `X-Proxy-Headers`.
### Upstream allowlist matching
`allowed_base_urls` is empty (allow any target) or a list of entry spellings;
each entry admits exactly one authority plus that entry's path and its children.
Both sides are compared as written, with one normalization: the scheme's
default port is filled in when either side has no port. Entries are validated
at load into the only spellings that can ever match, and those spellings are
the contract:
- the scheme must be literally lowercase `http://` or `https://`;
- the host must be lowercase ASCII as written, with an optional numeric port in
1..65535 after a hostname;
- no userinfo, query, fragment or trailing slash;
- a port-less entry admits only the scheme's default port, never every port on
that host.
What the byte comparison does not do: lowercase the request's own host, strip a
trailing dot (`example.com.` and `example.com` differ), normalize a leading-zero
port (`:080` differs from `:80`), or apply IDNA folding (the punycode or Unicode
bytes must be spelled the same on both sides). A path prefix is compared on the
escaped path, so percent-encoding that decodes the same is still different
bytes.
The joined upstream path is not cleaned either. `.` and `..` segments from
`X-Proxy-Path` or the incoming path travel verbatim, and an upstream that
normalizes request paths resolves them after the proxy has already allowed the
base. `X-Proxy-Path: /../../admin` under an entry for `http://host/v1` is sent
as `/v1/../../admin` and reaches `/admin` at such an upstream, outside the
allowlisted path prefix. The authority stays allowlisted either way; only the
path prefix can be left.
Client identity labels are retention-bounded before the record enters the live
ring, analytics or durable store: classified client and language 256 bytes,
user agent 1024 bytes, and `client_ip` 128 bytes. Longer values
are cut on a UTF-8 boundary and end with `…`; normal labels are unchanged.
`client_ip` records the first `X-Forwarded-For` element when that header is
present and the socket peer otherwise; the value is diagnostic, never verified,
and a directly reachable caller can set it.
Finalized and pending record text is retention-bounded at the record boundary
before it enters the live ring, pending view or durable store:
- path 1024 bytes, method 32 bytes;
- provider label, model, provider model, provider request id,
provider server, finish reason, error type, error code, error message, request
tool choice, response format, service tier and last turn role 512 bytes each;
- each tool name 128 bytes with at most 64 retained in order;
- debug session id 128 bytes;
- conversation identifiers 512 bytes before the namespace prefix
(retained identifier 514 bytes).
Absorbed-attempt provider model, provider
request id, provider server, error type, error code and error message use the
same 512-byte bound; provider labels and attempt error messages are bounded at
the same record boundary. Longer values are cut on a UTF-8 boundary and end
with `…`; normal values are unchanged. These record bounds do not alter debug
capture bodies, retained headers, content previews or request-parameter JSON.
One caller setting `X-Proxy-Limit-*` affects every key/client of that provider.
Absent headers preserve limits. A present `X-Proxy-Max-Concurrency` or
`X-Proxy-Timeout-Ms` value follows Go's HTTP field parsing: leading and
trailing optional whitespace is removed before the handler reads it, so a
whitespace-only value arrives empty and is treated as absent, while a padded
numeric value is read as its trimmed spelling. `0`, `off`, `none` or
`unlimited` clear a `X-Proxy-Limit-*` policy.
Windows accept Go durations from 1 second through 24 hours, plus `1d`.
Anti-burst and its ramp duration are operator API policy (`/admin/throttle`, the
dashboard and MCP), not header-driven; a header update changes only the three
caps and preserves the stored pair. The ramp is inert unless `anti_burst` is
true, the concurrency target is at least 2 and `ramp_duration` is above 0, and
its duration uses that same window grammar.
Malformed limit values reject the
request without applying any of its limit headers. A header update edits only
that provider's single-provider, unscheduled rule; operator rules scoped by
model, client, key hash, conversation or schedule are untouched. The operator
console shows every policy, its schedule and its source.
Request rates are strict sliding windows over the
configured length, so no window ever admits more than the configured count;
token rates remain continuously refilled budgets. See
[Pause and Limits](routes.md#pause-and-limits) for charging and window
behavior.
Supported proxy-control headers never forward upstream. Injected names must be
valid, unique case-insensitively, and not reserved control, framing
(`content-length`), hop-by-hop or Host names; values must be valid HTTP field
values. Ordinary hop-by-hop and Connection-nominated headers are removed.
Precedence depends on the request path:
| Path | Header precedence, lowest to highest |
| --- | --- |
| HTTP relay, including Anthropic translation | Forwarded values → extracted authentication → configured provider headers → explicit injection → matching request overrides. Configured/injected authentication can override the extracted key. |
| HTTP model discovery | Constructed authentication → configured provider headers. Explicit injection and request overrides are not applied. |
| Cursor run/model discovery | Configured headers → matching request overrides (runs only) → required protocol/framing headers and extracted authentication when a key is present. Explicit injection is not applied. |
The provider label is server-derived from the base host's registrable domain;
IPs retain their authority/port, and nonregistrable hostnames remain hostnames.
Optional provider aliases apply afterward. There is no supported
`X-Proxy-Provider` override.
`Host` is the one name with a different mechanism rather than a different
precedence: it is accepted in a provider `headers` map and written to the
request's host field, not its header map, because `net/http` sends the former
and ignores the latter. A configured host, port included, is the authority the
upstream sees on both paths above. The explicit `X-Proxy-Headers` map and an
`X-Proxy-Auth-Header` still refuse `Host`, and no request override may set it.
## URL and auth examples
For the HTTP relay, a base ending in `/v1` and request path
`/v1/chat/completions` do not produce
`/v1/v1/chat/completions`: the join removes the shared leading segment.
A path override still joins to the selected base. Use `X-Proxy-Query` rather
than embedding a query in the base URL.
For a native Anthropic-compatible endpoint, use its documented base/model/key
and set translation plus the appropriate upstream auth/path/version headers:
```sh
curl http://127.0.0.1:8080/v1/chat/completions \
-H "X-Proxy-Base-URL: $NATIVE_BASE_URL" \
-H "X-Proxy-Key: $NATIVE_API_KEY" \
-H 'X-Proxy-Auth-Header: x-api-key' \
-H 'X-Proxy-Auth-Prefix;' \
-H 'X-Proxy-Path: /messages' \
-H 'X-Proxy-Format: anthropic' \
-H "X-Proxy-Headers: {\"anthropic-version\":[\"$NATIVE_API_VERSION\"]}" \
-H 'Content-Type: application/json' \
-d "{\"model\":\"$NATIVE_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"hello\"}]}"
```
The example assumes JSON-safe model/version values. The semicolon in
`-H 'X-Proxy-Auth-Prefix;'` is important: curl sends a present empty header.
`-H 'X-Proxy-Auth-Prefix:'` instead omits it, leaving the proxy's Bearer default.
See the [official curl header guide](https://everything.curl.dev/http/modify/headers.html).
An unauthenticated local upstream needs no dummy API key.
## Relay, retries and model discovery
Ordinary requests relay the original body bytes unless a configured request
override, sub-conversation strip or ensured-tools rule rewrites them; the
request is buffered up to `max_request_bytes` before metadata extraction and
sending. Larger requests are rejected with HTTP 413 without an upstream send.
A decoded output cap
above 1,000,000 tokens is rejected with HTTP 400 the same way, before any
upstream send and at both trust boundaries - the original body's decode and
the re-decode of a translated body - for any spelling (`max_tokens`,
`max_completion_tokens` or `max_output_tokens`).
A body that combines the
Responses `max_output_tokens` spelling with a Chat `max_tokens` or
`max_completion_tokens` spelling is rejected with HTTP 400 before an upstream
send because the wire families are ambiguous. More than one top-level member
that folds to the same cap name is also rejected with HTTP 400 before an
upstream send, including case variants and escaped spellings: the relay sends
every occurrence, so accepting duplicates would let the validated and recorded
value differ from the bytes the upstream reads.
A valid request is also
rejected with HTTP 400 before an upstream send when it carries more than 10,000
entries in a `messages`, `tools`, `functions`, `input`, `stop`,
`stop_sequences` or `modalities` array, more than 10,000 members in a
`metadata` or `stream_options` object, more than `request_tool_calls_max`
aggregate `tool_calls` members across every message (default 1,024), more than
10,000 distinct top-level
members, or more than 10,000 distinct `logit_bias` keys merged across duplicate
top-level objects. Those preflight bounds apply to native and translated
requests alike.
A translated request also passes through its protocol codec,
which rejects with HTTP 400 more than 10,000 entries in a Responses `include`
array or in a tool `allowed_callers` array, or more than 4,096 blocks in an
Anthropic `system` list.
A native passthrough without a configured body
rewrite does not apply those codec-only collection bounds: the body is relayed
unchanged, so there is no materialization to bound. A configured
`request_overrides` rule, a sub-conversation strip or an ensured tool can still
rewrite that body, but those rewrites are not a codec decode.
Content composition has two independent limits. A content array above 4,096
parts is not rejected for that reason on the byte-transparent passthrough
path: within the aggregate budget it still relays unchanged, but composition
skips it, so it contributes no dashboard composition counts. On a translated
request the codec applies its own limit: Chat drops the oversized content
with a warning, while Responses and Anthropic reject the request with HTTP
400; `translation_loss: reject` turns any recorded loss into a 400.
Composition reads the Chat `messages`/`system`
and Anthropic `messages`/`system` shapes; a byte-transparent Responses body's
`input`/`instructions` items contribute no composition counts (their model,
stream flag and output cap are still recorded).
Separately, the aggregate
content-part count over every message content array, a top-level `system`
array and every Responses `input` item's nested `content`/`output` array is
capped at 16,384. Each Responses annotation array is capped at 4,096 members,
and their aggregate inside those content parts is capped at 10,000.
Anthropic citation arrays are capped at 10,000 members per array, and every
citation array in one document shares a 10,000-member aggregate.
Each nested
content array is capped at 4,096 blocks, and the nested content blocks of
one document share a separate 10,000-block aggregate; the citation and
nested-block aggregates are independent, so their combined total can reach
20,000. A document block's citations configuration object is reported as a
dropped field instead of being decoded into text citations. A valid request
above any of these limits is rejected with HTTP 400 before an upstream send.
Invalid JSON is passed through on the nontranslated path.
Syntactically valid JSON whose strings carry raw invalid UTF-8 is bounded the
same way: valid UTF-8 and JSON escapes decode to no more bytes than their
spelling, while the standard decoder replaces each invalid byte with a
three-byte U+FFFD.
A body whose decoded strings would exceed its uploaded size
by more than a small internal allowance is rejected with HTTP 400 before an
upstream send; smaller invalid sequences remain accepted on the passthrough
path, where a tracked value that fails the identity rules is still dropped
rather than rejected.
Anthropic translation requires the whole request body to
be valid UTF-8 and rejects a raw invalid byte before decoding it.
The proxy can retry transient transport failures, 429 and 5xx before
returning the final response, and retryable in-band error events on an open
stream before any generation content was relayed.
Durable quota/billing 429s follow `quota_pause_mode` instead of the transient retry ladder:
`off` (default) surfaces them immediately; `retry` parks the triggering request
and every new request to the same OpenAI-wire provider behind a recovery
gate that closes after `quota_pause_recovery_successes` consecutive successful probes (the default 1
closes on the first 2xx); a provider `Retry-After` on the quota response paces the
next probe like any other 429; `manual` installs an indefinite provider pause
that only an operator resumes. Quota re-sends never burn the transient retry
budget.
Provider retry hints and operator holds can substantially extend total wall
time.
With `suppress_client_retries` the proxy declares itself the client's sole retry authority: every
error response the relay returns then carries `x-should-retry: false`, the official OpenAI SDK
convention (python, node, go and ruby honor it from source), which overrides
the SDKs' 408/409/429/5xx auto-retry defaults and any `Retry-After`, including a
provider hint that would otherwise relay verbatim.
No response exists to carry the directive on transport-level failures,
clients that ignore the header (the Vercel AI SDK / opencode stack, whose
retry policy is status- and body-text driven) keep their own policy, and
mid-stream SSE errors never auto-retry in official SDKs and cannot carry new
headers.
Success responses are untouched, and the setting hot-reloads for new
responses.
One classifier owns every error decision: the retry ladder, the storm
observer, the health counts and the client-facing class all read the same
canonical class (`rate_limit`, `quota`, `unavailable`, `unreachable`,
`timeout`, `content`, `client_error`, `proxy`, `canceled`, `upstream`),
computed from the status, the provider's own structured type/code, and - for
a transport failure with no response at all - the transport kind, which is
`unreachable` or, for a timed out send, the narrower `timeout`.
Transport evidence outranks the proxy-minted arm, so a failure with no
provider answer is never filed as a proxy fault and never advised against a
retry. Message text plays no part except for that transport timeout
narrowing.
A rate limit announced by a non-429 status (a gateway answering `tier_limit_exceeded` or `resource_exhausted`
with a 400) is therefore retried, counted in the rate-limit series (and,
because the request still failed, in the error rate), and labeled `rate_limit`
instead of `client_error`.
With `client_error_class: header` every error status the relay returns also carries `X-Proxy-Error-Class: ` and a
class-derived `X-Should-Retry: true|false` before the status line commits (the body is never
rewritten; relayed errors stay verbatim), while `suppress_client_retries` still wins with a flat
`false`. The directive is the class's own retry verdict, not a re-reading of the
status:
| Class | `X-Should-Retry` | Why |
| --- | --- | --- |
| `rate_limit` | `true` | transient flow control at any status |
| `unavailable` | `true` | transient capacity or server failure |
| `unreachable` | `true` | transport failed with no usable response, including a fresh send whose peer closed the socket |
| `timeout` | `true` | the send or read deadline fired |
| `content` | `false` | a 2xx with nothing usable; retried on the quality/thinking budgets instead |
| `quota` | `false` | waiting does not clear it; `quota_pause_mode` owns the reaction |
| `client_error` | `false` | the caller's own fault |
| `proxy` | `false` | millivolt's own failure |
| `canceled` | `false` | the local client left |
| `upstream` | `false` | unclassified upstream failure, final by default |
Two settings extend the built-in vocabularies with gateway-specific
spellings, additively: `rate_limit_error_classes` and `quota_error_classes` (plus the existing `retryable_error_classes` for in-band
transient errors). Entries are matched case-insensitively against the parsed
envelope's type and code; validation rejects a spelling that names a
built-in of a different class, so a built-in classification can never be
moved or weakened (the one exception is `retryable_error_classes` naming a built-in client-fault
spelling, the documented override for a gateway that mislabels transient
failures).
Reload applies to new classifications, and the published predicates render
per call, so they re-read stored history through the same effective
vocabulary.
A failed response's reason is read from the provider's own body, in two layers.
The OpenAI/Anthropic envelope (`{"error":{"message","type","code"}}`,
`{"type":"error","error":{…}}`, or a plain-string `"error"`) always wins and
supplies type, code and message.
When the body carries no such envelope, the
failure path still recovers a human-readable reason from the members gateways
commonly use:
- a top-level `detail`, `message`, `msg`, `reason` or
`error_description` string;
- a FastAPI validation `detail` array (the first
item's `msg`);
- Cloudflare's `errors` array (`message`, with its numeric
`code`).
Nested lookups are depth-bounded, a wrong-typed member is ignored
rather than stringified, and every reason is truncated to the same record bound
as an envelope message.
A body that is valid JSON but carries no extractable
reason stays classified as `provider_error` with no message, and an empty body
stays unclassified - only the status remains.
This recovery runs only where the
response is already known to have failed; the success paths keep treating
those same bodies as non-errors, so a 2xx completion carrying a top-level
`message` is never mistaken for a failure.
An error body that cannot be read at all does not erase what the status already
decided. A `429` keeps both its status and the rate-limit series, because the
status alone is flow control; other statuses fail closed to a `502`, and a
provider-decided type survives a later body-read or translation failure instead
of being replaced by the proxy's `stream_read_error`.
SSE pacing may insert comment keepalives. A known queue/hold wait may commit HTTP
200 before the final upstream outcome, so a subsequent failure must be signaled
in-band. Bounded terminal/quality handling can replace a degenerate completion
with an error.
Relay framing follows SSE's own join rule: the data lines of one
event are joined with a newline, so a multi-line (pretty-printed) payload stays
one event and a synthetic blank separator is inserted only when the event's
joined data is already one complete JSON document, or before the `[DONE]`
sentinel.
The completeness check is incremental, so one long event costs linear
time in its bytes rather than quadratic in its line count; a bare scalar counts
only when it is exactly `true`, `false`, `null` or one well-formed number, so a
prose line that merely starts with one of those bytes (`the cat`, `3 apples`)
stays open for its continuation lines. Two single-line
events a provider sent without a separator therefore
stay two events instead of merging into an unparseable payload.
Non-streaming quality retries may make another upstream request;
the adopted attempt's response is spooled under the fixed 1 MiB quality buffer
before the client status line commits, so the retry is delivered as one
ordinary response with no post-commit keepalive gap or partial payload.
A stream truncated before any generation content was relayed is re-sent on
the same committed connection, while a truncation after relayed content
surfaces the in-band error for the client to retry. A trailing partial line
that a rescue or the degenerate terminal boundary drops was never delivered,
so it neither counts as relayed content nor as the record's answer content
or first-answer time, and it does not veto the re-send; only a delivering
path feeds and writes it, exactly once.
Only the truncation class is decided before
that feed, and only when the analyzer's non-committing tail classifier says the
line carries no completion terminal marker: a final Chat `[DONE]` or non-null
`finish_reason` without a trailing newline is fed rather than dropped.
A fed tail is delivered only when it supplies a complete answer or tool
call; an empty or reasoning-only completion resolves through the degenerate
boundary, which withholds or replaces the terminal bytes. A terminal tail is
classified before the pre-content rescues, so it is never re-sent.
A delivered tail is written as its own complete SSE event: a pending event
is closed first and a blank boundary follows, on clean EOF and on a read
error. Answer content that the source places after its finish marker is
therefore relayed as a later event, and a client that stops at the finish
reason may ignore it.
A terminator-less partial line
that carries neither a completion terminal marker nor a completion is not
delivered, so the stream reports truncation rather than a completion or
provider error the client never saw. A standalone in-band error line is not a
delivery terminal and is dropped the same way, and a `[DONE]` on a Responses
feed is not its terminal marker, so it is also dropped as truncation.
A translated streaming request whose 200 response body is not SSE is still
relabelled `text/event-stream`: the upstream bytes are never converted into SSE
frames, and the relay surfaces the non-SSE body as a truncated-stream error
rather than translating it.
A provider error event streamed
in-band on the 200 is retried the same way while nothing content-bearing was
relayed: the same availability and timeout vocabulary the status path treats
as retryable (500 `server_error`, the 503 overloaded/unavailable family, the
timeout spellings, the equivalent Anthropic/gateway spellings including
`bad_gateway`, `gateway_timeout` and `upstream_error`, and the
`engine_overloaded` capacity code) is dropped before the wire and the request
re-sent, so the client never sees the failure.
A specific capacity code
overrides the generic
`invalid_request_error` wrapper, because the code names a transient overload
rather than a malformed request; authentication, permission, not-found and
too-large types stay final.
The `retryable_error_classes` setting extends that vocabulary with operator-supplied type or code
strings for gateway-specific spellings (durable quota/billing classes are
rejected at load); rate limits in-band (no `Retry-After` exists there), durable
request/billing classes, unknown types, Responses-API feeds once any of
their events was relayed (per-response sequence numbers), and errors after
relayed contract bytes relay verbatim for the client to handle.
Mid-thinking rescues (`thinking_retries`) extend the re-send to requests that die or end
without an answer during the reasoning phase: a truncation or upstream read
error after reasoning-only output, a cleanly finished reasoning-only
completion (the withheld terminal region is replaced by the fresh attempt),
and a retryable in-band error that arrived after reasoning-only output.
The fresh stream appends behind the already-relayed reasoning, which is
auxiliary display text; answer content, tool calls, refusals, `finish_reason: length` (the
client's own token cap) and Responses-API feeds (per-response sequence
numbers) are never rescued.
An exhausted budget surfaces the
`reasoning_only` class as a provider error (`upstream_error`) on both
surfaces: in-band on the streaming connection, and as an HTTP 502 error
envelope for non-streaming requests.
Stream accounting has two accepted limits. A source that never reports a
numeric tool-call index is counted per id-bearing fragment; a provider that
repeats the same id across continuation fragments can overcount the calls, and
the fallback is unchanged until such a provider shape is observed.
Provider reasoning tokens are not cross-checked against output tokens: a
reasoning count greater than the reported output is accepted, the derived
answer count is `max(0, output - reasoning)`, and the record keeps the provider's own totals. A
future observed shape that violates the relation needs a targeted validation
rather than a universal rejection.
Neither transparency nor exactly-once upstream execution is unconditional.
Before relay or translation, the upstream request is subject to the configured
`request_overrides` (see [request overrides](configuration.md#request-overrides)):
matching rules set, replace or remove upstream headers as the last header
stage, and their body fields set `max_tokens` / `max_completion_tokens` on
Chat Completions or Anthropic client bodies before translation, whatever the
target (a Responses client body is not rewritten), fail closed. A configured
`sub_conversations` strip also removes tracked body fields before relay
([tracked body params](#tracked-body-params)). Only the upstream request
changes; the client-visible contract is unchanged.
### Scheduling and timing
The provider-plus-key group controls simultaneous upstream sends and its waiting
queue. A separate limit policy can apply to the whole provider or narrow to
models, clients, key hashes and conversations, with an optional recurring
server-local schedule; a provider-only rule combines all keys and clients.
Queue capacity and wait limits can reject work locally;
the configured `queue_retry_after` duration (for example `2s`, never a bare
integer) supplies the caller's retry hint. Operator-hold time is excluded from
the ordinary queue-wait limit. A hold's own queue cap still applies.
For the HTTP relay, `upstream_timeout` limits waiting for response headers,
not the streaming body.
`X-Proxy-Timeout-Ms` instead gives each upstream send a fresh deadline covering
headers and body. Neither is a total end-to-end deadline across queueing, holds,
retries and backoff; client cancellation is the way to stop the whole request.
The request-body progress bound applies only while the body is being read.
Reaching EOF or closing the body clears the connection read deadline, so a
long-lived response stream is never cut by the upload guard.
The [Cursor bridge](adapters.md#cursor-connect-bridge) instead applies the tighter
configured/per-request timeout only until send headers arrive, without a body
deadline on its parked run. Model discovery ignores the per-request timeout
and uses `models_discovery_timeout` for the whole discovery operation.
Buffered whole-document reads (translated non-streaming responses and discovery
pages) use `whole_document_read_timeout` when the send has no deadline of its
own and `whole_document_read_max` as the ceiling even against a longer
`X-Proxy-Timeout-Ms`. A shorter caller deadline still wins. Configuration
validation keeps `models_discovery_timeout` below the ceiling, so its
configured budget is untouched.
Provider Retry-After/reset hints are a floor on the wait, not a replacement
for adaptive exponential backoff. A long hint is not clamped by the ordinary
adaptive cap and can therefore outlast it. A short hint does not reset the
doubling, so a 1s "retry shortly" cannot re-hammer a struggling upstream.
When a 429 carries no header hint but its bounded error body contains an
RFC3339 reset instant (for example "... resets 2026-11-03T23:11:00Z"), that
instant is used as the same bounded hint: a provider-specific code such as
`tier_limit_exceeded` then waits for its window instead of exhausting the
retry budget in seconds and surfacing to the client.
A finalized request retains its absorbed attempts and final outcome in one
record. Every retried or final attempt stores the same upstream-response
information: the provider request id, server, processing time, echoed model,
rate-limit state and the redacted response headers. A provider-side request id
therefore matches to the exact upstream attempt that produced it, including
failures the client never saw.
Transport failures have no HTTP response, so
their attempt entries leave those fields empty.
An in-band 429 is the one
attempt entry that carries the in-band flag: when a keepalive already committed
the client's success 200, the upstream status cannot replace it, so the record
describes the committed 200 and keeps
`{status_code:429,at,in_band:true}` to preserve the distinct rate-limit
signal. The record likewise keeps the committed 200 for every other in-band
failure, including a failed retry re-send whose wire status never reached the
client.
Only a 429 mints one; any other in-band provider error stamps the
record's own type and status instead, and a failed body read behind the commit
keeps the provider-decided type rather than replacing it.
The entry carries no retry or error detail, is not counted as a retry, and
is excluded from retry disclosures and error enumerations.
Recovered upstream failures can
count as errors even when the final status succeeds. A final, retried or
in-band 429 is a separate affected-request signal.
A client that disconnects before a final outcome is recorded as status 499
with `client_disconnected`; a plain 499 is a flow event, but one carrying a
structured error counts as the upstream error it observed. That is the
provider's in-band stream failure the client aborted around, or a truncated
stream whose completion marker never arrived - an abort must be classified by
its outcome, never by which relay write happened to fail.
See [metric interpretation](routes.md#dashboard-data-and-request-inspection)
for duration, first-token latency and throughput.
### Error storm protection
Enable `storm_enabled` in Settings to add shared provider/model protection to
existing admission and retry pacing. Requests wait in the proxy with the caller
connected; this is not a client SDK queue. The proxy keeps the existing bounded
request bytes and credentials only for the request lifetime. Queues are in
memory, cannot survive a restart, and stop waiting when the caller cancels.
Each exact recorded model name within its resolved provider has a rolling
`storm_window`. A model qualifies when its sampled attempts meet
`storm_min_samples` and its selected-failure percentage reaches
`storm_error_percent`. Provider-wide protection requires **every active model**
to qualify, as well as the aggregate sample/rate threshold. Only models receiving
requests within that same window are active; an idle model or an unused model in
a provider catalog cannot affect that decision. This scope combines credentials
and clients. Dashboard model-display rewrites do not merge storm scopes.
The detector samples successful upstream attempts and selected failures, counting
absorbed retries separately. Incident details also count distinct logical
requests with a selected failure in the window; retries move the same request's
last-failure membership instead of adding another request.
Unselected statuses, non-retryable HTTP account/auth rejections, recognized
durable quota envelopes, caller cancellations and fired per-send deadlines
are excluded - quota outcomes never become storm evidence, even while a
quota gate is open. A selected 429 is a storm failure sample but remains a
separate rate-limit signal in dashboard request health. Configurable
response/stream failure detection observes a 2xx attempt once when its relay
completes, or before an existing quality re-ask. It never adds a replay
after meaningful output or native tool execution.
An open scope holds new sends until its cooldown expires, then admits one shared
probe. Failed probes multiply the cooldown, with configurable jitter and a cap;
valid larger upstream retry hints remain effective. Recovery needs the configured
number of consecutive successful probes.
The existing provider-plus-key queues,
operator holds and provider Limits still apply. The HTTP relay checks storm
readiness before acquiring key concurrency, then claims a probe only at the
actual send boundary. Readiness never reserves a probe while waiting for a key
slot.
Native parked runs remain resumable; fresh Cursor HTTP
handshakes use the shared native send owner and can retry before a run starts.
The quota pause (`quota_pause_mode: retry`) is a separate provider-wide gate in
the same scheduler: the first durable quota/billing 429 from an OpenAI-wire
provider opens it immediately (no sample thresholds), the triggering request
absorbs the 429 and re-sends per the same recovery cooldowns, and every new
request to the provider parks at admission.
The gate closes after `quota_pause_recovery_successes` consecutive successful probes (the default 1
closes on the first 2xx) and the queue drains; a provider `Retry-After` on the quota
429 is a floor on the next probe, bounded by the shared hint ceiling. Parked
sends obey `storm_max_queue` and `storm_max_wait`; beyond them the proxy answers HTTP 429 locally.
The gate appears in the storm banner as a quota incident with an operator
**Resume now** action; a manual-mode hold appears on the pause surface
instead. The gate is in-memory: a restart drops it and the provider's next
durable quota 429 re-arms it. Manual-mode holds persist like every operator
pause.
The ordinary HTTP retry budget remains `max_retries`. Selected failures during
an active storm may use up to `storm_max_retries` additional attempts across the
logical request, including quality re-asks.
Cursor runs the same verdict: its
fresh handshakes take the ordinary `max_retries` budget and, with protection
armed, the additional storm budget, and a retried success leaves the group at
one probe exactly like the generic ladder. Exhaustion returns the final upstream
failure. Retrying inference can repeat upstream execution or billing even when
no output arrived; this feature does not provide exactly-once processing. See
the [HTTP retry constraints](https://www.rfc-editor.org/rfc/rfc9110.html#section-9.2.2).
`storm_max_queue` bounds callers waiting at storm gates across the process;
ordinary scheduler queues have their own limits. `storm_max_wait` bounds each
storm wait, not total request wall time. Scope capacity and oversized scope
identifiers also fail closed. Local storm admission rejection returns HTTP 429
with `queue_retry_after`; a streaming connection already committed by keepalive
comments receives an in-band error instead. Successful retries remain one
finalized request record with its absorbed attempts and accumulated queue wait.
### Model discovery
`GET /v1/models` and `GET /models` are constructed discovery services, not
ordinary inference records. The proxy normalizes recognized model lists,
translates native formats, completes supported pagination, and optionally
enriches missing fields. Unrecognized successful list bodies remain verbatim;
primary discovery budget/errors fail explicitly, while optional enrichment can
fall back to the complete base list. Discovery can apply limit headers without
consuming an inference slot. Its time/byte/page/entry limits are
configuration-owned.
Primary and enrichment entries share one `models_discovery_max_entries` budget
(default 10,000); enrichment degrades to the base list when the remaining
budget cannot hold its metadata source. The rendered list is bounded by
`models_discovery_max_bytes` plus a fixed per-entry normalization allowance
(512 bytes for each configured entry) and is encoded without HTML escaping,
so literal `<`, `>`, and `&` keep their wire size. A rendered list past that
derived bound fails closed before any response bytes are written.
Discovery addresses the format's own models endpoint regardless of translation:
`/v1/models` on the shared base join, Anthropic's pagination query, or Cursor's
native unary model service. The translator's inference path is not reused
because both inference endpoints are POST-only upstream. An explicit
`X-Proxy-Path` or `X-Proxy-Query` still replaces the endpoint and query, while
the incoming request query is not forwarded.
A primary upstream non-200 response, including a redirect, becomes a 502
discovery error. Discovery authenticates with the resolved target auth
header and prefix, the same construction as a relayed request, and has no
protocol-specific fallback: an Anthropic endpoint that requires `x-api-key` must be
called with `X-Proxy-Auth-Header: x-api-key` and an empty `X-Proxy-Auth-Prefix`.
The ordinary HTTP relay returns upstream redirects rather than following them.
Forwarding also changes transport headers and may decompress an upstream gzip response;
do not compare transport framing as a byte-identical contract.
## Main and sub-conversations
### Automatic grouping
An explicit `X-Proxy-Session` takes precedence, then a configured tracked body
param (see [tracked body params](#tracked-body-params)). Without either, the
tracker partitions requests by client label and credential hash, then compares
total user/assistant/tool turn counts against its open conversations. A
nonexpired conversation with the greatest previous count no larger than the new
count is the nearest match.
A reset or idle gap can start another conversation, and the configured open-session
cap evicts the oldest tracked conversation in that partition.
This is a heuristic for clients that send conversation history, not a content
comparison or a universal agent identity. Parallel tasks with similar turn counts
can be ambiguous. Tracker state is process-local, so use explicit session IDs
when predictable continuity matters and keep the client/key namespace stable.
Automatic grouping never establishes a parent/child relationship.
### Tracked body params
The request body is client data, and one field can double as the conversation
identity. When `sub_conversations` names the request's classified client, the
tracked field the client already sends (exemplar: opencode's `promptCacheKey`)
supplies that request's identity in the `k:` namespace: behind an explicit
`X-Proxy-Session`, ahead of automatic grouping. Params match only top-level
body fields, checked in order.
The first present field decides: a present
value that is not a JSON string counts as absent and the next field is
checked, while a present string that cannot be decoded or fails the identity
rules drops the identity for the request - no later field is consulted. A
top-level field that occurs more than once is ambiguous and drops the
identity the same way, leaving the request on automatic grouping.
The body is relayed unchanged unless the entry sets `strip`, which removes the
entry's configured fields from the top level of the client request body
before relay or translation. It applies to any JSON body, including a
Responses client body.
A request translated to the Anthropic wire gains a top-level `prompt_cache_key` carrying
the tracked value. That field is a proxy tracking extension, not an
Anthropic cache control, and only upstreams that tolerate unknown top-level
fields accept it.
An absent or dropped identity is never a client
error. A tracked identity never declares a parent: `X-Proxy-Parent-Session`
still requires its explicit session pair, and the session wins identity. The
identity rules, name grammar, caps and the settings surface live in
[configuration](configuration.md#sub-conversation-tracking).
### Declaring a parent
A parent sends `X-Proxy-Session: main-task`. Its child sends its own
`X-Proxy-Session: worker-1` plus `X-Proxy-Parent-Session: main-task`, using the
same client label and API key. Send IDs without the stored `s:` prefix.
When declaring a parent, both headers must occur once, contain nonempty valid
UTF-8 without controls, differ from each other, and fit 512 bytes after trimming.
Self-parent/invalid declarations return 400. The parent header is proxy-only;
it neither edits the body nor instructs an upstream to create a child agent.
The server resolves observed relationships within
`(client, key_hash, conversation_id)`, including history outside the selected
view. The explorer reports main/sub/unresolved counts and role/parent metadata.
A missing parent is **not observed**, not an invented main. Conflicts, cycles or
ambiguous reused IDs remain unresolved. “Main” means no parent declared in
observed history, not independently verified orchestration.
Parent links preserve the full representable client/key namespace. Conversation
filters remain exact leaves, never descendant filters. A changed credential can
change the namespace. Historical parentage cannot be inferred from absent data,
message similarity, or ordinary conversation-continuity fields.
---
# Upstream compatibility, adapters and token refresh
The client-facing protocol is separate from the upstream protocol. Select an
adapter with `X-Proxy-Format`; do not infer one from a provider label.
See [protocol](protocol.md) for routing/auth headers and
[operations](operations.md#known-limits) for current limitations.
## Client and upstream compatibility
millivolt accepts three client wire families (Chat Completions, Responses and
Messages) plus the Cursor Connect bridge, and translates between them and the
supported upstream families. The client protocol is selected by
the request path, or explicitly by `X-Proxy-In-Format`; the upstream protocol
is selected by `X-Proxy-Format`. Translation runs only when both are known and
differ.
An absent `X-Proxy-Format` never enters the IR, and an explicit format
equal to the client protocol also stays passthrough. Passthrough is not an
unconditional byte-identity promise: configured `request_overrides`, a
`sub_conversations` strip, or ensured tools can still rewrite the upstream
body before it is sent (see
[request overrides](configuration.md#request-overrides)). Without any of those
configured, the body bytes are relayed unchanged.
| Client path | Client protocol | `X-Proxy-Format` | Upstream protocol |
| --- | --- | --- | --- |
| `POST /v1/chat/completions` | OpenAI Chat Completions | `openai` or absent | OpenAI Chat Completions, passthrough when both are Chat |
| `POST /v1/chat/completions` | OpenAI Chat Completions | `openai-responses` | OpenAI Responses |
| `POST /v1/chat/completions` | OpenAI Chat Completions | `anthropic` | Anthropic Messages |
| `POST /v1/chat/completions` | OpenAI Chat Completions | `cursor` | Cursor agent.v1 |
| `POST /v1/responses` | OpenAI Responses | `openai` | OpenAI Chat Completions |
| `POST /v1/responses` | OpenAI Responses | `openai-responses` or absent | OpenAI Responses, passthrough when both are Responses |
| `POST /v1/responses` | OpenAI Responses | `anthropic` | Anthropic Messages |
| `POST /v1/messages` | Anthropic Messages | `openai` | OpenAI Chat Completions |
| `POST /v1/messages` | Anthropic Messages | `openai-responses` | OpenAI Responses |
| `POST /v1/messages` | Anthropic Messages | `anthropic` or absent | Anthropic Messages, passthrough when both are Anthropic |
xAI/Grok uses the ordinary OpenAI-compatible relay, not a separate native
adapter. Other providers and local model servers can use that same path when
they expose a compatible API; there is no fixed provider registry. Native
translation covers the fields listed under
[cross-protocol translation](#cross-protocol-translation), not every endpoint
or arbitrary provider protocol.
Clients must support custom routing headers or middleware that adds them.
For native formats, supply the adapter selection and the upstream's required
authentication, version and path headers; the
[Anthropic example](protocol.md#url-and-auth-examples) and
[Cursor login setup](#cursor-login) show these choices. Provider profiles enrich
headers, metadata and ensured tool signatures but do not select an adapter or
grant provider access.
## Cross-protocol translation
Cross-protocol translation goes through one canonical request, response and
stream model in [format/ir](../internal/format/ir/ir.go). Each wire family has
one decoder and one encoder; the IR is the single source of truth for roles,
content blocks, tool calls and results, reasoning artifacts, stop reasons and
usage. Same-protocol requests never enter the IR: they relay unchanged unless
a configured request override, sub-conversation strip or ensured-tools rule
rewrites the upstream body first.
Carried wherever the target protocol represents them: text, images (URL,
base64 or file id), documents, audio where the target accepts it,
system/developer/user/assistant turns, function tools, tool choice, parallel
tool use, tool calls and results, structured outputs, reasoning effort and
summary, stop sequences, sampling parameters, metadata, usage and cache-token
accounting.
A malformed Chat `stop` value is rejected before the upstream send on a
translated request. Absent, null, a string and an array of strings are the
accepted shapes; any other type returns 400 instead of silently dropping the
stop filter.
An array element that is null or not a string is also rejected:
Go would otherwise decode it as an empty stop sequence. The same element rule
applies to Anthropic `stop_sequences`. Native Chat passthrough is not decoded:
the body is relayed unchanged, so `stop: 5` reaches the upstream and its
response is returned. An empty string and an empty array still mean no stops.
Protocol-only artifacts are preserved for the same family and reported when a
target cannot consume them:
- Anthropic thinking and redacted-thinking blocks keep their opaque signatures.
A Responses reasoning item has no Anthropic signature: a translated request
or non-streaming response emits its visible text with an approximation
warning instead of a `thinking` block, while a translated stream opens a
`thinking` block with an empty `signature`. The proxy keeps the reasoning
text and does not invent a signature; resending an empty-signature block is a
client choice and the target may reject it.
- OpenAI Responses reasoning items keep `encrypted_content` and item ids.
A cross-family encrypted reasoning artifact is never replayed into another
provider's field. A non-streaming surface drops it with a warning; a stream
has no warning sink and drops it silently.
- A Responses reasoning item exposes two independent text channels: the
summary parts and the content reasoning text. A translated stream emits both
as they arrive, so text the source places in both channels appears in two
target reasoning blocks. The proxy does not deduplicate across channels: a
native client accumulating the source observes the same two channels, and
merging them would corrupt genuinely distinct summary and content text.
- Responses namespace tools are flattened, not dropped. A client may group
function tools under `{"type":"namespace","name":"mcp__context7","tools":[...]}`;
the target receives each member as an ordinary function whose name is
`__`, and a returned call names the group again in the
client's `function_call` item. The flattened spelling is truncated to the
target name class and suffixed with a hash when it is longer than
Chat's 64-character limit, and two members that flatten to the same string
receive a numeric suffix, so every declared tool stays addressable.
- Provider-executed tool definitions (web_search, file_search, mcp,
image_generation, tool_search and similar) are optional to the client's
remaining tools and are dropped with a `tool.` warning when the target
cannot execute them; `translation_loss: reject` fails the request instead.
- A freeform `custom` tool is dropped the same way. Its input is a grammar,
not a JSON schema, and the client expects a `custom_tool_call` item back, so
declaring it as a function would hand the model a tool whose call the client
cannot dispatch.
- Provider-executed blocks such as server tool use and web search results have
no cross-protocol equivalent and are dropped with a warning.
- A Responses `max_tool_calls` request field has no cross-protocol equivalent.
A native Responses relay forwards it byte for byte, so the provider still
caps its own tool calls. A translated request drops it with a
`max_tool_calls` warning and `translation_loss: reject` turns that loss into
a 400 before any upstream send. The translated turn therefore has no
provider tool-call cap unless the client also sets one the target protocol
understands.
- A Responses item `phase` is a wire hint with no target equivalent. A
non-streaming request or response records one `input.phase` or `output.phase`
warning. A translated stream keeps one `output.phase` warning only in the
decoder's diagnostic state; no client header or stored record carries it.
The phase text never reaches the client wire.
- A `response.output_text.delta` for a part the source never announced is
dropped, but the matching done event or the terminal document recovers the
full text, so only the incremental gap is lost. A Chat `[DONE]` without a
finish reason maps to `response.incomplete` for Responses and to a clean
`message_delta` with `end_turn` for Anthropic; both are accepted limits with
no reproduced client failure.
- An Anthropic target derives its terminal from content when the canonical
reason maps to `end_turn`: a non-empty refusal becomes `refusal`, a tool call
becomes `tool_use`, and a refusal wins when both appear. The non-streaming
response reports the approximation in `X-Proxy-Warnings`; a stream has no
warning channel.
- A canonical output block with no Responses content part, such as a file or
audio block carried from Anthropic, is dropped from the output with one
`output.` warning. Text around the dropped block is preserved.
- Responses provider-executed tool calls in the shared tool-item vocabulary
(`custom_tool_call`, `local_shell_call`, `mcp_call`, `computer_call`,
`tool_search_call` and their peers) are tool activity: they count toward a
`tool_calls` finish even though no cross-protocol content block exists for
them, and a translated record still counts them (retaining streamed names)
when the target wire drops the item. Provider-executed generation artifacts
(`program`, `program_output`, `image_generation_call`,
`code_interpreter_call`) are output presence
instead: a completed turn that carries only one of them is not an empty
completion, while its finish stays the source's stop rather than
`tool_calls`, because the item is generated source or media rather than a
model tool invocation. Extending the tool vocabulary waits for a
demonstrated client need.
- `previous_response_id`, `item_reference`, `background`, `conversation` and
prompt templates require provider-side state and are rejected when the
target cannot honor them. `store: true` is honored by Chat Completions and
Responses targets and rejected by Anthropic. A Chat or Anthropic request
translated to a Responses target sets `store: false` when the source never
chose storage, because the Responses default retains data for at least 30
days while the source protocol has no such opt-in and the proxy has no
retrieval path. Explicit source values are preserved. A Responses source's
implicit storage cannot be honored on Chat or Anthropic, and no response id
is returned for later retrieval.
- Every system or developer message before the first emitted conversation
turn, together with the Responses instructions, becomes the top-level
Anthropic `system` prompt. A mid-conversation system run stays in `messages`
only after an emitted user turn and before an assistant turn or the end; any
other position is lifted to the prompt with one approximation warning. A
later developer turn has no Anthropic role and is lifted to the prompt with
an approximation warning, which reports the positional change.
- An Anthropic target drops leading assistant turns because the first message
must use the user role, reporting one `messages.leading_assistant` warning.
A tool result that answered a dropped leading call is dropped with one
`messages.orphan_tool_result` warning, so that result never reaches the
wire. A source-supplied result that matches no call in the source history is
content, not an artifact of the drop: it passes through unchanged as a
`tool_result` even though no preceding `tool_use` exists, with no warning.
Pairing happens before the drop: a result that names a dropped call stays
paired to it and is dropped with that call rather than retargeted to a later
same-name call, because the name may answer the dropped call and retargeting
could answer the wrong tool. A system or developer turn before
the first emitted turn is still lifted to the top-level prompt even when a
dropped assistant or an empty user turn preceded it.
- Consecutive same-role turns are merged. A merged user turn keeps Anthropic's
results-first rule: when the later turn would otherwise put text before a
`tool_result`, the results move to the front with their relative order
preserved and one `approximated:messages.tool_result` warning reports the
reorder. An already-ordered or text-only merge keeps source order and warns
nothing.
- A non-clean non-streaming terminal can carry a missing, null, whitespace or
partial tool-call argument value the source never finished: a Chat finish
other than `stop`, `tool_calls` or `function_call`, or a Responses
`incomplete` or `cancelled` terminal, including a `completed` response that
still carries an incomplete reason. The proxy drops only the unusable call
with one `tool_calls.arguments` or `function_call.arguments` warning and
keeps the text, refusal and source finish. A Responses `failed` terminal is not covered
by that rule: it returns the provider error envelope and discards the output,
so no block or warning is produced. A clean terminal still maps an omitted,
null or whitespace value to the empty object, but fails closed on a partial
or otherwise invalid value. Structural failures such as a missing name or id
remain fatal on every terminal, even when the arguments are also unusable.
- A streamed Responses `function_call` that omits `call_id` uses its item id as
the target tool id. When both identities are absent, the block starts with a
stable `call_proxy_` id instead of an empty id, so Chat does not
write an empty tool call and Anthropic does not reject the block. An
announcement that carries both identities accepts either spelling on the
terminal item. A terminal item that uses the other identity kind, an item id
after a call-id announcement or the reverse, matches a unique open announced
call with the same name. A terminal entry that repeats an announced call
whose block already closed is skipped before open matching when its identity
and name agree; an id-less entry also requires its effective argument value
to agree. An id-less repeat is skipped before its array position is resolved,
so a closed call announced at a foreign position cannot claim a different
open call. Formatting differences and object key order do not affect that
agreement, but numeric literals are compared as written, so `1` and `1.0`
do not agree. A value above the canonicalization size, nesting or
member-separator bounds keeps the byte-exact rule, as does a partial or
otherwise invalid value, so formatting and key-order differences in such a
value do not agree. A named terminal item matches only an open announced call
with the same name; several same-name candidates are disambiguated only by
the announcement at the terminal position, and no same-name candidate starts
a new call instead of taking a differently named open call. An id-less
terminal item then matches a unique open announced call with the same name,
then the announcement at its own position among same-name candidates, then a
lone or first still-open id-less announced call in output order, so a
reordered terminal array completes the announced block rather than
synthesizing a second call. If no open call matches, an id-less entry whose
name and effective argument value agree with a closed id-less call is
skipped as that call's repeat. An unannounced id-less call at a colliding
announced position cannot be distinguished from a repeat and is dropped
rather than assigned to a different call. An identified terminal call also
matches only a same-name candidate whose identity kind is compatible: a
candidate keyed by the same identity kind must carry that identity, while a
cross-kind candidate stays eligible under the mixed-spelling rule. Position
alone never moves arguments to a conflicting identity. The non-streaming
decoder still rejects a call with both identities absent.
- A legacy Chat `message.function_call` is treated as answer-shaped for retry
safety because the deprecated shape can represent either a tool invocation or
a final answer, and it is not counted as a tool call. A Chat source that uses
only that shape therefore records no tool-call count even though the call
translates.
- Anthropic `metadata.user_id` is limited to 512 characters. A longer Chat or
Responses `user` value is rejected before any upstream send rather than
truncated, because truncation can collide two distinct identifiers.
Chat/Responses `service_tier: priority` maps to Anthropic `speed: fast` with
an approximation warning; `fast` maps without one. A translated stream takes
its tier and metadata from the source start event. A value that appears only
in the terminal document is not applied retroactively, because the target
`message_start` and earlier chunks are already on the wire.
- An Anthropic document `title` is limited to 500 characters. A longer source
filename is dropped with a `document.title` warning while the document bytes
still translate; strict mode rejects the loss before any upstream send.
- A response metadata map has no field on the Anthropic Message object, so a
non-streaming translated response drops it with a warning. A translated
Anthropic stream cannot report response-side warnings after its status line
is committed.
- An image or document inside an Anthropic assistant request turn is
schema-valid because assistant request content shares the message content
type. The request encoder keeps it and reports an `assistant.image` or
`assistant.document` approximation warning; runtime acceptance of assistant
media is not verified. Anthropic's response `Message.content` union has no
image or document block, so the response encoder drops it with a
`response.assistant.image` or `response.assistant.document` dropped warning.
User media is carried without that warning.
- A Responses `input_file` with inline `file_data` but no media-type
declaration has one inferred from a data-URL prefix, a `.pdf` or common
plain-text filename extension, or a base64 `%PDF-` signature. Unrecognized
bytes are dropped with one warning while a file id or URL on the same part
survives; a part with no remaining payload is dropped. Chat file parts keep
their existing PDF-only default.
- A Chat response carries one refusal field, so multiple IR refusal parts are
concatenated in block order. An empty part contributes nothing and never
clears earlier text.
- A translated response has no audio representation: the IR has no audio output
block, so a Chat `message.audio` or streamed `delta.audio` and a Responses
`output_audio` content part never reach a target wire. The non-streaming
Responses decoder reports one `content.output_audio` warning; the Chat
decoder carries no audio block and reports no warning. Native same-protocol
relay keeps the audio unchanged. Mapping a transcript to assistant text
would be a fidelity feature, not a lossless translation.
- A translated non-streaming response assumes a schema-valid source identity.
A source that omits `id` or `model` yields an empty string on the target
rather than an invented identity. The Responses envelope carries protocol
defaults for request-echo fields (`parallel_tool_calls: true`, `tools: []`,
`tool_choice: "auto"`, null `instructions`/`temperature`/`top_p`,
`store: false`); a translated response cannot echo the source request's
parameters without threading the request IR into the encoder.
- A Responses output tool-call `call_id` has no schema length bound, so a
provider id longer than 64 characters is relayed unchanged. If the client
replays that call as request history, the Responses request boundary enforces
its 1..64 character reference rule and rejects the request with 400. The
proxy never shortens or drops the output id, because that would break the
correlation the client already received.
- Responses `include` options other than `message.output_text.logprobs` and a
non-string Chat metadata value have no target equivalent and are dropped
with a warning. The logprobs include maps to the target's request field, but
translated responses do not carry provider logprob details.
- Deprecated Chat `functions` and `function_call` are converted to the
canonical tool list and tool choice; a legacy per-assistant `function_call`
pairs with its `role: function` reply by function name. Chat `prediction`,
audio output, non-text `modalities`, and `web_search_options` have no target
equivalent and are dropped with a warning.
- A request-history tool call without an id gets a generated id and reports the
recorded approximation. Tool ids are pre-reserved once per request across
every message and item before any message is decoded: a generated id can
never collide with an explicit id anywhere in the request, including one that
appears later, and a repeated explicit id fails closed with HTTP 400. The
deprecated per-assistant Chat `function_call` keeps its function name as the
id because that shape carries no id; a repeated same-name legacy call is
inherently ambiguous and remains the documented exception. Pairing keeps the
kind of reference. A non-empty
`tool_call_id` answers the unanswered call with that exact id, so an explicit
id is never shadowed by another call's function name; when no id matches, a
generated call may still be addressed by the function name its client supplied
in place of the missing id. A message `name` (with no `tool_call_id`) addresses
a unique generated call. A reply with neither resolves only while exactly one
generated call is unresolved and no other call is unresolved. Any reference
matching more than one unresolved call fails closed, and an explicit call is
never answered by its name alone.
- A `json_schema` response format must carry a valid JSON object `schema` to
translate; a schema-less, null, non-object or invalid `json_schema` schema is
rejected with 400 because a cross-protocol target has no meaningful mapping
for one. A native same-protocol Chat request is relayed unchanged, so the
passthrough path still forwards it.
- A non-function tool type and malformed tool arguments are rejected before any
upstream send.
`translation_loss` selects the policy. `warn` (default) drops the optional
fields this proxy models but the target cannot represent, and reports each one
in `X-Proxy-Warnings`. Provider-specific extension fields outside that modeled
vocabulary are not inspected or reported; the passthrough path relays them
unchanged.
`reject` fails the request with 400 naming the fields before any
upstream send,
and it refuses every recorded loss, including a documented approximation such
as a clamped temperature, a moved developer message, a generated tool-call id
or a `json_object` response format, not only a field the target cannot
represent at all.
The header is proxy-owned: a provider response header of the same name
is never forwarded. Its whole value is bounded at 1024 bytes; an over-cap list
is cut at the byte budget, possibly inside the final field, and ends with
`; truncated`. Response-side warnings
are never fatal; a non-streaming response reports them in `X-Proxy-Warnings`,
while a streaming response cannot add a header after the status line is
committed and drops them. The field is described under
[settings](configuration.md#finding-a-setting).
A provider error envelope that arrives inside HTTP 200 is translated too, but
the client status becomes 502: a translated client would read 200 as a
successful completion. A non-200 translated error body is buffered before the
client sees it, at most 64 KiB, under `whole_document_read_timeout` (default 2
minutes), so a provider that holds an error stream open can delay the client
until that bound expires.
A non-200 status that forbids a response body (1xx,
204 or 304) cannot carry the translated envelope, so it fails closed as 502
`upstream_error` naming the upstream status instead of committing a status
whose error body the client cannot receive. Pre-commit failures (routing,
request validation, translation, or an oversized translated body) answer in the
client protocol's own error envelope.
Non-streaming translation reads the complete upstream 200
body and the translated document under `translated_response_max_bytes` each
(default 32 MiB); exceeding either fails closed with 502 `response_too_large`
before any client bytes. Translation buffers the whole document, so the
setting also bounds one in-flight translation's memory.
Streaming translation is event-driven and bounded. Each target encoder owns its
protocol's ordering invariants: Anthropic opens a block before any delta and
closes it before `message_delta`; Responses emits `response.created`, per-item
lifecycle events and one terminal event; Chat emits the role chunk, deltas, one
`finish_reason` chunk and one `[DONE]`.
The official Anthropic event flow closes
every content block before `message_delta`. A nonconforming source that sends a
content delta after `message_delta` is not given a runtime special case, because
the terminal contract and the target encoder ordering still own the turn.
A truncated upstream is closed with the
client protocol's own error event, never a fabricated clean finish.
The Responses encoder can emit `response.cancelled` for a canonical cancelled finish. The
current `ResponseStreamEvent` union lists completed, failed and incomplete terminals, while
the `Response` status enum includes cancelled; the event is kept as a
forward-compatible extension and a strict client may treat it as unknown.
Synthetic Responses error and terminal frames emitted on a
committed socket after an accepted non-completion status restart
`sequence_number` at 0 and 1; relayed events before that keep their original
sequence. The degenerate terminal-region substitution emits the same error plus
`response.failed` pair at 0 and 1 even when earlier streamed events carried
higher numbers, because the relay does not retain the provider's counter. Both
reset cases are documented contract gaps, not reproduced client failures; the
terminal event is final for the turn.
- A Chat stream chunk is decoded from its first choice only. A translated
request never emits `n` above 1 because only one choice can be translated,
so the limit is unreachable for compliant traffic; a noncompliant upstream
that interleaves several choices in one stream would mix them in the
translated stream.
- A Chat stream must carry `chat.completion.chunk` documents. A full
completion document, or a choice carrying `message` instead of `delta`, is
rejected instead of becoming a successful empty answer; a chunk without
`object` stays accepted when it carries a delta.
- A source in-band error followed by the protocol terminal is deliberate: Chat
and Responses clients receive the provider error and then `[DONE]` or the
terminal event, matching the source order. The relay does not suppress the
terminal after an error.
- A streamed tool `arguments` value that is exactly `null` is emitted literally
and fails the block-close validation, while the non-streaming decoder
normalizes it to `{}`. The stream cannot rewrite bytes it already delivered,
and no official provider shape emits this value, so the inconsistency is an
accepted limit rather than a normalization path.
- Anthropic `content_block_start` text is not counted as answer content,
first-answer time or a preview. The Anthropic wire is exempt from the OpenAI
degenerate substitution, so a completed start-text-only stream stays healthy
and records no answer content. Official start text is normally empty; this is
an accepted accounting limit.
Streaming citations are not translated. A Responses
`response.output_text.annotation.added` event is not decoded, and a streamed
target does not emit citations or annotations; non-streaming responses do carry
url citations. A citation stream event is a separate feature, not a silent
claim of streaming parity.
A non-streaming translated Responses message item carries
`status: "incomplete"` when the response-level status is not `completed`, and
`status: "completed"` otherwise. Function-call items always carry
`status: "completed"` because every emitted call has validated arguments. The
response-level `status` still owns the turn lifecycle.
In a translated stream,
`response.output_item.added` carries `status: "in_progress"` and each
`response.output_item.done` carries the status at which that item closed:
`"completed"` for a normal close and `"incomplete"` for an aborted item. The
terminal response output preserves those per-item statuses; only an item still
open at an incomplete or failed terminal is rendered `incomplete`, and a failed
terminal omits an open call whose arguments were never validated.
Parallel tool calls translated to Anthropic open their indexed content blocks
concurrently (for example start 0, start 1, delta 0, delta 1, stop 0, stop 1).
The blocks carry the index the streaming contract requires, and SDKs accumulate
by index; a strict assumption of sequential block lifecycles would need a
separate encoder design, not a per-request special case.
A streamed Responses tool call is validated by its terminal class. A
completed terminal fails closed on a partial function-call value. Incomplete
and cancelled terminals do the same, because both are normal turn boundaries
and an executable empty object must never close a call the model did not
finish. On those two terminals a call with missing, null or whitespace
arguments is also rejected unless the source explicitly sent `{}`.
When validation fails, the target abort closes non-tool blocks but
leaves an open tool block unclosed: Anthropic emits no `content_block_stop` over
empty input, and Responses emits no done function-call item with empty
arguments. The proxy error frame terminates the stream, so a client that keeps
reading cannot execute an empty call.
A failed, errored or truncated terminal
instead drops only the unparseable call and keeps the provider error or
truncation. On Anthropic and Responses, an already-announced open call is left
unclosed on those three terminals: Anthropic emits no `content_block_stop`,
Responses emits no done function-call item, and the proxy error frame ends the
stream.
The Chat target already streamed the announcement with empty arguments,
and Chat reports a cancelled finish as `stop`, so that accepted limit is not
distinguishable there. A Chat `[DONE]`
without a preceding non-null finish reason is a clean transport terminator but
an empty finish: a Responses target receives `response.incomplete` with
`incomplete_details: null`, and the record reports `finish=other`. That
degenerate classification is deliberate rather than a fabricated completion.
## Bundled compatibility profiles
The generated [configuration example](../proxy.example.yaml) enables the Grok,
OpenCode Zen and Cursor profiles and the opencode sub-conversation tracking
exemplar (the `promptCacheKey` tracked param, strip on). `config.Example()` in
[internal/config](../internal/config) owns the exact profile mappings, header
values and that tracking entry; built-in `config.Default()` remains
provider-neutral. The image installs the example into fresh config volumes.
Existing saved configurations are not silently replaced.
These are operator-selected compatibility snapshots, not verified current public
API fingerprints or permission to use an account service. They contain no keys
and do not choose the upstream URL or protocol adapter for a request. Review
them for your integration and remove or adjust any unwanted profile in Settings.
The Grok profile supplies client-identity/request-ID headers and model-modality
enrichment through the model discovery endpoint. Model `version` is not context
length, and the profile does not invent that mapping. Standard usage detection
and USD-tick scaling already handle recognized cost fields; custom `cost_keys`
must name USD values rather than bypassing that scaling. Streaming cost requires
the upstream to report usage; missing cost is not evidence of a free request.
See xAI's [model reference](https://docs.x.ai/developers/rest-api-reference/inference/models)
and [cost tracking](https://docs.x.ai/developers/cost-tracking).
The Cursor profile supplies CLI compatibility, allowed-tool and request-ID
headers. Its ghost-mode field is a supplied wire preference, not a privacy or
retention guarantee. Cursor's [public authentication documentation](https://cursor.com/docs/cli/reference/authentication)
does not establish a stable public contract for this native bridge or its
compatibility headers. Keep authorization and current service support separate
from a successful local fixture test.
The OpenCode Zen profile supplies the opencode CLI's client identity on the
ordinary OpenAI-compatible relay: the CLI's composite `User-Agent`,
`x-opencode-client` and `x-opencode-project`, plus the CLI's per-conversation
`x-opencode-request` and `x-opencode-session` headers, which the
`{{opencode-msg-id}}` and `{{opencode-ses-id}}` templates mint fresh per
request in the CLI's 26-character identifier format (provider headers
override client-forwarded values; the ids carry no session state, so
replacing a genuine client's own ids is functionally neutral). The profile
also sets `ensure_tools: [bash, read]`.
A check on 2026-09-24 found zen's
free tier gated on that wire signature rather than on TLS, header order or
credentials: with the id headers in shape and a chat `tools` array containing
the `bash` and `read` tool names, the gated free models served even under
the anonymous `public` bearer.
`ensure_tools` injects missing names as inert
stubs. When the client sent no tools of its own, an absent, null, or `auto`
`tool_choice` is forced to `none`, while an explicit `none` stays; a
`required`, named, or malformed choice skips injection entirely, because
forcing `none` would contradict the client and injecting would let the model
call an undeclared stub. When the client already declares at least one tool,
the configured names are added to that list and an explicit `required` or
named `tool_choice` stays in force.
A body that already satisfies the
signature is relayed unchanged, apart from any configured request override or
sub-conversation strip that runs before it. The stubs are Chat-shaped, so they
are injected only into an OpenAI Chat client body on a Chat outbound wire; an
Anthropic, Responses, Cursor, or arbitrary body is never rewritten.
The same check found the signature also requires a streaming request
(`"stream": true`): non-streaming requests to the gated free models get the
provider's `FreeTierError` 403, so clients of those models must stream (the
CLI always does). That observation is about that day, not an entitlement: the
profile is not an account, key or permission, and there is no zen login
helper. Zen's models are listed in
[OpenCode's Zen documentation](https://opencode.ai/docs/zen/).
Every profile is editable operator configuration, in the private config file
or in Settings under Providers (each card edits the field maps, ensured tool
names and upstream headers, with hot reload): changing a signature tool set
or a header value is an edit, never a code change.
To reproduce another first-party client's signature for a different
provider, work from evidence top down: capture the client's own requests
against a local logging upstream, strip one signal at a time against the
real endpoint (TLS ClientHello, header order and values, body fields,
credentials) until only the gate remains, then express the surviving
identity as a provider profile - static headers directly, per-request
dynamic values as `{{uuid4}}`, `{{platform}}` or a new mint placeholder in the single template
expander, and required tool names as `ensure_tools`.
The zen profile above is one worked example of that recipe,
not a hardcoded mechanism.
## Anthropic-compatible Messages
`anthropic` works in both directions: as an upstream selector for a Chat,
Responses or Anthropic client, and as the inbound protocol for
`POST /v1/messages`. Translation covers message history, system text,
images, tools and results, thinking blocks, cache control, selected parameters,
usage and streaming events. Fields with no target representation follow the
[translation loss policy](#cross-protocol-translation).
A configured [sub-conversation
tracking](configuration.md#sub-conversation-tracking) value is injected as a
top-level `prompt_cache_key`. That field is a proxy tracking extension, not an Anthropic
cache control: Anthropic caching uses `cache_control`, and the injection is only safe
for upstreams that tolerate unknown top-level fields.
The client's own spelling of the tracked field is
otherwise dropped by the allowlist. The caller
must supply the native endpoint's correct auth/version/path headers; the
[protocol example](protocol.md#url-and-auth-examples) demonstrates the mechanism.
Stored parameter and conversation-composition metadata describe the translated
upstream body, not an exact copy of the original request. For a native or
translated Anthropic body, the recorded stop count, tool choice, parallel-tool
flag, reasoning effort, response format and service tier come from the
Anthropic spellings (`stop_sequences`, the `tool_choice` type/name and
`disable_parallel_tool_use`, `output_config`, and `speed`) when the Chat
spelling is absent.
A body that combines the Responses `max_output_tokens` spelling with a Chat
`max_tokens` or `max_completion_tokens` spelling is rejected with HTTP 400
before translation or any upstream send. If the caller omits
all three output-limit spellings (`max_tokens`, `max_completion_tokens` and
`max_output_tokens`), translation supplies `anthropic_default_max_tokens`,
which can therefore appear in request details as a captured value.
When a request supplies both Chat spellings, translation prefers `max_completion_tokens`; the
proxy's request decode applies the same precedence when capturing the cap,
so the captured value always agrees with the one the translated body carries
upstream.
A translated Chat body always spells its output cap `max_tokens`: it has the
widest OpenAI-compatible support. OpenAI documents `max_tokens` as deprecated
and its o-series reasoning models require `max_completion_tokens` instead. A
native Chat client may send either spelling, and the decoder prefers
`max_completion_tokens`, but a translated Chat request still carries
`max_tokens` and can therefore be refused by a target that accepts only the
newer spelling.
An Anthropic target request forwards `temperature` (clamped to 0..1), `top_p`
and `top_k` for older and compatible upstreams. The current Anthropic
`MessageCreateParamsBase` request type no longer lists those sampling fields,
so a strict current endpoint may reject them. The forwarding is kept as a
compatibility choice rather than silently dropping client values; revisit it
if a live current endpoint is verified to reject them.
Non-streaming translation buffers the full response before committing status;
a translation failure can therefore return a real error status. A translated
stream is emitted in the client's protocol, not as OpenAI-shaped events, and it
observes original-source custom cost privately before final accounting.
Cache-inclusive native usage conversion has one owner; never add reasoning
twice when it is already part of output.
A valid native stop marker closes the turn. Missing completion/error state must
not be converted into a fabricated clean finish. Legitimate empty/refusal/native
tool turns are not permission to execute upstream tools again. General native
multi-line event assembly still has limits; fixture coverage is not a blanket
protocol-conformance claim.
Native model discovery translates and completes supported paginated lists under
the shared discovery time/byte/page/entry budget. Optional enrichment fills missing
fields without replacing upstream-owned values; it draws from the same
discovery entry budget and degrades to the base list when the remainder cannot
hold its metadata source.
Owners: [format/anthropic.go](../internal/format/anthropic.go),
[format/models.go](../internal/format/models.go),
[proxy/relay_transform.go](../internal/proxy/relay_transform.go), and
[proxy/models.go](../internal/proxy/models.go).
## Cursor Connect bridge
`cursor` is a text/tool bridge from OpenAI chat to the native agent.v1
bidirectional Connect/protobuf protocol. It currently requires an OpenAI Chat
client: a Responses or Anthropic client targeting `cursor` is rejected with a
clear 400 instead of receiving the wrong response shape. It is not a general multimodal bridge
or a promise of a stable public provider API. Check the provider's current
access terms and supported integration options before using account credentials;
the project does not establish permission on your behalf.
The run owns a bidirectional HTTP/2 transport, framed messages, heartbeat,
context/KV replies and one ordered event queue. Malformed known nested messages,
invalid usage or error terminal frames fail explicitly. A validated turn-ended
marker plus EOF remains a supported completion convention; bare EOF without
completion is not a clean success.
The event queue carries an internal retained
byte budget and the pending tool-call map internal count and payload byte caps;
the KV blob store carries per-run byte and entry caps.
The parked-run store enforces aggregate budgets across all parked runs:
retained pending plus queued event bytes, retained blob bytes, and blob
entries. It holds one per-run pending-plus-queued allowance free for the run
being actively driven, and it rejects admission or growth that would cross
an aggregate bound. The entry caps bound unique zero-length blobs that the
byte cap cannot charge.
Exceeding a run or aggregate parked bound fails closed with a stable
descriptive error instead of retaining unbounded memory; an over-cap history
fails the request and a refused park cold-starts the next turn. None of
these bounds is operator-tunable.
When the upstream requests a client tool, the proxy returns OpenAI tool calls
and parks the native stream. A later matching tool-result tail can resume that
same stream. Full route/auth/model/client scope prevents cross-identity reuse.
Registration, expiration and close share the run-store owner; client cancellation
ends active work, while a legitimate parked run lives until its configured TTL,
upstream close, capacity eviction or process restart.
For each new send, the tighter of `upstream_timeout` and
`X-Proxy-Timeout-Ms` bounds waiting for response headers. That wait deadline is
removed after headers; it is not an overall body deadline. Resuming an existing
parked stream does not open another send. The default path is the native Run
RPC; an explicit `X-Proxy-Path` is appended to the base with a leading slash,
without the HTTP relay's shared-segment deduplication. Query overrides and the
incoming query are not used.
A lost park cold-starts from provided history. The quality-retry path may re-ask
an empty continuation; a fresh Run is accepted only on HTTP 200, so any other
response to the re-ask is a failed re-ask and the void surfaces instead of an
acknowledgement being driven as a stream. Exhausted/invalid continuations
surface errors rather than silently completing.
Cursor's wire reports one
combined output total with no answer/reasoning split, so the generic
mid-thinking rescue
(`thinking_retries`) does not apply here: a Cursor turn that produced nothing
is the existing empty-turn re-ask's business. This is not exactly-once
execution, checkpoint persistence, or a guarantee that an upstream keeps a
parked stream alive.
Output usage comes from validated native counters. Input may use checkpoint
context occupancy or a documented estimate when unavailable; unreported cache
and reasoning splits are not invented. Fused model spellings are decomposed by
the existing protocol owner, not copied as raw native parameters. Stored model
metadata uses the canonical base ID, not the caller's fused spelling.
Model discovery uses the native unary usable-models service and returns the
constructed OpenAI list. Context metadata is not guessed from a static price or
capability table. Discovery budgets cover this path as well as optional enrichment.
Owners: [proxy/cursor_bidi.go](../internal/proxy/cursor_bidi.go),
[proxy/cursor_store.go](../internal/proxy/cursor_store.go),
[format/cursor_run.go](../internal/format/cursor_run.go), and neighboring
format/protocol tests. Keep protocol constants distinct from the configurable
[compatibility profile](#bundled-compatibility-profiles); built-in defaults remain
neutral and the example does not establish provider support.
### Cursor service and protobuf versions
The service name, transport version and codec are different layers, not two
protobuf implementations running together:
| Layer | What millivolt uses |
| --- | --- |
| Cursor service | `agent.v1.AgentService`: bidirectional `Run` and unary `GetUsableModels`. The `v1` is part of the service namespace. |
| Connect transport | `Connect-Protocol-Version: 1`. Run uses framed `application/connect+proto`; model discovery uses unframed `application/proto`. |
| Protobuf codec | The limited handwritten encoder/decoder in [proto.go](../internal/format/proto.go), using the adapter's mapped fields. There is no Go protobuf APIv1 or APIv2 dependency. |
The [Connect specification](https://connectrpc.com/docs/protocol/) defines its
version header separately from protobuf. Likewise, Go's
[APIv1/APIv2 distinction](https://go.dev/blog/protobuf-apiv2) and Buf's
[JavaScript runtime v2](https://buf.build/blog/protobuf-es-v2) are library versions,
not instructions to change `agent.v1` to `agent.v2` or protobuf syntax to proto2.
The small codec avoids a generated-runtime dependency, but requires maintaining
explicit field mappings and does not promise general protobuf conformance.
A read-only check on 2026-09-06 of the installed Cursor CLI bundle
`2026.08.11-e8db854` found `agent.v1` descriptors with proto3 semantics and
`@bufbuild/protobuf` **1.10.1**, not v2. This is evidence about that inspected
client artifact, not a claim about every client release or the current server.
No live login or inference was used for this check.
## First-login helpers
[grok-login.sh](../scripts/grok-login.sh) and
[cursor-login.sh](../scripts/cursor-login.sh) obtain initial account credentials
through interactive browser approval. Run them on a trusted Linux host or
workstation, not inside the minimal proxy image. Neither a running millivolt
instance nor the provider's CLI/desktop application is required by these scripts.
Account eligibility, permitted integrations and model access remain the provider's
decision; successful authentication does not establish subscription entitlement
or a free inference allowance.
The scripts do not write files or update client configuration. They print secret
credential headers to standard output, and approval links/progress to standard
error. Keep the terminal private: do not use shell tracing (`bash -x`), CI logs,
screenshots or shared output redirection. Review a downloaded script before
running it. These helpers have no command-line option parser; configure them
with the environment variables below, not `--help` or invented flags.
### Prerequisites and obtaining the scripts
Both need Bash, `curl`, `jq` and standard Linux command-line utilities. Cursor
also needs `openssl` and a curl build with HTTP/2 support; check that
`curl --version` lists `HTTP2` in Features, as described in the
[curl HTTP/2 guide](https://everything.curl.dev/http/versions/http2.html).
`xdg-open` is optional: without it, open
the printed approval URL yourself, including from another browser-capable device.
Source checkouts already contain both helpers under `scripts/`. Image-only
deployments can download just the desired helper on the host, without cloning
or rebuilding millivolt:
```sh
curl -fsSL https://raw.githubusercontent.com/LLM-4-People/millivolt/main/scripts/grok-login.sh -o grok-login.sh
curl -fsSL https://raw.githubusercontent.com/LLM-4-People/millivolt/main/scripts/cursor-login.sh -o cursor-login.sh
```
Use a new directory so downloads do not overwrite existing files. For a pinned
installation, replace `main` in those URLs with the reviewed commit or release
tag. Standalone downloads use `bash ./grok-login.sh` or `bash ./cursor-login.sh`
instead of the checkout paths below. Do not pipe a download directly into Bash.
### Grok login
From a source checkout:
```sh
bash scripts/grok-login.sh
```
Open the printed verification URL, check the account and approve the displayed
device code. The helper polls for approval and prints `Authorization: Bearer`
with the access token, `X-Proxy-Refresh-Token` when supplied by the provider, and
`X-Proxy-Base-URL: https://api.x.ai/v1`. This is an account-login token flow, not
creation of a pay-as-you-go `xai-` API key. It does not select a model or add a
native-format header; the ordinary OpenAI-compatible relay is used.
For SSH or a headless terminal, suppress automatic browser opening:
```sh
NO_OPEN=1 bash scripts/grok-login.sh
```
### Cursor login
From a source checkout:
```sh
bash scripts/cursor-login.sh
```
Open the printed login link and approve the account sign-in. The helper creates
a PKCE challenge, requests HTTP/2 when polling the account endpoint, and prints the access
token, optional refresh token, upstream base URL and `X-Proxy-Format: cursor`.
Use the printed format header to select the [Cursor bridge](#cursor-connect-bridge);
a provider profile by itself does not enable translation.
The same headless switch is supported:
```sh
NO_OPEN=1 bash scripts/cursor-login.sh
```
### Configure the client after approval
Keep the client's API base pointed at **millivolt**, such as
`http://127.0.0.1:8080/v1`, using your actual host/port or protected ingress URL.
The helper's `X-Proxy-Base-URL` instead identifies the upstream; do not replace
the client's millivolt URL with it.
| Printed value | Client configuration |
| --- | --- |
| Access token after `Authorization: Bearer ` | Put only the token in an SDK's API-key field, or use the complete printed Authorization header if configuring headers directly. Do not add `Bearer` twice. |
| `X-Proxy-Base-URL` | Add as a custom request header. |
| `X-Proxy-Refresh-Token`, when present | Add as a custom request header if using [automatic refresh](#token-refresh); the client must retain rotated credentials returned by the proxy. |
| `X-Proxy-Format: cursor` | Add for the Cursor helper only. |
Select a model your account is authorized to use. Keep tokens in private client
configuration or a secret store, not `proxy.example.yaml` or a Git-tracked file.
Millivolt does not register these credentials globally. If your ingress consumes
Authorization, follow the [separate upstream-key setup](reverse-proxy.md#credentials-and-access-control).
### Helper environment overrides
These affect the login process only, not server Settings or the saved client.
Use only origins and OAuth registrations you trust: they receive login material.
Defaults remain owned by each script.
| Variable | Helper | Meaning and default |
| --- | --- | --- |
| `NO_OPEN` | Both | Any nonempty value suppresses `xdg-open`; unset or empty permits it. The URL is still printed. |
| `GROK_LOGIN_TIMEOUT` | Grok | Approval-poll budget in seconds; `600`. Use a positive integer. |
| `XAI_AUTH_BASE` | Grok | Login origin; `https://auth.x.ai`. Does not change the printed inference base. |
| `XAI_OAUTH_CLIENT_ID` | Grok | OAuth registration; defaults to the client ID shared with the proxy's xAI refresh implementation. |
| `CURSOR_LOGIN_TIMEOUT` | Cursor | Approval-poll budget in seconds; `300`. Use a positive integer. |
| `CURSOR_API_BASE` | Cursor | Poll origin and printed inference base; `https://api2.cursor.sh`. |
| `CURSOR_LOGIN_BASE` | Cursor | Browser login origin; `https://www.cursor.com`. |
Timeouts are polling budgets, not strict end-to-end deadlines: a current HTTP
operation or sleep can extend elapsed time. Changing Grok's auth origin/client
ID does not change millivolt's xAI refresh endpoint/registration, so a custom login
can be incompatible with renewal. Cursor refresh targets the request's base URL
and still requires a recognized refresh mechanism. These overrides do not
establish support for arbitrary identity providers or compatible service clones.
### Login troubleshooting
| Symptom | What to check |
| --- | --- |
| Required tool not found | Install the named dependency on the host running the helper, not in the proxy image. |
| No browser opens | Open the printed URL manually. `NO_OPEN=1` deliberately disables automatic opening. |
| Cursor reports unsupported HTTP/2 or keeps waiting | Check `curl --version` for HTTP2 support and inspect its diagnostics. The helper keeps polling non-200 responses until its budget expires. |
| Approval times out, expires or is denied | Verify the account/browser approval, then rerun for a fresh login challenge. Never reuse an expired code. |
| No refresh token is printed | The helper warns and omits that header. Millivolt cannot renew without it; obtain another login when the access token expires. |
| Login succeeds but inference returns 401/403 | Check client headers, upstream base, account permissions and model availability. Login alone does not prove inference access. |
| Grok or Cursor repeatedly returns `token_expired` | The client must adopt the returned access token and any rotated refresh token, then retry. Repeating the old credentials does not complete handback. |
| Refresh is rejected | Recheck provider access and perform a new interactive login. Do not publish token responses or credential headers for troubleshooting. |
## Token refresh
With `auto_token_refresh` enabled, a request may supply its own
`X-Proxy-Refresh-Token`. The proxy can exchange an expiring JWT for supported
mechanisms; it stores neither a credential database nor reusable refresh state.
The JWT expiry check is not signature authentication.
**Grok and Cursor use the same inference handback behavior.** A successful
exchange returns HTTP 401 with error type `authentication_error` and code
`token_expired`. The error object includes `access_token` and, when the
exchange returned a different refresh token, `refresh_token`. The same
values are in the error message. No inference is sent or recorded for that
handback. The client stores the returned access token, adopts any returned
refresh token, and retries the original request.
Starting with **0.2.0**, Cursor no longer refreshes in place on inference requests.
Clients that previously relied on that behavior must now handle the same 401
handback as Grok. Starting with **0.3.1**, credentials are in that error body,
not `X-Proxy-Access-Token` / `X-Proxy-Refresh-Token` response headers. Update
client handling before upgrading; a generic client that only reads those
headers cannot complete automatic renewal. Retry only after adopting the
fresh access token, not by blindly replaying an old key.
Model discovery refreshes transparently instead of requiring the handback
workflow, but does not return updated tokens to the client.
Missing refresh tokens, unknown mechanisms, non-JWT keys or disabled
refresh leave ordinary forwarding unchanged. Exchange failure falls back to the
original upstream credential, allowing the provider to decide authentication.
Use the [first-login helpers](#first-login-helpers) to obtain initial credentials.
They are not proxy startup dependencies. Treat terminal output and returned tokens
as secrets; never paste them into issues or commit them. Adopt a returned
refresh token when present; if a successful exchange omits it, including when
the provider echoed the same refresh token, retain the existing refresh token
rather than replacing it with an empty value.
The implementation lives in [tokenrefresh.go](../internal/proxy/tokenrefresh.go).
Provider/account behavior can change; historic successful observations are not
compatibility or authorization guarantees.
---
# MCP server
The proxy's own `/mcp` endpoint exposes millivolt's operator and
observability API to an LLM, using the official
[Model Context Protocol Go SDK](https://github.com/modelcontextprotocol/go-sdk).
It is a client of a running proxy: it never starts, stops or restarts the proxy
process. Its operator tools can reconfigure a running proxy, but only through
the same credential and the same routes the dashboard uses.
The endpoint serves 25 tools from one implementation. It is operator-gated
and Bearer-only, enabled by default: `mcp_enabled: false` removes it from the
process (the route stays reserved, answering 404 and never inference; restart
to apply), and the limits are config fields in the `mcp` category. It
forwards the caller's own `Authorization: Bearer` credential on its internal
calls, so the dashboard session cookie is refused there.
The token length band matches the proxy's own boot check (16 to 512
bytes). A credential the proxy would refuse to arm with is not a
credential this server can use.
Redirects are refused outright. No operator route redirects, and `net/http` strips
`Authorization` only when the hostname changes, so a same-host different-port or
subdomain `307`/`308` would replay both the credential and the full request body
to the redirect target.
## URL placement
A URL-capable client (an editor agent, a hosted connector, an MCP-capable CLI)
needs no binary and no environment: it points at the proxy's `/mcp` endpoint
and presents the operator credential in an `Authorization` header, the same
credential the dashboard uses.
```json
{
"mcpServers": {
"millivolt": {
"url": "http://127.0.0.1:8080/mcp",
"headers": {
"Authorization": "Bearer the same value the proxy was started with"
}
}
}
}
```
A remote client uses the reverse-proxied HTTPS origin from
[the reverse-proxy guide](reverse-proxy.md), with `/mcp` appended:
```json
{
"mcpServers": {
"millivolt": {
"url": "https://your-hostname/mcp",
"headers": {
"Authorization": "Bearer the same value the proxy was started with"
}
}
}
}
```
The header is the whole setup: the endpoint reads no environment, starts no
process, and no session cookie or per-request state outlives the request. Every
request is independently authenticated, so a missing or wrong Bearer is denied
before any tool runs, and a request presenting only the dashboard session
cookie is refused with 403, because the caller's own credential is what the
internal calls carry.
`/mcp/` look-alikes are reserved: an unregistered path
answers 404 and is never forwarded to inference. The published Compose port is
loopback-only, so it is not a remote origin; remote clients terminate TLS at
the ingress as in the [reverse-proxy guide](reverse-proxy.md).
The transport is stateless and request/response only, and its limits are part
of choosing a client:
- `GET` and `DELETE` answer `405` with `Allow: POST`. There is no standalone
SSE stream and no session to delete.
- There is no session id: the endpoint never sends `Mcp-Session-Id`, and no
per-request state outlives the request. No session cookie is minted here
either, unlike `/admin`, `/session`, `/metrics` and `/dash`, so an MCP client's
cookie jar gains no dashboard credential; a client must not expect or send a
session id. The admitted credential does key an in-process cache of the SDK
server object, so repeated calls do not rebuild all 25 tool registrations;
admitting a different credential replaces it. The credential is only the
in-memory cache key: it is never logged or persisted.
- SSE resumption is not offered: no event store exists, so `Last-Event-ID`
replay is not available.
- The server never sends requests to the client: there is no sampling and no
elicitation, so a client that waits for a server-initiated request waits
forever.
- Every request must accept both types, for example
`Accept: application/json, text/event-stream`. A wildcard that covers both
(`*/*`, `application/*, text/*`) is accepted too; an `Accept` missing either
type, or one absent altogether, answers `400`.
- The accepted protocol versions are the bundled SDK's full set, newest first:
`2026-07-28`, `2025-11-25`, `2025-06-18`, `2025-03-26` and `2024-11-05`.
This build does not narrow them. A classic `initialize` is answered with the
requested version when it is one of the legacy versions (`2025-11-25` and
older, verified live) and with `2025-11-25` for any other requested version;
`2026-07-28` is served only through the SDK's per-request metadata form:
the request carries `Mcp-Protocol-Version: 2026-07-28`, matching `_meta` keys
`io.modelcontextprotocol/protocolVersion` and
`io.modelcontextprotocol/clientCapabilities`
(`io.modelcontextprotocol/clientInfo` is optional), and the `Mcp-Method`
header. `tools/call` additionally requires an `Mcp-Name` header equal to
`params.name`; a form missing any required piece answers `400`.
- The SDK's localhost DNS-rebinding check is deliberately off. It rejects a
loopback connection whose `Host` is not loopback, and the documented
reverse-proxied placement preserves the external `Host` while dialing the
proxy over loopback, so leaving it on would reject that placement. The
endpoint is Bearer-only and the operator gate re-validates the credential on
every request, which is the stronger check for an authenticated endpoint.
## Tools
`describe` is the one to call first for this instance's live schema; without
it, correct SQL is guesswork. `docs_list` is the one to call first for how
millivolt itself works: it indexes the same Markdown as the published site and
needs no proxy request.
The three docs tools read `internal/docs/docs.json`, which is generated from the
same Markdown as the published site by `scripts/build_site.py`. Refresh the
embedded copy with `python3 scripts/build_site.py --output site --write-manifest`
after any page, title or navigation change. The tools never call the proxy,
read storage, or use the operator credential. A full page is capped at 256 KiB
by `internal/mcp/tools_docs.go`.
| Tool | Purpose |
| --- | --- |
| `describe` | Live SQLite schema, the complete `requests`/`request_debug`/`meta` column reference, the exact health predicates, a value vocabulary for every filter dimension, the live known client/provider/model and tool names, the indexes, the complete result-limit set (the query row default and explicit `max_rows` ceiling, query bytes, log page band, default page size, explorer group and chart bucket caps, Prometheus line cap and capture byte budget), and the working SQL idioms for this build. |
| `docs_list` | Every published documentation page in navigation order, with its path, title, description, public URL, Markdown URL and section count. Starts the docs flow; local only. |
| `docs_search` | Ranked section search across titles, headings and body text, with path, anchor, heading context and a snippet. Every query word must match; the result reports its total and truncation. Local only. |
| `docs_read` | One page by path, or one anchored section by heading id. Full-page Markdown is capped at 256 KiB with an explicit marker. This is the canonical explanation of millivolt's routes, configuration, protocol and operator behavior. Local only. |
| `query` | One bounded `SELECT`, clamped on both rows and encoded size, truncating with an explicit marker. The only tool that can express a time range. |
| `values` | The distinct values one dimension has in durable history, most frequent first, so a filter value is discovered rather than guessed. |
| `explore` | Faceted breakdown over all matching history by one dimension, with at most 24 groups. No time window. |
| `chart` | Server-authoritative time series, at most 31 clock-aligned buckets. Cannot group. |
| `records` | One durable newest-first log page, with the paired `before_ms`/`before_id` cursor and the page's `model_canon` map. |
| `snapshot` | The dashboard's live snapshot: sequence, feed, counters, storage signal, recent records. With durable storage enabled, a full snapshot is capped at `2 * dash_log_rows`; with storage off the ring is the only history and the snapshot is not capped. |
| `prometheus` | The text exposition, which covers the ring rather than durable since-inception totals. |
| `audit_status` | Capture sessions, their scope and live capture counts, the scope vocabularies, and the storage prerequisite. |
| `audit_start` | Start or edit one capture session. Requires `confirm: "start capture"` and refuses when durable storage is off. |
| `audit_stop` | Stop one session by id, or every session with `stop_all: "stop every capture session"`. |
| `audit_captures_list` | Page the requests that have a stored capture, through a bounded `SELECT` over `requests WHERE debug = 1` whose `request_debug` sidecar exists and has not expired. Ids are canonical: empty and whitespace-padded record ids are excluded because `audit_capture_get` could not read them back. |
| `audit_capture_get` | Read one capture document by request record id. |
| `operator_state` | Pause, limits, quota/storm and restart status in one call. |
| `set_pause` | Create, edit or resume pause holds. |
| `set_throttle` | Set, merge or clear one provider's caps, including the optional anti-burst pair. |
| `resume_quota` | Close one provider's open retry-mode quota gate. |
| `set_config` | Revision-checked partial configuration patch. |
| `config_get` | The live configuration document: values, running overrides (the keys the process pins outside the file, which `set_config` strips), the canonical usage and model metadata field names, defaults, per-key schema, restart-required set and the revision a patch must echo. Credential-class provider headers and last-reload error text are redacted; use the operator CLI with `--reveal` for the raw bytes. |
| `reload_config` | Re-read the configuration file. |
| `purge_preview` | Count what a filter would delete, without deleting. |
| `purge` | Delete matching history permanently. |
The routes are a closed set owned beside the HTTP client, so no tool can name
an arbitrary path: nothing reaches the proxy's transparent inference
catch-all, and no restore-adjacent route is exposed. The HTTP endpoint
dispatches those same calls in process to the proxy's own gated handler, so
they never leave the process and the gate re-validates the forwarded Bearer.
`set_throttle` takes `provider` and any of `concurrency`,
`requests`/`request_window`, `tokens`/`token_window`, `anti_burst` and
`ramp_duration`; omitted dimensions keep their value.
A count needs its own
window only when the policy stores none for that dimension, a count of `0`
clears the dimension with its window, and a window without a count is refused
unless a count is already stored. `clear: true` drops the whole policy and
cannot be combined with a dimension. The proxy owns
the duration grammar and band and refuses an explicit on-ramp without an
active limit.
This tool addresses one provider and has no selector or schedule
fields. Scoped selectors and recurring schedules are available through the HTTP
API (`/admin/throttle`) and the dashboard console, not through MCP.
`set_pause` refuses a resume that carries a scope, and refuses `all` combined
with `new`, `clients` or `providers`. `config_get` and the `set_config` echo
redact credential-class provider headers and last-reload error text; the CLI's
`--reveal` flag is the explicit operator opt-in. A redacted document posted
back through `set_config` keeps the loaded secret: the marker is restored at the
write owner, so a read-then-repost round trip cannot erase a credential.
### Filters
`explore`, `chart`, `records` and the explorer facets share one grammar, owned in
one place. Filters are `dim:id`, repeated, split on the FIRST colon, AND-ed
across dimensions and OR-ed within one dimension. `dim` is one of `client`,
`provider`, `model`, `conversation`, `key`, `status`, `time`, `tool`, `error`.
`key` is the SHA-256 hash of the client key.
`status` is a single selector: an exact HTTP status code (`0` to `9999`) or a
live in-flight class (`streaming`, `paused`, `throttled`, `pending`). It is not
the explorer's status-class vocabulary, which travels as `status:2xx`.
Two bounds apply to `f=` itself: at most 64 filters, and each `dim:id` value at
most 2048 bytes. The list is refused whole when either is exceeded; no filter
is silently dropped.
An unknown dimension, malformed filter or unknown status is refused before the
request is sent.
`explore`'s `error_total` uses two units. With `dim: error` it is the union of
distinct affected requests represented by the error cards, including recovered
quality failures whose final status is 200; for every other dimension it is the
canonical error count. A card share is `N / error_total` and cannot exceed 100%.
Do not compare it across the two cases as one metric.
### Pagination
`records` and `audit_captures_list` page newest-first with a paired
`(before_ms, before_id)` cursor. Both fields or neither.
The proxy scans a bounded number of rows per call, so a page can be short or
empty while its `more` flag is still true. `exhausted` is true when a page that
was given a cursor returns that same cursor, and also when `more` is false:
treat it as "stop paging".
A first page has no cursor to repeat, so any returned
cursor counts as progress, including `(0, "")`. A non-advancing cursor means the
scan budget cannot reach the next row, and treating it as "keep going" is an
endless loop; an empty page with an advanced cursor is instead the normal shape
of "the budget ran out before a match", so page again.
Deep history needs this rather than `OFFSET`, which degrades on every page.
### Volume
A model cannot read ten thousand rows. Every listing defaults to a small page,
and every tool truncates with an explicit marker (`truncation.marker`) naming
what was withheld and the way to continue for that tool, because only some of
them have a cursor.
What is *not* claimed: the explorer caps groups at 24 and the chart caps buckets
at 31 SERVER-SIDE, so those drops happen inside the proxy and those tools'
`truncation.truncated` is `false` when groups were dropped. A model that trusts
that flag alone will read a short breakdown as a complete one.
### What each tool cannot do
These are the limits that otherwise send a model down the wrong path. They are
also in the tool descriptions, which is where a model reads them.
- `describe` first. Without the live schema, the column meanings, the health
predicates and the value vocabularies, correct SQL is guesswork.
- **`explore` has no time window.** It always folds all history. It is also
single-dimension: it cannot cross-tabulate provider by model. For either, use
`query` with a `WHERE` on `started_at` and a `GROUP BY`.
- **`chart` is the only windowed tool, and it cannot group.** A window broken
down by a dimension is a `query`.
- **`query` is the only tool that can express a time range.** There is no
`unixnow()` in this SQLite build, so a relative window is
`strftime('%s','now')*1000`, for example
`WHERE started_at >= strftime('%s','now')*1000 - 3600000` for the last hour.
Ready-to-run analysis SQL for exactly these time-ranged questions is
collected in the [operator recipes](routes.md#operator-recipes).
- An **unknown filter value is an empty result, not an error.** A misspelled
client, provider, model, tool, conversation or key produces a confident zero.
Call `values` before filtering on one of those. Model filters take the
**canonical** name the proxy folds from `model_rules` (which is what the
`values` tool returns), never a raw stored spelling.
- **`values(model)` is paged and folded.** The tool reads raw spellings in
bounded keyset pages, so a vocabulary larger than the proxy's
`storage_query_max_rows` no longer fails with 413, then folds and merges the
pages through `model_rules`. Past its documented raw-spelling,
canonical-name, or total page-query bound it fails explicitly rather than
returning a partial vocabulary. The ordinary `limit` still reports its own
explicit truncation marker.
- `time` is a **daypart bucket**, not a duration: `time:24h` matches nothing. The
buckets are `night`, `work`, `evening` and `weekend`.
- `status` takes the proxy's **status classes**, not HTTP shorthand: `2xx`,
`cancel` (499), `4xx`, `5xx` and `err` (anything outside those, including
3xx and below 200). There is no `3xx` class of its own.
## Audit semantics
Audit is the proxy's debug-capture subsystem.
**Capture requires durable storage.** With `db_path` empty, `audit_start` is
refused by the proxy with 503 `durable storage is disabled` and creates no
session; every capture read-back is a 404 because no capture document can
exist. `audit_start` and `audit_captures_list` therefore check the storage
signal first and refuse with that explanation, and `audit_status` reports it.
Starting a session that can never capture would be a silent no-op, which is
worse than a refusal.
**Scope is explicit.** An empty dimension matches anything; non-empty dimensions
are AND-ed and names within one dimension are OR-ed. There is no unrestricted
all-traffic scope, and there is no conversation dimension. Model names use the
proxy's native model normalization, so use the `known_models` values rather than
guessing spellings. A scope overlapping a live session is rejected (`409`)
rather than silently replacing it.
**Edits change only named dimensions.** With a session id, an omitted
`clients`, `providers` or `models` field preserves that dimension and an
explicitly empty array clears it. An omitted `duration` preserves the deadline
and an explicit empty string clears it.
A create response also carries
`created_id`, the id its own request installed, so concurrent starts resolve
their own sessions. `set_pause` follows the same presence rules for scope and
duration when `hold_id` targets an edit; without `hold_id`, an omitted create
scope is the documented global hold.
**Starting requires a confirmation.** `audit_start` takes
`confirm: "start capture"` because capture stores full request and response
bodies.
**Stopping does not delete, but a purge does.** Stopping a session, or all of
them, ends future captures only: a stopped session's documents stay readable by
record id until `debug_capture_ttl` expires. There is no capture-specific delete
endpoint, but a `purge` deletes the stored capture document of every request it
matches, in the same transaction as the request row. A filter that reaches a
captured request takes its evidence with it. Only a backup taken beforehand
restores it.
**A scope name must be known when the vocabulary is.** `audit_start` checks
every client, provider and model against the vocabulary `audit_status` reports.
When that vocabulary is non-empty, an unknown name is refused with the known
values offered, because a session scoped to a name that matches nothing would
report `enabled: true` and capture nothing.
An empty vocabulary is the
exception: a proxy that has seen nothing knows no names, so refusing every name
would make the tool unusable rather than safer and the check is skipped. A
vocabulary that cannot be read is not a skip: the start is refused, because the
guard cannot be applied at all.
**Read-back is by record id.** `audit_capture_get` takes the REQUEST RECORD id
(`requests.id`), not the session id. There is no listing endpoint, so
`audit_captures_list` discovers those ids by querying `requests WHERE debug = 1`
with an existing, unexpired `request_debug` sidecar and reports that it does so.
The listing also excludes an id that is empty or padded with whitespace: the
capture route trims the id before lookup while the sidecar key is stored
verbatim, so such an id could be advertised but never fetched. Every listed id
is therefore readable.
Captured documents are sensitive: credentials are
redacted from headers, but body content is not sanitized.
**Sessions survive a restart.** The proxy restores them from its own state, so a
start list may already contain sessions this server did not create.
## The destructive call
`purge` permanently deletes matching request history. The proxy has NO
server-side confirmation, NO confirm flag and NO two-step protocol. The operator
credential and this tool's own guards are the entire gate.
Three guards apply together, and each alone would be insufficient:
1. **The filter must constrain at least one field.** The bodyless request that
deletes ALL history is never sent by this tool; an empty filter is refused
before any request.
2. **The confirmation must be an exact phrase.**
`confirmation: "permanently delete the matching millivolt history"`. It is
also a required tool argument, so a call that omits it is rejected by the
schema before the handler runs.
3. **`preview_token` must be the token `purge_preview` issued for THIS filter.**
It is an opaque HMAC-bound token over the canonicalized filter, the count it
authorized and an expiry, so it cannot be forged, cannot be reused for another
filter, and expires **15 minutes after issue**. A bare count could not do this: it is only a number, so
two different filters matching the same number of rows authorized each other,
and a count obtained from anywhere - a `query`, a guess, a preview of a
different filter - satisfied the comparison. That was a live bypass, not a
theoretical one.
Always run `purge_preview` first and pass its `preview_token` back verbatim.
### What the guard does not prevent
Stated plainly, because an operator needs to know the size of the remaining
window:
- **The count compares a population, not row identities.** The count is
re-checked immediately before deleting, and the reviewed count travels with
the delete, where the proxy counts it again inside the delete transaction and
refuses with 409 when it moved. Nothing un-previewed is deleted by a refused
call, but two different populations with the same row count are
indistinguishable, and a refusal means previewing again.
- **A filter that reaches a captured request deletes its capture document too.**
The preview and the deletion both say so in their output.
Deleted history cannot be recovered from this API. The deletion preserves newer
completions and in-flight work and does not reset live error-storm protection.
`set_config` is the other consequential call, and it is a revision-checked
partial patch: `values` carries only the keys to change, the whole result is
validated before the file is written, CLI-overridden keys are stripped, and the
response names the keys that need a restart.
A supplied list or map replaces
that whole field, so call `config_get` first for structured keys such as
`providers` or `model_rules`, and echo the revision it returns.
A failed reload is reported as a save, not as an error. The proxy writes the file
and only then answers 500 with the reload failure in the body, so that response
is decoded as the committed document it is: `saved` is `true` and `error` carries
the reload message. Reporting it as a tool error would tell a model that a
mutation that succeeded had failed, and its retry would then fail with 409.
`reload_config` re-reads the configuration file. It is the one state-changing
route the dashboard never calls. The operator token itself is process-bound and
is not re-read.
## Errors
Every failure is a tool result the model can read and correct, carrying the
proxy's own `.error` text and the status: `401` (credential wrong or missing),
`403` (the proxy's operator plane is unarmed, or the request carried only the
dashboard session cookie), `404`, `409` (overlap or stale
revision), `413` (a result limit), and `429` with its `Retry-After`.
An
unreachable proxy, a timeout, and a body that is not the expected document are
reported the same way. The `.error` field is read regardless of `Content-Type`,
because the operator routes are inconsistent and some answer `text/plain` with
a JSON body.
The returned text is scrubbed of the operator credential first, because a
reflecting endpoint could otherwise put the `Authorization` value into its own
failure message and hand it to the model.
The scrub removes the literal token,
any contiguous 8-byte fragment of it (so a credential split across two JSON
string fields leaks no usable piece), percent escapes in any hex case
(including full-byte nested forms), the query `+` form of a space, and
JSON-escaped forms, including backslash-doubled text and surrogate pairs.
Each
form is decoded to a fixed point before matching: an encoding is either fully
seen or the excerpt fails closed. Text that still decodes after three escape
applications is not published partly decoded: the whole excerpt is replaced by
`[redacted]` instead.
That bound applies regardless of whether the credential
appears in the body: a heavily re-encoded failure with no credential in it
still loses its diagnostics, because a credential could be hidden past the
bound. The scan is bounded to the region that can still reach the excerpt, and
the raw bytes it reads are capped; a body whose whitespace collapse does not
reach the excerpt target inside that cap fails closed to `[redacted]` too, for
the same reason.
The excerpt itself is whitespace-collapsed and truncated to
4 KiB before the ` [truncated]` marker is appended, so the model-visible text
runs up to the 12-byte marker past 4 KiB. Redaction runs before the truncation,
so a credential straddling the boundary cannot survive as a fragment.
## Try it against a disposable instance
Run the endpoint locally against a disposable instance with the repository's
lifecycle owner, never the main instance on `:8080`, and drive it with one
stateless POST:
```sh
(
set -e
trap 'scripts/dev.sh stop' EXIT
export MILLIVOLT_OPERATOR_TOKEN='dev-instance-credential'
scripts/dev.sh
curl -fsS -X POST http://127.0.0.1:8081/mcp \
-H "Authorization: Bearer $MILLIVOLT_OPERATOR_TOKEN" \
-H 'Accept: application/json, text/event-stream' \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"0"}}}'
)
```
The answer carries the server identity. A full client session is the client
configuration above pointed at `http://127.0.0.1:8081/mcp`; a request without
the Bearer credential answers 401 from the operator gate.
## Security
The endpoint is driven with an operator credential, which is the whole
dashboard plane: pause, limits, configuration, restart, purge. Treat the client
configuration that carries it as sensitive and keep it out of shared
repositories. See [Security](../SECURITY.md).
## Verification
The unit suite uses a fake proxy that pins the exact method, path, query and
`Authorization` header of every tool, maps canned responses onto typed output,
and covers the error shapes, setup validation, truncation markers, the pagination
rule and the purge guards.
`tests/_go/internal/docs` checks the embedded copy against every source hash.
`tests/python/site.py` checks the published Markdown twins, `llms.txt`,
`llms-full.txt` and `docs.json` against the same page model. The paging rule has one owner shared with
`audit_captures_list`, so the two listings cannot disagree. One of its tests
runs outside that harness on purpose:
- the redirect test stands up a second listener as the redirect target and
asserts it received nothing, so "we refused the redirect" is an observation
rather than an assertion about an error string.
- the capture listing test executes the generated `SELECT` against a real
SQLite schema with empty, space-padded, tab-padded and NBSP-padded sidecars,
and asserts they are excluded while every returned id has an unexpired
sidecar and is the exact spelling `audit_capture_get` can fetch.
This page is mechanically pinned by `tests/python/mcp_docs.py`: the tool table
must match the `addTool` registrations, every " tools" mention must match
that count, every route the client owns must appear in the
[routes and debugging route table](routes.md#operator-and-data-routes), and this page
may name no operator path the client does not own.
The live integration test starts a private instance through
[scripts/dev.sh](../scripts/dev.sh) on a port and scratch database of its own, so
concurrent runs do not share a port or database, drives the tools through a real MCP session,
and runs a full capture cycle against neutral loopback traffic:
```sh
MILLIVOLT_MCP_INTEGRATION=1 scripts/check.sh go test -count=1 ./internal/mcp/...
```
It runs in CI as part of the isolated browser-fixtures step, which already has a
dev instance and an operator token; it is not part of the core source-and-unit
job, because that job should not have to start a second proxy.
It drives every registered tool against its own disposable instance.
The
state-changing tools are `set_pause`, `set_throttle`, `resume_quota`,
`set_config`, `reload_config`, `audit_start`, `audit_stop` and `purge`.
The
run exercises `set_pause` (a hold scoped to the fixture client, resumed at
once), `set_throttle` (a limit set and then cleared), `resume_quota`,
`set_config` and `reload_config` against the scratch config copy, a full
capture cycle (`audit_start` and `audit_stop`), and the purge path: the
missing confirmation, the wrong phrase, an unauthorized token and a token
issued for a different filter each refuse with the row count unchanged, and
then an authorized purge deletes the fixture rows.
The read-only tools are
`describe`, `docs_list`, `docs_search`, `docs_read`, `query`, `explore`,
`chart`, `records`, `snapshot`, `prometheus`, `audit_status`,
`audit_captures_list`, `audit_capture_get`, `operator_state`, `config_get`,
`values` and `purge_preview`; the run drives each of them.
A
successful purge and the config write are safe here because the instance, its
database and its config copy all belong to the test and are discarded.
The same run drives the proxy's own `/mcp` endpoint with a real streamable
HTTP client: `initialize`, `tools/list` (25 tools), and `describe` and
`records` returning the same documents the in-process session returns, plus the
auth boundary (401 without a credential, 401 for a wrong Bearer from the shared
gate, 403 for a cookie-only request or a wrong Bearer beside an admitted
cookie) and the reserved `/mcp/` look-alike answering 404, never inference.
---
# Operator CLI (mvctl)
`mvctl` is the human command-line client for millivolt's operator plane. It
calls the same documented `/admin/*` and `/metrics/*` routes as the dashboard
and reads them through the shared MCP operator client, so the origin check,
credential handling, redirect refusal, response bound and credential redaction
have one owner. It adds no server route, no server setting and no second
listener.
`mvctl` is a sibling of `cmd/proxy`, `cmd/stress` and `cmd/release`. It is
source-only: the container image still ships the proxy, and deployment
lifecycle stays with Compose.
## Build and run
```sh
go run ./cmd/mvctl status --url http://127.0.0.1:8081
go build -o mvctl ./cmd/mvctl
./mvctl --url http://127.0.0.1:8081 status
```
The origin is required: pass `--url` or set `MILLIVOLT_URL`. There is no
production default, so a command cannot silently address `127.0.0.1:8080`. A
trailing slash is normalized away. Only `http` and `https` origins are
accepted; userinfo, a path, a query and a fragment are rejected.
## Authentication
`mvctl` reads the operator credential from `MILLIVOLT_OPERATOR_TOKEN`, the one
variable the proxy's boot gate reads. It never accepts the token on argv,
because a command-line value is visible in `ps` and shell history. The shared
client validates the token's length band before sending a request.
```sh
export MILLIVOLT_OPERATOR_TOKEN='the-value-the-proxy-started-with'
mvctl status
```
Every request presents `Authorization: Bearer`. No cookie jar is kept: a
session cookie minted by one call is deliberately not reused, so a rotated
token takes effect on the next command. A missing or wrong credential is
reported with the operator gate's own status. A non-loopback origin should use
`https`; the credential is otherwise sent in cleartext.
## Output and exit codes
Human output is the default. `--json` (or `-j`) prints one JSON document, and
command output is stable enough for a pipeline. Diagnostics go to stderr as
`mvctl: ...`.
After argument parsing, `--json` turns a command failure into
`{"error": "..."}` on stdout so a caller can parse one stream. A usage error
detected while parsing the arguments still goes to stderr and exits 2, because
JSON mode is not established yet; a usage error a command reports after a
successful parse follows `--json` like any other failure.
| Exit | Meaning |
| --- | --- |
| 0 | success, including a preview that made no change |
| 1 | transport, HTTP or proxy-reported failure |
| 2 | usage error, unknown flag or invalid input |
| 130 | interrupted during a request, stream or watch |
`status` and `overview` print an allowlisted state projection with `--json`:
request rows and the full effective config are omitted, so use `requests` and
`config get` for those. Request rows in that projection would carry previews,
response headers and retry metadata that the dedicated commands keep out.
ANSI color is off when stdout is not a terminal. `--color`, `--no-color` and
`NO_COLOR` override the default. Request labels, model ids, error types and
proxy error text are sanitized before a terminal sees them, so a captured value
cannot inject an escape sequence.
## Global flags
| Flag | Meaning |
| --- | --- |
| `--url URL` | proxy origin; required unless `MILLIVOLT_URL` is set |
| `--timeout DURATION` | bound one request, default the shipped `mcp_timeout`; 0 disables the deadline |
| `--json`, `-j` | emit one JSON document |
| `--yes` | confirm a state-changing command |
| `--color` / `--no-color` | override terminal color detection |
| `--help`, `-h` | command or global help |
| `--version` | print the mvctl version and source revision |
`--` ends flag parsing; when no command has been named yet, the next word may
still name one. Boolean flags accept `--flag`, `--flag=true`, `--flag=1`,
`--flag=yes`, `--flag=on`, and the false spellings `false`, `0`, `no`, `off`.
## Commands
### View
| Command | Purpose |
| --- | --- |
| `mvctl status` | health, origin, dashboard build and headline metrics |
| `mvctl overview` | KPI, storage, pause, throttle and debug state |
| `mvctl requests [N]` | one page of the request log, default the shipped `dash_log_rows`; `N` must be inside that setting's schema band |
| `mvctl query "SELECT ..."` | bounded SQL through `/metrics/query` |
| `mvctl chart` | server-authoritative time series; `--window` is all history or whole minutes, `--filter` scopes it |
| `mvctl prometheus` | the Prometheus text exposition |
| `mvctl watch` | sample live counters and print a sparkline |
| `mvctl health` | check `/healthz` |
| `mvctl url` | print the normalized origin |
`requests` accepts `--before-ms MS` and `--before-id ID` together to fetch the
next page; the id is an opaque keyset position and may be empty, so it is sent
exactly as given. The command prints the next cursor when the page has more
rows. Its
JSON rows intentionally omit captured content previews, response headers and
retry attempts. Use the dashboard capture drawer when full bodies are needed.
`query` accepts `--query-timeout DURATION`; the default is the shipped MCP
query bound, and an explicit value must be greater than zero. The shared HTTP
client carries that shipped bound on its query path too, which is the one
`chart` takes. The statement must be a single `SELECT` without a semicolon,
using the same pre-check as the MCP query tool. The SQL text is sent as one `q`
parameter and is never logged.
`chart` prints the server-authoritative series: bucket geometry, percentile and
stat arrays, blended cost per million tokens and the buckets themselves. Its
fold runs through the client's query path with `--query-timeout` as the one
bound, so `--timeout` (the generic per-request deadline) never cuts it short.
`--window` takes `all` or a whole number of minutes; `--filter` takes
comma-separated `dim:id` pairs using the same dimensions as the log scope
(`client`, `provider`, `model`, `conversation`, `key`, `status`, `time`, `tool`,
`error`). `key` is the SHA-256 hash of the client key. A chart fold covers
finalized requests only, so a live in-flight status class is refused.
`watch` accepts `--samples N`, bounded by the command's settings surface, and
`--interval DURATION` (default the shipped `dash_poll_interval`). Interrupting
any request or stream exits 130.
### Config
| Command | Purpose |
| --- | --- |
| `mvctl config get [KEY]` | show one setting or every saved setting |
| `mvctl config keys [SUBSTRING]` | list schema keys |
| `mvctl config diff` | compare saved file values with built-in defaults |
| `mvctl config set KEY=VALUE ... [--yes]` | revision-checked save |
| `mvctl config reload [--yes]` | re-read the config file |
`config set` reads `GET /admin/config` for the current revision and the schema.
The schema's `kind` decides coercion: `bool`, `int`, `duration`, `bytes` and
`string` take their natural text form; `strings`, `providers`, `aliases`,
`model_rules`, `request_overrides` and `sub_conversations` take a JSON value.
A numeric-looking string stays a string. An empty value is never a boolean. An
unknown key, an empty boolean or an undecodable composite value is a usage
error before any write.
The server writes the file and then answers 500 when the follow-up reload
fails. `mvctl` treats that committed document as a warning, not a failure, and
prints the reload error without inviting a retry that would fail with a stale
revision.
Credential-class provider header values are redacted from `config get`, `config
diff`, the `config set` preview and its post-save document, in human output and
`--json`, using the shared header-name vocabulary. `--reveal` prints them. A historical
`last_reload` error is replaced by the redaction marker because a rejected value
can be quoted there. Token counters, token windows and rate-limit values are
measurements, not credentials, and are never redacted.
### Pause
| Command | Purpose |
| --- | --- |
| `mvctl pause [show]` | list holds and queued requests |
| `mvctl pause on --client X --provider P --yes` | create a scoped hold |
| `mvctl pause on --model M --conversation C --yes` | narrow a hold to a model or conversation |
| `mvctl pause on --new --yes` | hold clients first seen after the hold |
| `mvctl pause on --all --yes` | hold every client and provider |
| `mvctl pause on --id ID [flags] --yes` | edit one hold in place |
| `mvctl pause off --id ID --yes` | remove one hold |
| `mvctl pause off --all --yes` | remove every hold |
`pause on` requires a scope: `--client`, `--provider`, `--model`,
`--key-hash`, `--conversation`, `--new` or `--all`, unless `--id` names an
existing hold to edit. Set dimensions intersect and an unset dimension matches
any value, exactly as they do for a limit. `--duration` and `--max-queued` are
optional.
`pause off` deliberately rejects scope flags: the proxy identifies a hold by
`id`, and a scope-looking command that cleared every hold would be unsafe. Use
exactly one of `--id` or `--all`. A resume carries
only the hold id because the proxy has no scope-selective resume.
### Throttle and quota
| Command | Purpose |
| --- | --- |
| `mvctl throttle [show]` | list provider limits |
| `mvctl throttle set --provider P [flags] [--yes]` | merge limit dimensions |
| `mvctl throttle clear --provider P [--yes]` | drop a provider limit |
| `mvctl quota [show]` | show storm and quota state |
| `mvctl quota resume --provider P [--yes]` | close a retry-mode quota gate |
`throttle set` flags are `--concurrency`, `--requests`, `--request-window`,
`--tokens`, `--token-window`, `--anti-burst` and `--ramp-duration`. Windows and
ramps take a duration such as `1m`.
A count needs its window unless the policy already stores one for that
dimension, and a count of `0` clears the dimension with its window; a window
without a count is refused unless a count is stored. At least one dimension is
required.
The set and clear commands are provider-only. Scoped selectors and recurring
schedules are available through the HTTP API (`/admin/throttle`) and the
dashboard console, not through this CLI.
### Queue and capture
| Command | Purpose |
| --- | --- |
| `mvctl drain --all [--yes]` | release every parked request |
| `mvctl drain --provider P [--yes]` | release one provider's parked requests |
| `mvctl drain --model M [--yes]` | release one model's parked requests |
| `mvctl drain --client C [--yes]` | release one client's parked requests |
| `mvctl debug [show]` | list capture sessions and their bounds |
| `mvctl debug on --client C --yes` | start a scoped capture |
| `mvctl debug off --id ID --yes` | stop one session |
| `mvctl debug off --all --yes` | stop every session |
Exactly one drain scope is required. `debug on` requires a scope unless `--id`
names an existing session to edit, and it has no unrestricted all-traffic form,
matching the MCP tool: a bare enable would capture every request body.
Captures are sensitive even after header redaction, and a capture create needs
durable storage; the proxy answers 503 without it.
`debug off` identifies a session by `id` or `--all`, never by a scope flag,
and the two are mutually exclusive. `debug on` refuses a scope name the proxy
has not seen, because such a session reports enabled but captures nothing.
### History, backup and restart
| Command | Purpose |
| --- | --- |
| `mvctl purge [filters] [--all] [--yes]` | count first, then delete |
| `mvctl backup --out FILE` | download an archive |
| `mvctl restore --file FILE [--yes]` | inspect by default, apply with `--yes` |
| `mvctl restart --yes` | request a graceful restart |
`purge` filters are `--client`, `--provider`, `--model`, `--conversation-id`,
`--error-type`, `--status`, `--before-ms`, `--after-ms`, `--has-error` and
`--debug`.
Every value must parse, and a zero or unparsable numeric filter is a usage
error rather than an omitted constraint. `--all` cannot be combined with a
filter.
The command always calls `/admin/purge/count` first and prints the count; only
`--yes` calls `/admin/purge`, and an unfiltered purge also needs `--all`.
Before deleting, the command re-counts and refuses when the total changed since
the preview, then sends that count with the delete so the proxy checks it once
more inside the delete transaction. That compares counts, not row identities,
so the guarantee is a refused moved population rather than a row-identity
binding. A missing, fractional, negative or non-finite count is a failure,
never zero. A genuinely bodyless purge is the proxy's delete-everything form.
`backup --out FILE` is required; there is no default path. `--config` and
`--database` default to true (the command's settings surface) and are always
sent explicitly, so "both off" cannot silently become the server's both-on
default.
The download streams to a private temporary file, checks the `MVB1` magic,
syncs the file, renames, and syncs the directory entry only on success.
An existing file is refused unless `--overwrite` is passed. A short HTTP body
is a failed transfer, not a published archive. Backup and restore are bounded
by cancellation, not by `--timeout`.
`restore` defaults to `inspect=1`, which validates the archive without applying
it; `--inspect` forces that mode even with `--yes`. `--yes` applies it.
The preview decodes and preflights the whole archived database member on the
server, so it costs about as much as the apply's decode; `--yes` alone skips
the preview and applies directly.
The member selection (`--config`, `--database`) and the mode (`--config-mode`,
`--database-mode`; `merge` or `replace`) are always sent explicitly, and both
the inspect and apply output name the selected members and modes.
The default mode is `replace`, the proxy's documented default, so the preview
names the broader action instead of hiding it.
An inspect document's archived provider headers are redacted; `--reveal`
prints them. The apply response's `restart_required` keys are called out in the
output.
## Scope and non-goals
`mvctl` does not install, start, stop or update containers. It does not read or
write `proxy.yaml` directly, does not add a listener, and does not keep a second
copy of the proxy's routes, config schema, credential rules or backup grammar.
Those facts stay with `internal/mcp`, `internal/config` and
`internal/backup`. The command uses only the Go standard library and those
repository packages.
Because the shared operator client lives in `internal/mcp`, the resulting
binary links the MCP Go SDK and its OAuth2 and jsonschema dependencies. The
backup grammar and archive magic live in the dependency-free
`internal/backup/params` leaf, so `mvctl` does not link SQLite or zstd and never
opens a database.
---
# Operations
> [!WARNING]
> Read the [security boundary](../SECURITY.md) before exposing the listener.
The whole embedded dashboard is protected: every dashboard, `/metrics/*`,
`/admin/*` and `/mcp` request requires the `MILLIVOLT_OPERATOR_TOKEN` credential
and is denied while it is not configured. At the application gate, only the
unauthenticated `/healthz` liveness and `/readyz` readiness probes, the
`POST /session` login handshake, origin-root brand/PWA files and transparent
inference stay open; see the canonical [operator access](routes.md#operator-access)
list.
## Overview
This page is the operating overview. It covers health and readiness, durable
storage and accounting, the runtime cost model, and the known limits. Each
task-shaped page carries the detail.
| If you want to | Start with |
| --- | --- |
| Change a setting, an override or a provider alias | [Configuration](configuration.md) |
| Install, back up, upgrade or restart the deployment | [Containers and data](containers.md) |
| Call an operator route, or debug traffic | [Routes and debugging](routes.md) |
| Protect the listener | [Security](../SECURITY.md), [reverse proxy](reverse-proxy.md) |
| Measure the deployment | [Performance and resource usage](performance.md) |
## Health and readiness
Liveness (`/healthz`) answers "the process is up and serving HTTP". Readiness
(`/readyz`) additionally answers "this instance can persist": it fails with
`503` and a reason when the durable writer is wedged.
Two arms answer that: the
oldest record on the queue has waited, or no flush has succeeded, for longer
than one flush interval plus a progress window. The window is four flush
intervals clamped to at least 30 seconds and at most two minutes; the record's
own interval is allowed on top, so a record inside its own cadence never
convicts and a long configured interval cannot postpone the verdict by more
than that one cadence.
Every clock starts on arrival: the oldest record on the
queue ages the first arm, and while no flush has succeeded since boot the first
accepted record ages the second one, so a writer that never picks records up is
convicted all the same. Readiness also fails when
any durable flush has failed since boot.
Readiness stays red after the first failed flush even once writes recover,
because a lost batch is worth an operator's attention; restart or investigate.
With durable storage disabled both routes answer `200`, since there is no
pipeline to wedge.
The Compose files use `-readycheck` as the service healthcheck, so
`docker compose ps` reports `healthy` only for an instance that can persist.
Point an external load balancer or orchestrator at `/readyz` for the same
signal; both probe routes are unauthenticated and answer the exact body
`{"ok":true}` on success.
Storage state is also scrapeable at `/metrics/prometheus` (behind the operator
credential) as `llm_proxy_storage_enabled`,
`llm_proxy_storage_queue_depth`, `llm_proxy_storage_queue_capacity`,
`llm_proxy_storage_dropped_total{cause}` (`overflow`, `after_close`, `panic`,
`flush_failed`), `llm_proxy_storage_flush_failed_total`,
`llm_proxy_storage_last_flush_timestamp_seconds` and
`llm_proxy_storage_wedged`. The block is rendered on every scrape, outside the
memoized request aggregate, so queue and drop movement shows up immediately.
Alert on `llm_proxy_storage_wedged`, on a rising `flush_failed` counter, and on
`overflow` drops that are not explained by a burst.
## Storage and accounting
SQLite retains finalized records; the ring and pending registry support live
observation. Full bootstrap windows are capped in durable mode and older log
rows page from storage. Without storage, the ring is the available history.
The durable log order owns one secondary index, `idx_requests_log(started_at,
id)`. The log page pushes exact provider, client, key-hash, conversation and
canonical numeric status predicates into SQL and rechecks them on that scan;
model, status-class, tool, error and time-bucket filters stay Go-owned because
they match canonical/derived values, not raw stored text.
The export and purge
filter surface is different: it accepts exact provider, model, client,
conversation, error type, status code, `has_error`, `debug`, and time bounds,
with no key-hash selector, and its equality-only preview count or purge scans
the table instead of maintaining per-dimension composite indexes.
`BenchmarkStoreDiskBytes` measures
this on its deterministic synthetic fixture (10,000 and 100,000 seeded
records). The current per-record and total bytes are published in
[storage and retained projection](performance.md#storage-and-retained-projection);
that is a reproducible fixture scope, not a prediction of a deployment's disk
use or a capacity ceiling.
Dropping a superseded request-log index only moves its pages to SQLite's
freelist, so the first startup that finds and removes one also runs a one-time
`VACUUM` after the migration. Fresh and already-migrated databases skip it.
A
`VACUUM` needs free temporary space up to roughly twice the database size; if
it fails for space, locking or interruption, startup logs a warning and
continues with every row intact, but the freed pages stay on the freelist.
Reclaim them manually during a maintenance window: stop the proxy, run
`sqlite3 /data/proxy.db 'VACUUM;'` (or any SQLite client) against `db_path`,
and confirm `PRAGMA freelist_count` returns 0. The automatic pass is never
retried on a later boot, so a failed attempt needs this manual step.
Boot migrations also own a small set of one-shot data repairs for shapes older
binaries could write but current writers never produce: an empty-string JSON
document in `attempts` or `tool_names` is rewritten to the canonical `null`,
and a positive `ttft_ms` with `final_attempt_at = 0` (the pre-0.3.34
start-anchored TTFT, recorded with upload, queueing and retry backoff
included) is reset to 0 because the original per-attempt anchor was never
stored and cannot be recovered.
Each repair has its own `meta` marker
(`attempts_repaired`, `tool_names_repaired`, `ttft_anchor_repaired`): the
marker is written after a clean probe or after the rewrite, later boots read
one row instead of rescanning the table, and deleting the marker (or restoring
a database that predates it) re-arms the probe.
A repair that cannot run fails
boot closed instead of serving history its own consumers would reject.
The
first boot after an upgrade that finds rows to rewrite may pause while it
scans and rewrites them, and the totals rebuild follows: the rewrite advances
the projection epoch, which invalidates the persisted totals document. The
rewrite can transiently grow the WAL, so keep free space on the database
volume as for any other large write.
Enqueue is bounded and nonblocking. Overflow or write failures can lose durable
records even when HTTP succeeds. Bootstrap's
`storage:{enabled,dropped,totals_degraded}` reports cumulative process-local
loss and a failed startup history scan. Either appears in the status footer;
degraded since-inception totals count only rows committed since process start.
Increasing queue size absorbs bursts, not arbitrary sustained overload.
Storage overload remains an unresolved capacity boundary, not a lossless mode.
Measure it with the [isolated stress workflow](../CONTRIBUTING.md#performance-evidence):
successful responses, exact durable accounting and process-local drops are
separate acceptance criteria. A short successful burst does not establish
sustainable writer throughput or an absolute request-capacity ceiling.
`storage_query_timeout` bounds interactive dashboard reads (chart, explorer,
log/bootstrap, capture reads) and the ad-hoc query route. A capture read waits
first for the in-flight sidecar save of the same request, bounded by the
drawer client's own request context, and only then starts the read budget, so a
slow save cannot spend the budget the read needs.
Startup projection
preload, the boot totals rebuild, an administrative purge, and a backup merge's
totals scan run under a separate internal ten-minute maintenance budget: they
legitimately read or rewrite full history and must not inherit the interactive
bound.
The maintenance budget still honors an earlier caller cancellation or deadline,
and its expiry fails the operation closed (a purge transaction rolls back). It is a
fixed implementation guardrail, not a configuration field.
One shared analytical projection retains compact data and sorted metric orders
proportional to stored history. Initial preload costs startup CPU/RAM; fast
dashboard reuse is not free startup or bounded total-history memory.
Concurrent independent proxy writers sharing a database are not a supported
coherent-KPI deployment.
Expired debug sidecars are hidden from reads immediately by their TTL. Physical
reclamation deletes bounded batches per writer tick and continues on a
near-term tick, so a large expired set drains without holding the sole writer
connection; the sidecar byte cap accounting is updated exactly for the rows
removed.
Scheduler groups and some observed-identity sets also retain historical labels.
Bounded request queues do not bound all process memory; long-running workloads
with many unique identities need their own memory/cardinality measurements.
From a source checkout with Python installed, use the existing online backup
into a new destination:
```sh
python3 scripts/backup_db.py proxy.db /path/to/new-backup.db
```
The helper opens the source read-only and includes committed WAL state. Do not
plain-copy an open database or omit its WAL. Protect backups, debug captures and
exports as sensitive data.
This helper backs up the database, not configuration. Operators can also
download a self-checked config and/or database archive from
[Settings backup and restore](containers.md#settings-backup-and-restore). For image-only
deployments and both persistent volumes, use
[container backup and restore](containers.md#image-only-backup-and-restore).
## Performance and footprint
Separate deployment size, cold startup, idle footprint, request overhead and
loaded throughput. They answer different questions; a small image is not a RAM
limit, and a cached dashboard read is not a cold-start measurement.
The runtime contains a stripped Go binary with embedded UI assets, dependency
notices and the generated configuration, on a minimal non-root base. There is
no frontend build/runtime dependency in deployment. Docker's reported image
size describes an image artifact, not compressed network transfer, persistent
volume growth or the process's resident memory.
On the request path, one bounded body read/metadata decode feeds shared routing
and accounting. Upstream connections are pooled; streamed observation and durable
enqueue avoid a per-request history scan or synchronous SQLite write. On the
dashboard path, the initial HTML embeds bootstrap state and shared analytical
projections/cache slots avoid building a separate history copy for each view.
These implementation choices reduce repeated work, not every cost of a request.
RAM still includes active request/response buffers, connection pools, the live
ring and pending state, and the history/cardinality costs documented under
[storage](#storage-and-accounting). Raising queue/body/pool limits or concurrency
changes that footprint. Retained history and observed identities have no universal
process-memory cap; configuration limits must be evaluated together.
The startup preload decodes every stored row once, so its transient allocations
(SQL strings, attempts and tool-name JSON) exceed the retained projection several
times over. The process returns those burst pages to the OS immediately after the
preload completes; idle RSS after readiness reflects retained state, not the
startup burst.
The projection's per-record retained footprint is measured
reproducibly by `BenchmarkProjectionFootprint` (retained and transient bytes per
record over a seeded scratch database) and `BenchmarkProjectionProcessRSS`
(process RSS after readiness), without a live instance; the
[performance workflow](../CONTRIBUTING.md#performance-evidence) owns the commands.
`go_memory_limit` is the operator's steady-state RSS lever: a soft cap on Go
runtime managed memory (heap, stacks, runtime structures) applied once at
startup.
With the setting at 0, an operator `GOMEMLIMIT` wins and otherwise a
detected finite cgroup memory ceiling of at least 80MiB (v2 `memory.max` or v1
`memory.limit_in_bytes`) becomes a soft limit at 80% of that ceiling so a
container's own bound actually reaches the Go runtime; smaller ceilings are
ignored and, with no `GOMEMLIMIT`, the runtime stays unbounded.
A set value
overrides `GOMEMLIMIT`, not `GOGC`, for the process lifetime. `GOGC` remains
independent.
The limit is soft - exceeding it makes the collector
more aggressive (capped near 50% of CPU time) instead of failing requests - and
bounds Go-managed memory only, never SQLite's page cache or OS overhead.
The [performance and resource usage](performance.md) page is the single source
for the current measured tables, hardware, method, reproduce commands and
limitations. This section owns the implementation cost model above and the
`go_memory_limit` behavior. Use the [isolated performance workflow](../CONTRIBUTING.md#performance-evidence)
when adding evidence. Short local-upstream bursts and microbenchmarks are useful
regressions, not sustained production capacity, provider-latency or lossless
storage guarantees.
## Known limits
- Recognized provider fields drive usage/cost; this is not independent billing.
- Non-streaming usage/cost inspection keeps a bounded prefix, so larger valid
responses can forward completely without complete accounting.
- Optional non-streaming translation reads the complete upstream 200 body and
the translated document under the configured `translated_response_max_bytes`
each (default 32 MiB). Exceeding either fails closed with 502
`response_too_large` before any client bytes; discovery budgets do not
bound the path.
- General multi-line SSE accounting/native event assembly remains incomplete;
some split terminal/content shapes can affect classification, not just metrics.
- SQL result limits do not bound every intermediate SQLite allocation.
- Automatically grouped sessions are not proof of agent ancestry. Explicit parent
declarations and exact-leaf filtering are described in [protocol](protocol.md).
- Existing values lost by older storage formats cannot be reconstructed.
- Extreme numeric data can produce explicit accounting/encoding errors rather
than invented saturated prices or token counts.
---
# Configuration
How the configuration surface is owned, found and edited: the generated example,
the settings schema, request overrides and sub-conversation tracking.
> [!WARNING]
> `provider_aliases` is a durable history rename, and a `request_overrides`
> header value is operator-supplied and stamped onto a request whose
> destination the caller chooses with `X-Proxy-Base-URL`. Back up before an
> intentional rename, and review both sections before enabling them.
## Configuration and CLI
[proxy.example.yaml](../proxy.example.yaml) is generated from the canonical
`config.Example()` and schema documentation. `Example()` starts with neutral
`config.Default()` values and adds the enabled
[Grok, OpenCode Zen and Cursor compatibility profiles](adapters.md#bundled-compatibility-profiles)
and the opencode sub-conversation tracking exemplar (the `promptCacheKey`
tracked param, strip on). It is not mutable repository state. Source
deployments copy it to ignored local `proxy.yaml`; images bundle their own copy
for fresh config volumes. Do not start Settings against the committed example.
The proxy's flags are listed once, with their registration defaults and usage
text, in the [settings reference](settings.md#proxy-cli-flags). The effective
`-listen` value is validated after every override, and a handoff child rewrites
`-pid-file` with its own pid on boot so a supervisor keeps tracking the live
process. The three output-only flags are mutually exclusive and exit without
loading local config, opening a database or starting runtime services.
Precedence is built-in defaults, then the YAML file, then CLI overrides.
A missing file loads defaults; verify the path rather than assuming a typo
created a new empty overlay.
On startup, unknown keys, unknown nested fields and
extra YAML documents are dropped and the file is rewritten without them. An
invalid recognized value fails startup without rewriting the file, and an
unparseable file is moved to `PATH.invalid` and replaced with the documented
defaults, except an ambiguous duplicate top-level key, which fails closed and
stays in place.
Because replacing it with defaults would silently widen the
bind and drop the origin allowlist, an unparseable file first has `listen` and
`allowed_base_urls` salvaged from it block by block and applied over the
defaults; both `(unparseable yaml)` and ` (salvaged)` are reported in the
boot log and `last_reload`.
A named key with no usable value, or a salvaged
value the config rejects, fails startup with the original file still in place.
Reload rejects an invalid document without rewriting it.
Settings and restore merge treat a missing file as defaults, while
an invalid or unreadable document fails closed and is reported instead of
substituting a default.
Healthcheck is stricter: a named file that cannot be
read fails the probe, and only an empty `-config` reaches the built-in
defaults. Backup capture is stricter: a missing config file is an
error because there is nothing to archive.
Settings POST still rejects invalid
inputs. CLI-overridden fields cannot be saved from Settings. Healthcheck loads
whatever file it is given read-only and never rewrites it.
To regenerate the public example and the settings reference after changing
their canonical owners:
```sh
go run ./cmd/proxy -print-example-config > proxy.example.yaml
go run ./cmd/docgen -write
```
The shared checks verify generated-file consistency. Built-in defaults remain
neutral; compatibility-profile values belong only in `config.Example()`, never
consumer fallbacks or private credentials. Optional mappings are mechanisms,
not provider registrations, for example:
```yaml
providers:
neutral.example:
cost_keys: [billing.charge_usd]
usage_keys:
input_tokens: prompt_toks
```
Use the derived provider label as the map key. The example file and Settings
schema document other maps, model rules and header templates; do not maintain
a second complete list of defaults here.
### Finding a setting
The generated [settings reference](settings.md) is the complete field list:
every setting with its type, default, accepted values and hot-reload or restart
behavior, grouped by the same categories as the Settings sheet.
[proxy.example.yaml](../proxy.example.yaml) remains the commented deployment
example.
`GET /admin/config` exposes the same field/category metadata alongside
saved values, neutral defaults, CLI overrides, the process-effective snapshot
and the current revision. Its `last_reload` section reports the most recent
configuration application: `ok`, `error` (empty when ok), `dropped_keys`,
`restart_required` and the unix-millisecond `at` timestamp.
Settings searches labels, keys and help across categories.
Durations and byte sizes use a magnitude plus unit (`2s`, `32MiB`) in YAML and
Settings; do not write a bare number and assume seconds. A bare integer is still
accepted as a count of bytes. Integer counts (retries, percents, tokens) stay
unitless numbers with the unit in the schema label when it is not obvious.
| Settings categories | Surface |
| --- | --- |
| Server, Request, Upstream, Queue & retry, Error storm protection | The proxy's traffic surfaces, in that order. |
| Quota pause, Conversations, Format translation | Provider reaction, grouping, and cross-protocol translation. |
| Storage, Backup, MCP, Dashboard | Durable state, its archives, the LLM tool endpoint and the live view. |
| Models, Providers, Request overrides | Grouping rules, upstream mappings and scoped rewrites. |
The category labels, their help and their membership are owned by
`schema.Categories()` and `Schema()` over `GET /admin/config`; the generated
example carries the same sections with each field's type, range and default.
The rows above are names only, not a second copy of either.
The destructive controls sit in their category and each confirms first: a
rebuild and restart is confirmed and single-flight, starting a capture through
MCP needs its `confirm` phrase and stopping every session confirms separately,
a database delete confirms in the console (and over MCP needs the exact phrase
plus the preview token for that filter), and a restore inspects before its
confirmed apply.
### Guardrails that are not settings
Every numeric guard is classified when it is added. A value becomes a setting
when an operator can reasonably need a different number for legitimate traffic.
It stays an internal constant when changing it independently would break an
invariant, encode a protocol maximum, or only tune an internal heuristic.
| Family | Examples | Why it stays internal |
| --- | --- | --- |
| Protocol maxima | Chat `stop` list of 4 | The wire defines the practical maximum; a longer list is clamped to it with an approximation warning. |
| Derived frame bounds | `NativeStreamFrameMax(eventMax)`, `ResponsesStreamFrameMaxBytes`, the proxy's translated frame buffer | Computed from the accepted source-event and retained-byte bounds plus the JSON escape expansion. A separate value could under-size a scanner, turning a valid frame into a translation error, or over-size it and defeat the retention bound. |
| Recursion and canonicalization budgets | Anthropic nested block depth 16, tool-argument canonical depth 64, separator budget, canonical byte cap | They select a safe fallback (the raw-byte fingerprint) or cap decoder recursion. They are algorithm facts, not wire acceptance. |
| Diagnostic echo bounds | 128-byte warning echo, 512-byte identity and provider-text retention, header echo caps | A request-sized diagnostic or identifier must never multiply across a stream or a stored row. Raising them cannot admit a request that is otherwise rejected; it only grows logs and frames. |
| Record retention | Retained tool names (64), tool-call ids (1024), path/method/model/error caps, capture header caps | They bound the ring, pending snapshots and durable rows. Counts still count past the bound; only identities stop being retained. |
| Metrics-only truncation | Non-streaming choice cap, SSE analyzer event buffer | The non-streaming choice cap never changes relayed bytes. The SSE analyzer cap can affect classification for events split across that bound, as the protocol limits note; raising it cannot admit a request that is otherwise rejected. |
| Cursor run and store budgets | Parked run count, per-run pending call/byte caps, blob store caps, aggregate store budget | They bound per-run and process memory for a bridge whose counts come from live tool batches, not operator intent. The per-run and aggregate caps are coupled. |
| Internal heuristic thresholds | Quality retry spool, error-body capture, discovery output allowance, storage sweep batch sizes, log scan and chart/explorer budgets | They decide whether an internal heuristic runs or how much work one pass may do. Changing them changes diagnostics or performance, not what a client may send or receive. |
The operator-facing bounds are settings: `request_tool_calls_max`,
`stream_tool_calls_max`, `translated_response_max_bytes`,
`models_discovery_max_entries`, `non_stream_capture_max`,
`stream_terminal_hold_max`, `stream_event_max_bytes`, `token_response_max` and
`upstream_response_header_max`. `non_stream_capture_max` trades accounting
accuracy for memory and never changes relayed bytes. A new guard must be
classified in this table when it is added, and a guard that becomes
operator-facing moves to the settings schema instead of keeping a parallel
constant.
### Saving settings safely
Settings collects a draft until Apply. Revert reloads the saved document, and
closing a dirty sheet keeps it open until Apply or Revert; an open dirty draft
also asks the browser to confirm a tab close or reload.
The server validates the
whole result and compares its revision before writing. A stale tab receives 409;
reload/review its values instead of resubmitting an old full form.
The sheet
renders closed vocabularies as selects from the schema's `choices` and checks
duration/byte text against the schema's numeric band before the POST; the server
remains the authority. With no writable config the field controls are disabled,
not only the Apply action.
API saves overlay supplied top-level keys on the saved config. Omitted keys
remain unchanged, but a supplied map/list replaces that entire field. For
example, `providers`, `provider_aliases` and `model_rules` are not recursively
patched. Read the current document and preserve wanted entries when editing a
structured field. Explicitly empty collections clear those fields.
The response separates saved values from the effective snapshot, reports
restart-required keys and carries the same `last_reload` section as GET; the
Settings sheet renders it as its last-apply row with apply time, failure,
dropped-key count and restart notice.
A successful file write can still be
followed by a reload failure; `saved:true` means the file changed even when
the response reports an error. Do not assume rollback.
The `writable`
indicator means a config path is configured, not that filesystem permissions
or every future write are guaranteed.
See [persistence](containers.md#persistent-state-and-configuration) and
[reload behavior](containers.md#reload-restart-and-shutdown) before changing startup-bound fields.
### Provider aliases and model grouping
`provider_aliases` merges provider labels in stored history at boot/reload and
rekeys new requests.
All aliases in one map commit as a single transaction, so
a trigger or constraint failure on a later alias leaves every earlier label
unchanged. It rejects self-maps and alias chains.
This is a durable rename, not
a temporary view: removing the alias does not reconstruct original labels.
Existing in-memory ring records can retain their previous label until they age
out or the process restarts. Back up before an intentional history rename.
`model_rules` instead controls display/grouping. The list is capped at 64 rules.
Its ordered exact, pattern and lowercase steps keep stored model spellings
unchanged; an empty rule list groups by those raw spellings.
The shipped pipeline folds case, strips one or more leading `vendor/` namespaces
and trailing `:tag` deployments, strips a trailing run of serving decorations,
and unifies `.` with `-` between digits.
A decoration is what a provider adds around the same weights:
| Decoration | Examples |
| --- | --- |
| Quantization or serving format | `-fp4`, `-nvfp4`, `-bf16`, `-int8`, `-q4_k_m`, `-gptq`, `-awq`, `-gguf` |
| Service tier | `-fast`, `-speed`, `-value`, `-free` |
| Release channel | `-preview`, `-latest`, `-beta`, `-alpha`, `-stable`, `-experimental` |
| Reasoning or effort word | `-thinking`, `-reasoning`, `-xhigh`, `-high`, `-medium`, `-low`, `-minimal` |
| Build stamp | `-1017`, `-20250929` |
Model tiers, fine-tunes, specifications and parameter counts stay distinct:
- Model tiers: `-flash`, `-pro`, `-mini`, `-ultra`, `-lite`, `-nano`, `-turbo`,
`-max`.
- Fine-tunes: `-abliterated`, `-rl`.
- Specifications: `-coder`, `-base`, `-chat`, `-it`.
- Parameter counts.
Add an exact rule above the shipped pipeline to split a group, or one it cannot
recognize to merge the spelling. The Settings editor supports reordering,
parking and previewing rules.
Raw export/delete predicates do not silently expand to a display group.
Debug's native-model normalization is separate from these rules.
### Request overrides
`request_overrides` is an ordered rule list that rewrites the upstream request
before relay; the default empty list leaves every request untouched. Each rule
has a scope and at least one action: `headers` sets or replaces upstream
headers, `remove_headers` deletes them, and `body` sets the output-token
ceilings `max_tokens` / `max_completion_tokens` (1..1000000) on a Chat
Completions or Anthropic client body before translation.
A Responses client
body is not rewritten by the body action because its cap is
`max_output_tokens`; header and removal rules still apply. A Chat client whose
target is the Responses wire has the rewritten cap translated to
`max_output_tokens`. Both override spellings address an Anthropic body's single
`max_tokens` field, with `max_completion_tokens` winning when both are set.
The
list is capped at 64 rules and validated like every
other setting: a rule needs at least one scope field and one action, duplicate
scope triples are rejected, and one header name cannot be both set and removed
in the same rule.
Scope matching is exact leaf. A rule applies when every non-empty scope field
equals the request's value exactly:
- `client` against the classified client label (`X-Proxy-Client` when sent,
except a label that only repeats the user agent's product name, which records
the fuller user-agent identity; otherwise inferred from SDK/user-agent
metadata).
- `provider` against the canonical label after `provider_aliases`.
- `model` against the recorded model id (for Cursor requests the base id the
dashboard records, not the fused spelling).
An empty scope field matches anything. There is no model-rule canonicalization,
regex or wildcard broadening. All matching rules apply in list order, and for
the same header or body field the later rule wins.
The client label is a claim, not an authenticated identity. The caller chooses
it, and the proxy never verifies it, so a request presenting another client's
label is treated as that client here and by every other client-scoped feature
(sub-conversations, Debug sessions, pause holds, the Cursor parked-run scope).
See [client identity](../SECURITY.md#client-identity-is-a-claim-not-an-authentication)
for what that means for a listener reachable by parties you do not trust. What
it cannot do is move a credential: credential-class header names are refused at
load and skipped at send, so the exposure is the policy attached to the label,
not the secret behind it.
Header rules are the operator's last word on the upstream wire: the resolved
merge applies after client-forwarded values, configured provider headers and
the client's `X-Proxy-Headers` map, so a matching override wins over all of
them.
Only the proxy-owned transport-mode names `accept-encoding` and `range`
are stripped from the final upstream request before send, whichever source set
them, so the transport keeps its own gzip negotiation and never sends a partial
range; the client header maps (`X-Proxy-Headers` and an
`X-Proxy-Auth-Header` naming one) also reject them with a 400 before any send.
The request-override list refuses two further header classes at load. Protocol
and framing names (`host`, `content-type`,
`content-length`, `transfer-encoding`, `te`, `connection`, `keep-alive`,
`proxy-authenticate`, `proxy-connection`, `trailer`, `upgrade`) and the whole
`x-proxy-` control prefix belong to the proxy and the transport.
Credential
names belong to whoever issued the secret: `authorization`, `cookie`,
`proxy-authorization`, and every name the shared credential vocabulary
classifies. That vocabulary judges the header's *role*, not a list of vendor
spellings: `auth`/`authorization`, `api-key`, `secret`, `password`,
`credential`, `bearer`, `jwt`, `signature`, `cookie`, `key` and `token` are all
credential-class wherever they appear, so `x-auth-key`, `x-amz-credential`,
`x-functions-key` and `x-subscription-key` are refused along with `x-api-key`.
Names that mention a key or a token but carry an identifier or a measurement
stay admitted: `x-key-id`, `idempotency-key`, `cache-key`, `x-prompt-tokens`,
`x-token-count`, and the `x-ratelimit-*` and `anthropic-ratelimit-*` families.
That breadth is deliberate. An override
value is operator-supplied and is stamped onto a request whose destination the
caller chooses with `X-Proxy-Base-URL`, so a secret in any unclassified header
name would be delivered to that caller's host; vendor spellings cannot be
enumerated in advance, which is why the check keys on role rather than a list.
The same vocabulary drives capture and debug
redaction, so a stored capture never keeps a header the wire carried. At
runtime the target's configured auth header is skipped as well, so a rule that
predates this check, or a future grammar slip, still cannot move a credential.
The client header maps reject `content-length` before any send so the transport
keeps ownership of message framing. Header
names are trimmed and canonicalized
case-insensitively, and values must be non-empty single-line header values.
Provider `headers` refuse neither class above. They are applied after the
extracted credential is placed, so a credential name (including the target's
own auth header) or an `x-proxy-` name is legal there and takes effect over the
forwarded and extracted value. Configured provider values are still stripped
for `accept-encoding` and `range` at the final request, exactly like override
values.
The body rewrite is fail closed. It applies to Chat Completions or Anthropic
client bodies before translation, whatever the target (a Responses client body
is not rewritten); when the body is not a JSON object, or the rewritten
result would exceed `max_request_bytes`, nothing is written: the original
bytes are used unchanged by the rewrite stage and are still translated when
translation is active, with one process log line recording the reason. Only a
passthrough relays them verbatim.
A successful rewrite replaces the body for
the request's whole life, so every relay attempt, absorbed retry and quality
re-send reuses it, and the recorded request cap is re-stamped from the
effective body.
A rule sets only the body
fields it names, and the re-read pair keeps the wire's documented precedence -
`max_completion_tokens` wins over `max_tokens` - so a rule setting only
`max_tokens` cannot raise a request whose client already sent
`max_completion_tokens`: set both fields to the same value to override either
spelling.
Cursor targets carry no
token ceilings on their wire, so the body section is skipped for them while
header rules still apply at the native send.
Overrides apply to inference sends only, never model discovery or token
refresh. The list hot-reloads: changed rules apply to new requests. The
Settings editor mirrors the server's checks and rejections live for each
rule, marks empty scope fields as matching any request, and blocks Apply
on the first invalid rule.
To raise the output budget for one provider, either write:
```yaml
request_overrides:
- provider: nano-gpt.com
body:
max_completion_tokens: 65536
```
or open Settings, select **Request overrides**, pick **raise the output budget
for a provider** from the add-a-rule menu, and fill the provider scope; the
other starter templates seed an empty rule and an upstream header for a
client.
### Sub-conversation tracking
`sub_conversations` gives requests from one client a conversation identity of
their own, taken from a value the client already sends in its request body
(exemplar: opencode's `promptCacheKey`).
The default empty list leaves tracking
fully off, and the feature is deny-by-default per client: an entry applies only
when its `client` equals the request's classified client label exactly (the same
leaf the request-overrides `client` scope matches; no wildcard, no case
folding), and a request from any other client is never inspected, leaving its
bytes and grouping untouched.
Each entry lists 1..4 exact JSON field names, matched only at the request
body's top level and checked in order; the first present field decides. A
present value that is not a JSON string counts as absent and the next field is
checked, while a present string that cannot be decoded or fails the identity
rules drops the identity for the request - no later field is consulted.
A value is kept only when it is non-empty after trimming, valid UTF-8, free of
control bytes and at most 512 bytes, the same bound explicit session
declarations satisfy.
The UTF-8 judgment runs on the value's raw bytes
before JSON decoding: the standard decoder coerces invalid UTF-8 inside
strings to U+FFFD, so a raw invalid-UTF-8 sequence drops the identity while
an escaped lone surrogate, whose bytes are ASCII, is kept and decodes to
U+FFFD.
The drop is conservative: the value's raw spelling is
scanned inside a bounded window, so a value whose raw span exceeds the window
(escape sequences spend more raw bytes than they decode to, and trimmable
edges count before trimming) is dropped even when its trimmed form would
pass.
An absent or dropped identity is never a client error: on the
passthrough wire the body is payload, and the request keeps automatic
grouping. A translated target still requires the whole request body to be
valid UTF-8, as [protocol](protocol.md#relay-retries-and-model-discovery)
describes.
The kept value groups the request under a `k:` conversation identity.
Precedence is the explicit `X-Proxy-Session` header first, then the tracked
value, then automatic client+key grouping; a request carrying both header and
tracked value keeps the header identity.
`X-Proxy-Parent-Session` still owns
parent linkage, and declaring a parent still requires the explicit session
pair, so a tracked identity never carries a parent. In the dashboard a `k:` id
is an ordinary conversation id: lineage, filters, saved views and the request
drawer read it like any other.
`strip: true` removes every configured field of the entry that is present at
the body's top level from the client body before relay or translation,
including a Responses client body; the default `false` forwards them
unchanged. Unlike the [request overrides](#request-overrides) body fields,
strip is not limited to Chat Completions or Anthropic inbound.
The removal
rides the same body-rewrite engine: one document decode when both features
match one request, the original bytes used unchanged by the rewrite stage when
the body is not a JSON object or the rewrite would exceed
`max_request_bytes`, and cursor targets skipping the body section.
Without
`strip`, the client's spelling reaches the upstream wire on a same-protocol
passthrough; a translated body carries the value in `prompt_cache_key` or drops
it.
When a tracked value was extracted and the request translates to the Anthropic
wire, the translated body gains a top-level `prompt_cache_key` carrying the
value. This is a proxy tracking extension, not an Anthropic cache field:
Anthropic caching uses `cache_control`, and an upstream that rejects unknown
top-level fields will reject the injected request. Injection follows extraction
alone: `strip` governs the tracked client-body fields, never the injected
field.
Validation runs at the load boundary for YAML and Settings saves alike: at most
16 entries, at most 4 params per entry, param names one JSON key segment each
(trimmed, non-empty, at most 64 bytes, printable ASCII, no quote, backslash or
control bytes), and duplicate clients or duplicate params within one entry are
rejected. The list hot-reloads: changed entries apply to new requests.
The Settings editor lives in the Conversations category and mirrors every check
live per entry card: classified-client autocomplete from observed clients,
ordered param rows with add/remove gated at the caps, the strip toggle, a count
line against the entry cap, the server's exact rejection wording, and an Apply
blocked on the first invalid card. To track opencode's prompt cache key and
strip it from the upstream body, write:
```yaml
sub_conversations:
- client: "opencode"
params:
- "promptCacheKey"
strip: true
```
---
# Settings reference
Settings and the proxy CLI write the same fields. The dashboard Settings sheet
saves them through `GET` and `POST /admin/config`, the YAML file holds them at
rest, and the proxy flags override a few of them at launch. Every block below
is generated from the code that owns it, so a setting's key, label, type,
bound, reload behavior and help cannot describe one thing here and another in
the dashboard. A `-` under accepted values means the schema declares no bound
and no closed vocabulary for that key.
Read [configuration](configuration.md) for precedence, the YAML file and the
settings schema, [containers](containers.md) for image mounts, restore paths
and the deployment lifecycle, and the [capture gallery](gallery.md) for the
rendered sheet on both surfaces.
## Server
Bind address, database, and ring depth restart to apply. shutdown_timeout applies at the next SIGINT/SIGTERM.
#### `listen`
Listen address · `string`
Default `":8080"` · Accepted - · Restart required
Host:port the HTTP server binds. Restart required.
#### `db_path`
Database path · `string`
Default `"proxy.db"` · Accepted - · Restart required
SQLite metrics store. Empty disables durability (ring only). Restart required.
#### `history_size`
History size · `int`
Default `10000` · Accepted 1..1000000 · Restart required
In-memory ring depth and startup backfill count. Durable history and paged request logs can extend beyond this window. Restart required.
#### `shutdown_timeout`
Shutdown timeout · `duration`
Default `5s` · Accepted > 0s · Hot-reloads
HTTP shutdown wait after active request contexts are canceled on SIGINT/SIGTERM. Storage draining follows separately. Must be > 0; applies at the next shutdown without restarting the listener.
#### `restart_drain_timeout`
Restart drain timeout · `duration`
Default `10m` · Accepted >= 0s (0s = wait indefinitely) · Hot-reloads
How long a dashboard-triggered restart waits for in-flight streams. Expiry aborts the restart and resumes accepting traffic; unfinished streams are not killed. New connections queue during a successful handoff.
#### `read_header_timeout`
Read header timeout · `duration`
Default `10s` · Accepted >= 0s (0s = no timeout) · Restart required
Slowloris guard on inbound headers. Restart required.
#### `idle_timeout`
Idle timeout · `duration`
Default `2m` · Accepted >= 0s (0s = no timeout) · Restart required
Keep-alive idle connections. No write timeout (it would kill SSE). Restart required.
#### `request_body_idle_timeout`
Request body idle timeout · `duration`
Default `30s` · Accepted > 0s · Hot-reloads
How long one inbound body read may wait for progress. Re-armed before every read and cleared at EOF or Close. Must not exceed request_body_read_timeout. Applies to requests started after reload.
#### `request_body_read_timeout`
Request body read timeout · `duration`
Default `10m` · Accepted > 0s · Hot-reloads
Absolute bound for reading one inbound request body after headers. Must be at least request_body_idle_timeout. The restore upload is exempt (idle bound only). Applies to requests started after reload.
#### `response_write_timeout`
Response write timeout · `duration`
Default `10m` · Accepted > 0s · Hot-reloads
Generic per-write bound for buffered responses. Streaming owners may arm a shorter bound. Must be at least sse_write_progress_timeout. Applies to requests started after reload.
#### `go_memory_limit`
Go memory limit · `bytes`
Default `0B` · Accepted 32MiB..1024GiB (0B = auto cgroup, else unbounded) · Restart required
Soft cap on Go runtime managed memory (heap, stacks, runtime), applied once at startup. With the setting at 0, an operator GOMEMLIMIT wins and otherwise a detected finite cgroup memory ceiling applies at 80% (ceilings below 80MiB are ignored); a set value overrides GOMEMLIMIT, not GOGC. Exceeding it turns the collector more aggressive (capped near 50% CPU), never fails requests. Does not bound SQLite's page cache or OS overhead. Restart required.
## Request
Inbound body cap, tool-call history bound, SSRF allowlist, optional body-preview, token-refresh response cap, and debug-capture retention.
#### `max_request_bytes`
Max request bytes · `bytes`
Default `32MiB` · Accepted 1KiB..1GiB · Hot-reloads
Inbound body the proxy will buffer to read routing metadata. Larger bodies are rejected with 413.
#### `request_tool_calls_max`
Max request tool calls · `int`
Default `1024` · Accepted 1..10000 · Hot-reloads
Most Chat tool calls across the whole message history of one inbound request, including the deprecated per-message function_call shape. Anthropic tool_use and Responses function_call history are bounded by their protocol collection limits instead. Real agent turns replay a handful; raise it only for clients that resend unusually long tool loops, because every call is materialized in memory during translation.
#### `allowed_base_urls`
Allowed base URLs · `strings`
Default `[]` · Accepted - · Hot-reloads
SSRF allowlist of upstream base URLs. Empty allows any target (single trusted operator). An entry admits only the same scheme, host and port once the scheme's default port is filled in, plus that entry's path and its children, so a port-less entry does not admit other ports on the host. Each entry must be an absolute URL whose prefix is literally lowercase http:// or https://, with a lowercase hostname, no userinfo, query, fragment or trailing slash, and a port after a hostname in 1..65535.
#### `capture_body_preview`
Capture body preview · `bool`
Default `false` · Accepted - · Hot-reloads
Store a short bounded preview of the last user prompt / response in metrics. Off by default so content is not persisted.
#### `debug_capture_ttl`
Debug capture TTL · `duration`
Default `24h` · Accepted 1h..168h · Hot-reloads
How long a full debug capture (bodies and headers) is kept after the request finishes. Operator debug sessions choose which traffic is captured; this is blob retention, not the session timer.
#### `debug_capture_max_bytes`
Debug capture max bytes · `bytes`
Default `1MiB` · Accepted 4KiB..16MiB · Hot-reloads
Cap on one debug capture: retained headers are charged first and the remainder is split between the request body and the client-facing response bytes. Past the cap the rest is dropped and the capture is marked truncated.
#### `debug_store_max_bytes`
Debug store max bytes · `bytes`
Default `512MiB` · Accepted 16MiB..8GiB (0B = unlimited) · Restart required
Total durable debug-capture sidecar cap. New captures are skipped once the next write would exceed it; TTL expiry frees capacity. Bytes, not characters. 0 = unlimited. Restart required.
#### `auto_token_refresh`
Auto token refresh · `bool`
Default `true` · Accepted - · Hot-reloads
Exchange an expired or expiring JWT when the client sends X-Proxy-Refresh-Token. All supported providers use inference handback: HTTP 401 with token_expired and access_token in the error body, plus refresh_token only when the exchange returns a different replacement. No inference is sent; adopt the returned credentials and retry, retaining the old refresh token if no replacement is returned. Model discovery refreshes transparently without returning tokens. Stateless: no reusable credentials are retained.
#### `token_response_max`
Token response max bytes · `bytes`
Default `1MiB` · Accepted 4KiB..16MiB · Hot-reloads
Token-endpoint response read during transparent credential refresh. Past it the refresh fails open and the original request is proxied unchanged.
## Upstream
Connection pool, header timeout, stream keepalives, model-discovery budgets and response-buffer bounds.
#### `upstream_timeout`
Upstream header timeout · `duration`
Default `5m` · Accepted >= 0s (0s = no header cap) · Hot-reloads
How long to wait for upstream response headers. Streaming bodies are not subject to this. Transient header timeouts are retried.
#### `upstream_dial_timeout`
Upstream dial timeout · `duration`
Default `30s` · Accepted > 0s · Hot-reloads
How long establishing a new upstream connection may take. Applies to new connections after reload; in-flight streams keep their transport.
#### `upstream_dial_keepalive`
Upstream dial keepalive · `duration`
Default `30s` · Accepted > 0s · Hot-reloads
TCP keepalive interval for upstream connections. Applies to new connections after reload.
#### `upstream_tls_handshake_timeout`
Upstream TLS handshake timeout · `duration`
Default `10s` · Accepted > 0s · Hot-reloads
How long the TLS handshake with an upstream may take. Applies to new connections after reload.
#### `models_discovery_timeout`
Model discovery timeout · `duration`
Default `30s` · Accepted 1s..5m · Hot-reloads
One upstream metadata budget for transparent key refresh, all model pages and optional enrichment. Never applies to LLM streams. New discoveries use reloaded limits.
#### `models_discovery_max_bytes`
Model discovery bytes · `bytes`
Default `8MiB` · Accepted 4KiB..64MiB · Hot-reloads
Aggregate response bytes read after transport decoding across model pages, unary RPCs and enrichment. Primary overflow fails; optional enrichment is discarded. This is not a strict heap limit.
#### `models_discovery_max_pages`
Model discovery pages · `int`
Default `100` · Accepted 1..1000 · Hot-reloads
Maximum upstream model responses per discovery, including a unary RPC and optional enrichment. Limits are captured once, never reset between pages.
#### `models_discovery_max_entries`
Model discovery entries · `int`
Default `10000` · Accepted 100..1000000 · Hot-reloads
Model entries one discovery request may retain across every page and enrichment source. The byte budget alone cannot bound the decoded map cost of a compact catalog, so a source past this count fails closed instead of returning a partial list. Raise it for providers that publish unusually large catalogs.
#### `translated_response_max_bytes`
Translated response max bytes · `bytes`
Default `32MiB` · Accepted 4KiB..1GiB · Hot-reloads
Whole buffered non-streaming upstream response a cross-protocol translation may read, and the translated document it may write back. Past it the request fails with 502 response_too_large rather than relaying a prefix that breaks the translated shape.
#### `upstream_response_header_max`
Upstream response header max bytes · `bytes`
Default `1MiB` · Accepted 4KiB..64MiB · Hot-reloads
Upstream response headers the proxy accepts. Past it the request fails as a transport error. The upstream client is rebuilt on reload, so new requests use the new value while in-flight streams keep the client they started on.
#### `non_stream_capture_max`
Non-streaming capture max bytes · `bytes`
Default `1MiB` · Accepted 16KiB..64MiB · Hot-reloads
Prefix of a non-streaming response body kept for usage and cost parsing. Larger documents still relay in full; their accounting may be incomplete. This is a capture cap, not a forwarding cap, and it is per in-flight non-streaming request outside max_inflight_response_bytes.
#### `stream_terminal_hold_max`
Stream terminal hold max bytes · `bytes`
Default `64KiB` · Accepted 4KiB..1MiB · Hot-reloads
Bytes withheld at the end of a relayed SSE stream while the relay decides between verbatim release and an in-band degenerate-error substitution. Past it the hold is abandoned and the bytes are released verbatim. A single complete event line can exceed it by up to stream_event_max_bytes. Raise it only if a legitimate terminal region is larger than the default.
#### `stream_event_max_bytes`
Stream event max bytes · `bytes`
Default `2MiB` · Accepted 2MiB..16MiB · Hot-reloads
Largest SSE event accepted on the passthrough relay and by every translated stream decoder: a single line, and the accumulated data of a multi-line event. The translated frame buffer is derived from this value, so both bounds move together. The floor is the built-in accepted size; raise it only for providers that emit larger events.
#### `max_conns_per_host`
Max conns per host · `int`
Default `512` · Accepted >= 0 (0 = unlimited) · Hot-reloads
Total connections per upstream host. Sized for concurrent SSE streams, not requests/sec.
#### `max_idle_conns`
Max idle conns · `int`
Default `512` · Accepted >= 0 (0 = unlimited) · Hot-reloads
Process-wide idle connection pool.
#### `max_idle_conns_per_host`
Max idle conns per host · `int`
Default `512` · Accepted >= 1 · Hot-reloads
Idle connections kept per upstream host. Cannot exceed max conns per host when that is set. net/http treats 0 as a default of 2, not unlimited, so this must be >= 1.
#### `idle_conn_timeout`
Idle conn timeout · `duration`
Default `90s` · Accepted >= 0s (0s = never expire) · Hot-reloads
How long an idle upstream connection is kept before closing.
#### `upstream_stream_idle_timeout`
Upstream stream idle timeout · `duration`
Default `10m` · Accepted > 0s · Hot-reloads
Cancel an upstream response body after this much silence. Releases the handler, upstream connection and scheduler slot. New response bodies use the reloaded value.
#### `whole_document_read_timeout`
Whole-document read timeout · `duration`
Default `2m` · Accepted > 0s · Hot-reloads
Fallback bound for one whole-document upstream read when the caller supplied no deadline. Buffered reads only; streaming SSE is never subject to it.
#### `whole_document_read_max`
Whole-document read maximum · `duration`
Default `10m` · Accepted > 0s · Hot-reloads
Ceiling for whole-document upstream reads. Must cover whole_document_read_timeout and models_discovery_timeout.
#### `sse_keepalive_interval`
SSE keepalive interval · `duration`
Default `15s` · Accepted 1ns..10m · Hot-reloads
How often a quiet streaming socket gets a `: keepalive` comment (queue wait, operator hold, mid-stream gaps). Range 1ns..10m. Applies to streams started after reload.
#### `sse_write_progress_timeout`
SSE write progress timeout · `duration`
Default `1m` · Accepted > 0s · Hot-reloads
Per-event and keepalive write bound for a client-facing SSE stream. A stalled reader fails that write instead of pinning shutdown. Must not exceed response_write_timeout. Applies to streams started after reload.
## Queue & retry
Per-key concurrency, transparent 429/5xx retry and error classification, client error-class disclosure, and quality re-asks.
#### `max_concurrent`
Max concurrent · `int`
Default `0` · Accepted >= 0 (0 = unlimited) · Hot-reloads
In-flight sends per provider+key. 0 is unlimited.
#### `max_queue_size`
Max queue size · `int`
Default `1024` · Accepted >= 0 (0 = unlimited) · Hot-reloads
Queued waiters per group. 0 is unlimited. It also feeds the default queue cap of a new pause hold.
#### `max_inflight_requests`
Max in-flight requests · `int`
Default `10000` · Accepted 0..1000000 · Hot-reloads
Process-wide inference admission bound. Requests above it are refused with 503 and Retry-After instead of buffering their bodies. 0 = unlimited. Hot-reloads.
#### `max_inflight_body_bytes`
Max in-flight body bytes · `bytes`
Default `256MiB` · Accepted 1MiB..64GiB (0B = unlimited) · Hot-reloads
Process-wide budget for buffered inbound request bodies. Admission charges the bytes actually read and refuses with 503 when the next read would exceed the budget. A declared Content-Length reserves nothing, so an idle connection cannot hold the budget. Bytes past max_request_bytes are neither charged nor retained: that body is answered 413 by the cap owner. A refused request drains at most its remaining cap window so an over-cap body still answers 413 rather than a retryable 503. 0 = unlimited. Hot-reloads.
#### `max_inflight_response_bytes`
Max in-flight response bytes · `bytes`
Default `256MiB` · Accepted 1MiB..64GiB (0B = unlimited) · Hot-reloads
Process-wide budget for buffered upstream response bodies: whole-document translation of non-streaming responses, the non-streaming 200 quality spool (at most 1 MiB plus one 32 KiB chunk), and model discovery. Admission reserves the declared Content-Length when it fits the bounded read cap, otherwise that cap, and refuses with 503 and Retry-After before reading when the budget would be exceeded. The reservation is released once the buffer is written, translated, or abandoned. 0 = unlimited. Hot-reloads.
#### `max_queue_wait`
Max queue wait · `duration`
Default `10m` · Accepted >= 0s (0s = unlimited) · Hot-reloads
How long a request may sit in the queue before 429. 0s is unlimited. Operator-hold time is excluded.
#### `max_retries`
Max retries · `int`
Default `5` · Accepted 0..100 (0 = no retries) · Hot-reloads
Transparent retries of transient 429 / 5xx / transport failures. Durable quota 429s follow quota_pause_mode instead of this budget.
#### `queue_retry_after`
Queue Retry-After · `duration`
Default `2s` · Accepted 0s..24h · Hot-reloads
Retry-After hint sent when our own queue is full or the wait is exceeded. HTTP Retry-After is integer seconds, so the value must be a whole number of seconds.
#### `base_backoff`
Base backoff · `duration`
Default `1s` · Accepted > 0s · Hot-reloads
Initial adaptive delay. Doubles each attempt and each consecutive failed request, up to max backoff. A provider Retry-After/reset is a floor (never retry sooner) and does not reset this doubling. Cannot exceed max backoff. Must be > 0.
#### `max_backoff`
Max backoff · `duration`
Default `2m` · Accepted > 0s · Hot-reloads
Cap on adaptive backoff. Does not clamp a provider Retry-After / rate-limit-reset (daily limits are often 30–60m). Must be > 0.
#### `quality_retries`
Quality retries · `int`
Default `1` · Accepted 0..3 (0 = disabled) · Hot-reloads
Transparent re-attempts of a degenerate 200 (empty completion, empty tool_calls), a truncated stream that relayed no content yet, a retryable in-band provider error dropped before the wire, and Cursor's empty-resume re-ask. 0 disables.
#### `thinking_retries`
Thinking retries · `int`
Default `1` · Accepted 0..3 (0 = disabled) · Hot-reloads
Transparent rescue of requests that die or end without an answer during thinking: a stream truncated or reset after reasoning-only output, a cleanly finished reasoning-only completion, a retryable in-band provider error that arrived after reasoning-only output, and a non-streaming reasoning-only body. The fresh attempt appends after the relayed reasoning; answer content and tool calls are never rescued, and finish_reason length never is. Each rescue re-sends the full request (billed). 0 disables.
#### `retryable_error_classes`
Retryable error classes · `strings`
Default `[]` · Accepted - · Hot-reloads
Operator extensions to the built-in retryable in-band error vocabulary: exact type or code strings (case-insensitive) that a provider reports inside a 200 and that authorize a transparent re-send while no generation content was relayed. Built-ins already cover server_error, the 503 overloaded/unavailable family, api_error, overloaded_error, timeout_error and common gateway spellings; list custom gateway spellings here. Entries add to the built-ins and never move one; a durable quota/billing class is rejected at load, and in-band rate limits, unknown classes and errors after relayed content stay verbatim.
Reload applies to new rescue decisions.
#### `rate_limit_error_classes`
Rate-limit error classes · `strings`
Default `[]` · Accepted - · Hot-reloads
Operator extensions to the built-in transient rate-limit vocabulary (rate_limit_error, rate_limit_exceeded, tier_limit_exceeded, resource_exhausted, rpm/tpm spellings and friends). A gateway that announces its limit under a custom type or code string (case-insensitive) is then retried transparently within the retry budget, counted in the rate-limit series, and classified rate_limit for clients. Unlike a bare 429, a non-429 status carrying the class still counts as a failed request on the health surface. Entries add to the built-ins and never move one; a durable quota/billing spelling is rejected at load.
Reload applies to new classifications and re-reads stored history through the same vocabulary.
#### `suppress_client_retries`
Suppress client retries · `bool`
Default `false` · Accepted - · Hot-reloads
Make the proxy the clients' sole retry authority: stamp x-should-retry: false on every LLM-relay error response. The official OpenAI SDKs (python/node/go/ruby) honor it over their 408/409/429/5xx auto-retry and over Retry-After, including relayed provider hints; the proxy's own retry ladder has already run by then. Cannot suppress transport-level failures, and clients that ignore the header (the Vercel AI SDK / opencode stack) keep their own policy.
#### `client_error_class`
Client error class · `string`
Default `"off"` · Accepted `off`, `header` · Hot-reloads
Negotiate millivolt's canonical error class to clients on the LLM relay surface. off leaves relayed responses exactly as the provider sent them. header stamps X-Proxy-Error-Class (rate_limit, quota, unavailable, unreachable, timeout, content, client_error, proxy, canceled, upstream) and a class-derived X-Should-Retry: true|false on every error status before the status line commits, so a client can tell a rate limit announced by a 400 from a client fault and a durable quota block from transient backpressure without parsing provider envelopes. The body is never rewritten.
suppress_client_retries still wins while it is on; mid-stream SSE errors cannot carry new headers. Reload applies to new responses.
## Error storm protection
Bounded recovery queues for affected providers or exact model names. Hot-reloads; policy changes clear observations and wake waiters. Banner visibility changes do not reset protection.
#### `storm_enabled`
Enable protection · `bool`
Default `false` · Accepted - · Hot-reloads
Detect elevated upstream failure rates and queue affected sends for bounded recovery probes. Opt-in; requests can still fail on cancellation, exhausted retry budgets, queue limits or permanent errors. Retries can repeat upstream execution and billing.
#### `storm_provider_enabled`
Provider protection · `bool`
Default `true` · Accepted - · Hot-reloads
Gate every model on a provider only when all active models individually meet the sample and failure thresholds. Active means a request in the same detection window; idle models expire.
#### `storm_model_enabled`
Model protection · `bool`
Default `true` · Accepted - · Hot-reloads
Gate an affected exact recorded model name within its provider. Dashboard model grouping does not broaden this scope.
#### `storm_banner_enabled`
Show incident banner · `bool`
Default `true` · Accepted - · Hot-reloads
Show affected providers/models, safe error categories, failure percentages, queue state and recovery timing at the top of the dashboard. Changing visibility does not reset protection.
#### `storm_window`
Detection window · `duration`
Default `1m` · Accepted 1s..1h · Hot-reloads
Rolling time range for successful and eligible failed upstream attempts. Models with no requests in this window expire from provider-wide checks.
#### `storm_min_samples`
Minimum samples · `int`
Default `20` · Accepted 1..1000000 · Hot-reloads
Minimum successful or eligible failed upstream attempts in the detection window before a model qualifies. Provider-wide protection requires every active model to qualify. Retries count as separate attempts; this is not a distinct-request percentage.
#### `storm_error_percent`
Failure threshold · `int (percent)`
Default `50` · Accepted 1..100 · Hot-reloads
Minimum percentage of sampled upstream attempts with a selected failure. Applied to each exact model after the minimum sample count; all active models must qualify for provider-wide protection.
#### `storm_initial_backoff`
Initial recovery delay · `duration`
Default `5s` · Accepted 1ms..1h · Hot-reloads
Initial storm cooldown before a recovery probe. Cannot exceed maximum recovery delay. Valid upstream Retry-After/reset hints still apply independently.
#### `storm_max_backoff`
Maximum recovery delay · `duration`
Default `2m` · Accepted 1ms..1h · Hot-reloads
Cap on exponentially increased storm cooldowns. Must be at least initial recovery delay. Does not shorten upstream Retry-After/reset hints.
#### `storm_backoff_multiplier`
Recovery delay multiplier · `int`
Default `2` · Accepted 2..10 · Hot-reloads
Multiply the storm cooldown after a failed recovery probe, up to maximum recovery delay.
#### `storm_jitter_percent`
Recovery delay jitter · `int (percent)`
Default `20` · Accepted 0..100 (0 = no jitter) · Hot-reloads
Random spread as a percentage of the storm cooldown to separate recovery attempts. 0 disables jitter.
#### `storm_recovery_successes`
Recovery successes · `int`
Default `2` · Accepted 1..100 · Hot-reloads
Consecutive successful recovery probes required before queued traffic resumes normally.
#### `storm_max_queue`
Maximum queued requests · `int`
Default `1000` · Accepted 1..100000 · Hot-reloads
Maximum requests waiting for storm recovery. Excess work is rejected locally; requests and their bounded bodies stay in process memory while waiting.
#### `storm_max_wait`
Maximum queue wait · `duration`
Default `5m` · Accepted 1ms..24h · Hot-reloads
Maximum duration of each storm gate wait, not total request duration across all retries and other queues. Caller cancellation may end the wait earlier.
#### `storm_max_scopes`
Maximum tracked scopes · `int`
Default `2048` · Accepted 2..100000 · Hot-reloads
Bound on provider and exact-model detector scopes retained in process memory. Increasing it does not establish a total process memory ceiling.
#### `storm_max_retries`
Additional storm retries · `int`
Default `20` · Accepted 0..1000 (0 = no additional retries) · Hot-reloads
Additional retries beyond max_retries only while a relevant storm is active and the failure matches a selected eligible transient condition. Does not replay meaningful emitted upstream content. 0 keeps the ordinary retry budget.
#### `storm_status_codes`
HTTP failure statuses · `strings`
Default `- "500" - "502" - "503" - "504"` · Accepted - · Hot-reloads
Exact status strings selected for detection and additional storm retries: 429 or 500..599, without duplicates. Empty disables HTTP status triggers. 429 is opt-in because limits may be credential-specific; durable quota/billing 429s remain excluded from storm evidence (quota_pause_mode owns their reaction).
#### `storm_transport_errors`
Transient transport failures · `bool`
Default `true` · Accepted - · Hot-reloads
Include transport failures already eligible for the shared retry policy. Caller cancellation and permanent transport errors are excluded.
#### `storm_stream_errors`
Response and stream failures · `bool`
Default `true` · Accepted - · Hot-reloads
Include recorded in-band, stream and quality failures to protect future sends. Reports a generic response error without retaining error bodies for the banner. Never replays meaningful emitted upstream content.
## Quota pause
Provider-wide reaction to a 429 carrying a durable quota/billing error (insufficient_quota/credits, spend limits): surface it immediately, park the provider behind a recovery gate, or hold it until an operator resumes. OpenAI-compatible wire only; Anthropic and Cursor upstreams keep the off behavior.
#### `quota_error_classes`
Quota error classes · `strings`
Default `[]` · Accepted - · Hot-reloads
Operator extensions to the built-in durable quota/billing vocabulary (insufficient_quota/credits, spend and hard limits, and the numeric business codes of some gateways). A matching type or code string (case-insensitive) is a durable account condition: the ordinary retry ladder never absorbs it, quota_pause_mode owns the reaction, and it counts as an error rather than flow control. Entries add to the built-ins and never move one; a spelling already built into another class is rejected at load.
Reload applies to new classifications and re-reads stored history through the same vocabulary.
#### `quota_pause_mode`
Mode · `string`
Default `"off"` · Accepted `off`, `retry`, `manual` · Hot-reloads
Reaction to a 429 carrying a durable quota/billing error (insufficient_quota/credits, spend limits; in-band occurrences included): off surfaces it immediately; retry opens a provider-wide recovery gate - the triggering request absorbs the 429 and re-sends per the storm recovery delays, new requests park on the gate (bounded by storm_max_queue/storm_max_wait), and the gate closes after quota_pause_recovery_successes consecutive successful probes (the default 1 closes on the first 2xx); manual creates an indefinite provider pause that only an operator resumes.
The built-in classification is fixed (quota_error_classes may add unknown spellings); only the reaction is configurable. OpenAI-compatible wire only; Anthropic and Cursor upstreams keep the off behavior.
#### `quota_pause_recovery_successes`
Recovery successes · `int`
Default `1` · Accepted 1..100 · Hot-reloads
Consecutive successful recovery probes that close a retry-mode quota gate. 1 releases the queue on the first 2xx. Meaningful only in retry mode.
## Conversations
How the request log groups turns into conversations.
#### `conversation_idle_gap`
Idle gap · `duration`
Default `30m` · Accepted > 0s · Hot-reloads
A gap longer than this starts a new conversation for the same client+key. Generous so thinking pauses inside one task don't split it. Must be > 0.
#### `conversation_max_open`
Max open · `int`
Default `64` · Accepted 1..100000 · Hot-reloads
Cap on concurrently-open conversations per client+key. Oldest is evicted past it.
#### `sub_conversations`
Sub-conversation tracking · `sub_conversations`
Default `[]` · Accepted - · Hot-reloads
Per-client tracking of a request body field that carries a sub-conversation identity, exemplified by opencode's promptCacheKey.
Each entry names one classified client exactly (no wildcard, no case folding) plus 1..4 exact JSON field names matched only at the request body's top level; the first present field decides (a duplicate top-level key drops the identity): a present value that is not a JSON string value counts as absent and the next field is checked, while a present string that cannot be decoded or fails the identity rules (non-empty after trimming, valid UTF-8, free of control bytes, at most 512 bytes) drops the identity for the request - no later field is consulted - and a dropped identity is never a client error.
The drop is conservative: a value whose raw spelling exceeds the bounded scan window is dropped even when its trimmed form would pass. The value groups the request under a k: conversation identity, after the X-Proxy-Session header (s:) and before automatic client+key grouping; X-Proxy-Parent-Session still owns parent linkage. Requests translated to the Anthropic wire gain a top-level prompt_cache_key carrying the value.
strip: true removes the entry's configured fields from the top level of the client body before relay or translation, including a Responses client body; the default false forwards the body unchanged. Unlike the request-overrides body fields, strip is not limited to Chat or Anthropic inbound. Param names are exact printable-ASCII JSON key segments up to 64 bytes, no quotes or backslashes; duplicate clients and duplicate params within an entry are rejected. Caps: 16 entries, 4 params per entry. Default: an empty list, tracking fully off.
Hot-reloads.
## Format translation
Cross-protocol translation policy, streamed tool-call bound, Anthropic defaults and Cursor agent.v1 bridging.
#### `anthropic_default_max_tokens`
Anthropic default max tokens · `int (tokens)`
Default `4096` · Accepted 1..1000000 · Hot-reloads
Injected into translated Anthropic requests when the client sent none of max_tokens, max_completion_tokens or max_output_tokens.
#### `stream_tool_calls_max`
Max streamed tool calls · `int`
Default `1024` · Accepted 1..10000 · Hot-reloads
Most tool calls one translated response stream may carry: concurrent open blocks for Anthropic and Responses, distinct calls for Chat (its stream has no per-call close event). A stream past the bound ends with a translation_error. Raise it only for providers that emit unusually large parallel tool batches.
#### `translation_loss`
Translation loss policy · `string`
Default `"warn"` · Accepted `warn`, `reject` · Hot-reloads
What a cross-protocol translation does when the target cannot represent a client field. warn (default) drops optional fields and reports them in X-Proxy-Warnings; reject fails the request with 400 naming the fields before any upstream send and refuses every recorded loss, including a documented approximation. Structural features are always rejected when they cannot be honored. A response-side loss is never fatal: a non-streaming response reports it in X-Proxy-Warnings, and a streaming response drops it without a header. Hot-reloads.
#### `cursor_default_context_window`
Cursor default context window · `int (tokens)`
Default `0` · Accepted >= 0 (0 = omit (client default)) · Hot-reloads
Optional context-window hint stamped onto every model in GET /v1/models when X-Proxy-Format is cursor. 0 omits the field.
#### `cursor_park_ttl`
Cursor park TTL · `duration`
Default `3h` · Accepted > 0s · Hot-reloads
How long a parked Cursor Run stream stays open waiting for tool results. Must be > 0. Expiry closes the stream; the next continuation cold-starts.
#### `cursor_heartbeat_interval`
Cursor heartbeat · `duration`
Default `5s` · Accepted 1ns..10m · Hot-reloads
How often the proxy pings a Cursor agent.v1 run (idle parked runs included). Range 1ns..10m. Applies to runs created after reload.
## Storage
Durable SQLite write pipeline. Restart to apply.
#### `storage_write_chan_cap`
Write channel cap · `int`
Default `8192` · Accepted 1..1000000 · Restart required
Buffered records before overflow drops. Restart required.
#### `storage_batch_cap`
Batch cap · `int`
Default `256` · Accepted >= 1 · Restart required
Max records per SQLite transaction. Cannot exceed the write channel cap. Restart required.
#### `storage_flush_interval`
Flush interval · `duration`
Default `500ms` · Accepted > 0s · Restart required
Partial-batch flush cadence. Must be > 0 (a zero ticker panics). Readiness convicts a wedged writer after one interval plus a progress window of four intervals clamped to 30s..2m. Restart required.
#### `storage_query_timeout`
Query timeout · `duration`
Default `10s` · Accepted > 0s · Restart required
Bounds dashboard SELECT queries. Must be > 0. Open and AggAPI bind at start (restart required); pause/throttle persist reads the live snapshot.
#### `storage_query_max_bytes`
Query result bytes · `bytes`
Default `8MiB` · Accepted 4KiB..64MiB · Restart required
Maximum JSON bytes returned by /metrics/query; also bounds SQLite string/blob values. Oversized results fail with HTTP 413, never truncate. Does not cap all SQLite working memory. Restart required.
#### `storage_query_max_rows`
Query result rows · `int`
Default `10000` · Accepted 1..100000 · Restart required
Maximum rows returned by /metrics/query. Oversized results fail with HTTP 413; use SQL filters or LIMIT. Does not limit dashboard aggregates or exports. Restart required.
## Backup
Download or restore a self-checked archive of the saved config, the SQLite history, or both. Restore inspects first and the apply is confirmed. Merge keeps live defaults and live request ids; replace loads and validates the backup, writes canonical YAML, and preserves CLI-overridden listen and db_path.
#### `backup_max_bytes`
Backup size cap · `bytes`
Default `1GiB` · Accepted 1MiB..4GiB · Hot-reloads
Maximum size of one Settings backup download or restore upload. Larger archives are refused. Downloads and restores stream through private staging files; this is the admit/reject ceiling, not a heap guarantee.
## MCP
The operator-gated /mcp LLM tool endpoint: mcp_enabled registers the route or keeps it reserved (404, never inference), and the limits bound what its tools return to a model. Restart to apply; the route table and the endpoint's services are built once at boot.
#### `mcp_enabled`
Enable MCP endpoint · `bool`
Default `true` · Accepted - · Restart required
Register the streamable HTTP MCP endpoint at /mcp for LLM tool clients, operator-gated like /metrics and Bearer-only: a session cookie alone is refused. Off keeps the route reserved (404, never inference); the /mcp/ look-alike reservation and the operator gate stay either way. Restart required.
#### `mcp_page_size`
Page size · `int`
Default `50` · Accepted 10..500 · Restart required
Default page for the records, values and audit capture listings the endpoint serves to a model. Restart required.
#### `mcp_query_max_rows`
Query result rows · `int`
Default `200` · Accepted 1..100000 · Restart required
Default client-side row limit when a query call omits max_rows. An explicit max_rows may raise it up to this setting's maximum; the proxy's own storage_query_max_rows still answers 413 first. Restart required.
#### `mcp_query_max_bytes`
Query result bytes · `bytes`
Default `128KiB` · Accepted 1B..64MiB · Restart required
Client-side cap on the encoded size of one query tool result. Keeps whole rows, truncating with an explicit marker rather than silently. Restart required.
#### `mcp_capture_max_bytes`
Capture max bytes · `bytes`
Default `256KiB` · Accepted 1B..64MiB · Restart required
Size above which a stored capture document is withheld whole from the audit_capture_get tool; a document that large is more than a model should read in one call. Restart required.
#### `mcp_query_timeout`
Query timeout · `duration`
Default `2m` · Accepted > 0s · Restart required
Bounds one proxy read the endpoint makes that is not the proxy's own bounded SQL reader (a full-history chart or explorer fold can take seconds). Must be > 0. Restart required.
#### `mcp_timeout`
Request timeout · `duration`
Default `30s` · Accepted > 0s · Restart required
Bounds every in-process call the endpoint's tools make to the proxy, including mutations. Must be > 0. Restart required.
## Dashboard
Live-view cadence and request-log window. Hot-reloads; open dashboards pick up the next tick.
#### `dash_log_rows`
Request log rows · `int`
Default `60` · Accepted 10..500 · Hot-reloads
Page size of the live request log (newest first). Scrolling near the bottom loads more, including older durable history.
#### `dash_poll_interval`
Poll interval · `duration`
Default `5s` · Accepted > 0s · Hot-reloads
KPI / pause / SSE-fallback poll cadence. Must be > 0.
#### `dash_chart_refresh`
Chart refresh · `duration`
Default `15s` · Accepted > 0s · Hot-reloads
How often the traffic chart re-fetches server-authoritative buckets and period totals. Must be > 0.
#### `dash_explorer_stale`
Explorer stale after · `duration`
Default `15s` · Accepted > 0s · Hot-reloads
How old a cached explorer breakdown may be before it is re-fetched. Must be > 0.
#### `dash_background_refresh`
Background refresh · `bool`
Default `false` · Accepted - · Hot-reloads
Keep the dashboard current while its tab is hidden: the poll runs at the browser's throttled background cadence (about one wake per minute on Chromium) and a Web Lock keeps the SSE feed from being frozen. Desktop Chromium only; Android and iOS still suspend background pages at the OS level. Off keeps the current pause-while-hidden behavior.
## Models
Model grouping rules - an ordered rewrite pipeline merging spelling variants of the same model in every grouped surface. Records keep their exact spelling; hot-reloads.
#### `model_rules`
Model grouping rules · `model_rules`
Default:
```yaml
- {mode: lower, from: "", to: ""} - {mode: pattern, from: "^(?:[a-z0-9][a-z0-9._-]*/)+", to: ""} - {mode: pattern, from: "(?::[a-z0-9._-]+)+$", to: ""} - {mode: pattern, from: "(?:-(?:[a-z]{0,2}fp\\d+|bf\\d+|int\\d+|nf\\d+|[a-z]?q\\d+(?:_[0-9a-z]+)*|gptq|awq|gguf|exl2|mlx|onnx|ggml|fast|speed|value|free|preview|latest|beta|alpha|stable|experimental|thinking|reasoning|xhigh|high|medium|low|minimal|(?:\\d{4}|\\d{8})))+$", to: ""} - {mode: pattern, from: "(\\d)\\.(\\d)", to: "$1-$2"}
```
Accepted - · Hot-reloads
Capped at 64 rules. Ordered rewrite pipeline that merges spelling variants of the same model in every grouped surface (explorer model dimension, scope filters, debug checklist, the Logs download and Settings delete-filter model optgroups, and the request-log leaf’s displayed name). Each rule is one step: exact (whole-string merge old → new), pattern (regex rewrite of all occurrences, $1 capture refs), lower (fold case).
Rules apply once in order; the shipped default folds case, strips one or more leading `vendor/` namespaces, strips trailing `:tag` deployments, strips a trailing run of serving decorations (quantization or format such as `-fp4`, `-q4_k_m`, `-gptq`; service tier such as `-fast`, `-speed`, `-value`; release channel such as `-preview`, `-latest`; build stamp such as `-1017`), and unifies `.` with `-` between digits. Model tiers such as `-flash` or `-pro`, fine-tunes such as `-abliterated` or `-rl`, and parameter counts stay distinct.
An explicitly empty list groups by the exact stored spelling. Stored values remain unchanged; purge/export match them exactly and request details show the original. Debug matching uses separate native base-model normalization, not these display rules. Hot-reloads.
## Providers
Optional per-provider JSON field-name maps: usage/cost response keys, the models metadata endpoint merged into /v1/models, ensured tool names injected into relayed chat bodies, and optional upstream wire headers. Empty = auto-detect.
#### `providers`
Provider field maps · `providers`
Default `{}` · Accepted - · Hot-reloads
Per-provider cost/usage JSON field paths, optional models_path/models_keys enrichment, ensured tool names and headers (a string map). Field names only, never static prices. Custom cost_keys must report USD; leave cost_in_usd_ticks to automatic unit conversion. ensure_tools names function tools injected as inert stubs into relayed Chat Completions client bodies on a Chat outbound wire missing them.
When the client sent no tools of its own, an absent, null, or auto tool_choice is forced to none; an explicit none stays; a required, named, or malformed choice skips injection entirely. A first-party client's tool-signature gate still recognizes the request. Header values expand {{uuid4}} to a fresh UUID, {{platform}} to the server's Rust-style os; arch pair, and {{opencode-msg-id}}/{{opencode-ses-id}} to fresh opencode-format identifiers; unknown templates stay literal. On HTTP relay requests, configured headers override forwarded/extracted-auth values and X-Proxy-Headers wins.
accept-encoding and range are proxy-owned: the X-Proxy-Headers map rejects them with 400, and configured or override values are stripped from the final request. Model discovery and the Cursor bridge construct headers separately and do not apply X-Proxy-Headers. Headers can contain secrets; keep runtime configuration private.
#### `provider_aliases`
Provider aliases · `aliases`
Default `{}` · Accepted - · Hot-reloads
Map an old provider label to a canonical label. Rewrites existing stored history at boot and on reload; new requests use the canonical label. Existing in-memory records keep their prior label until they age out or the process restarts. Removing a mapping does not restore rewritten history. Chains and self-maps are rejected.
## Request overrides
Scoped rewrites of the upstream request: set, replace or remove HTTP headers and set output-token ceilings per client, provider and/or model. Rules match exactly, all matching rules apply in list order, and the later rule wins. Default: an empty list, the feature fully off.
#### `request_overrides`
Request override rules · `request_overrides`
Default `[]` · Accepted - · Hot-reloads
Capped at 64 rules. Ordered rules that rewrite the upstream request before relay. A rule matches when every non-empty scope field equals the request's value exactly: client as the classified client, provider as the canonical label after provider_aliases, model as the recorded model id; an empty scope field matches anything. All matching rules apply in list order, and for the same header or body field the later rule wins.
headers sets or replaces upstream headers (names are canonicalized, values must be non-empty single-line), remove_headers deletes them, and body sets max_tokens / max_completion_tokens (1..1000000) on a Chat Completions or Anthropic client body. Credential and protocol headers are rejected in an override at load (provider headers may use them; they apply after the credential).
Credential class is judged by role, not by a list of vendor spellings: any name carrying authorization/auth, api-key, secret, password, credential, bearer, jwt, signature, cookie, key or token is refused, so x-auth-key, x-amz-credential, x-functions-key and x-subscription-key are covered. Identifier and accounting shapes stay admitted: key-id, cache-key, idempotency-key, and the token counters (x-prompt-tokens, x-token-count) plus the x-ratelimit-* and anthropic-ratelimit-* families. Protocol and framing names (host, content-type, content-length, transfer-encoding and the rest) and the whole x-proxy- control prefix are refused as well.
The override applies after provider headers and the client's X-Proxy-Headers map and wins over both, with two enforced exceptions: the target's resolved auth header and Authorization are never set or removed by an override, and Accept-Encoding and Range are stripped after the override merge because the transport owns them. It covers inference sends only, never model discovery or token refresh.
Body rewrites apply to Chat Completions or Anthropic client bodies before translation, whatever the target (a Responses client body is not rewritten), and re-stamp the recorded request cap; the Cursor bridge reads no body fields, so body is skipped for cursor targets while header rules still apply. An empty list leaves every request untouched. Hot-reloads.
## Proxy CLI flags
The flag set in `cmd/proxy` owns these blocks. Every flag is parsed before any
configuration is loaded, so the default below is the registration default, not
the effective value after a config file or another override.
#### `-config`
`string`
Default `"proxy.yaml"`
path to config file (empty for built-in defaults)
#### `-print-config`
`bool`
Default `false`
print documented built-in defaults as YAML and exit without loading configuration
#### `-print-example-config`
`bool`
Default `false`
print the documented example with enabled provider profiles and exit without loading configuration
#### `-version`
`bool`
Default `false`
print application and source build metadata as JSON and exit without loading configuration
#### `-listen`
`string`
Default `""`
override listen address (e.g. 127.0.0.1:8081); keeps instances on distinct ports
#### `-db-path`
`string`
Default `""`
override storage DB path ("" = config value; "none" = disable durable storage)
#### `-pid-file`
`string`
Default `""`
write the process pid to this file after boot
#### `-healthcheck`
`bool`
Default `false`
probe GET /healthz on the configured listen address and exit nonzero on failure (Docker HEALTHCHECK); loads config read-only
#### `-readycheck`
`bool`
Default `false`
probe GET /readyz on the configured listen address and exit nonzero when storage is not ready (Compose readiness); loads config read-only
---
# Containers and data
This page covers the image, Compose operation, persistent volumes, backups,
versions and the restart lifecycle. The settings surface is in
[configuration](configuration.md).
> [!WARNING]
> This is an offline backup: stop callers first, allow active work to finish,
> and keep the service stopped while copying. Never copy an open SQLite database
> as plain files. Restore into a new project and fresh volumes, leaving the
> original deployment and backups untouched.
## Containers
The [Dockerfile](../Dockerfile) builds Linux `amd64` and `arm64` targets and runs
as UID/GID `65532:65532`, with no source tree, Go toolchain or shell in the
runtime image. JavaScript/CSS are embedded in the Go binary; Node and Python
are test tools, not runtime dependencies. The binary is stripped of debug and
symbol tables.
The final stage copies only that binary, dependency notices, the generated
example and prepared state directories onto the minimal distroless base.
The [Compose quickstart](../README.md#docker-compose) downloads the image-only
[compose.yaml](../compose.yaml). Normal deployment never builds from source.
Local builds use the separate
[development Compose](../CONTRIBUTING.md#container-development).
A registry tag is usable only after its publishing workflow succeeds.
### Compose operation
Keep the same Compose project name to reuse its named volumes. The normal file
names the project `millivolt`; an explicit `-p` overrides it.
The deployment inputs are:
- `MILLIVOLT_IMAGE` (image reference)
- `MILLIVOLT_PORT` (loopback host port)
- `MILLIVOLT_OPERATOR_TOKEN` (the operator credential, forwarded into the
container environment only)
- `MILLIVOLT_STOP_GRACE_PERIOD` (container stop grace)
- `MILLIVOLT_MEM_LIMIT` (container memory ceiling, default `1g`)
- `MILLIVOLT_PIDS_LIMIT` (process/thread ceiling, default `512`)
Their defaults live in [compose.yaml](../compose.yaml), not server
configuration. Supply them consistently through the environment or an ignored
local `.env` file. `docker compose config` shows the resolved deployment.
The memory ceiling is also the source of the runtime's own soft limit: with
`go_memory_limit` at its `0B` default, a detected finite cgroup ceiling at or
above 80 MiB becomes a soft limit at 80% of itself, so the container bound
reaches the Go runtime instead of leaving it unbounded.
Raise the ceiling with `history_size`, request buffers, concurrency or observed
identity cardinality; a ceiling under 80 MiB is ignored by the runtime and
leaves only the hard OOM kill. `pids_limit` bounds process and thread
amplification, not goroutines.
The image carries a Dockerfile `HEALTHCHECK` that probes the unauthenticated
`/healthz` through the binary's `-healthcheck` mode, so `docker compose ps`
reports health without any operator credential. Both Compose files replace that
probe with the [readiness check](operations.md#health-and-readiness), which is
strictly stronger.
```sh
# Start or adopt a newly pulled image while preserving volumes.
docker compose pull
docker compose up -d
docker compose ps
docker compose logs --tail 100
# Stop without removing containers or data.
docker compose stop
# Remove this project's containers and network, retaining named volumes.
docker compose down
```
Do not add `--volumes` or `-v` to `down` unless deliberately deleting its
configuration and history. Changing the Compose project name selects different
volumes and can look like missing data. Back up before an upgrade; the existing
image can be replaced, but a database migration is not necessarily reversible.
See Docker's [Compose lifecycle](https://docs.docker.com/reference/cli/docker/compose/up/)
and [volume removal](https://docs.docker.com/reference/cli/docker/compose/down/)
documentation for the underlying behavior.
Do not add a build section to the deployment file. The
[development Compose](../CONTRIBUTING.md#container-development) owns
source builds, its private port and its separate project identity.
The Compose service drops Linux capabilities, disallows privilege escalation,
uses a read-only root filesystem and a bounded temporary filesystem. These
controls do not add application authentication or a tenant boundary. Access to
a rootful Docker daemon through the `docker` group is effectively root access;
see Docker's [Linux post-installation guidance](https://docs.docker.com/engine/install/linux-postinstall/).
### Persistent state and configuration
| Container path | Persistence and permissions |
| --- | --- |
| `/data` | Writable database directory, including SQLite WAL/SHM sidecars. Mount the directory, not just `proxy.db`. |
| `/config` | Writable private configuration directory. The image bundles `/config/proxy.yaml` from the generated example; if deliberately absent, the application uses neutral built-in defaults. |
Fresh named config volumes receive the image's bundled example, owned by
UID/GID `65532:65532` with mode `0600`. No separate configuration download is
required. Replacing an image does not overwrite an existing saved configuration;
review example-profile changes deliberately when upgrading.
These bundle semantics apply to images built from this revision. Existing release
images are not changed by a main-branch update; consult the documentation at the
selected release tag for older images.
Existing volumes and host bind mounts must be owned by the runtime UID, or the
database file must already be mode `0600`. Read/write permission alone is not
sufficient: startup narrows a broader database mode to `0600` and fails closed
when it cannot, naming the file path, owner and current mode. A foreign-owned
`0600` database is left unchanged; a broader foreign-owned file must be pre-set
to `0600` or chown'd to `65532:65532`.
Bind mounts hide bundled files: supply a config yourself when using one, or
intentionally accept the application's missing-file defaults. Use private
directory permissions and mode `0600` for a supplied config; with rootless
Docker or user namespaces, account for the host UID mapping. Do not make
sensitive directories world-writable to work around ownership failures.
A first launch cannot prepare its own bind-mount host directories. When a
bind-mount target does not exist, Docker creates it as `root:root` mode `0755`,
which the runtime UID cannot write, and startup fails with SQLite's
`unable to open database file` error. The log names the database path, the
directory state and the runtime UID.
Pre-create the host `data/` and `config/` directories owned by `65532:65532`
before the first start, or `chown -R 65532:65532` them after a failed start,
then start the container again. Named volumes avoid this entirely: Docker seeds
them from the image with correct ownership.
Settings saves by creating a temporary file beside the resolved config and
atomically renaming it; a symlinked config path is followed to its final
target, so the link stays in place and the target receives the document. The
restore rollback publishes through the same writer, so a failed restore of a
symlinked config keeps the link too.
A read-only mount, or a bind mount of only the config file, cannot support that
save contract. Mount the whole writable directory, or deliberately use
read-only configuration and update it outside the dashboard. Do not mount the
committed example as a mutable runtime config.
The container listens on `:8080`; restrict publication on the host with
`-p 127.0.0.1:8080:8080`. Binding to loopback *inside* the container prevents
ordinary published-port access. Docker port publication does not add operator
authentication. Do not share one database among independent proxy containers.
Upstream URLs are reached from the proxy container, so `localhost` in
`X-Proxy-Base-URL` means that container, not the Docker host. Use an upstream
hostname reachable on its network, such as another Compose service name.
Host-service access needs an explicitly configured host-gateway mapping or
another reachable host address; it is not included in the supplied deployment.
Arguments after the image replace its complete default command. If overriding
a server flag, also keep `-config /config/proxy.yaml -db-path /data/proxy.db`
unless intentionally selecting other paths or disabling storage. The
`-print-config` and `-print-example-config` commands print their respective
documents without starting a server.
Replace the container to upgrade or adopt startup-bound settings; the dashboard
cannot rebuild an immutable image. Keep the same volumes and back them up before
upgrades. If only a saved config file changed, ordinary `up -d` need not replace
the container. Apply startup-bound changes explicitly:
```sh
docker compose up -d --force-recreate millivolt
```
Container stop sends SIGTERM, which cancels unfinished requests before
draining storage. Allow enough stop grace time for the configured shutdown and
queued writes; forced termination can prevent that drain. No stop/flush can
recover records already dropped under storage overload.
### Image-only backup and restore
The supplied deployment can back up both named volumes without a source checkout,
Python, a shell inside the image or a helper image. This is an **offline** backup:
stop callers first, allow active work to finish, and keep the service stopped
while copying. Run from the deployment directory with the same Compose project
and environment used to start it. If you use `-p`, repeat that selection on every
Compose command. Do not concurrently replace or restart the source container.
The following saves the original Compose file, the pulled image's registry digest,
and both volume archives in a new private directory. It expects the supplied
single-service layout, `/data` and `/config`, with a saved `proxy.yaml` present.
Privately retain any local `.env` or other deployment inputs as well.
```sh
(
set -eu
umask 077
backup_dir="$(mktemp -d ./millivolt-backup.XXXXXXXX)"
printf 'Backup directory: %s\n' "$backup_dir"
container_id="$(docker compose ps -q millivolt)"
test -n "$container_id"
image_id="$(docker inspect --format '{{.Image}}' "$container_id")"
docker image inspect --format '{{index .RepoDigests 0}}' "$image_id" > "$backup_dir/image.txt"
cp compose.yaml "$backup_dir/compose.yaml"
docker compose stop millivolt
test "$(docker inspect --format '{{.State.Status}} {{.State.ExitCode}}' "$container_id")" = 'exited 0'
docker compose cp millivolt:/data/. - > "$backup_dir/data.tar"
docker compose cp millivolt:/config/. - > "$backup_dir/config.tar"
docker compose start millivolt
)
```
A failed stop or copy exits without automatically restarting the service; inspect
the failure before resuming. Copy the completed backup to protected independent
storage. The data archive includes any WAL/SHM files, not just `proxy.db`. Never
use this file-copy procedure while SQLite is open. An archive cannot recover
records already dropped or writes lost through forced termination.
Restore into a **new project and fresh volumes**, leaving the original deployment
and backups untouched. Enter the completed backup directory and an unused
loopback host port when prompted. The original supplied Compose file is required:
do not substitute resolved `docker compose config` output, custom external-volume
names or bind mounts, which can point the new project at existing data.
```sh
(
set -eu
umask 077
printf 'Backup directory to restore: '
IFS= read -r backup_dir
test -s "$backup_dir/data.tar"
test -s "$backup_dir/config.tar"
MILLIVOLT_IMAGE="$(head -n 1 "$backup_dir/image.txt")"
case "$MILLIVOLT_IMAGE" in *@sha256:*) ;; *) exit 1 ;; esac
printf 'Unused loopback host port: '
IFS= read -r MILLIVOLT_PORT
case "$MILLIVOLT_PORT" in ''|*[!0-9]*) exit 1 ;; esac
test "$MILLIVOLT_PORT" -ge 1
test "$MILLIVOLT_PORT" -le 65535
export MILLIVOLT_IMAGE MILLIVOLT_PORT
restore_dir="$(mktemp -d ./millivolt-restore-XXXXXXXX)"
restore_project="$(basename "$restore_dir" | tr '[:upper:]' '[:lower:]')"
cp "$backup_dir/compose.yaml" "$restore_dir/compose.yaml"
restore_compose() {
docker compose --env-file /dev/null -p "$restore_project" -f "$restore_dir/compose.yaml" "$@"
}
existing_containers="$(docker ps -aq --filter "label=com.docker.compose.project=$restore_project")"
test -z "$existing_containers"
existing_volumes="$(docker volume ls -q --filter "name=^${restore_project}_")"
test -z "$existing_volumes"
restore_compose create millivolt
restore_compose cp --archive - millivolt:/data < "$backup_dir/data.tar"
restore_compose cp --archive - millivolt:/config < "$backup_dir/config.tar"
restore_compose up -d millivolt
restore_compose logs --tail 100 millivolt
printf 'Restored project: %s\nCompose directory: %s\nHost port: %s\n' "$restore_project" "$restore_dir" "$MILLIVOLT_PORT"
)
```
Keep the tar streams intact and retain `--archive` on restore: the private config
must remain owned by runtime UID/GID `65532:65532` with mode `0600`; the restored
database is subject to the same ownership and mode check at boot.
Check Settings and retained history on the new private port before redirecting
clients. Save the printed project name and deployment inputs for later commands;
the example bypasses automatic `.env` loading during recovery.
Docker documents [tar-stream copying](https://docs.docker.com/reference/cli/docker/container/cp/)
and [Compose archive mode](https://docs.docker.com/reference/cli/docker/compose/cp/).
The repository's isolated container check exercises stopped-volume backup and
restore into fresh volumes, including settings, history and file permissions.
Source deployments can instead use the live SQLite
[online-backup helper](operations.md#storage-and-accounting) or the Settings archive
described next.
### Settings backup and restore
The dashboard Settings Backup category downloads and restores a self-checked
`.mvb` archive. Choose config, database, or both. The archive is a packed
SQLite snapshot (`VACUUM INTO`, so free pages are not copied) and/or the
schema YAML, then zstd-compressed with a content checksum and an outer SHA-256
of the uncompressed payload.
Encoding streams each member into the compressor while hashing and counting it,
writes the archive once into a staging file, and self-checks that file before
returning it; an archive that fails decode or semantic checks is never emitted,
so a produced backup is never a silent corrupt file.
The database member is read from the packed snapshot file in bounded chunks
instead of being copied whole into memory, so the download cost does not grow
with a heap copy of the database. A source file that changes size or cannot be
read fails the encode before an archive is emitted.
Restore streams the upload through the `backup_max_bytes` ceiling into one
private staging file and decodes it with the same header, zstd, SHA-256 and
member-framing checks.
When a live store can stage it, the database member is written straight into a
private file on the live database's volume instead of being materialized on the
heap; a restore that selects the database member without a store fails closed
before staging, and a member the request did not select is stream-discarded
after its framing checks.
Config bytes stay buffered within the same 16 MiB config-document bound
`Config.Validate` enforces on the canonical rendered YAML, so a config that
validates can never exceed what an archive may retain. A config member that
declares a larger size is rejected before its bytes are copied, and encoding
refuses to emit one, so an older archive whose config member exceeds that bound
fails restore. The bound covers only the config member; the database member
stays streamed under the payload ceiling.
Restore validates that hash, the zstd frame, member framing and config
`Validate()` before applying anything. A selected database member is decoded
into its private staging file first, then must pass `PRAGMA integrity_check`;
only that checked file is published as the pending snapshot, which restore then
preflights for admission before the config member is written.
Decode holds the uncompressed payload to the same `backup_max_bytes` ceiling as
the upload (and to the Schema maximum, whichever is lower), so a small
compressed file cannot expand past what the operator configured; an over-cap
payload answers 413 with the upload's wording and stages no pending snapshot.
Download lists how many live settings differ from default and how many stored
requests would be packed. Picking a file first inspects the archive: created
time, size, member sizes, each config key as default vs modified vs the live
file, and the database request span when a live store can stage the member.
Inspect loads the live file through the same strict, bounded reader used by
config reload and restore, not the boot repair path: an unknown nested field,
an extra YAML document, a non-regular file, or a file above the 16 MiB document
bound fails the inspect instead of comparing against a repaired or partial
view.
Inspect decodes and counts the database member in place and computes its
overlap with the live store without writing another copy. Without a live store
it reports `present:true`, the decoded `bytes`, `inspectable:false` and the
reason, and produces no request span.
Config **replace** loads and validates the backup, writes canonical YAML, and
preserves CLI-overridden `listen` and `db_path`. Config **merge** overlays only
keys that differ from `Default()` onto the live file, so live custom values
stay unless the backup also customized them. An omitted `config_mode` or
`database_mode` defaults to replace; an explicitly empty or whitespace value is
rejected as malformed rather than treated as absent.
Either way the restored config hot-applies through the same owner as a Settings
save, is reflected in `last_reload`, and reports restart-required keys.
Database **replace** streams the decoded file into a pending snapshot next to
`db_path`, syncs it, and atomically renames it into place; Open admits the
pending file only after it checks again, and the restore waits for process
restart.
A `db_path` that is a symlink is resolved to its real target first, so staging,
admission, the rollback set and the state marker all belong to the file SQLite
actually opened; the link stays intact and points at the replacement. A
combined merge compares the same resolved identities rather than the two
spellings.
Database **merge** attaches the decoded file read-only and inserts request ids
that are not already in the live store immediately (live rows win on conflict);
the attached snapshot rejects writes, while the live database stays writable
for the merge, and no restart is required. Its request rows, debug sidecars and
since-inception totals commit as one transaction, so a failed totals write
leaves no partial merge behind. Truncated or bit-flipped files are rejected.
The `backup_max_bytes` setting is the encoded-archive and the decoded-payload
ceiling alike; encoding enforces it while the stream is written, so an over-cap
archive is removed instead of partially served, an over-cap upload answers 413
before decoding, and a decoded payload over it answers 413 before staging a
snapshot.
Restore phases are ordered, not atomic. The database member runs first: a
replace stages the decoded file and preflights it with the same schema,
writer-INSERT and request-scan value checks boot runs before touching the live
database, bounded by the same maintenance budget as boot maintenance, and a
merge commits rows into the running store.
One restore owns the whole transaction from the first uploaded byte, so two
restores cannot stage and decode two full members on the database volume at
once; an inspect shares that serialization because it stages the same members.
Only then is the config written and hot-applied.
A failed stage, preflight, or merge leaves the config file and live snapshot
unchanged, and a failed config write or reload rolls the config back and
restores a replace's previous pending slot. A successful live merge has no
rollback: if the later config phase fails, the merged rows stay applied.
A crash between the database and config phases is likewise not atomic; the
pending snapshot or completed merge can be durable while the config file still
has its previous bytes.
Preserved `.incoming`,
`.incoming.invalid-`, `.incoming.invalid-interrupted-`,
`.previous`, `.snapshot-state`, and `.millivolt-restore-pending-*` files next to
`db_path` are boot-recovery evidence and may need operator inspection after such
a crash; do not remove them while a restore may have been in flight. Quarantine
names carry a unique token and keep their sidecars beside them, so repeated
rejections accumulate instead of overwriting the previous one.
The `.millivolt-restore-pending-*` copy is a hard link to the snapshot that was
queued before the restore; it is created before the successor replaces the slot,
so the pending queue is never empty, and a crash on either side leaves a
recoverable snapshot instead of an orphan that the 24 hour staging sweep
eventually deletes.
A volume without hard links (exFAT, vfat, and several FUSE or SMB mounts)
rejects `link(2)`; there the slot is moved aside instead, still without copying
the snapshot bytes, and a crash leaves only the preserved sibling, which boot
re-admits. When the stage fails before the successor replaces the slot the
restore drops the copy again, and when a boot finds a rejected successor or an
empty slot it renames the preserved copy back onto the slot and admits once, so
the queued snapshot is neither hidden nor swept.
A combined config and database merge answers 400 when the restored config's
effective `db_path` differs from the running store's file, because the rows
would be written to a database the next boot does not open; merge the database
separately or use database replace.
At boot, a pending snapshot is admitted only after the same schema, trigger,
writer-insert, request-scan value and capture-id preflight `Open` runs. The
trigger fence rejects every unowned trigger in the snapshot, not only those
attached to `requests`: a trigger body is arbitrary SQL, so one attached to
`meta` or `request_debug` can delete or rewrite request rows.
The capture-id fence rejects a `request_debug` id that is NULL, non-text or
blank: such a row can never be fetched by id or reached by the expiry sweep,
yet still counts against the debug byte budget. Restore merge applies the same
fence before it imports any row.
Recovery consults the durable `.snapshot-state` marker rather than inferring a
transition from the files present. A markerless rollback set from a pre-marker
release cannot be classified: when a live main exists, recovery keeps it and
removes the `.previous` debris.
Discarding a non-empty `.previous-wal` in that markerless state logs a startup
warning, because its committed frames cannot be distinguished from debris and
may be lost. Marked pending and committed recovery behavior is unchanged.
Both directions stage database-sized transient files in the live database's own
directory, so capacity follows the data volume rather than a hardened
deployment's small `/tmp` tmpfs. Budget roughly one compressed upload plus one
packed database copy during a restore, and for a replace also the pending copy
at `db_path`.
The download's `VACUUM INTO` snapshot directory is removed after the archive is
encoded, including on failure or a client disconnect. A restore removes its
private upload file and decoded database directory after inspect or apply on
every path: a normal response, an invalid or over-cap archive, a validation or
apply failure, or a client disconnect. A config-only upload may use the process
temp directory because it carries no database-sized member.
A packed file already on the staging volume (VACUUM dest, decoded member,
pending `.incoming`) is integrity-checked in place, and inspect, overlap,
replace and merge consume it by path, so none writes a second database-sized
copy. A container's `/tmp` is a small tmpfs by design and never caps backup
size.
## Versions and images
[VERSION](../VERSION) is the application release source. The dashboard asset
fingerprint is separate: it detects changed embedded frontend bytes, not a
semantic release or a complete backend change.
`main` is promotion-only. Full source, browser and live MCP integration, and
native container checks run on all `testing` pull requests, code-bearing
`testing` pushes, manually dispatched `testing` gates, and manually pushed `v*`
tags; docs and site-only pushes use `docs.yml`. They also run on Dependabot's
`main` security PRs.
An ordinary promotion pull request from `testing` requires its head to be the
current `testing` tip, `main` to be already merged back into `testing`, and a
successful full-gate record for that exact commit: the canonical `check.yml`
push run's own check suite, with `unit`, `browser` and both native container
legs.
When a final docs-only commit follows the closest gated `testing` commit, it
reuses that gate if every path since it is docs/site-only; the fallback selects
the closest gated testing commit from canonical `testing` push runs, with no
commit-count bound. The proof polls pending exact check runs a bounded number of
times and fails closed if they are still pending.
For a code-bearing head, let the `testing` `check.yml` push gate finish before
opening or re-running a promotion PR. A docs/site-only head has no canonical
`check.yml` push run of its own, so wait for `docs.yml` or only the `required`
docs validation. It does not rerun the suite.
A code-bearing promotion must also have a `VERSION` newer than every existing
release tag. Docs/site-only promotions skip the version and full-gate proofs,
validate docs, workflow policy and the site in `required` within its ten-minute
budget, and do not rebuild images or cut a release.
Merge with a merge commit. The merged-PR release workflow tags that reviewed
merge commit and publishes the image; a delivered code-bearing main push is an
idempotent fallback. It never pushes `main`.
Dependabot security pull requests are the only other exception and normally do
not cut a release by themselves; an untagged pending `VERSION` can publish. A
manually pushed `v*` tag ships exactly as tagged.
If the version tag already points at the released commit but its GitHub release
is missing, the publish job fails closed with a repair instruction instead of
reporting a successful re-run. Repair the existing tag's release after verifying
the image; the job never moves a tag or republishes images. The full maintainer
checklist lives in [Contributing](../CONTRIBUTING.md#branch-workflow).
`go run ./cmd/proxy -version` (or `/millivolt -version` in the image) prints
JSON with `version`, `revision`, `modified`, and `go_version`. Unavailable VCS
metadata is reported honestly as revision `unknown` or `modified:null`.
The registry target is `ghcr.io/llm-4-people/millivolt`. Check the
[container package](https://github.com/LLM-4-People/millivolt/pkgs/container/millivolt)
and [image workflow](https://github.com/LLM-4-People/millivolt/actions/workflows/check.yml) for what has
actually been published; a version in a checkout does not prove an image exists.
Use a published digest to pin exact bytes. Version tags identify releases but
are not intrinsically immutable; moving branch and stable aliases can change.
The [workflow](../.github/workflows/check.yml) runs source, race and browser
checks, then native container smoke checks on Linux `amd64` and `arm64` for all
`testing` pull requests, code-bearing `testing` pushes, manually dispatched
`testing` gates, and manually pushed `v*` tags, and for Dependabot's `main`
security PRs. The `unit`, `browser` and `container` jobs start together. Docs
and site-only pushes use `docs.yml` instead.
A normal promotion PR proves the current testing tip, merged-back main and the
full-gate record for that commit, or for the closest gated testing commit with
only docs changes after it, and requires a successful
[escalation](../.github/workflows/escalation.yml) run (every Go test under the
race detector plus a five-run clock-sensitive flake sweep) for the reviewed
head; docs-only promotions validate the docs, workflow policy and site
directly.
The merged promotion PR carries the reviewed merge commit and releases it; a
delivered code-bearing `main` push only reuses the same reviewed merge and is
idempotent. Publication is limited to this repository's merged promotion pull
requests, delivered pushes to `main`, matching release tags, and
`testing`-branch pushes (which publish only the moving `testing` tag).
The release `publish` job waits for `required`. The `publish-testing` job waits
for `unit`, `browser` and `container` instead, because a testing push has no
promotion proof to wait for.
A release run whose `VERSION` is already tagged, such as a Dependabot security
merge, normally does not publish; if main carries an untagged pending `VERSION`,
that run can publish it. Open or unmerged pull requests do not publish images; a
merged promotion PR does.
| Image tag | Meaning |
| --- | --- |
| `main` | Moving production image for the released current `main` revision; updated by a non-prerelease versioned release cut from a merged promotion PR or a delivered main push. A prerelease does not move it, and a manually pushed release tag does not either. |
| `testing` | Moving development image after a successful testing-branch push; never a release. Its OCI version label reports the embedded `VERSION`, which can lag the latest release. |
| `sha-` | Release tag for the exact cut revision built and published; the version bump is part of the reviewed promotion merge, so this is a main commit. Testing pushes do not publish it, while a manually pushed tag publishes it for the tagged revision. |
| Version from `VERSION` | Created by the matching `v`-prefixed release tag on the promotion merge commit, or by a manually pushed tag. |
| `latest` | Moving stable alias updated by every non-prerelease release, including a release cut by a merged promotion PR, a main push or a manually pushed tag. Prereleases and testing pushes do not update it. A tag push moves the version, `sha-` and `latest` tags only; `main` stays where the last main-branch release put it. |
Release builds request an SBOM and max-mode provenance. The image and its
attestations are published before the tag and release are updated; the workflow
never pushes `main`. A failed image build does not leave a public release
pointing at a missing image. Verify
the published manifest and attestations with `docker buildx imagetools inspect`,
verify the repository-associated artifact attestation with
`gh attestation verify oci://ghcr.io/llm-4-people/millivolt: -R LLM-4-People/millivolt`,
and pin the digest for deployment. A tag name alone is not an integrity check.
Set `MILLIVOLT_IMAGE` to the desired full image reference or published digest
before the Compose pull/up commands. The `0.x` version series is initial
development, not a promise of stable APIs across releases.
GitHub initially publishes GHCR packages privately, even when their repository
is public. A package administrator must make the package public separately and
verify an anonymous pull. Repository linkage grants workflow access, not public
visibility; see GitHub's
[package access documentation](https://docs.github.com/en/packages/learn-github-packages/configuring-a-packages-access-control-and-visibility).
## Reload, restart and shutdown
`SIGHUP`, `POST /admin/reload`, a successful Settings save, and a backup
config restore use the same hot-reload owner. Invalid reloads preserve the
running config. Reloadable transport/scheduler settings affect new work
without closing active transports.
Startup-bound consumers keep their boot settings until restart. Settings reports
`restart_required`; its effective configuration snapshot can contain new values
that those consumers have not adopted yet. The schema/example identify which
fields require restart.
Rebuild & restart requires Go on PATH and the original source tree available at
its compiled-in path. Moved/trimpath builds may report it unavailable. It builds
the working tree exactly as it is: uncommitted, unstaged and untracked files are
compiled, so a restart is not a review gate and a dirty tree ships as-is.
The rebuilt binary is written into the process temp directory, which must be
writable (`TMPDIR`, or `/tmp` when unset); an unusable directory is refused with
that directory named rather than as a build fault.
The choreography builds first, stops acceptance while retaining the listening
socket, drains active handlers, flushes storage, then hands the socket to a
ready child. A flush failure aborts and resumes the old process. A drain timeout
still flushes and hands off: the in-flight requests are cut, but everything
finalized before them is durable before the child opens the database. The flush
has its own bound rather than sharing the drain window.
The handoff also takes the config transaction lock for its whole flush fence, so
a start while a reload, Settings save or backup restore is running is refused
with `409` naming the operation instead of queueing behind it; the fence itself
is bounded by `restart_drain_timeout` plus the flush bound.
A restore holds that lock across its streaming upload, so `POST /admin/reload`
and a Settings save answer `409` while one is in flight rather than waiting for
a client-paced upload to finish.
Parked native-adapter runs are process-local and do not survive a successful
restart.
Ordinary SIGINT/SIGTERM shutdown is different: it cancels active stream contexts,
then shuts down HTTP and drains/closes storage. Do not describe it as preserving
unfinished streams. Already-dropped records cannot be recovered by any flush.
Starting another matching proxy executable on an occupied port can trigger
same-binary port reclamation (SIGTERM, then escalation). This is not the graceful
rebuild workflow; choose distinct ports and databases for separate instances.
---
# Routes and debugging
This page covers the operator and data routes, access control, pause and limits,
error storm and quota protection, debug and preview capture, export and deletion,
restart, and the dashboard data and request inspector.
> [!WARNING]
> A genuinely bodyless `POST /admin/purge` deletes the whole history, and a
> restore applies data before the config phase; both are irreversible without a
> backup. Exports, debug captures and retained headers can carry sensitive
> content.
## Operator and data routes
The routes below belong to millivolt. They are operator-gated unless the row
is marked open; the canonical open-path list is in
[operator access](#operator-access). Read handlers enforce their supported
methods; wrong-method requests must not fall through to inference. The
namespace is standardized: `/metrics/*` and dashboard routes are the protected
read plane, and `/admin/*` is the operator action plane; the credential gates
both.
| Route | Action / important contract |
| --- | --- |
| `GET /healthz` | Unauthenticated liveness probe for Docker HEALTHCHECK and load balancers. Reports process/HTTP liveness only, no storage depth. Trailing-slash look-alikes (`/healthz/...`) are reserved and answer 404, never inference. |
| `GET /readyz` | Unauthenticated readiness probe. `200` when the durable pipeline can persist; `503` with a reason when the writer is wedged or a flush failed since boot. Compose uses it through `-readycheck`. Trailing-slash look-alikes are reserved and answer 404, never inference. |
| `GET /favicon.ico` | Unauthenticated ICO brand mark. Browsers fetch this without Authorization; it is registered on the mux so it never reaches the inference catch-all. |
| `GET /favicon.svg`, `/apple-touch-icon.png`, `/icon-*.png` | Ungated SVG/PNG icons for tabs, home screens and the web app manifest. |
| `GET /manifest.webmanifest` | Ungated web app manifest (name, standalone display, 192/512 icons). |
| `GET /sw.js` | Ungated service worker. Precaches brand icons/manifest; cache-first only for a `/dash/` URL carrying exactly the stamped version, network-first with cache fallback for unversioned or stale `/dash/` URLs and brand paths. Navigations to `/` are fetched live (never cached: login vs bootstrap) with a static offline fallback. Its never-cache list is generated at load from the same canonical namespace table that drives the gate and the reserved routes, so the three cannot drift; it never intercepts `/`, `/index.html`, `/healthz`, `/readyz`, `/models`, `/sw.js`, `/admin/*`, `/session/*`, `/metrics/*`, `/mcp/*` or `/v1`. |
| `POST /session` | The one open operator route: exchanges the credential for the session cookie the dashboard's live feed needs. Throttled like every gated route. Lives outside `/admin` so an external authentication layer can front the admin plane without gating millivolt's own login. |
| `GET/POST /admin/config` | Schema/file/effective state; save `{revision,values}`. Stale revision returns 409, and so does a save while a restore holds the config transaction (its upload is client-paced). Saved-but-reload-failed is explicitly reported. GET and save responses carry a `last_reload` section: `ok`, `error`, `dropped_keys`, `restart_required`, `at`. GET returns provider header values verbatim, because the settings sheet must round-trip them on save; redaction applies to the model-visible `config_get` tool and the operator CLI instead (see [cli](cli.md) and [mcp](mcp.md)), where `--reveal` is the opt-in. Posting a redacted document back is safe: a credential header whose value is the marker keeps the loaded secret, while a posted real value replaces it. The same applies to `set_config`, so a model can read and re-post without erasing credentials. |
| `POST /admin/reload` | Re-read config and report restart-required keys. Answers 409 while a restore holds the config transaction. |
| `GET/POST /admin/restart` | Status / rebuild. `GET ?watch=1` streams progress; concurrent starts are rejected, and so is a start while a config transaction (reload, Settings save, backup restore) holds the reload lock: `409` names the operation. |
| `GET/POST /admin/pause` | Inspect/add/edit/resume holds. POST requires `paused`; optional ID targets one hold. GET reports `queued` (admission and send-gate waiters, the populations an operator drain can release) and `storm_waiters` (requests parked by the storm/quota gate before admission, which only the gate opening or a quota resume frees). |
| `GET/POST /admin/quota` | Inspect storm state (shared with the banner) / `POST {"provider","resume":true}` closes a retry-mode quota gate; parked sends resume immediately. |
| `GET/POST /admin/debug` | Inspect/add/edit/stop capture sessions. POST requires `enabled`; starting a session requires durable storage (503 `durable storage is disabled` otherwise); optional ID targets one session. |
| `GET/POST /admin/throttle` | Inspect and edit limits. A policy is named by its opaque `id` or by its selector. Scope dimensions `providers` (the singular `provider` field is still accepted), `models`, `clients`, `key_hashes` and `conversations` intersect; an empty dimension matches any value. An optional schedule uses `days` (0 = Sunday), `start_minute` and `end_minute` in server-local time; equal minutes mean all day and a start after the end spans midnight. POST accepts `concurrency`, `requests`/`request_window`, `tokens`/`token_window`, `anti_burst` and `ramp_duration`; supplied limits merge. A count requires its own window only when the policy has no stored window for that dimension: a fresh count without one is refused, while a count may raise an already windowed dimension alone. A count of `0` clears that dimension and its window. A window without a count is a refused no-op unless a count is already stored. `clear:true` drops the whole policy. Anti-burst and its ramp require a single-provider policy without a schedule. GET also returns `known_providers`, `known_clients` and `known_models` for selector pickers. |
| `POST /admin/queue/drain` | Release parked requests without waiting for their limit, pause hold or retry window, covering both the admission queue and the send gate. Requires exactly one scope: `all:true`, `provider`, `model` or `client`. Released requests answer with a retryable 503, never a provider error; a request already exchanging bytes with the provider finishes normally. |
| `GET /admin/debug/capture?id=` | Load an unexpired durable debug sidecar; absent/no-store returns 404. `&download=1` serves the same document as a gzip artifact named `millivolt-debug-.json.gz`, falling back to the record id when `captured_at` does not decode. |
| `POST /admin/purge/count` | Preview the same deletion/export predicate; traffic can change the count afterward. A genuinely bodyless POST counts the whole set, mirroring the bodyless delete; otherwise the body is `{"filter":{...}}`. Filters are body-only, so any query key is rejected with 400 before the body is read. |
| `POST /admin/purge` | Delete matching finalized records. A genuinely bodyless POST deletes everything: that is the deliberate full-history command, it answers `200` rather than `404`, and any query key must be moved into a body instead. Otherwise the body is `{"filter":{...},"reviewed_count":N}`: an empty/invalid filter is rejected, and a supplied reviewed count is counted again inside the delete transaction, refusing with 409 when the row set moved. `status_code=0`, `debug=0` and `has_error=0` mean "no constraint" on that dimension, not a match for zero or false. The response is `{"deleted":N,"buffer_removed":N}`. Filters are body-only: any query key is rejected with 400 before the body is read, so `?provider=...` can never broaden the no-body full delete. |
| `GET /admin/backup` | Download the self-checked `.mvb` Settings backup archive of config and/or database members. `?config=1` and `?database=1` select members, defaulting to both when no key is given; an explicit empty selection is an error. The query grammar is strict: parse errors, repeated or unknown keys, and malformed values all return 400. |
| `POST /admin/restore` | Restore an uploaded `.mvb` Settings backup archive, validated before anything is applied. `?inspect=1` previews the archive (created time, member sizes, the config values plus their changed-key lists against defaults and the live config, and the database request span when a live store stages the member) without applying anything. A store-less preview still reports the member's presence and decoded size with `inspectable:false` and the reason. `?config=1` and `?database=1` select members as on backup; `?config_mode=` and `?database_mode=` take merge or replace and default to replace when omitted; an explicit empty or whitespace value is rejected as malformed. Success reports the `restart_required` keys. The upload is exempt from `request_body_read_timeout` (the idle bound still applies) and an expired read deadline answers 408. The query grammar is strict: parse errors, repeated or unknown keys, and malformed values all return 400. |
| `GET /metrics/export` | Download all or exactly filtered finalized records as a gzip artifact named `millivolt-logs-.json.gz`; gunzip it for the JSON array. `status_code=0` and `debug=0` mean "no constraint" on that dimension. A debug capture rides as a sibling `debug_capture` member only for a row that has a stored sidecar, so debug bodies appear only while durable storage is live; a ring-only export never carries them, and debug bodies stay sensitive either way. |
| `GET /metrics/query?q=` | Restricted SELECT with timeout/output limits; 503 when durable storage is disabled. Not a hostile-query sandbox. |
| `GET /metrics/bootstrap` | Dashboard state and full/incremental recent-record snapshot. |
| `GET /metrics/live/stream` | Replayable finalized SSE feed plus ephemeral pending lifecycle/reset events. |
| `GET /metrics/agg/chart`, `/explorer`, `/log` | Scoped history chart, faceted explorer, and durable log paging. The Requests-card `s=` selector accepts an exact code or a live in-flight class on all three, but only the explorer folds pending rows: the chart answers a live class with `400 invalid scope` (its buckets are finalized-only, so an accepted class could only read as "no traffic"), and the log keeps its standing empty page so the archive pager does not stop. |
| `GET /metrics/prometheus` | Prometheus exposition over the in-memory ring, not durable since-inception dashboard totals. |
| `GET /metrics/pprof/` | Standard Go profiling handlers behind the same credential as every `/metrics` route: the index listing plus the `profile` (CPU), `heap`, `allocs`, `goroutine`, `block`, `mutex` and `threadcreate` profiles at `/metrics/pprof/`. Unregistered subpaths answer the reserved namespace 404, never inference. See the capture workflow below the route table. |
| `POST /mcp` | Streamable HTTP MCP endpoint, operator-gated like `/metrics` and enabled by default. `mcp_enabled: false` removes it from the process; the route stays reserved, answers 404 and never reaches inference. Serves the 25 MCP tools, Bearer-only and stateless. See [MCP transport](#mcp-transport) below. |
Pause/Debug/Limits successes may include a persistence warning: runtime state
was applied, but saving it failed. Do not retry as though the mutation rolled
back. With storage disabled, Pause and Limits keep their memory-only state
intentionally. Debug is the exception: starting a session answers 503 because
with no store every matched request would retain bodies for a sidecar that
could never be fetched.
### MCP transport
The gate admits the session cookie on gated paths, but this endpoint refuses a
cookie-only request with `403`. It forwards the caller's own Bearer on its
in-process calls to the routes above, so no session cookie or per-request state
is retained between requests.
The admitted credential keys an in-memory cache of its SDK server object. The
cache is never logged or persisted, and it lets repeated calls avoid rebuilding
the 25 registrations. `GET` and `DELETE` answer `405`.
`/mcp/...` look-alikes are reserved and answer `404`, never inference. A
reserved-namespace spelling whose separator is single-encoded (`%2f` or `%5c`,
any hex case) is refused the same way. A double-encoded separator (`%252f`) is
not a reserved-namespace spelling: it decodes to a literal `%2f` inside one
segment, so it can reach neither the MCP handler nor a reserved namespace and is
ordinary inference traffic.
Three tools, `docs_list`, `docs_search` and `docs_read`, answer from the
embedded documentation index and make no proxy request.
The profiling subtree is operator-gated like every `/metrics` route: an
unarmed plane answers 403, a wrong credential 401, and a reserved-namespace
spelling with an encoded separator answers the reserved 404. `go tool pprof`
cannot send an `Authorization` header, so fetch the profile with curl and
analyze the file against a locally built binary. Against the dev instance
([`scripts/dev.sh`](../scripts/dev.sh), loopback `127.0.0.1:8081`):
```sh
go build -o /tmp/millivolt-profiling ./cmd/proxy
curl -fsS -H "Authorization: Bearer $MILLIVOLT_OPERATOR_TOKEN" \
-o /tmp/cpu.pprof 'http://127.0.0.1:8081/metrics/pprof/profile?seconds=30'
go tool pprof /tmp/millivolt-profiling /tmp/cpu.pprof
```
The `seconds=N` parameter on `/metrics/pprof/profile` is the capture window:
that one request stays open for the whole window while samples accumulate. The
valid range is 1 to `response_write_timeout` (600 by default); an absent or
unparseable value uses 30 and a larger one is lowered to the bound, so a
capture never outlives the delivery contract unnoticed.
A process runs one CPU profile at a time, so a second concurrent `profile`
request fails with 500 until the first finishes. On `/metrics/pprof/heap`,
`gc=1` forces a garbage collection cycle before the sample.
The proxy's own `/mcp` endpoint serves this whole surface to an LLM, using the
same credential and the same routes. It adds no route of its own and dispatches
each tool call in process through the same gate, so its forwarded Bearer is
re-validated exactly like an external request.
For its irreversible call, the
operator credential and the tool's own guards are the entire gate: an empty
purge filter, a purge without the exact confirmation phrase, and a purge
without the `purge_preview` token issued for that same filter are each refused.
The preview token expires 15 minutes after it is issued, so a purge more than
15 minutes after its preview needs a fresh one.
It also refuses a capture session on a proxy without durable storage, and an
unknown name in a capture scope or a throttle provider is offered the known
values instead. The throttle provider check fails open when the proxy's
vocabulary is empty or cannot be read; the capture scope check accepts any name
when the vocabulary is empty but refuses when the vocabulary cannot be read.
It never follows a redirect, and it presents the credential only in the
`Authorization` header. See the [MCP server guide](mcp.md).
### Operator access
`MILLIVOLT_OPERATOR_TOKEN` is the single operator credential for the whole
dashboard. It is read once at boot from the process environment, never from
proxy.yaml, so it cannot leak through Settings, `-print-config`,
`/admin/config` output, snapshots or logs.
The gate is deny by default and lives in one chokepoint in front of every
route. This is the canonical open-path list; [Security](../SECURITY.md) and the
[reverse-proxy guide](reverse-proxy.md) summarize it instead of maintaining a
second list.
- Open: `GET /healthz`, `GET /readyz`, the `POST /session` login handshake (the
one open operator route), origin-root brand/PWA files (`/favicon.ico`, icons,
`/manifest.webmanifest`, `/sw.js`) and transparent inference. Provider
credentials ride the same header name and are never inspected by the gate.
- Gated: the dashboard HTML and `/dash/*` assets, every `/metrics/*` surface
and every `/admin/*` route, including unclassified future admin paths and
unsupported methods, plus the `/mcp` endpoint and its reserved `/mcp/*`
look-alikes. A gated MCP request also needs its credential as a Bearer
header: the session cookie alone is refused there (`403`), because the
caller's credential is forwarded on the endpoint's internal calls.
Present the credential as `Authorization: Bearer `. A valid Bearer
request on a dashboard, admin, metrics or dash route also mints a session
cookie, because the dashboard's live feed uses EventSource, which cannot send
Authorization headers. The `/mcp` endpoint deliberately returns no cookie: its
caller forwards the credential on the endpoint's internal calls.
The cookie is HttpOnly (script never reads it), SameSite Strict, scoped to
12 hours, and signed with a key derived from the operator token, so a
container restart with the same credential keeps the session. Rotating the
token invalidates outstanding cookies.
The cookie carries `Secure` only when the request arrived over TLS or
any `X-Forwarded-Proto` token names a secure scheme (`https` or `wss`). The
header is a list: the canonical NGINX sample overwrites it with this server's
`$scheme`, and an ingress that appends its own scheme after the client's
`http` must not downgrade the cookie - hence any token, not the first. A
loopback plain-HTTP dashboard keeps working
while a TLS presentation gets the flag.
Every response to a secure presentation also carries
`Strict-Transport-Security: max-age=15552000` (180 days), set once at the
outermost server wrapper from the same secure-presentation test. There is no
`includeSubDomains`, because millivolt cannot know whether sibling or parent
hostnames are TLS-capable. A plain-HTTP response carries no header at all
(browsers ignore it there), so the dev instance is unaffected.
The lifespan is
deliberately under a quarter: moving the dashboard off TLS becomes reachable
again within that window instead of pinning a browser for a year.
Do not expose the listener over plaintext networks anyway.
The login page served for unauthenticated dashboard visits exchanges the
entered value for that cookie through `POST /session` and works without JavaScript; API
clients can call the same endpoint or simply send the Bearer header on every
request. An expired cookie re-prompts in the dashboard or reappears as the
login page on navigation.
Unregistered `/admin/*`, `/session/*`, `/metrics/*` and `/mcp/*` paths are reserved: they
answer 404 and are never forwarded upstream. The gate also refuses 404 when an
owned-namespace path is written with a percent-encoded separator (`%2f` or
`%5c`, any hex case), a duplicated separator (leading or interior) or a dot
segment: the mux cannot route it to the reserved subtree unchanged, so it
would otherwise fall through to inference.
A double-encoded separator
(`%252f`) is not a reserved-namespace spelling: it decodes to a literal `%2f`
inside one segment, matches no reserved route, and is ordinary inference
traffic.
Failed authentications are throttled per source IP. The immediate hop is the
only trusted one, so the throttle key is the rightmost `X-Forwarded-For`
element (the address that hop appended) and `RemoteAddr` when the header is
absent; a client can prepend anything, and an unparseable element is skipped.
The shipped [NGINX example](../deploy/nginx.conf) resolves its own peer with
`real_ip` first, so the rightmost element is the real client. A directly
exposed instance lets a client choose its own key, which only ever relaxes the
throttle, never the credential decision: after five rejected credentials the IP
is locked out with a window that doubles from 10 seconds to a 15 minute
ceiling.
The throttle counts only
presented-but-wrong credentials, so an expired-cookie dashboard polling in a
background tab cannot lock its own operator out, and a verified credential is
always admission: presenting the correct Bearer during a lockout clears the
budget, because the lockout exists to rate-limit guessing, not to lock the
operator out of their own dashboard.
The map is hard-capped at 4096 source
IPs with idle expiry, so spoofed-address floods cannot grow it; reaching the
cap is logged once per episode, because from then on a new source evicts a
tracked one. Crossing the five-failure threshold is logged once per source.
A lockout response is `429` with a coarse rounded `Retry-After` and a generic JSON body,
except `GET /` (and `/index.html`): the sign-in page is still served, read-only, with the
same `Retry-After`, the form intact and the lockout named in its notice slot, because
the form is the only route back to a credential and `POST /session` keeps working
through the lockout.
With the variable unset or empty the deny-all state holds: the whole dashboard
returns 403, the open `POST /session` handshake cannot mint a cookie, and only
`/healthz` and `/readyz`, brand/PWA files and inference respond.
A nonempty value shorter than 16 or longer than 512 bytes fails the boot
instead of silently weakening the gate.
The credential is process bound: config reload does not re-read it,
so rotating the value requires a real process restart (a supervisor restart
or a fresh deployment; the dashboard's restart handoff inherits the parent
environment, so it keeps the old credential by design).
The dashboard asks for the value once per browser tab, keeps it in
sessionStorage, attaches it to gated calls only and drops it as soon as the
server rejects it. Non-browser callers send the header directly, for example
`curl -H "Authorization: Bearer $MILLIVOLT_OPERATOR_TOKEN" -XPOST .../admin/purge`. Environment variables are readable by same-user processes through `/proc`,
so run the proxy under a dedicated account.
### Pause and Limits
Pause creates holds; it does not disconnect already-running sends. Matching new
sends and retries wait until the hold expires or is resumed. Choose all traffic,
any combination of the five scope dimensions, or clients that were unseen when a
new-clients hold was created. "New" does not mean new requests or new
conversations.
A hold scopes on the same five dimensions a limit does: providers, models,
clients, upstream key hashes and conversations. Values inside one dimension are
alternatives, an unset dimension matches any value, and set dimensions
intersect - the same rule limits use, because the scheduler matches both against
one request identity. When named clients and new are combined, either client
condition may match.
A hold has its own duration and queue cap; the cap defaults to `max_queue_size`
when the hold does not set one. Excess waiting work can be refused;
an indefinite hold needs an explicit resume. Existing holds can be edited in
place or resumed individually, while Resume all clears all holds. Overlapping
scopes are rejected instead of silently replacing a different hold.
The HTTP shape is closed. A resume carries only `paused:false` and an optional
`id`; supplying a scope, duration or queue cap beside it is a 400, because the
proxy has no scope-selective resume and accepting one would clear more holds
than the caller named.
`all:true` cannot be combined with `new` or any scope
dimension; the global hold already covers them. Named clients plus `new`
remains a valid union. An edit restates the dimensions it supplies and preserves
the ones it omits, so supplying an explicit empty array is how a dimension is
cleared. These are enforced by the handler, not only by the MCP and CLI
wrappers.
The known client/provider/model vocabularies (the pause and capture pickers,
and the snapshot a new-clients hold compares against) are bounded at 4096 names
per dimension.
Client labels and model ids come from the request, so they are
evictable: past the cap the least recently observed name makes room for the
newest, so a client that keeps sending requests outlives a burst of one-shot
labels that arrived after it.
Names an operator documents - a hold scope, a debug session, a limit policy
(including one installed from a request's `X-Limit-*` headers or restored at
boot) - are pinned and are never evicted by later traffic, so a flood
of distinct labels cannot hide the client you scoped a hold to.
Pins are derived from the documents that exist, not accumulated: resuming a
hold or ending a debug session releases its names back to ordinary eviction.
When a full vocabulary is entirely pinned, a new observed name is dropped
rather than displacing operator intent, and a scope that cannot be listed is
named in the log instead of disappearing silently.
A hold scopes on the caller's claimed client label, so it separates your own
tools from each other, not trusted callers from untrusted ones; a caller that
sends another client's label is parked, or not parked, as that client. See
[client identity](../SECURITY.md#client-identity-is-a-claim-not-an-authentication).
Limits applies to an intersection of conditions: providers, models, clients,
key hashes and conversations. Values inside one condition are alternatives; an
empty condition matches any value, so a rule with no condition covers every
request. A request must satisfy every non-empty condition, and every matching
policy charges its own lease, so the tightest cap wins.
A provider-only rule keeps the per-key concurrency and fairness behavior
below; adding any other condition narrows the same machinery without
changing it. Conditions select the request's recorded identity, so models
match the request spelling, keys match the SHA-256 digest, and conversations
match the selected leaf.
A rule may also carry a recurring schedule: `days` (0 = Sunday), `start_minute`
and `end_minute` in the server's local timezone. Equal minutes mean all day, and
a start after the end spans midnight, with the after-midnight portion belonging
to the previous day. An empty schedule is always active. Outside a rule's window
it does not charge, settle or queue, and a request that matches no other rule
proceeds normally.
A policy's identity is the digest of its normalized conditions and schedule.
Editing either one therefore creates a new identity, and two rules cannot
silently share counters. Provider-only policies keep the version 1 persisted
shape and still accept the singular `provider` field; a rule with any other
condition persists its selector arrays. `clear:true` removes the whole policy
and cannot be combined with a limit dimension: the handler refuses the
ambiguity instead of dropping the supplied value and reporting success.
Choose concurrency, requests per window, tokens per window and the optional
anti-burst ramp. The console shows whether a policy came from the console or a
request header, its schedule, and available budget and in-flight/queued state.
Queued counts every request the scheduler parks for that policy - admission
waits and send-gate waits behind a 429/5xx pacing window alike - not only work
blocked by the configured cap. Send-gate waits are paced by the provider's retry
hint and the retry budget; `max_queue_size` and `max_queue_wait` bound only the
admission FIFO.
Provider admission is fair across keys: a release or refill
rotates across the groups actually waiting on the shared provider gate
(concurrency or the request/token bucket), granting one request per group per
pass. A group parked only on its own pacing window, hold, anti-burst ramp or
per-key cap is outside that fair set and does not delay another key's fast
path. A release settles the gate and drains once, and the drain stops as soon as
a grant refills a saturated provider concurrency cap.
Header semantics are in
[protocol](protocol.md#routing-headers). Persisted policy does not mean the live
scheduler's counters and in-flight requests survive a restart.
**Drain queue** releases every request the scheduler has parked, or only those
whose recorded provider, model or client matches the one named in the console,
without waiting for a limit window, pause hold or retry pacing to open. The
selector reads each parked request's own identity, so it also releases work
parked by a hold, by pacing, or by a provider without a limit row. The overview
shows the total parked count.
Each released request answers with a retryable 503
and is recorded as `queue_drained`, never as a provider error; a request already
exchanging bytes with the provider finishes normally. The API also accepts the
same three scopes for scripted drains.
Anti-burst is opt-in and scoped to each provider+key group. It is available
only on a single-provider rule without a schedule, because a ramp is defined
over the provider-plus-key groups the scheduler owns; the server rejects the
pair on any other rule.
With `anti_burst` enabled and a concurrency target of at
least 2, a group starts at one concurrent request and gains allowance linearly
until it reaches the rule's `concurrency` target after `ramp_duration`. It is
inert unless all three are set: the flag, a target of at least 2, and a duration
above zero.
`ramp_duration` accepts the same 1s..24h window grammar as request and token
windows, including `1d`; empty, `0` or `0s` turns it off. Requests beyond the
current allowance stay in the existing per-key FIFO. A fully idle group whose
idle period reaches `ramp_duration` starts a new epoch on its next grant, while a
shorter idle gap continues the existing one.
Enabling the ramp while a group
already has in-flight work anchors its epoch at the current occupancy, so
running streams are not stalled. Changing the flag, duration or concurrency
target resets every group's epoch.
Header updates change only the three cap
dimensions and preserve the stored anti-burst pair. Clearing the last active cap
drops the whole policy, ramp included. The limit composer carries the checkbox
and the ramp chips; applying sends both, and a form whose caps are all zero
sends the pair off rather than an invalid on-ramp.
Operator limit edits persist synchronously and report a warning when the store
rejects the write. Header-driven limit changes apply in memory immediately and
are persisted by one coalescing background writer: a burst of changes becomes
the latest whole-policy document, with bounded retry and backoff after a store
failure. A closed store is not retried - the generation stays dirty and the
writer stops - while a busy one keeps the ladder. That document also carries
the anti-burst pair, and one written before
those fields restores them off.
A crash can therefore lose the most recent
header-driven changes, and a failed save never rejects inference. Graceful
shutdown and restart flush the latest policy before storage closes or the
restarted process opens the database. Provider labels are capped at 255 bytes
and the persisted throttle document at 8 MiB; normal provider labels derived
from base URLs remain below both bounds.
Admission is bounded end to end.
- `max_inflight_requests` is the process-wide
inference ceiling (default 10000).
- `max_inflight_body_bytes` is its buffered
body budget (default 256 MiB; admission charges the allocation hint, the
declared length bounded by `max_request_bytes` and the read hint, and reconciles
it against the bytes actually read, so a short body refunds the difference and a
chunked one is charged as it arrives).
- `max_inflight_response_bytes` is its
budget for buffered upstream response bodies (default 256 MiB): whole-document
translation of non-streaming responses, the non-streaming 200 quality spool
(at most 1 MiB plus one 32 KiB chunk), and model discovery.
A request-slot refusal is answered with `503` and `Retry-After` before the body is read.
The body budget charges the allocation hint at admission and reconciles it
against the bytes actually read; when the hint cannot be covered the slot is
still taken, and the first read decides without retaining the upload. The
per-request cap still owns the status: a refused request drains at most the
remaining `max_request_bytes` window so an over-cap body answers the definitive `413` instead
of a retryable `503`, while a body that fits the cap answers `503`.
Request-body reads must make progress within
`request_body_idle_timeout` and finish within the absolute
`request_body_read_timeout`, so a stalled peer releases its reservation
without cutting off a slow but progressing upload. The `POST /admin/restore`
upload is exempt from the absolute bound only: a full database archive can
legitimately outlast it on a slow link, while the idle bound still cuts a
client that stops sending, and an expired deadline answers `408` with its own
message instead of the archive-cap `413`.
Buffered whole-document
response reads use `whole_document_read_timeout` when the send has no deadline
and `whole_document_read_max` as the ceiling even when `X-Proxy-Timeout-Ms` is
longer, so a slow or stalled body releases its response-buffer reservation; a
shorter caller deadline wins and configuration validation preserves the
`models_discovery_timeout` budget below the ceiling. The three admission
budgets accept `0` = unlimited and hot-reload.
`max_queue_size` (default 1024
per group) and `max_queue_wait` (default 10m) bound a held or saturated
provider's parked work; both accept explicit `0` = unlimited. Independent
per-key/header concurrency remains unlimited by default.
The scheduler retains at most 100000 distinct provider+key groups, evicting
the least-recently-idle group (no in-flight or queued work) to admit a new
key at the bound; a new key is refused only when every retained group has
active work, so a table full of active groups still fails closed. Recent
pacing/backoff state for evicted groups is kept in a separate 4096-entry
bounded LRU. The scheduler also retains at most 4096 header-created provider
limits, refusing new ones past that bound rather than growing process memory
without limit.
A new request body has the `request_body_read_timeout` budget, SSE writes have the `sse_write_progress_timeout` per-event
progress bound, and an upstream body that stops sending for `upstream_stream_idle_timeout` is canceled.
Request budgets are strict sliding windows: at most the configured count is
admitted in any trailing window of the configured length, so an idle provider
banks nothing and a busy one cannot spend future allowance as a present burst.
Charging the window is what makes the limit match a provider that counts
requests per rolling or fixed minute; a refilling budget could emit the whole
count in one instant and then refill through the same span, which is how a
`100 per minute` setting reached a provider's `101/100`.
Each admitted request is charged once, at proxy admission, not again for
each absorbed upstream retry. A count above 65536 is enforced by even
spacing at the same ceiling instead of one timestamp per slot, so the memory
a window holds stays bounded.
Token budgets remain refilling buckets: prompt tokens are reserved from the
captured characters divided by four and then settled against final reported
usage, and a reservation larger than the whole capacity may proceed while
credit is positive, leaving debt that refill or settlement repays. The
client's output ceiling (`max_tokens` and its spellings) is deliberately not
reserved: it is a ceiling, not expected usage, and reserving it would turn
the token budget into a worst-case concurrency cap.
A token budget therefore also sets the sustained
average rate: 500K tokens per 6h refills at about 23 tokens/s, so once the
initial budget is spent, later requests wait for refill unless actual usage
settles lower than the reservation.
### Error storm protection
Open Settings, select **Error storm protection**, and turn on **Enable
protection**. It is opt-in in both neutral defaults and the deployment example.
Choose the rolling window, minimum samples and failure threshold, then select
HTTP failure statuses, transient transport failures and response/stream failures.
Provider protection requires all models receiving requests in that same window
to meet the thresholds individually. A single affected model is held on its own
while other active models remain healthy. If only one model is active, that
model can establish a provider-wide incident. Provider and model protection can
also be enabled independently.
Configure initial/maximum recovery delays, multiplier, jitter, recovery successes,
additional retries, queue capacity, per-wait timeout and retained scope capacity
in that category. The generated [example](../proxy.example.yaml) supplies every
type, range and default. Existing Queue & retry limits and provider Limits still
apply; neither `storm_max_wait` nor a per-send deadline is a total request timeout.
Use a client timeout appropriate to the desired end-to-end wait.
Each clickable banner identifies its current scope and safe failure
category (HTTP status, transport error or response error), percentage/count of
failed upstream attempts, detection window, queued callers and next probe or
recovery progress.
Open an incident to inspect the distinct requests that
encountered a selected failure, failed attempts, total sampled attempts, error
category, queue count and next retry time. Provider entries also show affected
versus active models.
Distinct request counts follow each request's latest selected failure in the
rolling window, so repeated retries do not inflate that count. The percentage
still describes upstream attempts. These are process-local observations, refreshed
through the normal dashboard bootstrap/poll cadence; they are not chart history or the
Errors chart's distinct-request percentage. Scope counts can overlap where a
caller waits on both provider and model protection. Hide the banner independently
without disabling protection.
Changing a storm policy resets its samples/incidents and wakes waiters to
re-evaluate the new policy. Banner visibility alone preserves detector state.
A reset does not cancel work already sent or erase request history. Restart
starts detection afresh, and queued requests do not persist. History deletion
does not reset live outage protection.
Storage stays outside the detector: fixed 64 time buckets retain counters per
scope, with oldest-edge precision within approximately `storm_window / 64`.
`storm_max_scopes` includes provider aggregates and exact provider/model scopes;
stale unreferenced healthy scopes can be evicted, while active or referenced
scopes remain protected.
Scope identifiers over the internal 512-byte retention
bound and exhausted scope capacity reject admission rather than silently bypass
protection. These bounds are not a total heap ceiling: waiting callers retain
bounded request bodies, and existing ordinary queues remain separately sized.
The [protocol guide](protocol.md#error-storm-protection) defines retry safety,
quota exclusions, native handshakes and client-facing failures.
### Quota pause
Open Settings, select **Quota pause**, and choose the reaction to a 429 whose
error envelope is a durable account condition - `insufficient_quota`,
`insufficient_credits`, spend/hard limits, the recognized numeric billing codes
(the built-in classification is fixed and `quota_error_classes` may add
spellings; only the reaction is configurable):
- **off** (default): the 429 surfaces to the client immediately. Nothing is
parked or retried.
- **retry**: a provider-wide recovery gate opens on the first such 429 from an
OpenAI-wire provider. The triggering request absorbs the 429 and re-sends
per the error storm recovery delays; every new request to the provider
parks on the gate (bounded by `storm_max_queue` and `storm_max_wait`); the
gate closes after `quota_pause_recovery_successes` consecutive successful
probes (the default 1 closes on the first 2xx) and the queue drains. A provider
`Retry-After` on the quota 429 paces the next probe like any other 429.
- **manual**: an indefinite provider-scoped pause (an operator hold labeled
with the quota class) is created automatically; every request, triggering
one included, queues until an operator resumes it on the pause surface.
When a pre-existing hold already covers the provider, no duplicate is
created: the triggering request queues under that hold only if it parks
the triggering client too, and a hold that parks other clients only lets
the triggering request surface the 429 immediately.
The retry-mode gate shares the storm scheduler's parking, probes and pacing,
so storm recovery settings configure it without storm protection being
enabled. It surfaces in the storm banner as a quota incident with a
**Resume now** action (`POST /admin/quota`); manual-mode holds appear on the
pause surface with their reason and resume like any hold, and they derive their
provider pin through the same commit owner as a manual pause so the provider
stays offered in the scope pickers.
A quota 429 never arms the rate-limit pacing trip (the rate-limit health
series still counts the 429 itself), and quota re-sends never burn the
transient `max_retries` budget. They are bounded by `storm_max_wait` instead, measured from the
request's first park: when it expires the client receives the canonical
retryable `error storm protection: ...` 429 rather than waiting on a gate that never opens, and the
provider stays parked for the next request.
The retry gate is in-memory - a restart drops it and
the provider's next durable quota 429 re-arms it - while manual-mode holds
persist like every operator pause. In-band occurrences (the same envelope
inside a 200 stream) arm the pause the same way; that request already
answered, so it is never parked.
Translated Anthropic-format and Cursor
upstreams keep the off behavior in every mode. The built-in vocabulary is
fixed; `quota_error_classes` may add a gateway-specific spelling as a durable
quota class (it may never move a built-in spelling between classes, and it
re-reads stored history through the same vocabulary).
### Debug and preview capture
Debug selects clients, providers and/or models and starts a capture session for
a chosen duration or until stopped. At least one dimension is required; selected
dimensions intersect, with alternatives inside each dimension. It does not have
an unrestricted "all" switch. Sessions can be edited or stopped individually;
Stop all ends every active session. A stop carries only `enabled:false` and an
optional session `id`; a scope or duration beside a stop is a 400 rather than a
silent stop-everything. Conflicting scopes are rejected.
Debug model matching uses the native-model normalization in
[debug.go](../internal/proxy/debug.go), separate from display-only model rules.
Do not assume that every model spelling is an independent exact scope.
The session timer selects future captures. `debug_capture_ttl` separately
controls retained capture lifetime after completion, and
`debug_capture_max_bytes` bounds one capture document: retained headers are
charged first and the remainder is split between the request and the
client-facing response body. Oversized captures are marked truncated.
`debug_store_max_bytes`
(default 512 MiB) caps the whole capture sidecar; once the next write would
exceed it, new captures are skipped and TTL expiry frees capacity again.
Capture writes are counted in bytes, not characters.
The request drawer loads an
available sidecar only for a captured request; expired/absent captures are not
recreated from ordinary metrics. Completion is published before the sidecar is
encoded and saved, so a slow sidecar write delays neither the live end nor
durable accounting. A drawer fetch that follows the live end waits for that
request's in-flight save instead of answering a transient 404; a disconnected
client stops waiting.
Starting a capture requires durable storage. With `db_path` empty,
`enabled:true` without an ID answers 503 `durable storage is disabled` and
creates no session. A store-less process
also has no persisted session document to restore, so it never starts with an
enabled session; if one were present in memory, the request path still refuses
to build a capture tap, so no headers or bodies are retained for an unfetchable
sidecar. Editing or stopping a named existing session remains available.
Retained request headers use the same policy as recorded response headers:
- names are visited in sorted order for the budget;
- each value is capped at 4 KiB and marked with `…` when cut;
- at most 16 values are kept per name;
- ordinary headers share a 16 KiB total budget per header map;
- a header name is capped at 512 bytes;
- at most 256 names are retained per map.
A longer name or an extra
name is omitted without a marker, exactly like a value beyond the budget.
Credential headers (the
configured auth header, `Authorization`, cookies, and token-, API-key- or
secret-bearing names) become `[REDACTED]` before any truncation, so a cut
value is never a credential prefix. Headers or values beyond the budget are
omitted without a marker; the capture schema has no header-truncation field.
The capture JSON is encoded without HTML escaping, so `<`, `>` and `&` appear
literally while JSON control escapes remain.
The drawer's Debug section also carries a Download button. It fetches the same
capture with `download=1` and saves a gzip artifact named
`millivolt-debug-.json.gz`, using the record id in the name when
the capture does not carry a readable `captured_at`.
Captured request and response bodies render through the same bounded code
surface as record previews, with wrap, copy and expand controls; the
Download button remains the whole-capture path. The capture cards also
surface its schema and captured-at time, conversation and key correlation,
first-answer timing, usage, cost and attempt count.
`capture_body_preview` is a different opt-in feature: it retains short prompt/
response previews in the normal record without starting a Debug session. The
two features also retain different request bytes: on the passthrough wire
the record's prompt preview is the client's original body, captured at the
metadata decode before `request_overrides` or the sub-conversations strip
rewrites it, while a debug capture's request body is the rewritten upstream
body sent after any rewrite.
A translated Chat Completions or Anthropic target
re-decodes the translated body for its prompt preview; a translated Responses
target keeps the client-body prompt preview, because the generated item list is
not re-decoded for preview. The response preview follows the client-facing
response content on every target.
Both features can retain sensitive content
even when credential headers are redacted.
Exports/backups can outlive the configured retention period. See
[Security](../SECURITY.md#upstream-destinations-and-credentials).
### Export and deletion
The operator console's export section downloads finalized request records as
JSON, not the process's stderr log and not a complete administrative audit
trail. With durable storage it reads the database; otherwise it uses available
ring records.
Open export, choose either all history or matching records, and
review the previewed count before downloading.
Provider, model, client and
conversation merge the server's known vocabularies with whatever the live record
ring holds, so a quiet proxy can still filter by what has run since it started,
and each of them also accepts a typed value. Status is the one list-only
dimension, because a status code is not a name anyone can type reliably. A
debug-only export can include retained capture sidecars.
A debug capture is written beside its record as a `debug_capture` member, and
only when the row still has a stored sidecar. The artifact therefore carries
debug bodies only while durable storage is live: a ring-record export, or a
sidecar already reclaimed by the capture retention sweep, has the record alone.
The bodies remain sensitive either way.
The export predicate has no constraint for `status_code=0` and `debug=0`; both
are off rather than a match for the zero status or the false flag. A
genuinely bodyless `POST /admin/purge` deletes everything in one call, which is
why a mistyped query key is refused instead of silently ignored.
The download is always a gzip artifact named `millivolt-logs-.json.gz`;
gunzip it to read the JSON array of records. Compression is applied at the
export writer itself, so every export shape (filtered, all or debug-only,
from the ring or durable storage) downloads compressed the same way. This is a
breaking change for scripts that consumed the plain JSON download: there is
no uncompressed fallback, so scripted consumers must decompress the artifact
first.
A durable export holds one read snapshot for the whole stream, so the server
bounds it with an internal total budget. If the budget expires after bytes were
written, the download is aborted mid-artifact instead of completing as a
successful gzip: the saved file is partial and must be discarded. A caller
cancellation or shorter caller deadline is reported separately from this
server budget.
In the dashboard, both database deletions live in Settings, in the storage
category, and nowhere else. Delete matching uses the same filter/count owner
as the export, and its scope is the chosen filter; Delete everything wipes
the whole database. Deletion requires confirmation in the dashboard; it does
not happen when the sheet opens.
A selected filter makes a read-only
`POST /admin/purge/count` preview; the count first commits the writer's queued
batch into the read view, so a just-finalized request cannot read as empty.
That barrier is bounded by the query timeout: a wedged writer lets the preview
answer from the committed snapshot instead of spinning the sheet forever, and
the delete's transactional re-check stays the guard.
Neither the caller's deadline nor a client cancel fails the preview, even
when it expires before the barrier does: each step runs detached on its own
query budget, so a short dashboard deadline answers the same as an unbounded
one.
A ring-only deployment removes through the boundary the count saw, so a
record that finalizes during the delete stays behind instead of disappearing
uncounted.
Current traffic can still change the count
before the action, but the UI retains the previewed age cutoff rather than
silently moving it, and sends the rendered count with the delete so the
proxy re-checks it inside the delete transaction and refuses a moved row
set with 409 instead of deleting unpredicted rows.
A 409 drops the sheet's
preview and re-counts, so the operator confirms the new set instead of
repeating the refusal. The filter selections reset every time the sheet
refills, so a stale invisible filter never arms a destructive default.
Both actions are irreversible without a backup.
Every purge, backup and restore writes one audit line to the process log with
its terminal outcome, before the response is written: the action, the rendered
selection (`filter="all"` for a bodyless full delete, `filter="unparsed"` while
the body or query has not been accepted, and `members="unparsed"` likewise for
a backup/restore), the previewed count when
one was supplied, and the committed `deleted` / `buffer_removed` counts, or the
error.
Even a rejection at the method gate or the query grammar, before the
selection exists, leaves a line. Each value is quoted with Go string escaping,
so a label carrying a newline or a quote stays one token on one physical line. A
lost or truncated response therefore stays diagnosable, and a
retry can be compared against what was actually attempted. These lines are the
destructive-action trail; they are not a general request log.
Export/delete filters are exact raw record fields, not display-name model
canonicalization. The API also supports conversation/error-type/time filters
beyond the console's common choices. See the shared
[filter decoder](../cmd/proxy/log.go) and
[PurgeFilter](../internal/storage/purge.go).
Empty-body full deletion is
deliberate and destructive; never use malformed or empty JSON as an all-records
command. The count preview applies the same empty-filter rule, so `{}` answers
400 on both routes rather than counting the full set for an ambiguous body; a
genuinely bodyless count previews the whole set, exactly like the bodyless
delete.
The delete fence preserves newer completions and in-flight work. A completed
purge, and an expired debug-capture sweep, checkpoint-truncates the WAL after
the committed delete: `secure_delete` zeroes freed cells in the pages SQLite
writes, and the checkpoint removes the pre-delete frames that would otherwise
keep the old page images readable in the `-wal` sidecar. A reader holding an
older snapshot can defer the checkpoint; the writer's near-term tick retries it
while the reader is gone, and a clean shutdown makes the final attempt after the
read pool is closed.
### Settings and Restart
Settings only reads configuration when opened. Its search, category rail and
structured editors work on a local draft; Apply uses the
[revision-checked save contract](configuration.md#saving-settings-safely).
The console's system section reads restart availability and status when opened.
The restart action is disabled while a restart is in progress or rebuilding is
unavailable. Progress reports actual build/drain/flush/handoff phases; it is not
a simulated timer. After a successful handoff, reconnect/bootstrap refreshes
history, aggregates, operator state and config. Changed embedded frontend assets
trigger a page reload through that same refresh owner. The
[lifecycle section](containers.md#reload-restart-and-shutdown) distinguishes source rebuilds
from container replacement and ordinary shutdown.
## Dashboard data and request inspection
The KPI band is global. Explorer selections and the request status filter scope
the timeline and request log; selecting a card adds a filter and entity links
pivot within that shared route. Browser back/forward restores hash-based scopes.
Chart preset, timeframe, percentile and series visibility persist in browser storage.
These saved views are local browser state, not shared server settings.
The In-flight tile separates work that is sending from work the scheduler is
parking. Its summary names the two totals that fit the narrowest tile - active
and queued - and the composition rail under the value splits every live state
(sending, pending, operator-paused, throttled) by color; hovering the rail, or
focusing it with assistive technology, reads the exact per-state counts.
A queued request is one the proxy has not sent upstream yet: it can be
waiting in the admission queue (per-key concurrency, pacing, a provider cap
or an operator hold) or in the retry send gate behind a 429/5xx pacing
window. The request table marks the same state with the `throttled` pill, and the
status selector can filter on it.
Explorer dimensions are Providers, Models, Client, Conversations, Tools, Time,
Status, Errors and Keys. Their meanings are server-owned: model grouping and
time-bucket membership are not browser guesses. Conversations can show declared
parents and main/sub/unresolved counts; read the
[lineage contract](protocol.md#main-and-sub-conversations) before interpreting them.
Tools and error signatures can each have multiple memberships per request.
Tool cards count affected requests for N and share, with request-level card
metrics counted once per affected request; their calls KPI separately sums
request ToolCalls, so repeated same-name calls do not inflate the request count.
A request using several tools contributes to each tool card. Error/429 health
badges and every gallery share use distinct affected requests.
The [visual guide](dashboard.md) covers every timeline and its controls.
- Overview
is the summary: it carries no plot, and its tiles take the whole card (requests
over the blended token volume, tokens in/out/cached with their shares, spend
with the blended per-Mtok price, the errors / 429 pair, and average latency /
speed over every measured request with each metric's low-high sample range).
- Traffic compares requests/errors.
- Tokens separates available input/output/reasoning/cache
usage and tracks the cache-hit share of input on a percentage axis.
- Speed + latency
plots throughput on its own axis with TTFT and the decode window (first token to
last, the span decode speed measures) sharing a duration axis.
- Errors compares
error count/rate.
- Cost places reported USD spend beside request volume.
One percentile selection controls the timing series (the Overview keeps only
the timeframe control). Period readouts are computed from period samples,
not averages of bucket percentiles. Insufficient samples remain unavailable.
All time means the available retained history, not deleted, dropped or
never-observed requests, and the timeframe control never offers a window
wider than it.
The chart endpoint takes either a minute `window` (or
`all`) or an explicit `from_ms`/`to_ms` pair for calendar ranges; each payload
reports the resolved right edge `to_ms` and the unscoped `data_from_ms` extent
the control gates on.
### Request rows and details
The log starts with recent rows and pages older durable records as you scroll.
Live rows show in-flight/queue/retry progress; final rows retain their outcomes.
Retry disclosure opens absorbed attempts under the original request, not another
inference record.
The attempts log keeps the newest `max_retries` plus
`storm_max_retries` entries (at least one): a durable quota wait re-sends
without spending either budget, so the tail is what explains a long parked
request.
Click the detail arrow or a non-link row cell to open its
drawer; previous/next controls navigate requests without changing the scope.
The inspector presents available facts in four classified groups.
- Request covers
identity/status/times, time bucket, wire sizes, client/runtime, provider metadata,
model and captured parameters.
- Content covers conversation and parent ids, turn
and size composition, and prompt/response previews.
- Usage covers token splits,
generated tokens, timing, throughput, cost, transport decomposition and tools.
- Diagnostics covers queue/rate limits, retries, in-band upstream responses,
retained headers and Debug.
A summary readout carries the status, route, identity
and first numbers, and a final upstream failure renders as an error alert above
the groups. Only the selected group is shown; card headers collapse, and the
active group and collapse choices survive live updates.
A typed 429 is flow control, so its reason, code and message stay in the
rate-limit card instead of the error alert; a rate limit carried by a non-429
status is a final failure and renders the alert like any other.
An in-band upstream response that arrived after a committed success is disclosed
in Diagnostics rather than counted as a retry. Live pause and throttle state
appear as explicit rows while a request is in flight.
Copy actions cover the values that matter (identifiers, paths, models, headers,
error detail and captured bodies), field-list cards and the header's JSON
action. One clipboard path reports success or denial in a polite status line.
Long captured bodies use a bounded code surface with line count, wrap, copy and
expand controls; a long single line scrolls inside that surface instead of
widening the inspector. Narrow viewports get the full-width inspector with keys
stacked above values and touch-sized copy and close targets.
Recognized parameters include sampling and output limits, stop/logprob controls,
reasoning/verbosity, response format, tool choice and streaming flags. Explicit
zero/false parameters remain distinct from absent ones. Ordinary HTTP relay
metadata does not infer missing parameter defaults. Adapters differ: Anthropic
metadata comes from the translated body, including its configured output-limit
default, and Cursor stores the base model ID rather than its fused spelling. See
[adapters](adapters.md) before treating these fields as an exact representation
of the caller's original request.
Opening ordinary details does not fetch private request content. Prompt/response
previews appear only if captured, and Debug sidecars use their separate opt-in
and retention contract. Even ordinary labels, error text and headers can be
sensitive; do not publish request drawers from private history without review.
### Reading timing, cost and health
Request duration starts after upload and includes metadata work, queue/hold/retry
waits and response handling. TTFT is measured from the upstream attempt's own
send start to the first token (reasoning or answer), so every retry has its own
TTFT. The value excludes proxy scheduler queueing, request parsing and retry
backoff, but it does span the upstream transport window from request handoff
(dial, TLS, connection-pool wait, header wait).
A request that produced no
token, one that never sent an upstream attempt of its own (a parked cursor
resume), and any non-streamed response have no TTFT: the log shows a
placeholder, not 0 ms, and aggregates skip it.
Rows written before 0.3.34 stored `ttft_ms` anchored at request start (upload,
queueing and retry backoff included) and were zeroed once by the boot
repair, so they report TTFT as absent; their `started_at` and `first_token_at` timestamps remain,
keeping `pre_token_ms` and decode windows usable.
TPS is the single generation-speed measure: (generation tokens - 1) over the
first-token to last-token window. Average TTFT and average TPS are the
arithmetic means of those per-request samples. TPS is absent for fewer than
two tokens, a non-positive window, or a response that did not stream (no
token timestamps). The compact surfaces (the KPI band and the Overview
timing tile) abbreviate this unit as `t/s`; roomier readouts spell it `tok/s`.
The numerator assumes the first observed content chunk carries one token; a provider that bundles several tokens into that chunk can
still overstate the rate. These are provider-attempt measurements, not claims
about total user wait or system-wide throughput.
Money stays USD in storage and APIs. The shared UI formatter uses cents below
$1, including fractional cents, and dollars for larger magnitudes.
Usage/cost comes from recognized upstream fields or configured field
mappings; unavailable reporting and a displayed zero do not establish a zero
bill. Reasoning/cache splits depend on what the upstream actually reports.
When a stream carries no
usage object, or carries one whose output count is zero, the recorded output
token count is restored from the streamed chunk count, which includes
tool-argument fragments; the count is an estimate, not a provider report.
Chat tool calls are deduplicated by delta index while the retained index set
holds up to 1,024 distinct indexes. Once the set is full a further index
cannot be retained, so each of its fragments counts again; a hostile stream
of repeated indexes can overstate the tool-call count without growing memory
past the bound.
Errors count affected requests with a genuine failure, including recovered
upstream failures. Rate limits count requests that saw a rate-limit class on
the final, a retried or an in-band attempt: an HTTP 429, a 429 whose status
line a committed 200 could not relay, or a provider spelling the classifier
recognizes (such as `tier_limit_exceeded` on a 400); each request counts once.
The two signals can overlap: a bare 429 is not an error, but a non-429 status
that carries a rate-limit class failed the request and counts in both.
Client-local cancellation is likewise distinct from an upstream failure -
except a 499 whose record carries a structured error type: that observed a
failure before the client left, either the provider's in-band stream failure
(typically an error event the client received and aborted around) or a
truncated stream whose completion marker never arrived, independent of which
relay write failed. It counts as the upstream error it is.
Zero explorer health badges are hidden independently. Error-dimension card
headlines count failure events (final plus absorbed), while card N counts
distinct affected requests.
The gallery share denominator `error_total` is the
union of affected requests represented by the error cards, so a card's share
is N / `error_total` and can never exceed 100%. This denominator includes
recovered quality failures whose final status is not an error (for example an
absorbed empty-completion attempt), counting each affected request once even
when no request ends in an error status. For every other dimension
`error_total` remains the canonical affected-request error count. Do not treat
every displayed error number as the same metric.
The explorer gallery renders at most 24 cards. Its fold tracks exact
per-value accumulators for at most 8192 distinct values in the active
dimension; later values are combined into a synthetic `other` card with
aggregate count, cost, token, tool and health fields, competing for the 24
cards by that count.
The synthetic card carries `synthetic: true` on the wire,
renders as `other (grouped)`, and is not filterable. A real value named `other`
keeps its own exact card and filter; `other` is a display bucket only when the
card is synthetic.
Rail distinct-value counts, footer scope totals and
conversation summaries still cover the whole filtered history, so card counts
need not add up to them.
### Snapshots, live feeds and monitoring
The HTML includes the canonical bootstrap state so the first paint need not wait
for a second state request. Live SSE carries pending progress and finalization;
KPI/operator polling and configured aggregate refresh keep derived surfaces current.
Reconnects use sequence/feed identity and fall back to full snapshots when replay
cannot be trusted.
The footer distinguishes live/offline, active holds/Debug,
process-local storage drops, totals degraded by a failed startup history scan,
and a degraded operator-state restore.
The last one is the difference between
"nothing is held" and "a saved hold or limit document could not be read": the
served `pause` and `throttle` snapshots carry `restore_degraded`, so a truncated
document does not boot as a healthy, unpaused proxy with its saved rules silently
gone. The flag stays true for the life of the process; a restart that reads a
document which decodes is what clears it.
Each key holds one value, so the next
accepted operator write replaces the unreadable document rather than preserving
it: capture the bytes first if they are needed as evidence.
An offline indicator alone does not prove inference is down. None of these
mechanisms makes initial history preload instantaneous.
For integrations, use the documented read routes rather than scraping the DOM.
Bootstrap/live data is a recent observation window; scoped aggregate routes query
available history; JSON export provides finalized records; restricted SQL queries
durable data within configured time/output limits. The API is in initial
development, so pin and verify the selected revision for integrations.
Prometheus is a bounded-ring view. Its request/token/error totals are gauges that
can shrink after eviction, purge or restart, not monotonic lifetime counters;
do not apply counter-only `rate()`/`increase()` assumptions to them. Its TTFT
quantiles use the ring's sample contract, which differs from dashboard R7 history
percentiles. Metric names and units are exposed with HELP/TYPE metadata at
`/metrics/prometheus`; [prometheus.go](../internal/metrics/prometheus.go) owns them.
An unchanged finalized ring revision serves the exact last exposition without
another ring copy, aggregate or TTFT sort; any ring content transition rebuilds.
### Operator recipes
The gated query route (`GET /metrics/query?q=`, above) and the [MCP](mcp.md)
`query` tool both take exactly one `SELECT` with no statement separator, and
results are bounded by `storage_query_max_rows` and
`storage_query_max_bytes`; an over-budget result is an error on the route, so
keep wide scans behind a `started_at` window or a `LIMIT`.
The statement gate also denies the read-only capabilities SQLite exposes
outside ordinary tables and columns: `ATTACH`, `DETACH`, `PRAGMA`, `LOAD_EXTENSION`, the `sqlite_dbpage` and `dbstat`
table-valued functions, the whole result-bearing `pragma_*` family and the `readfile`/`writefile`
functions. Those would read raw pages, page statistics or the absolute
database path, which is not part of this console's surface.
The denial is lexical over the raw query text, string literals included, so a
keyword spelled inside a literal is refused too: `SELECT 'attach' AS x` and
`SELECT 'pragma_table_info'` both fail. A name only has to contain the bare
word at an ASCII boundary; `attachment` and `attach_x` pass.
The driver executes a multi-statement string wholesale, returning the rows
of the last statement, so both rules are load-bearing: without the `;`
refusal a second `ATTACH` statement would run behind a `SELECT` prefix the prefix
check accepts, and the keyword scan alone would not see it.
Column meanings come from the MCP `describe` tool. `started_at` is Unix milliseconds: a
relative window is `started_at >= strftime('%s','now')*1000 - 3600000`, and calendar math divides by 1000 and applies
SQLite's `unixepoch` modifier, with `localtime` for the host's zone.
The recipes below earn their keep in
the derived shapes no dashboard surface computes: the outlier ratio, the
write/read ratio, the latency decompositions. Their ingredient aggregates
are conceded up front, the way the decode window already is: the explorer
sums per-group counts and cache reads (its `N` and `Cache` fields) and a
per-group blended $/Mtok (its `cost_per_mtok` KPI, the same cost-reporting
in+out rule), and the chart's dec series samples the shared decode window
(`last_token_at - first_token_at`) that recipe 1 decomposes latency
around.
Several of the columns a recipe reads do overlap the aggregate
scans: `ttft_ms`, `reasoning_tokens`, `status_code`, `attempts`,
`output_tokens` and `cache_read_tokens` all appear in the dashboard's scan
lists. The recipes' private reading set, which no aggregate scan reads, is
`queue_wait_ms`, `first_answer_at`, `retry_after_ms`,
`cache_write_tokens`, the `chars_*` columns, `total_tokens`,
`req_max_tokens`, `duration_ms` and `retries`. Each recipe is a
question, the SQL that answers it, and how to read the result.
**Where did the time go on a slow request?**
```sql
SELECT started_at, duration_ms, queue_wait_ms, retries,
first_token_at - started_at AS pre_token_ms,
ttft_ms,
last_token_at - first_token_at AS decode_ms,
started_at + duration_ms - last_token_at AS tail_ms
FROM requests
WHERE started_at >= strftime('%s','now')*1000 - 3600000
AND first_token_at > 0
ORDER BY duration_ms DESC
LIMIT 20
```
`pre_token_ms` is the wait the caller felt, including queue and retry waits;
`ttft_ms` is measured from the adopted upstream attempt's send start, so a
retried request shows a small ttft beside a large `pre_token_ms`.
Rows written before 0.3.34 carry a start-anchored `ttft_ms` and were zeroed once
by the boot repair, so their `ttft_ms` reads 0 (absent) while `pre_token_ms` and `decode_ms` stay
usable from the stored timestamps. `decode_ms` is the streaming window and `tail_ms` what
remains after the last token, where a millisecond of rounding noise is
normal. A request that produced no token, or was not streamed, has nothing
to decompose, hence the `first_token_at > 0` gate.
**How long did a model think before answering?**
```sql
SELECT started_at, model, reasoning_tokens,
first_answer_at - first_token_at AS thinking_ms
FROM requests
WHERE started_at >= strftime('%s','now')*1000 - 3600000
AND first_answer_at > 0
ORDER BY thinking_ms DESC
LIMIT 20
```
`thinking_ms` spans the first token of any kind, reasoning included, to the
first answer token; 0 means the first token already was the answer.
`first_answer_at` is streaming-only on the generic relay; cursor-format
non-streaming requests set it (the bridge synthesizes each delta and tracks
it via cursorTrackDelta), and such rows legitimately enter the gate because
`first_token_at` is stamped by the same helper. Tool-call-only streams keep
0 and stay outside the gate.
**How long does an agent idle between conversation turns?**
```sql
SELECT conversation_id, started_at,
started_at - LAG(started_at + duration_ms)
OVER (PARTITION BY conversation_id ORDER BY started_at) AS idle_ms
FROM requests
WHERE started_at >= strftime('%s','now')*1000 - 86400000
AND conversation_id != ''
ORDER BY conversation_id, started_at
```
`idle_ms` is the gap between one request's end and the next request's start
inside the same conversation; the first turn has no predecessor and stays
null. Order by `idle_ms DESC` instead to surface the biggest stalls first.
**What retried before the client saw a response?**
```sql
SELECT r.started_at, r.id, r.retries, r.status_code AS final_status,
j.value->>'status_code' AS attempt_status,
j.value->>'at' AS attempt_at,
j.value->>'retry_after_ms' AS retry_after_ms
FROM requests r, json_each(r.attempts) j
WHERE r.started_at >= strftime('%s','now')*1000 - 86400000
AND json_array_length(r.attempts) > 0
AND COALESCE(json_extract(j.value,'$.in_band'),0)=0
ORDER BY r.started_at, j.key
```
One row per absorbed retry, oldest first. The `in_band` filter excludes a 429
recorded behind a committed 200: that entry is health evidence, not a retry.
The `json_array_length` gate is required: `attempts` defaults to the JSON
literal `null`, and `json_each` over that literal still yields a row. `retry_after_ms` appears (nonzero)
when the provider sent a `Retry-After` header or a ratelimit-reset header
(`x-ratelimit-reset-requests`/`-tokens`, `anthropic-ratelimit-requests-reset`,
`anthropic-ratelimit-tokens-reset`), bounded by the proxy's hint ceiling.
**Did a conversation's cache writes pay for themselves?**
```sql
SELECT conversation_id, COUNT(*) AS requests,
SUM(cache_write_tokens) AS cache_written,
SUM(cache_read_tokens) AS cache_read,
CASE WHEN SUM(cache_read_tokens) > 0
THEN ROUND(1.0 * SUM(cache_write_tokens) / SUM(cache_read_tokens), 2)
END AS write_per_read
FROM requests
WHERE conversation_id != ''
GROUP BY conversation_id
ORDER BY cache_written DESC, cache_read DESC
LIMIT 20
```
`write_per_read` at or below 1 means the reads repaid the writes; reads with
no writes ride a cache somebody else warmed. A conversation with writes and
no reads paid for a prompt that never came back.
**When does traffic arrive?**
```sql
SELECT strftime('%w', started_at/1000, 'unixepoch', 'localtime') AS weekday,
strftime('%H', started_at/1000, 'unixepoch', 'localtime') AS hour,
COUNT(*) AS requests, SUM(cost) AS cost,
ROUND(AVG(duration_ms), 1) AS avg_ms
FROM requests
GROUP BY weekday, hour
ORDER BY weekday, hour
```
`weekday` is `0` (Sunday) through `6` and `hour` is two digits in the host's
local zone; drop `localtime` for UTC. Cost is USD, summed as stored.
**How far does the token estimate drift?**
```sql
SELECT model, COUNT(*) AS requests,
ROUND(AVG((chars_system + chars_user + chars_assistant + chars_tool) / 4), 1) AS avg_chars_est,
ROUND(AVG(input_tokens), 1) AS avg_input,
ROUND(AVG(1.0 * input_tokens /
((chars_system + chars_user + chars_assistant + chars_tool) / 4)), 3) AS input_vs_chars4,
ROUND(AVG(1.0 * total_tokens /
((chars_system + chars_user + chars_assistant + chars_tool) / 4 + req_max_tokens)), 3) AS total_vs_est
FROM requests
WHERE input_tokens > 0
AND chars_system + chars_user + chars_assistant + chars_tool > 0
GROUP BY model
ORDER BY input_vs_chars4 DESC
```
The scheduler reserves prompt chars/4 plus `req_max_tokens` when the client
sent one. `input_vs_chars4` compares real input tokens with the chars/4
share, and `total_vs_est` compares settled total tokens with the full
reservation. A ratio far from 1 on a well-represented model means the
reservation systematically over- or under-reserves for it.
**Which requests paid far above their model's blended rate?**
```sql
SELECT o.model, o.id, o.started_at,
ROUND(1e6 * o.cost / (o.input_tokens + o.output_tokens), 2) AS per_mtok,
ROUND(1e6 * a.cost_sum / a.tok_sum, 2) AS model_avg_per_mtok,
ROUND((1e6 * o.cost / (o.input_tokens + o.output_tokens)) /
(1e6 * a.cost_sum / a.tok_sum), 2) AS ratio
FROM requests o
JOIN (SELECT model, SUM(cost) AS cost_sum,
SUM(input_tokens + output_tokens) AS tok_sum
FROM requests WHERE cost > 0 GROUP BY model) a ON a.model = o.model
WHERE o.cost > 0 AND o.input_tokens + o.output_tokens > 0
ORDER BY ratio DESC
LIMIT 10
```
`per_mtok` is the request's own rate over its in+out tokens, and `ratio`
scales it against the model's blended average, computed over cost-reporting
requests only so unpriced traffic cannot dilute it. A ratio far above 1
marks expensive outliers such as cache misses or long-tail pricing; flip the
sort to find the bargains. SQLite has no `median()`, which is why the
blended average plus a sorted ratio stands in for it.
---
# Reverse proxy and remote access
Use a protected ingress when accessing millivolt beyond a trusted local machine.
The whole dashboard is gated by the shared `MILLIVOLT_OPERATOR_TOKEN`
credential; there is no tenant isolation and no per-user identity.
At the application gate, only the unauthenticated `/healthz` and `/readyz`
probes, the `POST /session` login handshake, origin-root brand/PWA files and
transparent inference stay open, so a load balancer can health-check the
service without holding the credential and a browser can reach the login
handshake. The canonical open-path list is in
[operator access](routes.md#operator-access); this guide only adds the ingress
boundary. Read [Security](../SECURITY.md) first.

The diagram is a text-equivalent summary of the example below: NGINX terminates
TLS and enforces access; millivolt owns routing, the operator session, and the
history; the upstream receives the selected credentials.
## NGINX setup
[deploy/nginx.conf](../deploy/nginx.conf) is the canonical example. It serves
millivolt at the origin root, with NGINX on the same host as the loopback-published
Compose service. Use a maintained NGINX release with the `http2 on` directive
(introduced in 1.25.1), not the deprecated `listen ... http2` syntax.
The sample splits two prefix locations because they do not share an access
model:
- `location /v1` is the OpenAI-compatible inference base: preserve provider
`Authorization` and routing headers, and allowlist API clients.
- `location /` covers everything else, including the dashboard and operator
plane (`/dash/`, `/metrics/`, `/admin/`) plus the application-open
`POST /session`, `/healthz`, `/readyz` and brand/PWA paths: allowlist
operators, and keep `Host` plus the session cookie so Settings origin checks
and the live feed work; the comparison is case-insensitive, so a
case-preserving ingress is fine.
The `/session` handshake is open at millivolt so login works, but this location
still allowlists it at the edge. A URL-capable MCP client uses this location
with `/mcp` appended and the operator Bearer header: the application's MCP
endpoint is Bearer-only and refuses the session cookie.
Shared streaming proxy settings live on the `server` so both SSE paths stay
unbuffered. Prefix `/v1` wins over `/` for `/v1/chat/completions` and
`/v1/models`; `GET /models` without that prefix follows `location /` unless you
add an exact `/models` block with the inference rules.
1. Start the normal [Compose deployment](../README.md#docker-compose), retaining
its loopback-only host port.
2. Copy the example into your NGINX HTTP configuration. Replace its example
hostname, certificate paths and the two TEST-NET allowlists. The supplied
address is a placeholder; unmatched clients are denied. The lists may differ
when operators and API clients are not the same sources.
Set `proxy_pass` to the chosen host port if `MILLIVOLT_PORT` is not 8080.
3. Validate with `nginx -t` before reloading your existing NGINX service. Confirm
a trusted client and a denied client on both `/` and `/v1`.
4. Use `https://your-hostname/` for the dashboard and
`https://your-hostname/v1` as the client API base. Keep sending the actual
upstream base/key as described in the [client setup](../README.md#connect-a-client).
The sample is a `server` block, included under NGINX's existing `http` context,
not a replacement for the entire installation configuration. It does not issue
certificates or configure your firewall. Do not expose the original proxy port
alongside the protected HTTPS endpoint.
### Streaming and request handling
The example disables response/request buffering and proxy retries so the edge
does not batch SSE events or independently replay inference. It leaves HTTP
statuses intact, disables edge compression/caching, and ignores special upstream
`X-Accel-*` controls for buffering, redirects, rate limits, caching and charset.
Millivolt uses SSE, not a WebSocket endpoint; no `Upgrade` block is needed.
It preserves the external `Host`, including a nondefault port, so browser-origin
checks on Settings and other mutations keep working.
It also overwrites `X-Forwarded-Proto` with this server's `$scheme`, so a TLS
presentation reaches millivolt as `https` and the session cookie carries
`Secure`; a client-supplied value cannot spoof it. Overwriting is the
recommended form. If your ingress appends its scheme instead
(`$http_x_forwarded_proto,$scheme`), the application still marks the request
secure when any token does, so a client-supplied `http` in first position
cannot downgrade the cookie.
The sample also resolves the client address with `real_ip`:
```nginx
set_real_ip_from 127.0.0.1;
real_ip_header X-Forwarded-For;
```
Without it, millivolt sees this hop's address as `RemoteAddr` for every request
and the per-source authentication throttle gets one bucket, so a single wrong
guess locks out every operator. `set_real_ip_from` lists the addresses allowed
to send a replacement address: this host's own loopback.
If another load balancer, tunnel or proxy fronts NGINX, add its address range
there and set `real_ip_recursive on` once every hop in the chain is listed.
`real_ip` walks `X-Forwarded-For` from the right and keeps the last untrusted
address, so a client that prepends a forged entry cannot displace the real one;
NGINX's default `$proxy_add_x_forwarded_for` then appends the resolved address
on the way inbound, and the per-source authentication throttle reads that last
element.
The record field is separate: `client_ip` records the first `X-Forwarded-For`
element when that header is present and the socket peer otherwise; the value is
diagnostic, never verified, and a directly reachable caller can set it.
The resolved address is a throttle key, not an identity: keep the location
allowlists in force. Those lists evaluate the resolved address, so a caller
that already reaches this host's loopback chooses what they see: they separate
remote networks from each other and from loopback callers, not loopback callers
from each other.
NGINX's body-size gate is
disabled in the sample: millivolt's existing `max_request_bytes` remains the
application limit, rather than an accidentally smaller duplicate at ingress.
Choose edge idle timeouts for your deployment. `proxy_read_timeout` limits the
gap between upstream reads, not the total lifetime of a stream.
These behaviors follow the official [proxy module](https://nginx.org/en/docs/http/ngx_http_proxy_module.html),
[HTTP core module](https://nginx.org/en/docs/http/ngx_http_core_module.html),
[HTTP/2 module](https://nginx.org/en/docs/http/ngx_http_v2_module.html) and
[realip module](https://nginx.org/en/docs/http/ngx_http_realip_module.html).
## Credentials and access control
The sample uses a trusted IP/VPN allowlist on each location, preserving
provider `Authorization` on `/v1` and the operator Bearer or session cookie on
`/`. Replace each placeholder with the addresses you intend to trust for that
plane.
If another load balancer is in front of NGINX, list its addresses in
`set_real_ip_from` and switch `real_ip_recursive` on, so the trusted chain
matches reality; a hop that appends to `X-Forwarded-For` without being listed
is ignored on the left of the real client.
TLS encrypts traffic but does not authorize callers. A secure presentation also
makes millivolt answer `Strict-Transport-Security: max-age=15552000` (180 days,
no `includeSubDomains`) so a browser pins the scheme; NGINX's own `add_header`
is not needed for it, and adding a second HSTS header at the edge would send
two values. See NGINX's
[access module](https://nginx.org/en/docs/http/ngx_http_access_module.html)
and [TLS configuration](https://nginx.org/en/docs/http/ngx_http_ssl_module.html).
If you instead add HTTP Basic authentication on `/v1`, `Authorization` contains
the ingress credential and cannot simultaneously contain the provider bearer
key. Supply the provider key through `X-Proxy-Key`, and clear the consumed
ingress header before forwarding with `proxy_set_header Authorization "";`.
The same conflict applies on `/`: a browser behind Basic auth cannot also
present the dashboard's Bearer credential in the same header, so gated actions
fail. Prefer an address/VPN allowlist as in the sample, or an ingress that
adds its own credential through a different mechanism and forwards
`Authorization` untouched.
Do not put Basic auth on `/v1` to "protect inference" while leaving `/` on
Bearer: that still overwrites the provider key. The same separation applies to
other ingress-only credentials: do not forward them to an LLM provider. Clients
must support your ingress authentication and the proxy's custom routing
headers.
The session handshake lives at `/session`, outside the admin plane, which
supports one specific pattern: an external authentication layer may front
`/admin/*` alone, because millivolt's own login stays reachable and the
mutation and settings routes become double-gated, the external layer plus
millivolt's credential.
Operators fronting `/admin` this way must still pass the session cookie and
`Host` through to the proxy, since the dashboard's fetch-based mutations rely
on both.
## Container networking and other ingress servers
When NGINX itself is a container, `127.0.0.1` points back to NGINX, not millivolt.
Attach it to the same private Docker network and use `http://millivolt:8080` as
its upstream instead. Only the edge needs a publicly reachable port. Do not bind
millivolt to loopback inside its container, which would prevent peer access.
`MILLIVOLT_PORT` changes host publication, not this internal service port.
For another ingress implementation, preserve these same boundaries: protect the
origin, keep inference and the operator plane as separate access rules, forward
the original authority and required custom headers, allow streaming without
response buffering, avoid automatic inference retries, and choose upload/idle
limits deliberately. Subpath hosting is not the supplied deployment: dashboard
assets and routes expect the origin root.
---
# Performance and resource usage
This page is the single source for the published performance and resource
numbers. It records what was measured, how it was measured, and what the result
does not prove. The [performance and footprint](operations.md#performance-and-footprint)
section in operations owns the implementation cost model and `go_memory_limit`
behavior.
These are scoped local measurements of the native proxy process, not deployment
guarantees. The page was regenerated from the result bundle on
2026-11-16 UTC.
## Artifact and idle footprint
| Artifact or state | Bytes | Display | Notes |
| --- | ---: | ---: | --- |
| Static amd64 binary | 21,663,904 | 20.7 MiB | Stripped, no CGO |
| amd64 runtime image | 38,022,232 | 36.3 MiB | Docker-reported content size |
| Compressed image archive | 9,887,262 | 9.4 MiB | `docker save` piped through `gzip -9` |
| Idle process, empty SQLite | 30,204 KiB | 29.5 MiB | After a 15-second quiet interval |
| Idle process CPU | 2 scheduler ticks | 0.067% | 2 ticks over 30 seconds |
The image size includes the distroless base, dependency notices, and the
embedded UI. It is not a compressed download size, a registry digest, or a
persistent-volume estimate. Idle RSS and CPU describe one process on this host.
A short sample cannot prove that a longer idle period uses exactly the same
resources.
## Method
| Item | Setting |
| --- | --- |
| Date | 2026-11-16 |
| Source revision | [`22c0b6b73655`](https://github.com/LLM-4-People/millivolt/commit/22c0b6b73655a349e086c345bc80907efbe16822) |
| Go toolchain | go1.27.4, `CGO_ENABLED=0` |
| Platform | Linux/amd64 under WSL2 |
| CPU | Intel(R) Core(TM) Ultra 9 285H, 12 visible logical CPUs |
| Visible RAM | 62 GiB |
| Configuration | Public example with private listen and database paths |
| Debug captures | Off |
| Body previews | Off |
| Browser and live observer | None during load stages |
| Upstream | Local synthetic server on loopback |
| Load shape | Closed-loop HTTP stages, then drain |
| RSS sampling | 100 ms |
| CPU unit | 100% equals one logical CPU |
| Accounting check | Exact stages reconcile rows, tokens and cost; a dropping stage reconciles durable plus dropped |
| Host load (1-minute) | 23.36 at start, 13.38 at end |
The public example configuration was used, with private listen and database
paths supplied by `scripts/dev.sh`. The method table records the harness
declarations for captures, previews, the observer and the upstream. CPU and RSS
describe the native Go process, not Docker's container-wide accounting.
Percentiles are nearest-rank.
## Fast-response throughput
Each stage used a fresh disposable database and the same synthetic response:
10 content frames, usage `100/10`, and fixture cost `0.001`. The upstream added no intentional delay. Each stage started
requests for 5 seconds, then let active requests drain. The final
row is a separate
10-second run used
to reach about 129,442 durable rows
without a storage drop.
| Client workers | HTTP requests | Requests/s | p50 | p95 | p99 | Proxy CPU | Peak proxy RSS | Durable rows |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | 6,879 | 1,375.7 | 0.66 ms | 1.09 ms | 1.66 ms | 52.6% | 61.5 MiB | Exact, 6,879 |
| 8 | 30,264 | 6,050.9 | 1.17 ms | 2.28 ms | 4.06 ms | 239.3% | 88.3 MiB | Exact, 30,264 |
| 32 | 53,081 | 10,567.2 | 2.55 ms | 6.01 ms | 8.63 ms | 374.3% | 93.0 MiB | Exact, 53,081 |
| 128 | 66,813 | 13,346.4 | 8.40 ms | 20.19 ms | 28.29 ms | 465.2% | 119.0 MiB | Exact, 66,813 |
| 128, 10 s | 129,442 | 12,931.5 | 8.50 ms | 21.30 ms | 31.58 ms | 459.7% | 124.5 MiB | Exact, 129,442 |
The exact fast-response point is 13,346.4 requests/s at 128 workers, with p95 20.19 ms and p99 28.29 ms. The 128-worker 10-second stage used 227.3% CPU and 33.7 MiB peak RSS in the separate load generator and synthetic upstream process.
All HTTP responses succeeded in these stages, including the
128-worker stage. That is not a
promise that every configuration and history size stays exact under overload.
The durable queue is bounded and best-effort; the deliberate overload fixture
below shows what a drop result looks like.
## Sustained capacity
The sustained capacity point is the highest tested closed-loop level that
completed a 300-second run with zero durable drops and exact
accounting. Each level started from a fresh disposable database. The upstream
response and client shape were the same as the fast-response table:
10 content frames, usage `100/10`, and fixture cost `0.001`.
| Workers | Duration | HTTP requests | Requests/s | p50 | p95 | p99 | Proxy CPU | Peak proxy RSS | Durable rows | Drops |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 32 | 300.00 s | 2,584,965 | 8,616.5 | 2.54 ms | 9.93 ms | 19.71 ms | 289.1% | 107.0 MiB | Exact, 2,584,965 | 0 |
| 48 | 300.01 s | 3,683,159 | 12,276.8 | 3.12 ms | 8.92 ms | 15.68 ms | 421.1% | 121.9 MiB | Exact, 3,683,159 | 0 |
| 64 | 300.00 s | 4,649,811 | 15,499.2 | 3.62 ms | 8.18 ms | 12.03 ms | 522.6% | 125.8 MiB | 4,602,957 kept | 46,854 |
| 128 | 300.03 s | 4,169,782 | 13,897.8 | 7.77 ms | 20.25 ms | 32.11 ms | 451.4% | 141.1 MiB | 3,775,709 kept | 394,073 |
The exact 300-second point is 12,276.8 requests/s at 48 workers, with p95 8.92 ms and p99 15.68 ms. The 64-worker point dropped 46,854 rows (1.0% of its requests). A level that drops is an overload observation, not a capacity result; the bounded durable queue is not lossless at every load.
## Streaming concurrency
The synthetic upstream emitted 10-frame response across approximately
1-second stream, followed by usage and a terminator. Each stage started its
fixed number of requests and then drained. Latency includes the intentional
upstream delay, so these numbers are not proxy-only overhead.
| Workers | HTTP requests | Requests/s | SSE events/s | Output tokens/s | Response KiB/s | p50 | p95 | p99 | First-byte p95 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | 5 | 1.0 | 12 | 10 | 0.6 | 1,009.61 ms | 1,010.49 ms | 1,010.49 ms | 2.22 ms |
| 8 | 40 | 7.9 | 95 | 79 | 4.7 | 1,007.21 ms | 1,009.82 ms | 1,010.22 ms | 3.59 ms |
| 32 | 160 | 31.4 | 377 | 314 | 18.8 | 1,007.57 ms | 1,016.44 ms | 1,059.19 ms | 8.23 ms |
| 64 | 320 | 62.9 | 755 | 629 | 37.6 | 1,008.33 ms | 1,037.34 ms | 1,054.23 ms | 31.17 ms |
| 128 | 640 | 124.5 | 1,494 | 1,245 | 74.4 | 1,009.33 ms | 1,074.57 ms | 1,107.40 ms | 69.27 ms |
| 256 | 1,280 | 248.0 | 2,976 | 2,480 | 148.2 | 1,010.96 ms | 1,093.08 ms | 1,119.54 ms | 89.49 ms |
| 512 | 2,560 | 490.0 | 5,880 | 4,900 | 292.9 | 1,008.64 ms | 1,150.71 ms | 1,153.58 ms | 145.75 ms |
| 1,024 | 3,072 | 494.8 | 5,938 | 4,948 | 295.7 | 2,016.27 ms | 2,144.02 ms | 2,157.64 ms | 1,133.32 ms |
| Workers | Proxy CPU | Peak proxy RSS | Upstream peak | Durable rows | Drops |
| --- | ---: | ---: | ---: | ---: | ---: |
| 1 | 0.2% | 31.7 MiB | 1 | Exact, 5 | 0 |
| 8 | 1.0% | 32.7 MiB | 8 | Exact, 40 | 0 |
| 32 | 3.7% | 36.7 MiB | 32 | Exact, 160 | 0 |
| 64 | 6.9% | 40.2 MiB | 64 | Exact, 320 | 0 |
| 128 | 13.2% | 49.8 MiB | 128 | Exact, 640 | 0 |
| 256 | 24.6% | 67.4 MiB | 256 | Exact, 1,280 | 0 |
| 512 | 48.6% | 103.7 MiB | 512 | Exact, 2,560 | 0 |
| 1,024 | 48.6% | 137.9 MiB | 512 | Exact, 3,072 | 0 |
These stages measure concurrency and end-to-end latency for that fixed fixture.
They do not measure SSE event rate, bytes per second, token throughput, TTFT,
chunk-size or hold-duration effects, sustained streaming, or the point where
streaming saturates. First-byte p95 is the first response byte, not the first
generated token.
The 1,024-worker stage reached
512 upstream connections
and remained exact. This is a configured concurrency result on one host, not a
universal request-capacity ceiling. The largest level also matches the example
configuration's `max_conns_per_host` of 512. No higher level, sustained
streaming stage, or automatic saturation sweep was run; see
[Limitations](#limitations).
## Streaming throughput
`scripts/perf.sh streaming` runs the longer end-to-end throughput ramp. Each
level starts a fresh private `scripts/dev.sh` instance and sends requests
through the real proxy to a local synthetic SSE upstream. Each level ran for
60 seconds
with 100 content frames over a roughly 1-second
stream.
| Workers | HTTP requests | Requests/s | SSE events/s | Output tokens/s | Response KiB/s | p50 | p95 | p99 | First-byte p95 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | 57 | 0.9 | 96 | 94 | 4.8 | 1,067.07 ms | 1,079.25 ms | 1,085.80 ms | 3.02 ms |
| 8 | 448 | 7.5 | 760 | 746 | 37.9 | 1,066.92 ms | 1,125.67 ms | 1,146.53 ms | 6.08 ms |
| 32 | 1,824 | 30.3 | 3,086 | 3,026 | 153.7 | 1,055.96 ms | 1,067.91 ms | 1,073.94 ms | 3.17 ms |
| 64 | 3,648 | 60.2 | 6,140 | 6,020 | 305.9 | 1,061.67 ms | 1,072.37 ms | 1,088.48 ms | 3.88 ms |
| 128 | 7,296 | 120.5 | 12,291 | 12,050 | 612.3 | 1,060.13 ms | 1,069.44 ms | 1,113.02 ms | 5.44 ms |
| 256 | 14,592 | 239.2 | 24,395 | 23,916 | 1,215.2 | 1,061.58 ms | 1,138.38 ms | 1,174.13 ms | 11.75 ms |
| 512 | 28,789 | 471.5 | 48,094 | 47,151 | 2,395.8 | 1,063.37 ms | 1,141.20 ms | 1,300.62 ms | 15.32 ms |
| 1,024 | 29,696 | 478.6 | 48,819 | 47,861 | 2,431.9 | 2,128.84 ms | 2,162.96 ms | 2,285.10 ms | 1,091.76 ms |
| Workers | Proxy CPU | Peak proxy RSS | Upstream peak | Durable rows | Drops |
| --- | ---: | ---: | ---: | ---: | ---: |
| 1 | 1.2% | 31.5 MiB | 1 | Exact, 57 | 0 |
| 8 | 5.5% | 37.1 MiB | 8 | Exact, 448 | 0 |
| 32 | 14.9% | 47.8 MiB | 32 | Exact, 1,824 | 0 |
| 64 | 28.2% | 59.0 MiB | 64 | Exact, 3,648 | 0 |
| 128 | 57.9% | 81.3 MiB | 128 | Exact, 7,296 | 0 |
| 256 | 115.1% | 114.9 MiB | 256 | Exact, 14,592 | 0 |
| 512 | 233.1% | 152.2 MiB | 513 | Exact, 28,789 | 0 |
| 1,024 | 239.9% | 189.4 MiB | 512 | Exact, 29,696 | 0 |
Throughput peaked at 478.6 requests/s at 1,024 workers. At 1,024 workers the upstream peak stayed at 512, throughput was 478.6 requests/s against 471.5 at 512 workers, and p50 latency rose from 1.06 s to 2.13 s. The extra clients queued behind the configured 512 upstream connections.
Every level completed with exact durable accounting unless its row reports
drops. This run started under the host load the harness recorded, so absolute
rates are comparable only within this page and this host.
## Durable writer and overload
These are isolated Go benchmarks from the test overlay, not HTTP runs. Every
row and drop count is verified against successful requests. The overload
benchmark deliberately fills the bounded queue, so its drop count is a fixture
result, not a recommended operating point.
| Path | Durable rows/s | Allocations |
| --- | ---: | ---: |
| Direct batch insert at the configured batch capacity | 62,921 | 425,396 B/op, 7,279 allocs/op |
| Queued `Record` plus `Flush` at the queue capacity | 56,055 | 13,594,117 B/op, 232,920 allocs/op |
| Overload fixture | Wire requests/s | Durable rows/s | Dropped rows |
| --- | ---: | ---: | ---: |
| 32 workers | 72,619 | 19,994 | 101,507 |
| 128 workers | 64,161 | 14,605 | 113,242 |
The overload fixture confirms the accounting invariant `durable + dropped =
successful requests`. Increasing queue capacity alone cannot make sustained
overload lossless. A higher HTTP success rate is not a complete-accounting
result unless the durable row count and drop delta agree.
## Storage and retained projection
The storage fixture stores complete records, indexes, projection triggers, and
marshaled response headers in SQLite. Disk bytes include the database, WAL, and
shared-memory sidecar after a final flush.
| Measure | 10,000 records | 100,000 records |
| --- | ---: | ---: |
| SQLite bytes total | 6.84 MiB | 49.20 MiB |
| SQLite bytes per record | 717.3 | 515.9 |
| Projection retained, built | 449.6 B/record | 470.8 B/record |
| Projection interned dimensions | 442.0 B/record | 429.1 B/record |
| Projection scanned decode | 580.3 B/record | 576.0 B/record |
| Projection transient allocation | 3,125 B/record | 3,434 B/record |
The projection process benchmark is a relative measure inside the test binary,
not the deployed proxy's RSS. At 100,000 records it
measured 94.6 MiB process RSS and a
66.0 MiB projection delta; at
10,000 records it measured
40.6 MiB process RSS and a
12.0 MiB delta. The test binary and its
linked packages are part of the absolute figure.
A fast-response run reached a 119.0 MiB
peak with 66,813 durable rows while no
dashboard projection was preloaded. Use the projection delta above for the
retained analytical cost, not the loaded-process peak as a substitute.
## Query and metadata paths
The query benchmarks seed a 100,000-row
scratch database and decode a 200-row
page. These are single-iteration measurements on a shared host, so small
differences are not meaningful.
| Query | Before cursor | With cursor | Allocations before cursor |
| --- | ---: | ---: | --- |
| Provider equality | 4.24 ms | 3.85 ms | 1.38 MB, 5,455 allocs |
| Model equality | 2.81 ms | 2.93 ms | 1.38 MB, 5,445 allocs |
| Client equality | 36.52 ms | 18.73 ms | 1.38 MB, 5,442 allocs |
| Key-hash equality | 33.68 ms | 18.28 ms | 23.5 KB, 265 allocs |
| Conversation equality | 40.16 ms | 21.73 ms | 23.2 KB, 264 allocs |
| Status equality | 38.98 ms | 20.99 ms | 698.1 KB, 2,942 allocs |
The request metadata path performs one bounded metadata decode; the allocation
column is measured per call.
| Metadata body | Time per request | Throughput | Allocations |
| --- | ---: | ---: | ---: |
| 1 KiB | 428 ns | 2,580.8 MB/s | 16 B, 1 alloc |
| 100 KiB | 4,808 ns | 21,314.0 MB/s | 16 B, 1 alloc |
| 1 MiB | 75,397 ns | 13,908.4 MB/s | 16 B, 1 alloc |
## Implementation benchmarks
These are the isolated benchmark results the design notes cite. They are current
measurements from the same run as the tables above, not a historical before and
after across revisions.
| Benchmark | Time | Bytes per op | Allocs per op |
| --- | --- | --- | --- |
| `BenchmarkAggregateFold/chart` | 2.74 ms | 136,736 B | 28 |
| `BenchmarkAggregateFold/explorer` | 4.38 ms | 209,696 B | 37 |
| `BenchmarkAggregateFold/explorer_tool` | 3.73 ms | 807,840 B | 38 |
| `BenchmarkAggregateFold/chart_window` | 413,833 ns | 17,616 B | 28 |
| `BenchmarkHandleLogPage/empty/limit_60` | 1.70 ms | 987,376 B | 3,533 |
| `BenchmarkHandleLogPage/sql_provider/limit_60` | 1.95 ms | 550,080 B | 2,645 |
| `BenchmarkHandleLogPage/go_status_class/limit_60` | 1.89 ms | 903,504 B | 2,686 |
| `BenchmarkHandleLogPage/go_status_class_5xx/limit_60` | 4.88 ms | 1,819,616 B | 21,125 |
| `BenchmarkHandleLogPage/go_tool_missing/limit_60` | 5.00 ms | 1,883,032 B | 29,360 |
| `BenchmarkHandleLogPage/empty/limit_200` | 4.26 ms | 2,889,608 B | 7,999 |
| `BenchmarkHandleLogPage/sql_provider/limit_200` | 4.92 ms | 2,888,720 B | 7,993 |
| `BenchmarkHandleLogPage/go_status_class/limit_200` | 4.14 ms | 2,890,912 B | 8,015 |
| `BenchmarkHandleLogPage/go_status_class_5xx/limit_200` | 3.88 ms | 1,889,744 B | 21,029 |
| `BenchmarkHandleLogPage/go_tool_missing/limit_200` | 4.72 ms | 1,957,512 B | 29,258 |
| `BenchmarkProjectionMetricAppend/history_100000` | 11,393 ns | 960 B | 5 |
| `BenchmarkProjectionMetricAppend/history_1000000` | 11,716 ns | 960 B | 5 |
| `BenchmarkProjectionMetricAppend/append_cross_100000` | 1.58 ms | 3,956,976 B | 14 |
| `BenchmarkProjectionMetricAppend/append_cross_1000000` | 1.54 ms | 3,956,976 B | 14 |
| `BenchmarkAggregateUnchanged/identity_marshal` | 33,009 ns | 6,456 B | 6 |
| `BenchmarkAggregateUnchanged/unchanged` | 1,338 ns | 0 B | 0 |
| `BenchmarkKPI/records_10000/miss` | 403,272 ns | 83,632 B | 6 |
| `BenchmarkKPI/records_10000/hit` | 1,024 ns | 32 B | 3 |
| `BenchmarkKPI/records_100000/miss` | 7.47 ms | 804,528 B | 6 |
| `BenchmarkKPI/records_100000/hit` | 1,986 ns | 32 B | 3 |
| `BenchmarkKPI/records_1000000/miss` | 91.84 ms | 8,005,296 B | 6 |
| `BenchmarkKPI/records_1000000/hit` | 2,153 ns | 32 B | 3 |
| `BenchmarkObserveModelNames4096` | 7.59 ms | 3,613,216 B | 70,975 |
| `BenchmarkObserveModelNameEvent/rules0` | 29,568 ns | 3,136 B | 19 |
| `BenchmarkObserveModelNameEvent/rules64` | 99,814 ns | 26,992 B | 29 |
| `BenchmarkObserveModelNameEventWired/rules0` | 15,688 ns | 3,136 B | 19 |
| `BenchmarkObserveModelNameEventWired/rules64` | 76,644 ns | 31,088 B | 30 |
| `BenchmarkSkipJSONString/plain/102400` | 9,647 ns | 0 B | 0 |
| `BenchmarkSkipJSONString/escaped/102400` | 8,821 ns | 0 B | 0 |
| `BenchmarkSkipJSONString/tail/102400` | 6,882 ns | 0 B | 0 |
| `BenchmarkSkipJSONString/plain/1048576` | 84,535 ns | 0 B | 0 |
| `BenchmarkSkipJSONString/escaped/1048576` | 86,790 ns | 0 B | 0 |
| `BenchmarkSkipJSONString/tail/1048576` | 67,966 ns | 0 B | 0 |
| `BenchmarkRequestMetadataShapes/escaped` | 629,819 ns | 35,552 B | 370 |
| `BenchmarkRequestMetadataShapes/many_messages` | 254,854 ns | 32,280 B | 30 |
| `BenchmarkRequestMetadataShapes/multimodal` | 168,469 ns | 16,240 B | 73 |
| `BenchmarkRequestMetadataShapes/tool_schema` | 85,121 ns | 3,480 B | 22 |
| `BenchmarkRequestMetadataShapes/digit_description` | 73,440 ns | 2,648 B | 6 |
| `BenchmarkAnalyzerFeed/content` | 3,503 ns | 0 B | 0 |
| `BenchmarkAnalyzerFeed/reasoning` | 2,881 ns | 0 B | 0 |
| `BenchmarkAnalyzerFeed/tool_call` | 6,972 ns | 240 B | 1 |
| `BenchmarkAnalyzerFeed/usage` | 3,131 ns | 0 B | 0 |
| `BenchmarkAnalyzerFeed/negative` | 1,798 ns | 0 B | 0 |
| `BenchmarkAnalyzerToolNameDedupe/10000` | 4.88 ms | 192,096 B | 10,121 |
| `BenchmarkAnalyzerToolNameDedupe/100000` | 48.39 ms | 694,264 B | 100,220 |
## Reproduce
`scripts/perf.sh` owns the repeatable local run: `bench`, `stress`, `streaming`,
`sustained`, `image`, or `all`. `scripts/perf.sh docs` runs the full set,
writes `perf/results.json`, and regenerates this page. It refuses a dirty tree
unless `PERF_ALLOW_DIRTY=1` is set, because published numbers must name a
commit.
The full run uses the documented levels and durations:
5 seconds for each
fast stage, 300
seconds for each sustained level, and
60 seconds for
each streaming throughput level. Override the level lists and durations with
`PERF_FAST_LEVELS`, `PERF_SUSTAINED_LEVELS`, `PERF_STREAMING_LEVELS`,
`PERF_FAST_DURATION`, `PERF_SUSTAINED_DURATION`, `PERF_STREAMING_DURATION`,
`PERF_STREAMING_HOLD` and `PERF_STREAMING_CHUNKS`. The bundle records what ran;
a changed run changes the page.
The individual commands below are the same evidence path. Use a fresh
`DEV_COPY_DB=0` instance for each workload. Never target the main instance on
port 8080. The stress harness verifies that the target is the private
`scripts/dev.sh` instance before sending traffic.
```sh
(
set -e
trap 'scripts/dev.sh stop' EXIT
export MILLIVOLT_OPERATOR_TOKEN='dev-instance-credential'
DEV_COPY_DB=0 scripts/dev.sh
go run ./cmd/stress -target http://127.0.0.1:8081 \
-concurrency 1,8,32,128 -duration 5s -stream-duration 0
)
```
The isolated benchmarks use the test overlay:
```sh
scripts/check.sh go test -run='^$' -bench='BenchmarkProjectionFootprint|BenchmarkStoreDiskBytes' \
-benchtime=1x -count=1 ./internal/web
scripts/check.sh go test -run='^$' -bench='BenchmarkProjectionProcessRSS' \
-benchtime=1x -count=1 ./internal/web
scripts/check.sh go test -run='^$' -bench='BenchmarkStorageWriter' \
-benchtime=2s -benchmem -count=1 ./internal/storage
scripts/check.sh go test -run='^$' -bench='BenchmarkQueryBeforeEquality' \
-benchtime=1x -benchmem -count=1 ./internal/storage
scripts/check.sh go test -run='^$' -bench='Benchmark(RequestMetadata|ProxyDurableContention)' \
-benchtime=2s -benchmem -count=1 ./internal/proxy
```
The image measurements use a disposable local image:
```sh
docker build --platform linux/amd64 -t millivolt-perf .
docker image inspect millivolt-perf --format '{{.Size}}'
docker save millivolt-perf | gzip -9 | wc -c
```
## Limitations
- The host was shared. Other workloads can change absolute rates and latency.
- The upstream was a local synthetic server. Provider latency, TLS, retries,
payload shape, and real identity cardinality are not represented.
- These are short closed-loop bursts, not sustained production capacity. No
result here is a capacity ceiling or a guarantee.
- The streaming concurrency table is a short ramp on a 10-frame, roughly
1-second fixture. The streaming throughput table is a
60-second
per-level ramp on a
100-frame, roughly 1-second
fixture. Both are synthetic-fixture measurements, not universal maxima. The
throughput table's
1,024-client level is above the
example `max_conns_per_host` of 512 and measures queueing; it is a
fixed level list, not an automatic saturation search. Its token rate is the
fixture's usage count, not provider-token throughput.
- The 300-second sustained sweep uses the fast-response fixture with no
intentional stream hold (`-stream-duration 0`). A level with durable drops is
an overload observation, not a maximum streaming rate. The streaming
throughput ramp is the only sustained held-stream run; it stops at its fixed
levels and does not search beyond the largest listed level.
- Performance checks are manual (`scripts/perf.sh docs`), not CI. Each level is
one run per invocation on one host; there is no repetition or variance study.
- Durable accounting is best-effort. The fast stages happened to remain exact;
the overload benchmark intentionally does not.
- Container-wide CPU/RSS, cold-start time, browser rendering, arm64 image size,
disk throughput, and a full dashboard preload over a large authorized history
were not measured for this page.
- The image archive is a local artifact, not a published registry digest.
- Do not compare these numbers directly across differing hosts, workloads, or
revisions.
---
# Security
## Deployment boundary
millivolt is a trusted-operator tool, not a multi-user security boundary. It
has no authorization roles or tenant isolation. The whole dashboard is gated
by the shared `MILLIVOLT_OPERATOR_TOKEN` environment credential and denied
while it is unset (see [operator access](docs/routes.md#operator-access)):
there is one operator identity, no per-user accounts and no audit trail of
who acted. Only the `GET /healthz` liveness and `GET /readyz` readiness probes,
the `POST /session` login handshake, origin-root brand/PWA assets and transparent
inference stay open at the application gate; the canonical open-path list is
[operator access](docs/routes.md#operator-access). The upstream API key is
forwarded for inference; it does not secure the dashboard or administration.
The supplied listen address binds all interfaces. Prefer an explicit loopback
listen for local use. Before network exposure, restrict access with network
controls and an appropriately authenticated ingress. Protect the entire operator
surface, including dashboard/bootstrap/live records, SQL, Debug, exports,
Settings, Pause, Limits, purge and restart. Inference clients can also change
provider-wide limits through request headers.
For Docker, publish on host loopback, for example `-p 127.0.0.1:8080:8080`.
A non-root container and a public image do not authenticate callers or make a
publicly reachable operator listener safe. Protect persistent config/data
volumes as sensitive state, including backups and debug captures.
Browser same-origin mutation checks and framing denial are defense in depth
behind the operator credential, not a substitute for it. The live metrics feed
does not opt into cross-origin
browser sharing; the dashboard uses a same-origin EventSource authenticated
by the HttpOnly session cookie. This does not
prevent direct non-browser reads or secure a publicly reachable listener.
Explicit non-browser calls remain supported; gated endpoints need the
`Authorization: Bearer` credential or a session cookie minted through
`POST /session`. The `/mcp` endpoint is the exception: it requires the
operator credential as a Bearer header and refuses a session cookie alone with
`403`, because the caller's credential is forwarded on its internal calls.
## Upstream destinations and credentials
`X-Proxy-Base-URL` selects an HTTP(S) destination. An unrestricted caller can
therefore make the proxy contact destinations reachable from its host.
`allowed_base_urls` constrains accepted upstream targets: an entry admits the
same scheme, host and port (with the scheme's default port filled in) and that
entry's path and its children. A port-less entry therefore admits the default
port only, not every port on that host. Use trusted destinations and
appropriate egress controls. This is not a DNS/IP isolation sandbox.
The proxy does not follow upstream redirects.
Ordinary metrics group credentials by SHA256 rather than retaining the raw key.
Known secret and configured authentication headers are redacted in retained
metadata/captures without changing forwarded headers. This does not sanitize
arbitrary text: request/response bodies, URLs, error messages, model names or
user-supplied labels can contain sensitive data. Preview capture and Debug
capture are opt-in, but enabling them can persist that data.
## Client identity is a claim, not an authentication
`X-Proxy-Client` (and the classified client derived from SDK and user-agent
headers when it is absent) is chosen by the caller. Nothing verifies it: a
request that sends another client's label is treated as that client by every
feature that scopes on client - `request_overrides` (so it inherits that
client's header and token-ceiling rules), `sub_conversations` (so it gets that
client's field mapping and strip list), Debug capture sessions, pause holds
including the "new clients" comparison, and the Cursor parked-run scope. The
recorded client, and therefore dashboard grouping and the `client` filter, also
follow the claim.
The recorded client address is likewise a hint. `client_ip` records the first
`X-Forwarded-For` element when that header is present and the socket peer
otherwise; the value is diagnostic, never verified, and a directly reachable
caller can set it. Do not treat `client_ip` as a verified source for
rate limiting, allowlisting or attribution outside the proxy. The operator
gate's brute-force limiter is a separate key: it reads the address the
immediate hop appended (the last `X-Forwarded-For` element) and falls back to
the socket address, so it distinguishes clients behind a correct ingress but a
directly reachable caller can still pick its own bundle.
The example ingress trusts exactly one loopback hop (`set_real_ip_from
127.0.0.1`, [deploy/nginx.conf](deploy/nginx.conf)) and its `allow` lists
evaluate that rewritten address. A caller that can already reach the loopback
listener therefore chooses the address the lists see: they separate remote
networks from each other and from local callers, not local callers from each
other. Treat a location allowlist as reachability control for a network, not as
identity.
Treat client scoping as a way to separate your own tools and traffic, not as a
security boundary between mutually untrusted parties. A proxy reachable by
someone who should not act as another client needs that separation enforced
outside these labels: distinct `allowed_base_urls`, distinct upstream
credentials per caller, or a separate proxy instance. Credential-class header
names are refused in `request_overrides` and redacted in retained data, so a
claimed label cannot by itself move an upstream secret; the exposure it does
create is the policy attached to the label, not the credential behind it.
Token-refresh handback responses contain credentials in the error body for the client to adopt.
Do not publish them, browser exports, databases, backups, local config,
login-script output or screenshots containing private history.
Provider favicons can make browser requests to Google's favicon service, exposing
the provider hostname and the browser's network address to that service.
`referrerpolicy="no-referrer"` omits the dashboard referrer, not the requested
hostname. Hostname syntax checks do not establish that every qualified provider
label is public. The documentation capture workflow blocks all external browser
requests; that isolation is not the ordinary dashboard's network policy.
## Administrative data and resource limits
SQL is restricted to read-only queries with execution/output limits, not a
sandbox for adversarial resource consumption. Debug has bounded capture sizes
and retention, but captured bodies remain sensitive. Exports and backups can
outlive those retention settings.
Metrics storage is asynchronous and best-effort. Queue overflow/write failures
can lose records; this is not a tamper-proof, complete compliance audit log.
Do not rely on the dashboard as the sole security or billing record. See
[operations and limitations](docs/operations.md#storage-and-accounting).
## Code scanning
GitHub code scanning (CodeQL) reports on pull requests and the default branch.
Alerts are triaged against the [deployment boundary](#deployment-boundary), not
dismissed for convenience. True positives are fixed and covered by regression
tests; accepted false positives record their rationale in the alert instead of
being hidden by a broad path exclusion. The current categories and their
trust-boundary evidence:
- Operator-only read-only SQL (`internal/storage/query_admin.go`). The query
console sits behind the operator gate, accepts one SELECT, and enforces
execution time, row, column and result-byte limits. The operator is already
trusted with the database, so this is not an unauthenticated SQL surface.
- JSON, SSE and gzip response writers. The operator surfaces behind the gate
encode the proxy's own output, and framing and compression state are bounded
by the response-size guard. Two writers are intentionally reachable without
the operator gate: the `/healthz` and `/readyz` probe JSON and the transparent
inference SSE relay, which streams the upstream response to a client that
supplied its own provider credentials. Neither exposes operator data (the
readiness reason names queue depth and flush failures only), so this category
is not an authentication bypass.
- Bounded allocations (`cmd/proxy/mcp.go`, `internal/sse/frames.go`). The MCP
recorder cap is the fixed `mcp.MaxResponseBytes` read budget (64 MiB) plus one
byte, not a configurable limit. The SSE frame's requested size is the
marshaled payload length plus the fixed framing overhead; `slices.Concat` may
round the capacity up to the allocator's size class. The payload is bounded by
the relay's scanner and encoder frame caps and the response-size guard. Input
is rejected at those bounds before the sized allocation, so neither is an
attacker-controlled unbounded size.
The operator-cookie and integer-conversion findings were true positives and are
fixed. The operator session cookie now sets `Secure` when the request arrived
over TLS or through an https-forwarding ingress. The jsonnum, config and
throttle paths now parse or bound at the target width, with regression tests.
The allocation findings are closed by construction, using overflow-safe capacity
owners. The operator SQL console is an accepted boundary, and the transparent
response-writer wrappers are false positives from interface fan-out; their
alerts are dismissed with a recorded reason. Dismissing or resolving a
code-scanning alert is a maintainer action in the GitHub security UI; this note
records triage rationale and does not dismiss any alert automatically.
## Scope
In scope: the proxy, dashboard, MCP server, container image, and repository
automation. Out of scope: findings that require a compromised host, an operator
who already holds the credential, a misconfigured ingress, or a third-party
provider. A report that depends on the documented trusted-operator boundary must
show how that boundary is crossed.
## Safe harbor
The maintainers will not pursue or support legal action for good-faith research
that stays within this scope, avoids privacy violations, service disruption, and
data destruction, and allows reasonable time to fix the issue before disclosure.
## Reporting a vulnerability
Do not publish credentials, private request content, or actionable exploit
details in a public issue.
Use the repository's enabled
[private vulnerability reporting](https://github.com/LLM-4-People/millivolt/security/advisories/new)
channel.
Include the affected revision, deployment assumptions, impact, and a minimal
reproduction using neutral local fixtures. Test only systems you own or are
explicitly authorized to assess.
## Supported versions
Security fixes target the latest published release. The `0.x` series is in
active development; older releases are not backported.
| Version | Supported |
| --- | --- |
| Latest published 0.x release | Yes |
| Older releases | No |
## Disclosure process
Reports are acknowledged on a best-effort basis; no response-time SLA is
promised. Maintainers will validate the report, prepare a fix, publish a GitHub
security advisory, and credit the reporter unless they prefer otherwise. Please
allow time for a coordinated fix before public disclosure.
Dependency review and a scheduled Go vulnerability scan run in CI. A clean scan
is not a security guarantee, and neither replaces a private report.
GitHub documents [private vulnerability reporting](https://docs.github.com/en/code-security/how-tos/report-and-fix-vulnerabilities/configure-vulnerability-reporting/configure-for-a-repository).
---
# Architecture and ownership
millivolt is one Go HTTP service with an embedded, no-build dashboard. The
client-facing protocol and upstream relay/translation are separate boundaries;
the [compatibility matrix](adapters.md#client-and-upstream-compatibility)
defines their supported combinations. [Contributing](../CONTRIBUTING.md)
covers change and verification rules; [protocol](protocol.md) and the
[operations overview](operations.md) describe public behavior.
## Source map
| Owner | Responsibility |
| --- | --- |
| [cmd/proxy](../cmd/proxy/main.go) | Config/CLI, dependency wiring, routes, process lifecycle. Restart, log, backup and operator-gate handlers have separate existing files. |
| [internal/config](../internal/config/config.go) | Config/defaults/validation, split by subject into `config_types.go`, `config_defaults.go`, `config_clone.go`, `config_load.go`, `config_validate.go` and `config_validate_overrides.go`; schema, YAML generation, revision-checked Settings, model-rule execution, and the credential-class header vocabulary and redactor in `credentials.go`. |
| [internal/settingsdoc](../internal/settingsdoc/settingsdoc.go) | Renders the generated [settings reference](settings.md) from `Schema()`, the defaults passed through the YAML writer and the proxy flag registration; [cmd/docgen](../cmd/docgen/main.go) exposes it as `-write` and `-check`, and the config suite fails when the page drifts. |
| [internal/backup](../internal/backup/archive.go) | Streamed self-checked operator archive format (VACUUM INTO snapshot + zstd + SHA-256), with byte- and file-backed database members. One decoder core serves byte and file callers; restore stages its upload and decoded database as private files instead of heap copies. `params/params.go` owns the backup/restore query keys, boolean spelling, restore modes and archive magic shared by the HTTP handlers and `mvctl`; the leaf imports no encoder, SQLite or zstd. |
| [internal/proxy](../internal/proxy/proxy.go) | Routing, admission integration, retry/relay, request metadata, scoped request overrides, native-run ownership, operator state/capture; inbound/outbound protocol selection and canonical paths in [protocol.go](../internal/proxy/protocol.go); the quota-pause reaction owner [quotapause.go](../internal/proxy/quotapause.go) (settleQuotaFailure/addQuotaHold) and GET/POST /admin/quota. |
| [internal/scheduler](../internal/scheduler/scheduler.go) | Provider/key admission, retry pacing, scoped holds, and limit policies selected by provider, model, client, key hash and conversation with optional recurring schedules; the provider-wide durable quota/billing recovery gate (quota facet, probe-paced recovery). |
| [internal/metrics](../internal/metrics/metrics.go) | Records, numeric/usage/outcome semantics, pending/ring lifecycle, observers and Prometheus. |
| [internal/sse](../internal/sse/analyzer.go) | Streaming content/usage/TTFT inspection for Chat, Responses and Anthropic SSE ([analyzer_anthropic.go](../internal/sse/analyzer_anthropic.go)), split into decode, feed, usage, terminal, error, fill and report modules; client-facing OpenAI SSE frames and completion envelopes. |
| [internal/format](../internal/format) | Explicit wire codecs (Chat, Responses, Anthropic) over the canonical [IR](../internal/format/ir/ir.go), Connect/protobuf framing, and the same-protocol translation facade. |
| [internal/storage](../internal/storage/store_open.go) | Asynchronous SQLite writing, schema/read fidelity, totals, purge fence, packed snapshots and bounded queries. |
| [internal/web](../internal/web/aggregate.go) | Embedded shell/assets, bootstrap, canonical history projection and chart/explorer/log aggregates; `aggregate.go` owns the core row model and each surface has its own `aggregate_*.go` module. The site build republishes `css/themes.css`, `js/theme_boot.js` and `js/theme_state.js` byte-for-byte, so both surfaces share one palette registry and one apply path. |
| [cmd/mvctl](../cmd/mvctl/main.go) | the human operator client: command dispatch, per-command flags, human and JSON rendering, and destructive-action guards. It reuses `internal/mcp` for the operator HTTP client, `internal/config` for setting kinds and bands, and `internal/backup` for archive and query grammar. |
| [internal/mcp](../internal/mcp) | The MCP tool server for the operator plane: one HTTP client owner (credential, request building, error extraction), one scope-filter owner, one tool registry served by the proxy's streamable HTTP `/mcp` endpoint, the destructive-call guards, and the streamable HTTP handler the proxy mounts at `/mcp`. It owns no analytical projection of its own; the three docs tools read `internal/docs` locally. |
| [internal/docs](../internal/docs/docs.go) | Embedded generated documentation index: strict decode of `docs.json`, ordered list, section read and ranked search for `docs_list`, `docs_search` and `docs_read`. The checked-in file is produced by `scripts/build_site.py --write-manifest`, and a source-hash test rejects a stale copy. |
| [scripts/gallery](../scripts/gallery) | The documentation capture registry, scene composition, capture manifest and generated gallery page. `tests/docs_capture.py` executes the scenes against a private redacted instance; `scripts/site_plates.py` upgrades registered images into the published plate chrome. |
## Logging
[internal/logging](../internal/logging/logging.go) is the one owner of process
log lines. It wraps the standard logger's sink and gives a request-scoped line
the record id minted at the read boundary, so the line joins its stored row,
capture and neighbouring lines. Process and operator-state lines pass no id.
`log.Fatal` exits and the earliest flag errors that precede the process logger
stay direct.
## Release identity and promotion
[version.go](../version.go) owns VERSION validation and SemVer precedence. The
[workflow](../.github/workflows/check.yml) owns the promotion gate and release
publication: `unit`, `browser` and `container` run the full gate on all
`testing` pull requests, code-bearing `testing` pushes, and manually pushed
`v*` tags, plus Dependabot's `main` PRs; docs and site-only pushes use
`docs.yml`.
The three gate jobs start independently and `required` aggregates them.
`required` reuses that gate for ordinary promotions by requiring the head to
be the current `testing` tip, `main` to be already merged back into it, a
successful full-gate record for that exact commit or for the closest gated
`testing` commit when every path since it is docs/site-only, and a VERSION
newer than every tag for non-Dependabot code-bearing promotions.
A non-Dependabot promotion also requires a successful
[escalation](../.github/workflows/escalation.yml) run (forced full-race sweep
plus the repeated clock-sensitive flake sweep) for the reviewed head; the
proof fails closed while that run is missing, pending, failed, from another
workflow, branch or commit. Pending exact check runs get a short bounded wait
and the proof fails closed if they are still pending; let the `testing` push
gate finish before opening or re-running a promotion PR.
Docs/site-only promotions skip the VERSION and full-gate proofs and validate
docs, workflow policy and the site in `required`. The `required` job has a
ten-minute budget for that path.
`publish` tags the reviewed promotion merge, publishes the image and creates
the release without pushing `main`; a manually pushed `v*` tag ships exactly
as tagged. `publish-testing` instead waits for `unit`, `browser` and
`container` directly, because a testing push has no promotion proof to wait
for. The `main-testing-first` ruleset adds the pull-request and merge-commit
requirements with no bypass actors.
[Contributing](../CONTRIBUTING.md#branch-workflow) owns the maintainer
checklist; [operations](containers.md#versions-and-images) owns the published
tag contract.
## Request and scheduling path
`readRequest` retains bounded original bytes and calls the single metadata
decoder. Ordinary content is measured without retaining decoded prompt/schema
trees; opt-in previews stay bounded. Exceptional decoder compatibility belongs
at that owner, not a second routing parser.
Before the typed decode, `parseLLMRequest` bounds decoded JSON-string
expansion: valid UTF-8 and JSON escapes decode to no more bytes than their
spelling, so only raw invalid UTF-8 can consume the small allowance (the
standard decoder writes a three-byte U+FFFD per invalid byte). A larger
expansion is rejected before materializing; malformed JSON keeps its
passthrough contract, and small invalid sequences still pass through.
`parseLLMRequestFast` is a narrow lexical pre-pass for a minimal top-level
object: `model`, `stream`, `messages` with printable-ASCII role and content
strings, and `system`. It extracts the same routing and composition fields
without the typed decode and delegates every other shape, including all
parameter collections, escapes, duplicates, content part arrays, and preview
capture, to the existing decoder and its bounds.
Differential tests pin byte-for-byte record and rejection equivalence.
`SkipJSONString` finds each candidate quote with one `IndexByte` and reads its
escape parity from the backslash run before it, so escaped content is one
forward pass instead of a restart per escape. The metadata preflight counts
message elements, content parts and tool calls in one walk instead of three.
Current scanner and metadata-shape costs are published under [implementation
benchmarks](performance.md#implementation-benchmarks)
(`BenchmarkSkipJSONString`, `BenchmarkRequestMetadataShapes`).
The streaming analyzer scans each frame once for its classifier keys and
classifies lexically wherever it can. Chat tool-call frames are scanned once
for the element index, id and `function.name`, with the typed decode kept only
as the fallback for a shape the lexical grammar rejects. Current frame and
dedupe costs are published under [implementation
benchmarks](performance.md#implementation-benchmarks)
(`BenchmarkAnalyzerFeed/tool_call`, `BenchmarkAnalyzerToolNameDedupe`). A
differential corpus and a fuzz target compare both paths over every analyzer
field they touch.
The Anthropic translator independently requires valid UTF-8 input, so it
cannot materialize the replacement expansion either. The Start stamp follows
body upload and precedes metadata/translation CPU. Upstream whole-document
buffers share one process-wide `max_inflight_response_bytes` reservation:
translated non-streaming responses, the non-streaming 200 quality spool, and
model-discovery pages. Admission reserves the declared Content-Length when it
fits the bounded read cap, otherwise that cap, and refuses with a retryable
`503` and `Retry-After` before reading the upstream body.
`resolveRequestOverrides` in
[request_overrides.go](../internal/proxy/request_overrides.go) merges the
matching `request_overrides` rules once per request, after routing metadata is
known, and its merge has exactly two application stages: the final header
stage of `buildUpstreamRequest`, and the cursor identity headers at the native
send.
Its body stamp is fail closed (an undecodable or oversize result relays the
original bytes with one log line, prefixed by the request id minted at the
read boundary so the line joins the stored row), and a nil resolution - the
feature off or nothing matched - leaves the request path byte-identical.
The sub-conversation strip is the single body-rewrite engine's second
consumer: it fires independently of override resolution, after the tracked
value is extracted from the buffered body and before translation, with
[conversation.go](../internal/proxy/conversation.go) owning the extraction and
the `k:` identity.
A provider's ensured tool names are the same engine's third consumer: they are
restricted to the OpenAI wire, because the injected stub tools are
chat/completions-shaped and must never reach a translated-anthropic or cursor
body, and they fire with any override and strip actions through one document
decode.
Immutable config/client snapshots hot-swap for new work; old transports finish
active requests. Admission is provider+key scoped, with persistent limit
policies layered at the same grant owner. A policy intersects its non-empty
selector dimensions and an empty dimension matches any value, so a
provider-only rule is the provider-wide cap while a narrower rule reuses the
same machinery.
Every matching policy charges its own lease, and a scheduled policy outside
its window neither charges nor queues. Acquisition release settles exactly
once. The provider gate and token-bucket epoch captured at admission keep
clearing/replacing a cap from charging another generation's reservation.
A provider-to-groups index in `Scheduler` keeps each group's slot in a
fair-waiting set: a group joins when its queued head was refused by the shared
provider gate (concurrency or the request/token bucket), not when it is parked
on its own pacing window, hold, anti-burst ramp or per-key cap.
A release settles the gate, then drains that set once in rotating
one-grant-per-group passes, starting after the last group served, so one key
cannot take the whole provider allowance while peers wait; the drain stops as
soon as a grant refills a saturated provider concurrency cap.
The lock-free fair count is the fast path's precise witness: an unrelated key
admits immediately while peers wait on their own predicates, and a group that
stops on one of those predicates leaves the set.
An anti-burst epoch starts a group at allowance 1 and raises it by
`floor((concurrency-1) * elapsed / ramp_duration)`, clamped to the target; a
fully idle period of at least the ramp duration resets the epoch on the next
grant, a shorter gap continues it, and changing the flag, duration or target
resets every epoch.
[tokenrefresh.go](../internal/proxy/tokenrefresh.go) separates provider
exchanges from response policy. Exchange adapters return credential data only;
the shared `resolveKey` ends successful inference refreshes with handback
before admission, live publication or recording. `resolveKeyTransparent` owns
the model-discovery exception. The token-refresh tests cover both mechanisms,
adoption/retry, absent replacement tokens, applicability/failure gates and
bounded exchange responses.
Retries, provider hints, hold waits and per-send deadlines have distinct
owners. Capture context error before canceling it for cleanup. Never retry a
canceled caller or replay meaningful emitted stream content. Quality failure
handling is explicitly exceptional; see [adapters](adapters.md) and the relay
tests.
Queue visibility has one owner across both park points: the bounded admission
FIFO and `WaitSend`'s send gate. Every parked request publishes the live
queued edge (`Throttled`) for the whole wait and clears it on the grant.
`Stats` reports send-gate waiters separately (`SendWaiters`) and the operator
snapshots add them to the global and per-provider queue totals, so a 429
pacing window reads as queued work rather than idle streams; a retrying owner
waiting out its own Retry-After hint is queued for the same reason. Send-gate
waits stay separate from the admission queue bound so `max_queue_size` keeps
its own meaning.
[scheduler/storm.go](../internal/scheduler/storm.go) owns bounded rolling
attempt counters, active-model evidence, cross-key provider/model gates and
exclusive recovery permits. Provider activation requires all models with
activity inside the same window to independently qualify. Old permits cannot
recover newer incidents. Initial admission waits for readiness without
reserving a probe; only the actual send boundary claims a probe after key
admission.
Storm policy changes reset observations and wake waiters without mutating
retained history. Each request carries an observation for its provider and
model, moving distinct failed-request membership between fixed buckets across
retries without retaining a request-ID map. No storm observation scans metrics
history or waits for SQLite.
[proxy/storm.go](../internal/proxy/storm.go) maps config once into scheduler
options and bridges existing relay/native send outcomes. A 2xx permit settles
after relay or before an existing quality re-ask or thinking rescue re-send,
preventing double samples and stream replay. Safe protocol-level reason labels
carry no provider error bodies. The banner uses the canonical bootstrap
builder and browser apply/render gates, with no client-side event arithmetic.
Storm scheduler/proxy/config regressions and the shared UI/browser suite cover
these boundaries.
Important coverage includes request metadata/parser fuzz tests,
`BenchmarkRequestMetadata`, `BenchmarkProxyRequest`,
`TestConcurrentStreamingAccounting`, transport lifecycle tests and scheduler
efficiency/race tests. Proxy requests must not perform history scans or wait
for SQLite accounting.
## Finalized records, pending lifecycle and replay
`PublishLive(begin/update)` snapshots pending state without adding durable
rows. The proxy registers its deferred `Record` before publishing `begin`, so
a subscriber that fails while handling the begin event cannot leave a
permanent in-flight row. `Buffer.Record` atomically finalizes the row, retires
pending state, updates its gauge/revision and emits end. Exactly one finalized
record per request is the accounting unit; proxy-owned IDs cannot collide with
reused client request IDs.
A debug capture registers its pending sidecar marker in `debugRuntime` before
that publish and encodes/saves after it, so the sidecar write delays neither
the live end nor durable accounting. A drawer fetch for the id waits on the
marker, then reads the authoritative store; a resolved marker without a row
keeps the normal 404.
`Buffer.Record`, `PublishLive` and `Backfill` share the one record-retention
pass ([internal/metrics/retain.go](../internal/metrics/retain.go)) at that
boundary. It bounds client-controlled paths/methods and provider-controlled
model/request/finish/error text, tool names and attempt metadata; normal
values stay byte-identical. The bounds and the UTF-8 cut marker are documented
in the [protocol](protocol.md).
Existing lock order keeps sequence assignment and finalized publication
ordered. A feed epoch changes on restart/deletion; queued fanout retains its
mutation epoch. Subscribers attach before snapshot capture and overflow
retires that subscriber so reconnect can replay, rather than silently dropping
permanent rows.
When more than one dashboard subscriber is attached, `Record` and
`PublishLive` attach one lazy observed-JSON cache to that published event
([internal/metrics/event_cache.go](../internal/metrics/event_cache.go)). The
first consuming stream encodes the observer result and every subscriber of
that same event reuses the retained bytes; an encode error is retained too, so
all of them close and resync. Zero or one subscriber keeps the direct
`json.Marshal` path with no cache allocation, and no cache outlives its event
or crosses a feed epoch.
`SnapshotRequest` is the shared atomic cursor/feed gate for bootstrap and SSE.
A foreign/stale cursor requires a full snapshot; deltas are never truncated.
Snapshots include current pending rows and their revision. Full durable-mode
snapshots cap the recent window, with archives fetched separately.
The browser's one upsert owner deduplicates finalized replays. Pending upserts
cannot overwrite finalized rows or a newer pending revision of the same ID.
The authoritative gauge revision is independent: an older event may still add
a missing different row without rolling back the gauge. Full bootstrap closes
the stream before capture; only the winning refresh reopens it. Stale
same-feed full SSE snapshots recapture through that barrier by
sequence/pending revision.
Tests live under [tests/](../tests/README.md), with Go package mirrors in
`tests/_go/`. Use the overlay-aware `scripts/check.sh go test` entry point.
See buffer snapshot/stream tests and the harness sections under
`tests/ui_harness/sections/` for the lifecycle and bootstrap races. Do not
collapse these row, gauge, epoch and request-ownership gates into a
whole-event drop rule.
## Durable writes and deletion
`Store.Recorder(buffer)` publishes the live final record and nonblocking
enqueue under one short producer fence. SQLite is the durable authority;
enqueue overflow and failed batches remain visible best-effort losses.
`internal/storage/health.go` is the one owner of that loss accounting and of
the pipeline's liveness. Every discard is counted under exactly one cause
(`overflow`, `after_close`, `panic`, `flush_failed`); `Dropped()` sums them
for the dashboard counter, and the exposition renders each cause so alerting
can separate overload from shutdown, a panic and a failing database.
The same owner publishes the oldest queued record's arrival and stamps every
flush attempt, so `/readyz` can tell a stalled writer from a merely idle one:
a non-empty queue whose oldest record, or last successful flush, has passed
one flush interval plus the wedge window (four intervals clamped to 30 seconds
and 2 minutes) reads wedged. The interval on top is the record's own cadence,
so a record inside it never convicts and a long cadence cannot hide a stuck
writer for longer than that.
The arrival stamp comes from the producer when the queue was empty as well as
from the writer, so a writer that dies before its first pickup still ages the
record, and with no flush since boot the first accepted record ages the
last-success arm too, so a zero flush stamp is not a blind spot. A failed
flush stays sticky. `/healthz` stays the liveness contract; readiness is the
stronger answer a scheduler needs.
`tests/_go/internal/storage/health.go` pins the cause split, the wedge/ready
transition and the disabled-store answer, and
`tests/_go/internal/metrics/prometheus_extra.go` pins that the storage block
is rendered per scrape rather than from the ring-revision memo.
The one writer owns transactions and incremental totals. Its insertion plan
binds arbitrary values, using exact zero/empty literals only for absent scalar
work, and preserves existing conflict/rollback semantics. Record columns,
migration, scanner and request-parameter presence bits must evolve together;
explicit zero/false is not absence.
Boot migrations also own one-shot data repairs for shapes current writers
cannot produce.
The empty-string `attempts`/`tool_names` document becomes the canonical
`null`. The pre-0.3.34 start-anchored `ttft_ms` (`ttft_ms > 0` with
`final_attempt_at = 0`) is zeroed because its request-start anchor was never
stored and cannot be re-derived; a snapshot merge applies the same TTFT
canonicalization so a pre-repair backup cannot reintroduce the shape.
Each repair is guarded by a `meta` marker and re-armed when that marker is
absent (a restored backup predating it). A repair that rewrites rows fires the
projection update trigger, advancing the epoch and invalidating the persisted
totals document so boot rebuilds it. Provider-alias repair applies every pair
in one transaction, in source-key order, so a later trigger or constraint
failure rolls earlier label rewrites back instead of leaving history
half-merged.
The durable log order is `(started_at, id)`, owned by the sole request-log
index `idx_requests_log`. The web log page pushes exact provider, client, key
hash, conversation id and canonical numeric status predicates into SQL; a
filtered page rechecks them while scanning the ordered index.
Model filters remain wholly Go-owned: canonical matching trims every stored
spelling and applies the configured rules, so raw SQL equality would silently
drop a padded or rewritten stored model that still matches. Status classes,
tools, errors and time buckets also stay in Go.
Within one dimension, repeated filters are OR alternatives and are pushed as
one SQL `OR` group when that dimension is exact; a group that cannot be pushed
stays Go-owned so the SQL candidate set never narrows below the group's union.
The ordered export and purge use the separate `PurgeFilter` surface (provider,
model, client, conversation, error type, status code, `has_error`, `debug`,
and time bounds; no key hash), whose equality-only preview count or purge
scans the table. Results, order and pagination stay exact without six
per-dimension composite indexes; a time bound becomes an index range.
An upgrade that actually finds and drops one of those superseded indexes
repacks the database once with `VACUUM`, because SQLite otherwise only moves
the freed index pages to its freelist. Fresh and already-migrated databases
skip it, and a failed reclaim logs a warning without failing startup: every
row stays intact and manual reclamation remains possible. A successful reclaim
resets the projection witness, because `VACUUM` may renumber implicit rowids.
The page handler classifies active filters: when every predicate is SQL-exact
it requests exactly the remaining rows, and Go-only predicates grow
geometrically from the page size to the 512-row batch cap within the 4,096-row
scan budget. Both preserve the exclusive `(started_at, id)` cursor and `more`.
The candidate statement selects only the columns the residual groups read, and
the derived error signature and tool labels are filled only when the plan
declares them. Current filtered and unfiltered page costs are published under
[implementation benchmarks](performance.md#implementation-benchmarks)
(`BenchmarkHandleLogPage`).
`Store.Clear` serializes administrative purges, captures an accepted-record
boundary and publishes one command. The writer flushes earlier records,
deletes matching records/sidecars and rebuilds totals transactionally, then
removes only matching ring rows through that boundary. New completions
survive.
Failed SQL does not mutate the ring; postcommit memory effects finish even
after caller cancellation. The atomic command check must not acquire the
producer lock for every ordinary consumed record.
A non-empty purge, and a drained expired-capture sweep, checkpoint-truncates
the WAL after the commit: `secure_delete` cleans the pages SQLite writes, and
truncation removes the pre-delete frames that would otherwise keep the old
page images in the `-wal` sidecar.
The checkpoint is bounded and best effort; a busy reader raises a pending flag
that the writer's near-term tick retries, and `Store.Close` makes the final
attempt after the read pool is closed, without failing the committed deletion.
The restricted SQL console (`GET /metrics/query` and the MCP `query` tool) is
gated in `validateQuery`: one `SELECT`, no `;`, no
`ATTACH`/`DETACH`/`PRAGMA`/`LOAD_EXTENSION` word, and no
`sqlite_dbpage`/`dbstat`/`pragma_*`/`readfile`/`writefile`.
The scan is lexical over the raw text, string literals included, and the read
pool's `mode=ro` pins only the main database; SQLite still opens an `ATTACH`ed
file writable.
Follow-up gap: the pinned modernc.org/sqlite v1.61.0 exposes an authorizer per
physical connection (`AuthorizerRegisterer`, installed through
`RegisterConnectionHook`) and a `_defensive=1` DSN mode, and this build wires
in neither. An authorizer policy at read-pool open plus defensive mode would
turn the keyword list into defense in depth instead of the only owner.
Operator backup and restore stage database-sized work as private files.
`backup.EncodeFile` streams the packed snapshot, encoded archive and
self-check in bounded chunks. On restore, the handler streams the capped
upload into one private file, and `backup.DecodeFile` shares the byte
decoder's header, zstd, SHA-256 and member-framing owner while writing the
database member into a unique staging directory on the live database's volume.
Config bytes remain buffered; the database member does not.
Inspect and overlap integrity-check or attach that file in place, replace
streams it into the pending snapshot and atomically renames it, and merge
attaches it to the held writer transaction. Decode and validation finish
before an apply path takes the writer connection.
The upload and decoded database files are removed on every handler path,
including over-cap, invalid archive, apply failure and client disconnect.
A boot-time sweep reclaims only known staging names older than a safety age,
examining the directory in bounded batches up to a per-boot entry budget; the
live database, pending snapshot, rollback set, state marker and quarantined
admission evidence are outside that registry. A rejected or interrupted
pending file is preserved under a unique `.incoming.invalid-` or
`.incoming.invalid-interrupted-` name with its sidecars, so a later
rejection cannot overwrite earlier evidence.
A pre-existing pending snapshot is hard-linked aside before the successor
replaces the slot: the restore drops that link again on its own rollback, and
a boot that rejects the successor renames the link back onto the slot and
admits it once, so the queue entry is never orphaned into the sweep. The
attached inspect/merge and decoded source use the shared `SQLiteURI` with
`mode=ro`, so a snapshot path with URI metacharacters stays literal and writes
through the attached schema fail.
Relevant coverage: storage writer/purge/request-parameter tests,
`BenchmarkStorageWriter` and `BenchmarkProxyDurableContention`. The latter
deliberately overloads storage and separates durable throughput from drops.
Non-stream responses decode once into one analysis: quality outcome, usage,
model/id, finish reason, tool names, bounded preview and the top-level error
envelope share that pass, and the relay applies the same value at commit.
Cost-bearing bodies add the existing provider-keyed cost pass; streaming
analyzer semantics are unchanged.
## History projection and aggregate math
One durable analytical projection retains compact contributions, request IDs,
interned dimensions and immutable sorted metric row orders. It is not a second
database or a per-view history cache. Startup preload and incremental catch-up
share its builder. SQLite transactions capture rowid high-water/destructive
epoch/schema authority; destructive changes rebuild.
Failed or canceled reads never publish stale rows or advance authority. Folds
check the caller's context once per 4,096-row block, and the ring visitors use
the same cadence, so an abandoned request stops without publishing a partial
result.
The `has429` flag (and any future vocabulary-derived classification) comes
from the installed vocabulary, not from a stored column, so the projection
stamps the metrics classifier revision and forces a full reclassification
rebuild when a reload changes it. The chart and explorer memos include the
same revision in their key, so neither a stale projection nor a pre-reload
memo can answer a post-reload request.
Materialization runs off the projection lock with one single-flight builder;
concurrent readers keep serving the previous complete snapshot and wait for
that builder instead of repeating its scan. The builder's token is released by
a defer, so a panic in the scan, decode or interning step frees the waiters
instead of wedging them until restart. A destructive change builds a complete
replacement off-lock and swaps it under a short write lock.
The durable since-inception totals are one meta row written in the same
transaction as every insert, purge and merge on the held writer connection;
its recorded projection epoch is the mutation witness. A merge writes its
request rows, debug sidecars and totals document in one transaction, so a
failed totals write rolls the whole merge back instead of advancing the epoch
over partial rows. An unsupported external writer that advances the epoch
leaves the pair mismatched, so the next boot rebuilds rather than trusting it.
A boot with a changed epoch rebuilds once from the rows instead of trusting a
stale cache, and a full purge has no surviving rows to scan. Restart readiness
therefore no longer scales with the sum of a totals scan plus a separate
preload scan.
Chart and explorer each retain one immutable last-result slot. Keys include
validated view/scope, canonical model rules, finalized/database revisions and,
for explorer, pending revisions. Clock-aligned geometry also gates chart
reuse.
One in-flight fold per memo is shared by concurrent same-key callers: a
different key never waits, and a waiter that finds no published entry after
the leader ends recomputes as the next leader instead of inheriting its error.
The stability recheck hands over the fold's compiled rules identity instead of
re-reading the config and re-marshalling up to 64 rules. The current 64-rule
recheck cost is published under [implementation
benchmarks](performance.md#implementation-benchmarks)
(`BenchmarkAggregateUnchanged`).
The bootstrap KPI fold retains one exact totals slot keyed by the ring's
finalized revision and the store's durable accounting revision; live counters
are filled at serve time and the ring snapshot itself is never retained. The
unwritten set stays ring-derived. A record can be missing from the durable
store while never having been queued: an enqueue that overflowed the writer
channel and a batch whose insert failed are both dropped by design and counted
by cause in the storage health owner; `Dropped()` sums the causes for the
dashboard counter.
The writer queue is therefore not a superset of "possibly not yet durable",
and only the written-id set can say which ring rows are already durable, so
the O(ring) probe under `aggMu` is the exact dedupe boundary. Current
cold-fold costs at the default ring and the one-million-record ceiling are
published under [implementation
benchmarks](performance.md#implementation-benchmarks)
(`BenchmarkKPI/records_1000000/miss`); the ceiling is a non-default
`history_size`, and a correct-but-slow cold fold is the accepted cost of never
double counting.
SQLite data_version is read on one retained connection; unsupported external
writers are not made safe merely by cache invalidation. Log pages push exact
provider/client/key/conversation/status equalities into SQLite before
decoding; model filters are always Go-owned.
The Go scope matcher remains authoritative for canonical models, status
classes, errors, tools, time buckets and live-only classes. Ad-hoc SELECT
results are encoded once during the bounded scan and the endpoint writes that
array directly; row/byte cap failure still precedes every response byte.
Explorer uses dictionary IDs, scoped predicates and bitsets. The projection
also owns two derived tables keyed by its immutable build: the interned
conversation topology (identities, parent declarations, resolved issues and
public-ID groups) and the canonical model IDs/dictionary for the compiled
model-rule revision. Appends extend both by the new rows; destructive
replacement publishes fresh tables, and a rules change rebuilds only the model
table.
Requests mark their own rail/gallery flags on the shared topology, resolve
lineage metadata only for the returned conversation cards, and overlay
ring/pending declarations without mutating the published build. Projected
metric samples stay in shared sorted orders: single-valued dimensions retain
one group id per row, while tool/error membership spans preserve multi-valued
occurrences.
An error's identity is `type|code|msg`; a backslash or the `|` separator
inside a part is escaped, so arbitrary error text can never merge two distinct
errors into one dictionary id. Ordinary text keeps the historic spelling,
which preserves saved error-filter ids.
Exact group accumulators are capped at `explorerGroupBudget` (8192) distinct
values per fold. Values beyond the budget fold into one synthetic `other` card
carrying aggregate count, cost, token, tool and health fields; no per-value
accumulator and no per-card request-ID set is allocated for either an exact
card or them. An error card's affected count is a counter plus the previous
request id: one row's visits are contiguous, so that guard dedupes exactly
what the old ID set did while staying bounded regardless of history.
The card carries an explicit `synthetic` wire marker and no filter key, so a
real value sharing its display label keeps its own exact, selectable card. The
card ranking remains exact among tracked values, with `other` competing by its
aggregate count, and overflow memberships map to the reserved other slot so
their percentile samples still select correctly.
Rows store the group's 16-bit membership slot, not its dimension-wide
dictionary id, so the per-row table is two bytes and the selected-card lookup
is a bounded probe instead of a max-id-sized array; the current 100,000-row
provider fold cost is published under [implementation
benchmarks](performance.md#implementation-benchmarks)
(`BenchmarkAggregateFold/explorer`). Rail distinct-value counts, scope totals
and conversation summaries remain whole-history values and are not derived
from the truncated group set.
Selected groups use the same exact rank reader as chart buckets, and only top
cards derive percentile payloads. Pending dedupe retains only relevant pending
IDs, not another history-sized seen set. Chart's rank reader also produces the
occurrence-weighted avg/min/max in the same traversal, so a metric order is no
longer walked separately for stats.
The all-history order is a compacted immutable base plus a bounded
value-sorted pending delta (16,384 samples): a catch-up append folds only its
new samples, so it stays O(delta log delta) regardless of history, and
reaching the bound arms one background fold that runs off the request and
publishes the merged base under a one-assignment write. Readers keep the
current base and pending until then, so an interrupted fold only leaves the
published order in place.
The fold owns its armed flag behind a deferred release: an unwind there
(recovered and logged) clears the arm with the pending delta intact, so the
next append folds again instead of the flag staying set and pending keeping
the whole delta for the process lifetime.
An append that crosses the bound at one million rows is published under
[implementation benchmarks](performance.md#implementation-benchmarks), now off
the request (`BenchmarkProjectionMetricAppend`). Chart and explorer walk the
two runs through one merge iterator, so a fresh view allocates no
history-sized copy; a materialized merge stays cached per published snapshot
only for consumers that need a slice. A full rebuild replaces the base, clears
pending and discards a fold that has not published yet.
A retained last-65,536-row order serves bounded chart windows whose earliest
in-window durable row is inside that tail. It uses the same compacted base
plus bounded pending delta design and its own per-snapshot view holder, so a
sub-threshold catch-up merges only its new batch instead of copying the tail;
a rebuild of the newest 65,536 rows replaces its base, clears the pending
delta and releases samples that fell out of the tail.
Older or unscoped windows fall back to the all-history order, so the fast path
never changes rank or stat results. Chart membership is indexed from the first
row a bounded window accepts and sized only to that window's remaining span.
The current 15-minute view over 100,000 rows is published under
[implementation benchmarks](performance.md#implementation-benchmarks); the All
path keeps its historical allocation (`BenchmarkAggregateFold/chart_window`).
Both consumers share R7 rank/interpolation and the minimum-sample gate. Period
speed/TTFT percentiles use period samples, never means of bucket percentiles.
Prometheus and the stress harness instead share `metrics.Percentile`'s
nearest-rank contract; do not conflate these two intentionally distinct APIs.
Checked counts/sums/ratios prevent wrapped tokens or non-finite JSON. Cost
stays USD in storage/API; missing/unreportable data must not become fabricated
prices.
`Record.IsError` and `HasRateLimit` deliberately differ. Final/retried 429
counts once per affected request, not via the broad pacing flag. A real
failure and 429 may overlap.
A 429 whose status line a committed keepalive 200 hid is retained as one
in-band attempt entry, so the rate-limit count still sees the upstream status
while the same request counts as an error through its final `error_type`; that
entry is not a retry and adds no second error entry.
`metrics.IsNonRetryableQuotaErr` is the fixed durable quota/billing
classification vocabulary: the retry ladder, storm evidence and in-band
precedence all consult it, so the account condition never counts as a rate
limit. A plain 499 (the local client's own abort) is a flow event; a 499
carrying a structured error observed the upstream's in-band failure before the
client left and counts like its 200 twin.
The shared first-membership gate keeps duplicate tool/error occurrences from
duplicating these health counts without changing event/sample multiplicity.
Tests cover raw/projected/durable/ring/pending parity.
`internal/metrics/errorclass.go` owns the canonical classifier: one
`Classify(facts)` answers "what is this", and the retry ladder, the storm
observer, the Cursor transport, the health predicates, the client-facing class
and the published SQL all read it. `metrics.OutcomeIsError` (composed by
`Record.IsError`) owns the health arms, and the MCP describe document and the
durable purge filter render `metrics.IsErrorSQL`/`HasRateLimitSQL` per call
instead of carrying their own predicates, so an operator extension applied by
a reload reaches the published SQL too.
Classification reads the status AND the provider's own type/code, never
message text.
The two questions are separate and answer differently in the two documented
families: a 429 whose body names a durable quota/billing condition is
ClassQuota for the retry ladder and quota_pause_mode while the health surface
counts the 429 itself as flow control (not an error), and a rate-limit class
on any non-429 status is counted in the rate-limit series while the health
surface also counts the failed request as an error.
The health predicate reads the status and the error type only, so a spelling
carried solely in `error_code` (a shape the envelope parser does not produce -
it pairs every code with a type) can move the class but not the health answer.
The operator's
`rate_limit_error_classes`/`quota_error_classes`/`retryable_error_classes`
extensions are installed as one value (`metrics.InstallClassExtensions`,
installed by `proxy.New`/`proxy.Reload`) and every consumer reads the
effective vocabulary, including the browser mirror, which receives it in the
bootstrap payload when extensions are installed.
Extensions add only: config validation rejects a spelling that names a
built-in of another class; the one documented exception is
`retryable_error_classes` naming a built-in client-fault spelling, the
explicit override for a gateway that mislabels transient failures.
With `client_error_class: header` the relay's outer writer stamps
`X-Proxy-Error-Class` and a class-derived `X-Should-Retry` on every error
status before the line commits (`suppress_client_retries` still forces false);
relayed bodies stay verbatim.
See the files under `tests/_go/internal/web/`: `aggregate_projection.go`,
`aggregate_dimensions.go`, `aggregate_percentiles.go`,
`aggregate_explorer_fold.go`, `aggregate_rate_limit.go` and
`conversation_lineage.go`.
## Canonical observer meaning and conversation lineage
Go's compiled model-rule executor owns canonical grouping. Observer payloads
carry exact raw-to-canonical names and a rules-content revision; only full
authoritative snapshots install a new revision. Deltas, archive and Debug
names merge under that revision, with recapture on conflicts. Browser
dictionaries prune to loaded rows/selector needs. RE2 patterns do not execute
in JavaScript.
The installed observer provider is an immutable atomic holder, so SSE
record/end serialization never takes the ring lock. Canonical names are
memoized across observer calls under the compiled rules revision, and each
known-name vocabulary caches its sorted view under a set revision. Both caches
are bounded and invalidate on content change, so bootstrap polls do not re-run
the rules or re-sort thousands of names.
The single-name memo owns the whole payload under one mutex keyed by the rules
revision, so a repeat at event rate costs one lock and one lookup. Current
event and vocabulary memo costs at the 64-rule maximum are published under
[implementation benchmarks](performance.md#implementation-benchmarks)
(`BenchmarkObserveModelNameEvent`, `BenchmarkObserveModelNames4096`). The
canon the observer reads is memoized per live config pointer, so consecutive
events hand the compiled-rule cache the same backing array and it skips the
whole-set compare; the wired shape's current cost is published there too
(`BenchmarkObserveModelNameEventWired`).
Pinned names are derived from the live holds, debug sessions and limit
policies, never accumulated, so a removed document releases its scope. The
derivation reads and applies the whole roster under the owning document locks:
every document commit happens under the same lock and re-derives right after,
so a stale derivation can never overwrite a newer roster and unpin a live
scope.
Observer records carry server-local `time_bucket`; browser-local dates are
presentation only. Raw names remain raw for exact export/delete semantics.
The explorer's one iterative lineage resolver validates declarations by
client/key/conversation before applying rail/gallery scopes. The resolved
graph is a projection-owned table, interned by dimension slot; a request only
marks flags and overlays unprojected ring/pending declarations. Parent-only
dictionary names do not become observed memberships.
Conflicts, cycles and mixed namespaces remain unknown; missing parents are not
invented roots. Summary counts follow full-filter rail scope; conversation
cards follow the gallery cross-filter. Payloads retain small resolved fields,
not a per-request graph. Parent pivots remain exact namespace-qualified
leaves.
## Operator plane clients
The HTTP client in [internal/mcp](../internal/mcp) owns the operator
credential, request building, the bounded response read and the `.error`
extraction that reads the flat error body regardless of the declared
Content-Type. The credential is redacted from both the transport error and the
failure body.
Redirects are refused rather than followed: `net/http` strips `Authorization`
only when the hostname changes, so a same-host different-port `307` would
replay the credential and the request body.
Its route paths are one closed set beside it, so no tool can name a path
outside the operator plane: nothing reaches the transparent inference
catch-all, and no restore-adjacent route is exposed.
Three registry tools are local-only: `docs_list`, `docs_search` and `docs_read`
answer from the `internal/docs` index instead of constructing an HTTP request.
They share the generated page model with the published site and never touch
the operator credential.
The explorer's `f=dim:id` / `s=` grammar is owned once and shared by every
scoped tool, and the tools never loop: a non-advancing cursor is reported as
exhaustion because the aggregate scan budget is bounded per call. An empty
page is deliberately not exhaustion, because the proxy advances its cursor on
every row it scanned, including rows the scope excluded, so an empty page with
an advanced cursor is a page that must be continued.
The same client backs the whole tool registry. The proxy's own `/mcp` endpoint
injects a transport that dispatches each synthesized request straight to the
proxy's gated handler, with the caller's Bearer attached so the gate
re-validates it, and a named non-routable origin (`http://millivolt.internal`)
that is never dialed and never derived from the request Host.
The endpoint is stateless and Bearer-only: every POST gets a temporary server
session bound to that request's credential, nothing survives it, and the
dashboard session cookie is refused there because a cookie must not create an
internal call.
The SDK's own localhost DNS-rebinding `Host` check is deliberately off: the
documented reverse-proxied placement preserves an external `Host` while
dialing the proxy over loopback, so the check would reject that placement.
Protection therefore rests entirely on the Bearer, which the same gate
re-validates on every request; a missing or wrong credential is denied before
any tool runs. The consequence is explicit: the endpoint admits any `Host` on
the listener it is reachable from, with a valid credential, and any
`Host`-based allow rule has to sit at the ingress rather than here.
The destructive surface is guarded where the risk lives. A purge needs a
filter that constrains something (the bodyless delete-everything command is
never sent), an exact confirmation phrase, and the opaque `preview_token` that
`purge_preview` issued for that same filter, its count and an expiry.
A count cannot bind the two: it is only a number, so two filters matching the
same number would authorize each other.
The token is stateless and HMAC-bound under the operator credential, so
nothing is persisted, no server-side cleanup is needed, and a rotated
credential invalidates every outstanding preview. It carries its own expiry,
enforced when the token is verified, so a preview authorizes a deletion only
shortly after it was taken.
A refusal discloses no count, because a refusal that reports the live count is
a count oracle a model can retry its way through. The count is compared a
second time inside the delete transaction in `Store.purgeTransaction`, so a
population that moved refuses with 409 before any row is removed.
The remaining limit is stated rather than claimed away: a count compares a
population, not row identities, so a same-size different row set passes and a
refusal needs a fresh preview.
Capture refuses to start without durable storage, and checks a scope name
against the vocabulary the proxy reports, because either way the session would
report success and store nothing. Stopping a capture session deletes nothing,
and a purge deletes the stored capture document of every request it matches;
the tools say which is which rather than implying otherwise.
When that vocabulary is readable and non-empty, an unknown capture scope or
throttle provider is refused and the known names are offered; both checks
accept any name when it is empty, and the throttle provider check also accepts
one when the debug read fails, while the capture tool refuses when the same
read fails. See the [MCP server guide](mcp.md).
## Dashboard render owners
HTML embeds the canonical bootstrap builder's JSON, safely escaped in an inert
data block. Transit gzip is negotiated from `Accept-Encoding` with only the
`q` parameter read: unknown parameters are ignored, a bare `*` accepts gzip,
and an explicit gzip entry outranks the wildcard; an unparseable or
out-of-range qvalue refuses.
Deferred external assets download alongside HTML; the Settings sheet is the
one exception: its ordered `settings*.js` set is injected by the Settings
trigger on first open, one module at a time under the same versioned URL, and
single-flighted across concurrent opens. Saved hash/chart choices apply before
rendering; invalid/missing seed falls back to the endpoint. Dynamic HTML/state
is private/no-store.
Every local `/dash/` CSS/JS URL in the shell carries the content version as
`?v=`. On a hit, the service worker answers a `/dash/` request carrying
exactly that version from its stamped CacheStorage before the network; on a
miss it fetches once, stores a successful response, and returns it. This is
the one cache-first path.
A request with no `v`, a stale value, or more than one `v` stays network-first
with cache fallback, as does every brand/PWA path, so an old index cannot pin
new bytes under an old URL. The server serves the exact-version response
`private, max-age=31536000, immutable`, while a missing or stale version
revalidates with `no-cache`. A Bearer-authenticated response that also mints a
session cookie keeps the gate's stricter `no-store`. Immutable asset
representations and revalidation identities are precomputed.
Content version changes use the same winning bootstrap gate to reload once.
The cache-first path is asset caching only, never an offline app shell:
navigation to `/` stays live with the static offline fallback, and
live/operator routes are never cached.
Vanilla modules under [static/js](../internal/web/static/js) have distinct
owners:
- `core`: formatters and shared state.
- `feedback`: the one clipboard and toast path.
- `entities`: identities and record labels.
- `explorer`: routing and cards.
- `chart`: uPlot plans.
- `menu`: the one open/close, placement, dismissal and focus contract for menu
panels.
- `chrome_menus`: the header menu registry and the remaining theme tray, nav
and dimension class menus.
- `operator_console`: the header readout, console shell, section rail, shared
wording and bounded session change ledger.
- `operator_console`'s four policy modules `operator_limits`,
`operator_holds`, `operator_export` and `operator_overview`, each owning its
own section.
- `chrome_operator`: the mutation gate and request shape.
- `chrome_lists`: the operator vocabularies, model grouping, capture storage
gate and footer words.
- `chrome_restart`: the restart choreography.
- `chrome_settings`: the Settings shell and trigger, the accepted debug
snapshot, hold liveness and the download paths.
- `timeframe`: the chart time vocabulary, stored-extent availability and the
timeframe control.
- `log`: the live log frame.
- `log_rows`: the rows, day dividers and column registry.
- `log_archive`: the durable archive window.
- `log_drawer`: the detail drawer lifecycle and retry-attempt navigation.
- `log_detail`: the detail primitives.
- `log_debug`: the drawer's Debug capture section.
- `chrome_storm`: the storm banner, incident identity and details dialog.
- `theme_boot`: the palette ids, labels, order, defaults and the pre-paint guard.
- `theme_state`: normalization, the stored pair, the one apply path and the
browser chrome color, shared by the dashboard and the site.
- `theme`: the dashboard preview tray, comparator state and keyboard
lifecycle.
- `settings`: the lazily loaded Settings sheet.
- `live`: bootstrap/SSE/derived state.
`theme_boot.js` is the one registry: palette ids, labels, order and defaults,
plus the blocking guard that applies a stored pair before first paint.
`theme_state.js` owns normalization, the single stored shape, the one apply
path, the browser chrome color and the optional chart/menu callbacks. A commit
changes the live document pair; normalization also repairs storage at load when
the stored value is invalid.
`theme.js` owns only the dashboard tray: preview, comparator state and the
keyboard lifecycle. The site's `site_theme.js` renders the same registry into
its own header tray and reuses the same state module.
Hover or keyboard focus previews inside the tray only, and the polite live
region follows the keyboard cursor: pointer-induced focus never announces a
candidate. Arrow keys, Home and End move the cursor and its one tab stop;
Enter/Space or a click commits. Compact touch, short and narrow layouts hide
the comparator, preview line and hint while keeping the current caption,
surface control and all eight rows.
`css/themes.css` owns the complete surface, accent and material families every
palette resolves in both modes and the Black/Clear mode material fallbacks. The
site publishes these exact bytes and loads them after its generated tokens and
before `site.css`; its extra contrast rungs derive from the active mode tokens
with `color-mix`, so there is no second palette table.
`dashboard.css` owns the default Instrument Black base, the header chrome and
the theme tray, including the comparator/caption-bar layout. The dashboard
sheet is split by surface, linked in this order: `dashboard.css` (palette,
base, chrome and KPI band), `explorer.css`, `overlays.css` (drawer and
settings sheet), `charts.css`, `requests.css` (including the shared semantic
status fills) and `inspector.css`, with `themes.css` next, then `console.css`
for the readout and operator console only, and the vendored uPlot sheet last.
The link order is the cascade; `dashboard.css` remains the `:root` palette and
type owner: its `@font-face` rules load the embedded Instrument Sans and
JetBrains Mono faces from `static/fonts`, and its `--sans`/`--mono` tokens
name the families every surface consumes. The docs build publishes the same
files and adds Instrument Serif as the site-only display face for marketing
and documentation titles, so neither surface makes a third-party font request.
Each palette owns `--shadow-rgb` and `--veil-rgb`, and the mode veil tokens
compose that hue, so cool Clear palettes get cool elevation and scrims.
`chrome_settings.js` owns the Settings trigger, loader, open-state probe,
shared status line and the single close entry point; the lazy `settings*.js`
modules own the sheet's state, editors and apply/revert/backup/purge behavior.
A clean open at the same document revision and writable state still fetches
freshness but keeps the retained rows and baseline; revision/writable changes,
drafts and Revert refill. The model-rule preview reuses one traffic-example
cache keyed by the live revision, and its test spelling is preview-only: it
never revalidates the draft or marks the sheet dirty.
One memoized derivation serves scoped rows and shared readouts. O(1) ID
upserts, pure-replay suppression and rAF-coalesced live rendering avoid
repeated scans. Incremental log groups preserve scroll/selection; archive
paging uses paired timestamp/ID cursors and stays outside the live
arrival-ordered ring.
One single-entry memo reuses the merged ring+archive window within a live
revision, keyed by the live data and its revision plus the archive array
identity/length and the scoped-ring (or prior merged) array identity; archive
replacement or trim, reset, a live revision bump and a data replacement each
rebuild it.
The explorer memoizes its filter-set identity on the canonical,
order-independent scope (filter set plus status, excluding the `by/`
breakdown). A by-only hash change repaints the rail/gallery and fetches that
one breakdown while the scoped log window, chart and drawer keep their state.
Unchanged explorer renders write nothing beyond card string construction: the
rail and gallery HTML is compared before assignment, the shared dimension
trigger is rewritten only when its dimension/count presentation key changes,
and gallery layout sizing runs only after the gallery content actually
changes.
Server chart responses alone own buckets/totals at configured cadence.
Presets, series, money/unit formatters and visible dimension order each have
one registry. Derived series (error rate, cache hit, blended totals) recompute
from those server bucket sums; the browser never folds events into chart data.
Hidden lines keep stable columns; percentile/range changes reuse the plot.
Isolated samples use filled/ring points without coordinate jitter. Shared axis
formatting avoids duplicate labels. The percentile dropdown is the only
visible pXX label owner; conditional error/429 badges omit zero independently.
Operators use shared mutation/revision gates, lossless Settings collection,
stable event-path outside-click handling and a focus/inert dialog lifecycle.
CSS owns responsive breakpoints. Preserve accessibility, saved state and cheap
boot; jsdom tests do not replace real desktop/mobile canvas checks.
## Documentation captures
`scripts/gallery/data.py` is the one registry for every published application
capture: path, title, description, scene, viewport and setup. `tests/docs_capture.py`
owns the scene strategies, and `scripts/gallery/compose.py` composes the two
surfaces of each plate. A new view is one registry entry plus a scene, never a
second list in the capture harness or the site build.
The capture writes `docs/images/capture.json`: per-plate bytes, dimensions and
SHA-256, the verified record count and a digest of `internal/web/static`. The
digest is the freshness gate; a dashboard change fails `tests/python/gallery.py`
until the plates are recaptured. The same run regenerates `docs/gallery.md`.
`scripts/site_plates.py` is the one upgrade from a registered image to a plate.
It serves both a plain Markdown image and the generated page's semantic figure,
so the published gallery, the prose guide and the Markdown source agree about
number, station, route and the Black/Clear split legend.
`scripts/gallery/__init__.py` loads and validates the registry at import, so a
missing alt, duplicate path, unknown scene or bad viewport fails before a browser
starts. CI runs the same module through `tests/python/gallery.py`.
---
# Contributing
Start with the [architecture map](docs/architecture.md). Keep changes focused on
existing shared owners, include regressions, and use neutral fixtures rather
than real providers. Setup, safety and verification rules live in this guide.
Use the [issue tracker](https://github.com/LLM-4-People/millivolt/issues) for
non-sensitive bugs and proposals, and
[pull requests](https://github.com/LLM-4-People/millivolt/pulls) for changes.
Security reports belong in the private channel in [SECURITY.md](SECURITY.md).
## Support and first change
[SUPPORT.md](SUPPORT.md) routes usage questions, reproducible defects, concrete
proposals, and security reports. For a first change, fork the repository, branch
from `origin/testing`, update the existing shared owner, and add a targeted
regression. Run `scripts/check.sh` and report what ran. Open the pull request
against `testing`, not `main`.
Use the `good first issue` label only for documentation, tests, or tooling with
a clear scope and expected result. Do not label credential handling,
authentication, storage, protocol-core, or release work. Ask in the issue before
starting.
## Branch workflow
New features branch from and land on `testing` first, with their documentation
and relevant checks in the same change. Fetch the current remote branch before
starting follow-up work. Use feature branches targeting `testing` for review,
or push a completed, checked feature directly there when that is authorized.
`main` is promotion-only. Do not develop on it, push it directly, or bypass its
rules. The `main-testing-first` ruleset requires a pull request and the
`required` check, allows merge commits only, and has no bypass actors. The
`long-lived-branch-integrity` ruleset also blocks deletion and non-fast-forward
updates. Administrators have no bypass either. When release automation needs new
behavior, change the workflow in a pull request; never add a bypass actor or use
an administrator override. Inspect the active ruleset with:
```sh
ruleset_id="$(gh api repos/LLM-4-People/millivolt/rulesets \
--jq '.[] | select(.name == "main-testing-first") | .id')"
gh api "repos/LLM-4-People/millivolt/rulesets/$ruleset_id" \
--jq '{name,enforcement,bypass_actors,current_user_can_bypass}'
```
It must report `enforcement: active`, an empty `bypass_actors`, and
`current_user_can_bypass: never`.
### Commit as you go
Commit each completed fix locally as soon as its targeted check passes. This
includes every individual finding from a review, audit, or multi-agent round:
do not hold those edits in the working tree until the end, and do not squash
several findings into one final commit. If a fix depends on an earlier
uncommitted change, commit that coherent change first, then commit the
dependent fix. Keep unrelated user changes out of the commit. A push still
waits for the full push gate on the exact commit being pushed.
A commit subject also feeds the generated release notes: every subject whose
type is not one of `build`, `brand`, `checks`, `chore`, `ci`, `deps`, `doc`,
`docs`, `merge`, `refactor`, `release`, `style`, `test`, `tests`, `tooling` or
`version` becomes a Highlight on the release that first contains it, and
internal types are folded under "Other changes". Write the subject for an
operator reading the release page.
Completed feature pushes to `testing` publish only the moving development image.
All pull requests to `testing`, all code-bearing pushes to `testing`, manually
dispatched `testing` gates, and manually pushed `v*` tags run the full gate.
Docs and site-only pushes to `testing` run the smaller documentation gate in
`docs.yml`.
An ordinary promotion pull request to `main` runs only `check.yml`'s lightweight
`required` promotion gate. It verifies the head is the current `testing` tip,
`main` is already merged back into `testing`, and the full gate already
succeeded for that commit: the canonical `check.yml` push run's own check
suite, with `unit`, `browser` and both native container legs.
A final docs-only commit after the closest gated `testing` commit reuses that
gate when every path since it is docs/site-only. Every non-Dependabot promotion
also starts [`escalation.yml`](.github/workflows/escalation.yml) on its head and
must carry a successful `race` job for that exact commit. The proof waits under
a bounded budget and fails closed. The fallback selects the closest gated
testing commit from canonical `testing` push runs, with no commit-count bound.
For a code-bearing head, let the `testing` `check.yml` push gate finish before
opening or re-running a promotion PR. Pending exact check runs get a short
bounded wait and the proof fails closed while they remain pending. A
docs/site-only head has no canonical `check.yml` push run of its own, so wait
for `docs.yml` or only the `required` docs validation. A Dependabot security
pull request targets `main` directly and therefore still runs the full gate.
### Promote testing to main
1. Fetch and align the branches. `testing` must contain the feature work and
`main` must be an ancestor:
```sh
git fetch origin
git switch testing
git merge --ff-only origin/testing
```
If `main` is not an ancestor of `testing`, merge `origin/main` into
`testing` first and resolve conflicts before continuing.
2. For a code-bearing promotion, bump `VERSION` on `testing` to a version
newer than every existing `v*` tag. Normal SemVer precedence applies; a
prerelease is lower than the same stable version. Commit the bump
separately:
```sh
printf '\n' > VERSION
git add VERSION
git commit -m "bump version to "
git push origin testing
```
A docs/site-only promotion may skip this step. `required` detects that every
changed path is covered by the docs/site filter (for example Markdown,
`docs/`, `site-src/`, the docs and Pages workflows, and `LICENSE`), and the
merged main push stays docs-only.
3. Open a pull request from `testing` to `main`. For a code-bearing head, wait
for the `testing` `check.yml` push gate to finish first. A docs/site-only
head has no canonical `check.yml` push run of its own, so wait for `docs.yml`
or only the `required` docs validation. Then wait for `required`. The proof
polls pending exact runs a bounded number of times and fails closed while
they remain pending, so opening or re-running the PR early only produces a
red check. `required` does not rerun the full `testing` gate. For
every non-Dependabot
promotion it verifies that the head is the current `testing` tip and that
`main` is already merged back into `testing`; code-bearing promotions must
also have a successful full-gate record for that exact commit, or for the
closest gated `testing` commit when every path since it is docs/site-only,
and `VERSION` newer than every release tag. A non-Dependabot promotion also
needs the escalation workflow's `race` job to succeed for this head; it
starts with the PR, and `required` waits for it under a bounded budget.
If it fails with "Promote the current testing tip", push the final commit to
`testing` and reopen or update the PR. If it fails with "Merge main back
into testing", merge `origin/main` into `testing` first. If it fails with
"changes non-docs paths after the gated commit", push the final code commit
and let its full gate finish. If it fails with "VERSION must be newer", bump
`VERSION` above every tag and push the commit. If GitHub does not create the
`testing` push run, dispatch the full gate on the current tip and wait for it
to succeed; the proof accepts that run as an equivalent testing gate:
```sh
gh workflow run check.yml --ref testing
```
4. Merge with a merge commit:
```sh
gh pr merge --merge
```
Squash and rebase are disabled on `main`; they would replace the reviewed
testing history instead of preserving it.
5. The merged-PR release run starts as soon as the promotion merge lands; wait
for its publish job (and the Pages run, when applicable) to finish, then
merge `main` back into `testing` so the next promotion is up to date:
```sh
git fetch origin main
git switch testing
git merge --ff-only origin/main
git push origin testing
```
If `testing` moved after the promotion merge, merge `origin/main` normally
and resolve conflicts before the next promotion.
For a code-bearing promotion, the merged-PR release run creates the annotated
release tag on the reviewed merge commit and publishes the image. It never
pushes `main`. A delivered `main` push is an idempotent fallback; a manually
pushed `v*` tag remains the explicit recovery path. Do not push a `v*` tag as
part of feature work; the manual tag path is only for an explicitly verified
release and must match `VERSION`.
Dependabot version updates target `testing`. Dependabot security updates target
the default branch and are the only exception to the testing-ancestry check;
they also skip the version check. They still run the full gate and the
`required` check. A security-only merge normally publishes no image and creates
no tag because `VERSION` is already tagged; if `VERSION` is still untagged, that
merge can publish the pending version. Merge `main` back into `testing` and
include the fix in the next versioned promotion.
Docs and site-only pushes to `testing` skip image work. A docs/site-only
promotion to `main` skips the `VERSION` and full-gate proofs, validates the
Markdown, generated site, workflow policy and docs contracts in `required`, and
the merged-PR release path records the change as docs-only, so it does not
rebuild images or cut a release. A Pages-relevant change deploys the site from
the `main` push, or from the Pages `workflow_dispatch` fallback when that push
workflow is not delivered.
`AGENTS.md` is intentionally local and ignored by the publication guard. Keep its
working instructions aligned with this shared guide; do not force-add private
instruction files to publish workflow policy.
## Development dependencies
Linux is supported. Version sources are:
- Go: [go.mod](go.mod).
- Node: [.node-version](.node-version).
- Python: [.python-version](.python-version).
- `jq`: the workflow contract suite executes the promotion gate's shell
predicates, so the core checks need the `jq` binary on `PATH`.
- Browser tooling: [requirements-dev.txt](requirements-dev.txt).
- Documentation site: [requirements-docs.txt](requirements-docs.txt).
The proxy embeds its vanilla JavaScript/CSS and needs no frontend build or Node
runtime in deployment. Node/jsdom, Python/Playwright, Python-Markdown and PyYAML
are development and documentation tools.
```sh
npm ci --ignore-scripts
python3 -m venv .venv
. .venv/bin/activate
python -m pip install --require-hashes -r requirements-dev.txt
python -m playwright install chromium
```
On fresh Linux installations, Chromium may also need system libraries; use
`python -m playwright install --with-deps chromium` with appropriate installation
permissions. Go race checks require `CGO_ENABLED=1` and a working C compiler;
the regular proxy build does not require a native SQLite driver.
## Checks
```sh
scripts/check.sh
```
The shared script owns the core verification commands. They include offline
documentation-link, publication-ignore and typography regressions (Git is
required), the generated-site contract, the documentation readability check and
the workflow/automation policy.
That policy suite runs the workflow files of the ref under test. A
`pull_request` run evaluates its own head revision rather than the base
branch's, so a workflow change is judged as this pull request ships it. Review
the workflow diff itself, since the run that validates it was configured by the
same commit; only the post-merge `main` state proves what landed.
`npm test` runs the jsdom suites and the request-log merge regression probe
separately. All tests and disposable harnesses
live in [tests/](tests/README.md), with suffix-free physical filenames. Go's
temporary overlay adds its required virtual `_test.go` names; Python unit
modules live in `tests/python/`. Use `scripts/check.sh go test` for targeted Go
checks and `scripts/check.sh go vet` or `scripts/check.sh go mod tidy` to include
test code and dependencies. Raw `go test ./...` does not discover the relocated
tests and can report success without running them. Re-run checks after your
last edit. Concurrency changes need race coverage. JavaScript unit checks do
not establish real canvas rendering, layout, focus or viewport behavior.
### Documentation site
The public site is generated from the existing Markdown, not maintained as a
second documentation tree. `site-src/` owns the layout, styles, and brand
assets; `scripts/build_site.py` renders the pages, rewrites repository links,
and emits the search index, sitemap, 404 page and the machine-readable docs
outputs described below. The generated `site/` directory is ignored and never
committed.
```sh
python3 -m pip install --require-hashes -r requirements-docs.txt
python3 -B -O -m tests.python.site
python3 scripts/build_site.py --output site
```
Both requirements files are hash-pinned and every install command passes
`--require-hashes`, so resolution is reproducible and a substituted file fails
the install. After changing a pin, regenerate the hash block for that version
from the PyPI JSON document (`https://pypi.org/pypi///json`,
`digests.sha256` of every non-yanked file, one `--hash=sha256:` line each), keep
the pins sorted as they are, and run the install above once to prove the file.
For a local preview, serve `site/` from a separate terminal with
`python3 -m http.server --directory site 8082`; do not point it at the proxy or
use the main `:8080` instance. The `docs.yml` workflow builds the site for
documentation changes, and `pages.yml` deploys from `main` through the official
GitHub Pages Actions flow. Repository-level Pages enablement and the
`github-pages` environment are one-time administrator settings.
The shell every page loads before content renders is owned by `SHELL_ASSETS` in
`scripts/build_site.py`: `tokens.css`, `themes.css`, `site.css`,
`theme_boot.js`, `theme_state.js`, `site_theme.js` and `site.js`. Each is
published at a content-hashed name such as `site.11c6502e41.css`.
`tokens.css` is derived from the dashboard palette. `themes.css`,
`theme_boot.js` and `theme_state.js` are published byte-for-byte from
`internal/web/static`, so the dashboard and the site share one palette owner,
one registry and one state module. `tests/python/site.py` checks that identity,
the load order and the registered palette ids.
GitHub Pages sets its own `Cache-Control`, so the URL is the only cache key this
repository controls. A page cached from the previous deployment names the
previous hash, and it must find those bytes. The build republishes the last
`SHELL_RETENTION` generations of each shell file, regenerated from the revisions
that produced them, which is why the Pages workflow checks out full history.
A checkout without history still builds the current page correctly but cannot
retain earlier generations. Do not add a hand-written reference to an unhashed
name; `tests/python/site.py` rejects one.
### Machine-readable documentation
`scripts/build_site.py` drives `scripts/site_llm.py`, which writes a Markdown
twin beside every rendered page, plus `llms.txt`, `llms-full.txt` and
`docs.json`. The MCP server embeds a checked-in copy at
`internal/docs/docs.json`; never edit that file by hand.
Regenerate it after any Markdown, title, navigation or description change:
```sh
python3 scripts/build_site.py --output site --write-manifest
```
`tests/python/site.py` fails when the embedded copy is stale, when a page does
not advertise its Markdown twin, or when a generated URL names a missing file.
The MCP tools `docs_list`, `docs_search` and `docs_read` read that index and
make no proxy request.
### Documentation prose
Write for a reader who is scanning. Keep one idea per paragraph and roughly 110
words or fewer. Use a list or table when a sentence carries three or more
parallel items, and split a paragraph when it changes subject.
`tests/python/docs_readability.py` enforces the word limit on hand-written
Markdown. `docs/settings.md` is generated from the schema and is excluded.
The site is published at `https://millivolt.llm4people.dev/`. That origin has two
owners, and neither is a file in this repository: the Pages custom domain in the
repository settings, which is what GitHub serves and certifies, and the DNS
record, which is a `CNAME` from `millivolt` to `llm-4-people.github.io` with the
proxy disabled. `scripts/build_site.py` warns when a build for `SITE_URL` names
a different host than `DEFAULT_BASE_URL`, because that is what a half-finished
migration looks like. GitHub ignores a `CNAME` file in an Actions deployment, so
this repository does not ship one.
### Brand plates
The README covers, specification strips, Open Graph card and GitHub social
preview are generated, not hand-edited. `scripts/brand/facts.py` owns the
headline, the four specification facts and the sample-history values.
`scripts/brand/manifest.json` owns each target's dimensions and hash.
Regenerate every plate after a copy or fact change:
```sh
python3 -m pip install --require-hashes -r requirements-docs.txt
python3 -m scripts.brand.build
```
The PNG renders need Playwright's Chromium; the SVG strips do not.
`python3 -m scripts.brand.build --check` verifies every committed asset against
the manifest and re-renders both SVG strips from source. CI runs that check.
The tool count in the specification strip comes from
`scripts/mcp_registry.py`, the same parser the MCP docs test uses, so adding a
tool cannot leave the art behind.
Upload `.github/social-preview.png` under the repository's Settings, Social
preview after it changes. That upload is a GitHub setting, not a repository
file.
### Push gate
Never push `main` directly: open a promotion pull request and merge it with a
merge commit. Do not push `testing` until the exact commit you will push has
passed the same CI gate jobs. A request to push is not a waiver.
Package-scoped `go test` and `npm test` during development are not sufficient.
CI's source-and-unit step is the default core suite:
```sh
scripts/check.sh
```
A run may not start while a code-scanning finding is unresolved.
`scripts/security_gate.py` reads `code-scanning/alerts?state=open` from the
repository API in the `unit` job and again in `publish` before the version tag
is created. Fix the finding, or record a dismissal with a reason that explains
why the code is correct; a dismissed alert is not open, and the security page
keeps that reason beside the location. An API the gate cannot read fails
closed rather than passing.
CI sets `GOTOOLCHAIN=local`, so the host Go must match the `go` version in
`go.mod`. On a host with an older base Go, install Go 1.27.2 or select the
matching toolchain before running the gate; `GOTOOLCHAIN=local` alone cannot
upgrade an older installation.
Full-gate CI events also run the isolated browser fixtures (desktop and mobile)
in the `browser` job. Use `scripts/dev.sh` plus `scripts/check.sh browser` as
below, with `MILLIVOLT_OPERATOR_TOKEN` set. Re-run both after the last commit. A
red GitHub check after a push means the gate was skipped, stale, or unmonitored:
fix, re-run, and push only a passing tree, then watch the new run to completion.
That same `browser` job also runs the live MCP integration test, which
`scripts/check.sh browser` alone does not run. It is mandatory on the same
commit:
```sh
MILLIVOLT_MCP_INTEGRATION=1 scripts/check.sh go test -count=1 ./internal/mcp/...
```
The test starts a private dev instance through `scripts/dev.sh` on its own
loopback port and scratch database, drives every registered tool, and stops the
instance again, so it does not touch the instance the browser check uses.
Before every push, check open pull requests and report any that are awaiting
the user's review or approval. Do not approve or merge them without explicit
authorization. Use:
```sh
gh pr list --state open --json number,title,author,isDraft,reviewDecision,mergeable,statusCheckRollup,url
```
After pushing, monitor the GitHub Actions run for the pushed commit until every
required job completes successfully. The local gate cannot see the browser,
MCP integration, container, code-scanning, or dependency-review jobs. Find the run with
`gh run list --branch --limit 1`, then wait for it with
`gh run watch --exit-status`. A queued or in-progress run is not green.
If any job fails, treat the push as incomplete: fix the root cause, rerun this
gate, and push the fix. Do not leave `testing` or `main` red or start unrelated
work on top of a failing run.
For changes to configuration, update the Config/default/validation/schema owners
and regenerate the example as described in [configuration](docs/configuration.md).
Do not edit `proxy.yaml`: it may be a running operator's configuration.
## Isolated development
Never use an existing main instance on port 8080 for tests. Use the lifecycle
script, with cleanup even if verification fails:
```sh
(
set -e
trap 'scripts/dev.sh stop' EXIT
export MILLIVOLT_OPERATOR_TOKEN='dev-instance-credential'
scripts/dev.sh
scripts/check.sh browser --target http://127.0.0.1:8081
)
```
The dev instance shares your shell environment, so a local
`MILLIVOLT_OPERATOR_TOKEN` arms its operator gate; browser and stress harnesses
need the same value for gated fixture cleanup. An instance started without it
serves reads only and denies every mutation.
`scripts/dev.sh status` reports the private instance. The script builds first,
then replaces only its tracked dev process; a failed build leaves it running.
The tracked identity is the pid file. `status`/`stop` fall back to the listener
on the instance's port when the file names a dead pid. A rebuild child rewrites
the file only once it is fully booted, so a child that dies before readiness
would otherwise strand the instance. The fallback still requires the
process to carry this instance's exact `-pid-file` flag.
Starting/restarting development resets its disposable database and private config,
seeded from the public example by default. `DEV_CONFIG=proxy.yaml` explicitly
opts into copying an operator config; it is never selected implicitly.
UI-triggered rebuilds keep that private config and update the tracked PID.
Overrides are deliberately restricted: `DEV_PORT` is a non-production port,
`DEV_HOST` is loopback, and `DEV_DB` is a permitted scratch filename or `none`.
`DEV_COPY_DB=1` seeds a consistent online backup of local history: use it only
when that history is authorized for the task, and treat the scratch copy as
sensitive. `DEV_COPY_DB=redacted` instead derives the separately validated
documentation copy described below; `0` keeps fresh fixture state. Do not use
broad process kills or start an ad-hoc second main instance.
The browser fixtures share private-target/config/database and redirect guards.
They disable SSE and test bootstrap/poll rendering, not uninterrupted live-feed
delivery. They remove only their synthetic client records. Their assertions must
remain enabled under Python `-O`; the guard suite verifies this.
### Live Chrome DevTools verification
Frontend changes are additionally verified in a visible Chrome window through
Chrome DevTools - headful, never headless. Sign in with the dev operator token,
drive the real fixture traffic and the actual controls, and check console
output, element geometry, the accessibility tree and screen flow in the
rendered page. Narrow fixture data passes the harnesses but can hide
distribution problems (wide values, wrapping, density); the live session is
where those surface. Keep the dev instance and the window open while you look;
stop the instance with `scripts/dev.sh stop` when done.
Fixture traffic for these sessions has its own lifecycle owner, never a paid
provider and never the main instance on `:8080`:
```sh
scripts/dev-traffic.sh # bursts of neutral loopback traffic at the dev instance
scripts/dev-traffic.sh status
scripts/dev-traffic.sh stop
```
Bursts land about a minute apart so one-minute chart buckets gain a bucket
per burst; each run includes recovered 429s (rate-limited, not errors) and one
final 400 so both health surfaces carry data.
## Container development
Normal [compose.yaml](compose.yaml) only runs a published image. For a local
source build, use [compose.dev.yaml](compose.dev.yaml) from the checkout root.
It is a complete Compose file, not an overlay: project and service are both
`millivolt-dev`, and the container takes the project-scoped name Compose
derives from them.
```sh
docker compose -f compose.dev.yaml up -d --build
docker compose -f compose.dev.yaml logs --tail 50
docker compose -f compose.dev.yaml down
```
The development project keeps its containers and named volumes separate from a
normal deployment, including when `COMPOSE_PROJECT_NAME` is set outside the
checkout. Its `MILLIVOLT_DEV_PORT` and `MILLIVOLT_DEV_IMAGE` inputs are defined
only there, and it shares the deployment Compose's `MILLIVOLT_MEM_LIMIT` and
`MILLIVOLT_PIDS_LIMIT` bounds. Choose an unused development port if `scripts/dev.sh` is already
running.
Rerun the same `up --build` command after source changes. The resulting runtime
image has no compiler or source tree, so dashboard rebuild is unavailable.
`down` preserves its named volumes unless you explicitly request deletion.
This source-image workflow does not replace `scripts/dev.sh`: the host browser
and stress fixtures require that script's private process/database identity.
## Documentation screenshots
The published gallery uses recorded metrics from an explicitly authorized,
redacted local-history snapshot, not fabricated timing or cost values. Never
point capture tools at a main instance or serve a raw history copy for screenshots.
The source is the checkout's local `proxy.db`, opened read-only through the existing
online-backup helper. This workflow needs explicit authorization for that history.
From a checkout with browser dependencies installed and that authorized source:
```sh
(
set -e
export DEV_PORT=18081 DEV_HOST=127.0.0.1
export DEV_DB=/tmp/millivolt/millivolt-dev-docs.db
export DEV_CONFIG=proxy.example.yaml DEV_COPY_DB=redacted
trap 'scripts/dev.sh stop' EXIT
scripts/dev.sh up
python3 -B -O -m tests.explorer_check \
--target "http://${DEV_HOST}:${DEV_PORT}" --docs-history docs/images
)
```
Choose an unused permitted dev port. Before startup, `dev.sh` derives a new
metrics-only database: identifiers are replaced, content/headers/debug data are
removed, and recorded timing, token and cost values are retained. The raw online
snapshot is temporary and never served. Treat even the derived database as
sensitive; retained usage and timestamps can reveal activity patterns.
The view list is a registry, not a capture script. `scripts/gallery/data.py`
declares every plate: its published path, title, description, scene, named
viewport and setup. `tests/docs_capture.py` owns the scene strategies (page,
element, scoped explorer, absorbed retries, drawer, console, settings, chart,
window picker, theme tray). `scripts/gallery/compose.py` turns each scene into
one plate: the same document is photographed on Instrument Black and Instrument
Clear, then split along the diagonal from bottom-left to top-right. `applyState`
is the one theme mutation path, so an open drawer, a selected filter or an open
tray survives the second half.
The run writes `docs/images/capture.json`, which records every plate's bytes,
dimensions and SHA-256, the verified record count, and a digest of
`internal/web/static`. It also writes the generated `docs/gallery.md`. Commit
both with the images. `tests/python/gallery.py` fails when a registered plate
has no capture, a capture is not registered, a byte or dimension drifts, the
digest no longer matches the dashboard source, or the gallery page is stale.
A UI change therefore requires a recapture in the same branch, not a later one.
Capture last. The gallery is a picture of the shipped dashboard, so a markup,
CSS or layout change dates every image that shows the affected surface.
Recapture in the same branch once the last layout edit is in, never before it,
and recapture again if a later edit moves what an image shows. A recapture diff
is the review set: inspect every changed plate, and expect geometry changes
(for example the request table's 8px cell padding and retry sub-rows) to come
with them.
The capture mode revalidates every row and its provenance, plus exact public
configuration bytes, before and after browser capture. It permits only read-only
same-origin requests: no inference, POST, purge or Settings save.
Images are staged before publication, and the exit trap stops the private
process. Review every plate before committing. SSE is deliberately disabled, so
the images' offline indicator describes a static capture session, not current
service health. Captures keep the same category folders under `docs/images/`:
`overview`, `charts`, `explorer`, `requests`, `console`, `settings` and `theme`.
The site fails on any image under `docs/images/` without a Markdown consumer, so
a recapture that drops a view cannot leave an orphan behind.
The independent `--docs-images` option remains a strict synthetic regression.
Run it only with `DEV_COPY_DB=0` and direct those two images to a scratch directory,
not `docs/images`. It creates and cleans up its own six-request fixture and
rejects copied history. Neither capture mode is a performance or compatibility
benchmark; do not weaken one mode's admission checks to accommodate the other.
## Performance evidence
The published tables, method and limitations live in
[performance and resource usage](docs/performance.md). That page is the single
source for measured numbers; do not copy them into other documentation and do
not edit the page by hand. It is rendered from `perf/results.json`, and
`tests/python/site.py` fails when the two disagree.
`scripts/perf.sh` owns a repeatable local measurement run without touching the
main instance. It starts fresh private instances on a disposable port and stops
them through `scripts/dev.sh`:
```sh
scripts/perf.sh docs # every published measurement, bundle and page
scripts/perf.sh bench # isolated Go benchmarks
scripts/perf.sh stress # private fast and streaming ramps
scripts/perf.sh streaming # sustained end-to-end SSE throughput ramp
scripts/perf.sh sustained # fixed-duration fast-response capacity levels
scripts/perf.sh image # disposable image content and compressed size
scripts/perf.sh all # image, bench, then stress
```
`docs` refuses a dirty tree, runs the complete documented set, writes
`perf/results.json`, and regenerates `docs/performance.md`. Commit the bundle
and the page together. Never hand-edit either one; a missing stage, failed
response or unbalanced durable result aborts the run instead of publishing a
partial page.
The bundle is also the source of the facts the page describes: the commit, date
and platform, each stage's response shape (frame count, usage and unit cost),
the configured durations, and the harness declarations for captures, previews
and the observer. `cmd/stress` records the fixture shape in its workload line
and the collector validates it per stage, so a table and the sentence beside it
cannot describe different runs. The landing headline reads the same bundle;
`tests/python/performance.py` renders two bundles that differ only in those
fields and fails if any published sentence keeps its old text.
For request, SSE-event, response-byte and output-token rates, use the
`streaming` mode. It derives those rates from responses the client accepted
byte-for-byte, and it is still a synthetic-fixture measurement, not a universal
capacity claim.
For an individual stress run with the private dev instance already running:
```sh
go run ./cmd/stress -target http://127.0.0.1:8081 -concurrency 1,8,32,128 -duration 5s
```
The harness does not manage or reconfigure the proxy. It uses a local synthetic
upstream and checks exact bytes, durable rows/tokens/cost and the same-process
storage-drop delta.
A drop is a failed stage even if every HTTP request succeeded, unless the run
asks for an overload observation with `-allow-drops`. That mode still requires
`durable + dropped = successful requests` and still aborts on a response
failure. Read `go run ./cmd/stress -h` for workload/safety controls.
The ramp retains its cumulative fixture history across measured stages, then
removes only its unique client records after measurements finish. Cleanup has
its own bounded wait even after cancellation and revalidates private process,
config and database ownership before every deletion. A cleanup failure is
reported, never silently treated as success.
Report workload, duration, concurrency, configuration, contention and whether
storage/observers were active. Separate request throughput from durable
throughput. Closed-loop bursts, microbenchmarks and local upstreams do not
establish sustained production capacity. The current storage contract and
capacity caveats live in [operations](docs/operations.md#storage-and-accounting).
Use the existing `BenchmarkStorageWriter` and `BenchmarkProxyDurableContention`
for isolated writer/contention comparisons; neither replaces a real HTTP run.
For the analytical projection's memory, use the self-seeded
`BenchmarkProjectionFootprint` (retained and transient bytes per stored record)
and `BenchmarkProjectionProcessRSS` (process RSS after readiness) from the test
overlay; they need no running instance:
```sh
scripts/check.sh go test -run='^$' -bench='BenchmarkProjection' -count=1 ./internal/web
```
## Versioning and publication
[VERSION](VERSION) is the only semantic version source; do not duplicate it in
package manifests, build arguments or image metadata. Normal and prerelease
SemVer are supported without build metadata. A release tag must be exactly
`v` followed by that file's version. The release CLI validates the same contract:
```sh
millivolt_version="$(tr -d '\r\n' < VERSION)"
go run ./cmd/release -revision "$(git rev-parse HEAD)" -tag "v$millivolt_version"
```
Publication is automatic. A code-bearing versioned promotion releases when the
promotion PR merges, after the full gate has passed for the promoted commit, or
after the closest gated `testing` commit plus a docs-only delta.
A non-Dependabot promotion also requires a successful
[escalation](.github/workflows/escalation.yml) run for the reviewed head. The
promotion PR starts it automatically, and `required` waits for it under a
bounded, fail-closed proof. The merged-PR
event carries the exact merge commit; a delivered `main` push is an idempotent
fallback, so a dropped push event cannot leave the release untagged. A
Dependabot security PR targets `main` and runs the full gate itself rather than
reusing a `testing` result. The release `publish` job waits for `required`.
The `publish-testing` job waits for `unit`, `browser` and `container` instead,
because a testing push has no promotion proof to wait for:
1. A code-bearing promotion PR owns the version bump. `required` compares
VERSION against every existing `v*` tag with SemVer precedence and rejects a
non-Dependabot main PR unless VERSION is newer. Docs/site-only promotions
skip that comparison, validate docs, workflow policy and the site in
`required`, and stay release-free. If VERSION is already tagged, no
release is cut: this is the Dependabot security-merge case, and the next
versioned promotion releases the change.
2. The workflow creates the annotated `v` tag on the reviewed
promotion merge commit and builds and publishes the multi-arch image with
`main`, `sha-*`, `version` and (for non-prereleases) `latest` tags, including
an SBOM and max-mode provenance. The `sha-*` tag and OCI revision label name
the promotion merge commit that was built.
3. Only after the image exists does it push the tag and create the GitHub
release. It never pushes `main`. Release notes filter merge commits, dedupe
subjects, use the lowercase image path, include the digest, and link the
compare range. A failed image build leaves no public release pointing at a
missing image. `main` and `latest` exclude prereleases; the `testing` alias
never moves them.
Docs and site-only pushes to `testing` skip image work. They validate Markdown,
the generated site, workflow policy and docs contracts.
A docs/site-only main promotion also skips the VERSION and full-gate proofs but
runs the same Markdown, generated-site, workflow-policy and repository checks in
`required`. A Pages-relevant merged main push stays docs-only, so Pages deploys
without rebuilding images or cutting a release.
The merged-PR release path classifies the change and skips publication. A
code-bearing promotion must carry a VERSION newer than every existing tag and
cuts a release. A manually pushed `v*` tag
still ships exactly as tagged through the same
pipeline, with the image published before the release is created. A manual tag
must match VERSION. Code-bearing pushes to `testing` publish the moving
`ghcr.io/llm-4-people/millivolt:testing` image after the same checks and
container builds, without cutting a release or any version tag. Its OCI version
label reports the embedded VERSION, which can lag the latest release.
Manual validation of a candidate remains available:
```sh
scripts/check.sh release -revision "$(git rev-parse HEAD)" -tag "v$millivolt_version"
```
Run the shared checks before relying on a release. Confirm the hosted container
build and smoke tests succeed, then check the package visibility and anonymous
pull. Do not report a release/image as available merely because validation or a
local build passed. Image usage and persistence limits live in
[operations](docs/containers.md#versions-and-images). The gate clears inherited
Git routing/configuration variables before checking
source identity, so another worktree or external ignore policy cannot satisfy
the release checks accidentally.
Only push the intended release tag after that check succeeds. Main changes go
through the protected pull request path. The workflow publishes images
automatically and creates the release with notes generated from the commit log.
Native container checks run before the multi-platform publish job. After the
promotion run, verify the published tag and image:
```sh
gh release view "v$millivolt_version"
docker buildx imagetools inspect "ghcr.io/llm-4-people/millivolt:$millivolt_version"
```
### If a release fails
- Before the tag is pushed, no public release exists. Fix the failure and rerun
the main workflow or promote a new version.
- After the tag is pushed, keep the tag in place. If the GitHub release is
missing, create it from that tag or publish a new patch version. An automatic
tag push does not start a second workflow run.
- Do not move or delete a published version tag.
For local container verification, build with Docker, resolve the image's
immutable ID, then use the same fixture as CI:
```sh
docker build -t millivolt:check .
millivolt_image_id="$(docker image inspect --format '{{.Id}}' millivolt:check)"
scripts/check.sh container --image "$millivolt_image_id"
```
The shared container mode validates Compose defaults and owns only its
disposable, network-isolated Docker fixture, selected by immutable image ID.
It checks startup, HTTP, Settings and durable state across stop/start, then backs
up the stopped volumes and restores into another fresh container. Recovery must
retain settings, history and private file ownership/modes. It never publishes
a host port or reuses an operator volume. It complements, rather
than replaces, the host lifecycle in `scripts/dev.sh`.
## Change checklist
Use sentence case or lowercase in prose, headings and labels. Avoid em dashes;
preserve exact protocol names, identifiers, user data and license wording.
- Add a targeted regression for a reproduced defect; preserve existing coverage.
- Keep defaults, units, schemas and state transitions at their canonical owner.
- Measure allocations/latency when changing the hot path or analytical folds.
- Check saved views, mobile layout and accessibility for visible changes.
- Update the relevant existing documentation, not duplicate specifications.
- Exclude local configs, databases, tokens, captures and generated test artifacts.
- Preserve dependency licenses. Report vulnerabilities using [SECURITY.md](SECURITY.md),
not a public issue containing sensitive reproduction data.
Dependabot checks Go, npm, Python, Docker and GitHub Actions dependencies weekly
with grouped minor and patch updates. Review dependency changes and re-run
checks; runtime version files still require deliberate updates and verification.
The Docker `golang` base is deliberately left manual because its tag must move
with go.mod; distroless updates remain automated. Security updates target
`main` as described in the branch workflow. Dependency review runs on pull
requests, and a scheduled `govulncheck` workflow covers the Go module graph.
The scanner is pinned as a tool dependency in `tools.go`, so it resolves from
this module and `go.sum` covers it instead of a version being fetched fresh on
each run. These scans complement tests; a clean scan is not a security guarantee.
---
# Support
millivolt is pre-1.0 and maintained on a best-effort basis. There is no
response-time SLA.
## Read first
- [Documentation index](docs/README.md)
- [Quickstart](README.md#quickstart)
- [Operations](docs/operations.md)
- [Reverse proxy](docs/reverse-proxy.md)
- [Security](SECURITY.md)
## Where to ask
- **Usage, configuration, or deployment question:** open a
[usage question](https://github.com/LLM-4-People/millivolt/issues/new?template=question.yml).
- **Reproducible defect:** open a
[bug report](https://github.com/LLM-4-People/millivolt/issues/new?template=bug_report.yml).
- **Concrete proposal:** open a
[feature request](https://github.com/LLM-4-People/millivolt/issues/new?template=feature_request.yml).
- **Security vulnerability:** use the
[private advisory channel](https://github.com/LLM-4-People/millivolt/security/advisories/new).
Do not open a public issue.
Search existing issues before opening a new one. Include the millivolt version
or revision and a sanitized configuration shape. Remove credentials, private
prompts, databases, and host-specific paths.
---
MIT License
Copyright (c) 2026 millivolt contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
---
# Third-party notices
millivolt's [MIT license](LICENSE) covers the project's own work, not a
relicensing of dependencies.
## Embedded dashboard library
[uPlot 1.6.32](https://github.com/leeoniya/uPlot/tree/1.6.32) is vendored as
`internal/web/static/vendor/uplot.js` and `uplot.css`. Copyright (c) 2022 Leon
Sorokin. Its complete [MIT notice](internal/web/static/vendor/uplot.LICENSE)
is kept beside the assets and included in the embedded filesystem.
Preserve that notice when redistributing the library or built dashboard.
The notice was checked against the
[upstream versioned license](https://github.com/leeoniya/uPlot/blob/1.6.32/LICENSE).
Update the version, assets and notice together when changing this dependency.
## Embedded typefaces
The dashboard and the documentation site ship four WOFF2 faces in
`internal/web/static/fonts`: Instrument Sans (roman and italic), Instrument
Serif, and JetBrains Mono. They are Copyright 2022 The Instrument Sans Project
Authors, Copyright 2022 The Instrument Serif Project Authors, and Copyright 2020
The JetBrains Mono Project Authors. All three families are licensed under the
[SIL Open Font License 1.1](internal/web/static/fonts/LICENSE); the complete
notice and copyright lines are kept beside the files and included in the
embedded filesystem. The dashboard, the docs site and the generated brand
plates all consume these exact faces, so no third-party font request is made at
runtime. Preserve the notice when redistributing the fonts or a built
dashboard.
## Module and development dependencies
[go.mod](go.mod) and [go.sum](go.sum) identify Go dependencies; their upstream
module distributions carry their own licenses. [package-lock.json](package-lock.json)
pins development-only jsdom dependencies. [requirements-dev.txt](requirements-dev.txt)
pins Python browser tooling; Playwright selects its corresponding Chromium build.
[requirements-docs.txt](requirements-docs.txt) pins the Python-Markdown renderer
(BSD-3-Clause) used to build the static documentation site; PyYAML (MIT) is a
development dependency used by the workflow policy checks. Node, Python and
these build/test dependencies are not required to run the proxy.
## Container distribution
The [Dockerfile](Dockerfile) runs [the notice collector](scripts/licenses/main.go)
against the exact target binary's Go dependency graph. It preserves module,
package-ancestor and Go toolchain license, copyright, notice and patent files,
plus the embedded uPlot notice, under `/usr/share/licenses/millivolt`.
The deterministic `manifest.json` records versions, relative paths and SHA-256
content hashes. Missing notices and module replacements fail the build for
review; this collector does not classify licenses or grant additional rights.
The pinned distroless runtime retains its own files and dependency notices.
Its [upstream project](https://github.com/GoogleContainerTools/distroless)
documents the base-image contents. Node, Python and development dependencies
are not copied into the runtime image.
When distributing a binary by another route, generate and include the same
bundle with `go run ./scripts/licenses -target linux/amd64 -out NEW_DIRECTORY`
(or `linux/arm64`) and review the actual distribution. This source notice alone
is not a complete binary-release license bundle.
---