millivolt Install millivolt
OperateConfiguration
On this page

Configuration

How the configuration surface is owned, found and edited: the generated example, the settings schema, request overrides and sub-conversation tracking.

Configuration and CLI

proxy.example.yaml is generated from the canonical config.Example() and schema documentation. Example() starts with neutral config.Default() values and adds the enabled Grok, OpenCode Zen and Cursor compatibility profiles and the opencode sub-conversation tracking exemplar (the promptCacheKey tracked param, strip on). It is not mutable repository state. Source deployments copy it to ignored local proxy.yaml; images bundle their own copy for fresh config volumes. Do not start Settings against the committed example.

The proxy's flags are listed once, with their registration defaults and usage text, in the settings reference. The effective -listen value is validated after every override, and a handoff child rewrites -pid-file with its own pid on boot so a supervisor keeps tracking the live process. The three output-only flags are mutually exclusive and exit without loading local config, opening a database or starting runtime services.

Precedence is built-in defaults, then the YAML file, then CLI overrides. A missing file loads defaults; verify the path rather than assuming a typo created a new empty overlay. On startup, unknown keys, unknown nested fields and extra YAML documents are dropped and the file is rewritten without them. An invalid recognized value fails startup without rewriting the file, and an unparseable file is moved to PATH.invalid and replaced with the documented defaults, except an ambiguous duplicate top-level key, which fails closed and stays in place. Because replacing it with defaults would silently widen the bind and drop the origin allowlist, an unparseable file first has listen and allowed_base_urls salvaged from it block by block and applied over the defaults; both (unparseable yaml) and <key> (salvaged) are reported in the boot log and last_reload. A named key with no usable value, or a salvaged value the config rejects, fails startup with the original file still in place. Reload rejects an invalid document without rewriting it. Settings and restore merge treat a missing file as defaults, while an invalid or unreadable document fails closed and is reported instead of substituting a default. Healthcheck is stricter: a named file that cannot be read fails the probe, and only an empty -config reaches the built-in defaults. Backup capture is stricter: a missing config file is an error because there is nothing to archive. Settings POST still rejects invalid inputs. CLI-overridden fields cannot be saved from Settings. Healthcheck loads whatever file it is given read-only and never rewrites it.

To regenerate the public example and the settings reference after changing their canonical owners:

go run ./cmd/proxy -print-example-config > proxy.example.yaml
go run ./cmd/docgen -write

The shared checks verify generated-file consistency. Built-in defaults remain neutral; compatibility-profile values belong only in config.Example(), never consumer fallbacks or private credentials. Optional mappings are mechanisms, not provider registrations, for example:

providers:
  neutral.example:
    cost_keys: [billing.charge_usd]
    usage_keys:
      input_tokens: prompt_toks

Use the derived provider label as the map key. The example file and Settings schema document other maps, model rules and header templates; do not maintain a second complete list of defaults here.

Finding a setting

The generated settings reference is the complete field list: every setting with its type, default, accepted values and hot-reload or restart behavior, grouped by the same categories as the Settings sheet. proxy.example.yaml remains the commented deployment example. GET /admin/config exposes the same field/category metadata alongside saved values, neutral defaults, CLI overrides, the process-effective snapshot and the current revision. Its last_reload section reports the most recent configuration application: ok, error (empty when ok), dropped_keys, restart_required and the unix-millisecond at timestamp. Settings searches labels, keys and help across categories. Durations and byte sizes use a magnitude plus unit (2s, 32MiB) in YAML and Settings; do not write a bare number and assume seconds. A bare integer is still accepted as a count of bytes. Integer counts (retries, percents, tokens) stay unitless numbers with the unit in the schema label when it is not obvious.

Settings categories Surface
Server, Request, Upstream, Queue & retry, Error storm protection The proxy's traffic surfaces, in that order.
Quota pause, Conversations, Format translation Provider reaction, grouping, and cross-protocol translation.
Storage, Backup, MCP, Dashboard Durable state, its archives, the LLM tool endpoint and the live view.
Models, Providers, Request overrides Grouping rules, upstream mappings and scoped rewrites.

The category labels, their help and their membership are owned by schema.Categories() and Schema() over GET /admin/config; the generated example carries the same sections with each field's type, range and default. The rows above are names only, not a second copy of either.

The destructive controls sit in their category and each confirms first: a rebuild and restart is confirmed and single-flight, starting a capture through MCP needs its confirm phrase and stopping every session confirms separately, a database delete confirms in the console (and over MCP needs the exact phrase plus the preview token for that filter), and a restore inspects before its confirmed apply.

Guardrails that are not settings

Every numeric guard is classified when it is added. A value becomes a setting when an operator can reasonably need a different number for legitimate traffic. It stays an internal constant when changing it independently would break an invariant, encode a protocol maximum, or only tune an internal heuristic.

Family Examples Why it stays internal
Protocol maxima Chat stop list of 4 The wire defines the practical maximum; a longer list is clamped to it with an approximation warning.
Derived frame bounds NativeStreamFrameMax(eventMax), ResponsesStreamFrameMaxBytes, the proxy's translated frame buffer Computed from the accepted source-event and retained-byte bounds plus the JSON escape expansion. A separate value could under-size a scanner, turning a valid frame into a translation error, or over-size it and defeat the retention bound.
Recursion and canonicalization budgets Anthropic nested block depth 16, tool-argument canonical depth 64, separator budget, canonical byte cap They select a safe fallback (the raw-byte fingerprint) or cap decoder recursion. They are algorithm facts, not wire acceptance.
Diagnostic echo bounds 128-byte warning echo, 512-byte identity and provider-text retention, header echo caps A request-sized diagnostic or identifier must never multiply across a stream or a stored row. Raising them cannot admit a request that is otherwise rejected; it only grows logs and frames.
Record retention Retained tool names (64), tool-call ids (1024), path/method/model/error caps, capture header caps They bound the ring, pending snapshots and durable rows. Counts still count past the bound; only identities stop being retained.
Metrics-only truncation Non-streaming choice cap, SSE analyzer event buffer The non-streaming choice cap never changes relayed bytes. The SSE analyzer cap can affect classification for events split across that bound, as the protocol limits note; raising it cannot admit a request that is otherwise rejected.
Cursor run and store budgets Parked run count, per-run pending call/byte caps, blob store caps, aggregate store budget They bound per-run and process memory for a bridge whose counts come from live tool batches, not operator intent. The per-run and aggregate caps are coupled.
Internal heuristic thresholds Quality retry spool, error-body capture, discovery output allowance, storage sweep batch sizes, log scan and chart/explorer budgets They decide whether an internal heuristic runs or how much work one pass may do. Changing them changes diagnostics or performance, not what a client may send or receive.

The operator-facing bounds are settings: request_tool_calls_max, stream_tool_calls_max, translated_response_max_bytes, models_discovery_max_entries, non_stream_capture_max, stream_terminal_hold_max, stream_event_max_bytes, token_response_max and upstream_response_header_max. non_stream_capture_max trades accounting accuracy for memory and never changes relayed bytes. A new guard must be classified in this table when it is added, and a guard that becomes operator-facing moves to the settings schema instead of keeping a parallel constant.

Saving settings safely

Settings collects a draft until Apply. Revert reloads the saved document, and closing a dirty sheet keeps it open until Apply or Revert; an open dirty draft also asks the browser to confirm a tab close or reload. The server validates the whole result and compares its revision before writing. A stale tab receives 409; reload/review its values instead of resubmitting an old full form. The sheet renders closed vocabularies as selects from the schema's choices and checks duration/byte text against the schema's numeric band before the POST; the server remains the authority. With no writable config the field controls are disabled, not only the Apply action.

API saves overlay supplied top-level keys on the saved config. Omitted keys remain unchanged, but a supplied map/list replaces that entire field. For example, providers, provider_aliases and model_rules are not recursively patched. Read the current document and preserve wanted entries when editing a structured field. Explicitly empty collections clear those fields.

The response separates saved values from the effective snapshot, reports restart-required keys and carries the same last_reload section as GET; the Settings sheet renders it as its last-apply row with apply time, failure, dropped-key count and restart notice. A successful file write can still be followed by a reload failure; saved:true means the file changed even when the response reports an error. Do not assume rollback. The writable indicator means a config path is configured, not that filesystem permissions or every future write are guaranteed. See persistence and reload behavior before changing startup-bound fields.

Provider aliases and model grouping

provider_aliases merges provider labels in stored history at boot/reload and rekeys new requests. All aliases in one map commit as a single transaction, so a trigger or constraint failure on a later alias leaves every earlier label unchanged. It rejects self-maps and alias chains. This is a durable rename, not a temporary view: removing the alias does not reconstruct original labels. Existing in-memory ring records can retain their previous label until they age out or the process restarts. Back up before an intentional history rename.

model_rules instead controls display/grouping. The list is capped at 64 rules. Its ordered exact, pattern and lowercase steps keep stored model spellings unchanged; an empty rule list groups by those raw spellings. The shipped pipeline folds case, strips one or more leading vendor/ namespaces and trailing :tag deployments, strips a trailing run of serving decorations, and unifies . with - between digits. A decoration is what a provider adds around the same weights: a quantization or serving format (-fp4, -nvfp4, -bf16, -int8, -q4_k_m, -gptq, -awq, -gguf), a service tier (-fast, -speed, -value, -free), a release channel (-preview, -latest, -beta, -alpha, -stable, -experimental), a reasoning or effort word (-thinking, -reasoning, -xhigh, -high, -medium, -low, -minimal), or a build stamp (-1017, -20250929). Model tiers (-flash, -pro, -mini, -ultra, -lite, -nano, -turbo, -max), fine-tunes (-abliterated, -rl), specifications (-coder, -base, -chat, -it) and parameter counts stay distinct. Add an exact rule above the shipped pipeline to split a group, or one it cannot recognize to merge the spelling. The Settings editor supports reordering, parking and previewing rules. Raw export/delete predicates do not silently expand to a display group. Debug's native-model normalization is separate from these rules.

Request overrides

request_overrides is an ordered rule list that rewrites the upstream request before relay; the default empty list leaves every request untouched. Each rule has a scope and at least one action: headers sets or replaces upstream headers, remove_headers deletes them, and body sets the output-token ceilings max_tokens / max_completion_tokens (1..1000000) on a Chat Completions or Anthropic client body before translation. A Responses client body is not rewritten by the body action because its cap is max_output_tokens; header and removal rules still apply. A Chat client whose target is the Responses wire has the rewritten cap translated to max_output_tokens. Both override spellings address an Anthropic body's single max_tokens field, with max_completion_tokens winning when both are set. The list is capped at 64 rules and validated like every other setting: a rule needs at least one scope field and one action, duplicate scope triples are rejected, and one header name cannot be both set and removed in the same rule.

Scope matching is exact leaf. A rule applies when every non-empty scope field equals the request's value exactly: client against the classified client label (X-Proxy-Client when sent, except a label that only repeats the user agent's product name, which records the fuller user-agent identity; otherwise inferred from SDK/user-agent metadata), provider against the canonical label after provider_aliases, and model against the recorded model id (for Cursor requests the base id the dashboard records, not the fused spelling). An empty scope field matches anything. There is no model-rule canonicalization, regex or wildcard broadening. All matching rules apply in list order, and for the same header or body field the later rule wins.

The client label is a claim, not an authenticated identity. The caller chooses it, and the proxy never verifies it, so a request presenting another client's label is treated as that client here and by every other client-scoped feature (sub-conversations, Debug sessions, pause holds, the Cursor parked-run scope). See client identity for what that means for a listener reachable by parties you do not trust. What it cannot do is move a credential: credential-class header names are refused at load and skipped at send, so the exposure is the policy attached to the label, not the secret behind it.

Header rules are the operator's last word on the upstream wire: the resolved merge applies after client-forwarded values, configured provider headers and the client's X-Proxy-Headers map, so a matching override wins over all of them. Only the proxy-owned transport-mode names accept-encoding and range are stripped from the final upstream request before send, whichever source set them, so the transport keeps its own gzip negotiation and never sends a partial range; the client header maps (X-Proxy-Headers and an X-Proxy-Auth-Header naming one) also reject them with a 400 before any send.

The request-override list refuses two further header classes at load. Protocol and framing names (host, content-type, content-length, transfer-encoding, te, connection, keep-alive, proxy-authenticate, proxy-connection, trailer, upgrade) and the whole x-proxy- control prefix belong to the proxy and the transport. Credential names belong to whoever issued the secret: authorization, cookie, proxy-authorization, and every name the shared credential vocabulary classifies. That vocabulary judges the header's role, not a list of vendor spellings: auth/authorization, api-key, secret, password, credential, bearer, jwt, signature, cookie, key and token are all credential-class wherever they appear, so x-auth-key, x-amz-credential, x-functions-key and x-subscription-key are refused along with x-api-key. Names that mention a key or a token but carry an identifier or a measurement stay admitted: x-key-id, idempotency-key, cache-key, x-prompt-tokens, x-token-count, and the x-ratelimit-* and anthropic-ratelimit-* families. That breadth is deliberate. An override value is operator-supplied and is stamped onto a request whose destination the caller chooses with X-Proxy-Base-URL, so a secret in any unclassified header name would be delivered to that caller's host; vendor spellings cannot be enumerated in advance, which is why the check keys on role rather than a list. The same vocabulary drives capture and debug redaction, so a stored capture never keeps a header the wire carried. At runtime the target's configured auth header is skipped as well, so a rule that predates this check, or a future grammar slip, still cannot move a credential. The client header maps reject content-length before any send so the transport keeps ownership of message framing. Header names are trimmed and canonicalized case-insensitively, and values must be non-empty single-line header values.

Provider headers refuse neither class above. They are applied after the extracted credential is placed, so a credential name (including the target's own auth header) or an x-proxy- name is legal there and takes effect over the forwarded and extracted value. Configured provider values are still stripped for accept-encoding and range at the final request, exactly like override values.

The body rewrite is fail closed. It applies to Chat Completions or Anthropic client bodies before translation, whatever the target (a Responses client body is not rewritten); when the body is not a JSON object, or the rewritten result would exceed max_request_bytes, nothing is written: the original bytes are used unchanged by the rewrite stage and are still translated when translation is active, with one process log line recording the reason. Only a passthrough relays them verbatim. A successful rewrite replaces the body for the request's whole life, so every relay attempt, absorbed retry and quality re-send reuses it, and the recorded request cap is re-stamped from the effective body. A rule sets only the body fields it names, and the re-read pair keeps the wire's documented precedence - max_completion_tokens wins over max_tokens - so a rule setting only max_tokens cannot raise a request whose client already sent max_completion_tokens: set both fields to the same value to override either spelling. Cursor targets carry no token ceilings on their wire, so the body section is skipped for them while header rules still apply at the native send.

Overrides apply to inference sends only, never model discovery or token refresh. The list hot-reloads: changed rules apply to new requests. The Settings editor mirrors the server's checks and rejections live for each rule, marks empty scope fields as matching any request, and blocks Apply on the first invalid rule.

To raise the output budget for one provider, either write:

request_overrides:
  - provider: nano-gpt.com
    body:
      max_completion_tokens: 65536

or open Settings, select Request overrides, pick raise the output budget for a provider from the add-a-rule menu, and fill the provider scope; the other starter templates seed an empty rule and an upstream header for a client.

Sub-conversation tracking

sub_conversations gives requests from one client a conversation identity of their own, taken from a value the client already sends in its request body (exemplar: opencode's promptCacheKey). The default empty list leaves tracking fully off, and the feature is deny-by-default per client: an entry applies only when its client equals the request's classified client label exactly (the same leaf the request-overrides client scope matches; no wildcard, no case folding), and a request from any other client is never inspected, leaving its bytes and grouping untouched.

Each entry lists 1..4 exact JSON field names, matched only at the request body's top level and checked in order; the first present field decides. A present value that is not a JSON string counts as absent and the next field is checked, while a present string that cannot be decoded or fails the identity rules drops the identity for the request - no later field is consulted. A value is kept only when it is non-empty after trimming, valid UTF-8, free of control bytes and at most 512 bytes, the same bound explicit session declarations satisfy. The UTF-8 judgment runs on the value's raw bytes before JSON decoding: the standard decoder coerces invalid UTF-8 inside strings to U+FFFD, so a raw invalid-UTF-8 sequence drops the identity while an escaped lone surrogate, whose bytes are ASCII, is kept and decodes to U+FFFD. The drop is conservative: the value's raw spelling is scanned inside a bounded window, so a value whose raw span exceeds the window (escape sequences spend more raw bytes than they decode to, and trimmable edges count before trimming) is dropped even when its trimmed form would pass. An absent or dropped identity is never a client error: on the passthrough wire the body is payload, and the request keeps automatic grouping. A translated target still requires the whole request body to be valid UTF-8, as protocol describes.

The kept value groups the request under a k: conversation identity. Precedence is the explicit X-Proxy-Session header first, then the tracked value, then automatic client+key grouping; a request carrying both header and tracked value keeps the header identity. X-Proxy-Parent-Session still owns parent linkage, and declaring a parent still requires the explicit session pair, so a tracked identity never carries a parent. In the dashboard a k: id is an ordinary conversation id: lineage, filters, saved views and the request drawer read it like any other.

strip: true removes every configured field of the entry that is present at the body's top level from the client body before relay or translation, including a Responses client body; the default false forwards them unchanged. Unlike the request overrides body fields, strip is not limited to Chat Completions or Anthropic inbound. The removal rides the same body-rewrite engine: one document decode when both features match one request, the original bytes used unchanged by the rewrite stage when the body is not a JSON object or the rewrite would exceed max_request_bytes, and cursor targets skipping the body section. Without strip, the client's spelling reaches the upstream wire on a same-protocol passthrough; a translated body carries the value in prompt_cache_key or drops it.

When a tracked value was extracted and the request translates to the Anthropic wire, the translated body gains a top-level prompt_cache_key carrying the value. This is a proxy tracking extension, not an Anthropic cache field: Anthropic caching uses cache_control, and an upstream that rejects unknown top-level fields will reject the injected request. Injection follows extraction alone: strip governs the tracked client-body fields, never the injected field.

Validation runs at the load boundary for YAML and Settings saves alike: at most 16 entries, at most 4 params per entry, param names one JSON key segment each (trimmed, non-empty, at most 64 bytes, printable ASCII, no quote, backslash or control bytes), and duplicate clients or duplicate params within one entry are rejected. The list hot-reloads: changed entries apply to new requests.

The Settings editor lives in the Conversations category and mirrors every check live per entry card: classified-client autocomplete from observed clients, ordered param rows with add/remove gated at the caps, the strip toggle, a count line against the entry cap, the server's exact rejection wording, and an Apply blocked on the first invalid card. To track opencode's prompt cache key and strip it from the upstream body, write:

sub_conversations:
  - client: "opencode"
    params:
      - "promptCacheKey"
    strip: true
search

Search the docs

Search titles, headings, and body text. Press / from anywhere.