millivolt Install millivolt
OperateSettings reference
On this page

Settings reference

Settings and the proxy CLI write the same fields. The dashboard Settings sheet saves them through GET and POST /admin/config, the YAML file holds them at rest, and the proxy flags override a few of them at launch. Every row below is generated from the code that owns it, so a setting's key, label, type, bound, reload behavior and help cannot describe one thing here and another in the dashboard. A - under accepted values means the schema declares no bound and no closed vocabulary for that key.

Read configuration for precedence, the YAML file and the settings schema, and containers for image mounts, restore paths and the deployment lifecycle.

Server

Bind address, database, and ring depth restart to apply. shutdown_timeout applies at the next SIGINT/SIGTERM.

Setting Default Accepted values Reload Description
listen (Listen address · string) ":8080" - Restart required Host:port the HTTP server binds. Restart required.
db_path (Database path · string) "proxy.db" - Restart required SQLite metrics store. Empty disables durability (ring only). Restart required.
history_size (History size · int) 10000 1..1000000 Restart required In-memory ring depth and startup backfill count. Durable history and paged request logs can extend beyond this window. Restart required.
shutdown_timeout (Shutdown timeout · duration) 5s > 0s Hot-reloads HTTP shutdown wait after active request contexts are canceled on SIGINT/SIGTERM. Storage draining follows separately. Must be > 0; applies at the next shutdown without restarting the listener.
restart_drain_timeout (Restart drain timeout · duration) 10m >= 0s (0s = wait indefinitely) Hot-reloads How long a dashboard-triggered restart waits for in-flight streams. Expiry aborts the restart and resumes accepting traffic; unfinished streams are not killed. New connections queue during a successful handoff.
read_header_timeout (Read header timeout · duration) 10s >= 0s (0s = no timeout) Restart required Slowloris guard on inbound headers. Restart required.
idle_timeout (Idle timeout · duration) 2m >= 0s (0s = no timeout) Restart required Keep-alive idle connections. No write timeout (it would kill SSE). Restart required.
request_body_idle_timeout (Request body idle timeout · duration) 30s > 0s Hot-reloads How long one inbound body read may wait for progress. Re-armed before every read and cleared at EOF or Close. Must not exceed request_body_read_timeout. Applies to requests started after reload.
request_body_read_timeout (Request body read timeout · duration) 10m > 0s Hot-reloads Absolute bound for reading one inbound request body after headers. Must be at least request_body_idle_timeout. The restore upload is exempt (idle bound only). Applies to requests started after reload.
response_write_timeout (Response write timeout · duration) 10m > 0s Hot-reloads Generic per-write bound for buffered responses. Streaming owners may arm a shorter bound. Must be at least sse_write_progress_timeout. Applies to requests started after reload.
go_memory_limit (Go memory limit · bytes) 0B 32MiB..1024GiB (0B = auto cgroup, else unbounded) Restart required Soft cap on Go runtime managed memory (heap, stacks, runtime), applied once at startup. With the setting at 0, an operator GOMEMLIMIT wins and otherwise a detected finite cgroup memory ceiling applies at 80% (ceilings below 80MiB are ignored); a set value overrides GOMEMLIMIT, not GOGC. Exceeding it turns the collector more aggressive (capped near 50% CPU), never fails requests. Does not bound SQLite's page cache or OS overhead. Restart required.

Request

Inbound body cap, tool-call history bound, SSRF allowlist, optional body-preview, token-refresh response cap, and debug-capture retention.

Setting Default Accepted values Reload Description
max_request_bytes (Max request bytes · bytes) 32MiB 1KiB..1GiB Hot-reloads Inbound body the proxy will buffer to read routing metadata. Larger bodies are rejected with 413.
request_tool_calls_max (Max request tool calls · int) 1024 1..10000 Hot-reloads Most Chat tool calls across the whole message history of one inbound request, including the deprecated per-message function_call shape. Anthropic tool_use and Responses function_call history are bounded by their protocol collection limits instead. Real agent turns replay a handful; raise it only for clients that resend unusually long tool loops, because every call is materialized in memory during translation.
allowed_base_urls (Allowed base URLs · strings) [] - Hot-reloads SSRF allowlist of upstream base URLs. Empty allows any target (single trusted operator). An entry admits only the same scheme, host and port once the scheme's default port is filled in, plus that entry's path and its children, so a port-less entry does not admit other ports on the host. Each entry must be an absolute URL whose prefix is literally lowercase http:// or https://, with a lowercase hostname, no userinfo, query, fragment or trailing slash, and a port after a hostname in 1..65535.
capture_body_preview (Capture body preview · bool) false - Hot-reloads Store a short bounded preview of the last user prompt / response in metrics. Off by default so content is not persisted.
debug_capture_ttl (Debug capture TTL · duration) 24h 1h..168h Hot-reloads How long a full debug capture (bodies and headers) is kept after the request finishes. Operator debug sessions choose which traffic is captured; this is blob retention, not the session timer.
debug_capture_max_bytes (Debug capture max bytes · bytes) 1MiB 4KiB..16MiB Hot-reloads Cap on one debug capture: retained headers are charged first and the remainder is split between the request body and the client-facing response bytes. Past the cap the rest is dropped and the capture is marked truncated.
debug_store_max_bytes (Debug store max bytes · bytes) 512MiB 16MiB..8GiB (0B = unlimited) Restart required Total durable debug-capture sidecar cap. New captures are skipped once the next write would exceed it; TTL expiry frees capacity. Bytes, not characters. 0 = unlimited. Restart required.
auto_token_refresh (Auto token refresh · bool) true - Hot-reloads Exchange an expired or expiring JWT when the client sends X-Proxy-Refresh-Token. All supported providers use inference handback: HTTP 401 with token_expired and access_token in the error body, plus refresh_token only when the exchange returns a different replacement. No inference is sent; adopt the returned credentials and retry, retaining the old refresh token if no replacement is returned. Model discovery refreshes transparently without returning tokens. Stateless: no reusable credentials are retained.
token_response_max (Token response max bytes · bytes) 1MiB 4KiB..16MiB Hot-reloads Token-endpoint response read during transparent credential refresh. Past it the refresh fails open and the original request is proxied unchanged.

Upstream

Connection pool, header timeout, stream keepalives, model-discovery budgets and response-buffer bounds.

Setting Default Accepted values Reload Description
upstream_timeout (Upstream header timeout · duration) 5m >= 0s (0s = no header cap) Hot-reloads How long to wait for upstream response headers. Streaming bodies are not subject to this. Transient header timeouts are retried.
upstream_dial_timeout (Upstream dial timeout · duration) 30s > 0s Hot-reloads How long establishing a new upstream connection may take. Applies to new connections after reload; in-flight streams keep their transport.
upstream_dial_keepalive (Upstream dial keepalive · duration) 30s > 0s Hot-reloads TCP keepalive interval for upstream connections. Applies to new connections after reload.
upstream_tls_handshake_timeout (Upstream TLS handshake timeout · duration) 10s > 0s Hot-reloads How long the TLS handshake with an upstream may take. Applies to new connections after reload.
models_discovery_timeout (Model discovery timeout · duration) 30s 1s..5m Hot-reloads One upstream metadata budget for transparent key refresh, all model pages and optional enrichment. Never applies to LLM streams. New discoveries use reloaded limits.
models_discovery_max_bytes (Model discovery bytes · bytes) 8MiB 4KiB..64MiB Hot-reloads Aggregate response bytes read after transport decoding across model pages, unary RPCs and enrichment. Primary overflow fails; optional enrichment is discarded. This is not a strict heap limit.
models_discovery_max_pages (Model discovery pages · int) 100 1..1000 Hot-reloads Maximum upstream model responses per discovery, including a unary RPC and optional enrichment. Limits are captured once, never reset between pages.
models_discovery_max_entries (Model discovery entries · int) 10000 100..1000000 Hot-reloads Model entries one discovery request may retain across every page and enrichment source. The byte budget alone cannot bound the decoded map cost of a compact catalog, so a source past this count fails closed instead of returning a partial list. Raise it for providers that publish unusually large catalogs.
translated_response_max_bytes (Translated response max bytes · bytes) 32MiB 4KiB..1GiB Hot-reloads Whole buffered non-streaming upstream response a cross-protocol translation may read, and the translated document it may write back. Past it the request fails with 502 response_too_large rather than relaying a prefix that breaks the translated shape.
upstream_response_header_max (Upstream response header max bytes · bytes) 1MiB 4KiB..64MiB Hot-reloads Upstream response headers the proxy accepts. Past it the request fails as a transport error. The upstream client is rebuilt on reload, so new requests use the new value while in-flight streams keep the client they started on.
non_stream_capture_max (Non-streaming capture max bytes · bytes) 1MiB 16KiB..64MiB Hot-reloads Prefix of a non-streaming response body kept for usage and cost parsing. Larger documents still relay in full; their accounting may be incomplete. This is a capture cap, not a forwarding cap, and it is per in-flight non-streaming request outside max_inflight_response_bytes.
stream_terminal_hold_max (Stream terminal hold max bytes · bytes) 64KiB 4KiB..1MiB Hot-reloads Bytes withheld at the end of a relayed SSE stream while the relay decides between verbatim release and an in-band degenerate-error substitution. Past it the hold is abandoned and the bytes are released verbatim. A single complete event line can exceed it by up to stream_event_max_bytes. Raise it only if a legitimate terminal region is larger than the default.
stream_event_max_bytes (Stream event max bytes · bytes) 2MiB 2MiB..16MiB Hot-reloads Largest SSE event accepted on the passthrough relay and by every translated stream decoder: a single line, and the accumulated data of a multi-line event. The translated frame buffer is derived from this value, so both bounds move together. The floor is the built-in accepted size; raise it only for providers that emit larger events.
max_conns_per_host (Max conns per host · int) 512 >= 0 (0 = unlimited) Hot-reloads Total connections per upstream host. Sized for concurrent SSE streams, not requests/sec.
max_idle_conns (Max idle conns · int) 512 >= 0 (0 = unlimited) Hot-reloads Process-wide idle connection pool.
max_idle_conns_per_host (Max idle conns per host · int) 512 >= 1 Hot-reloads Idle connections kept per upstream host. Cannot exceed max conns per host when that is set. net/http treats 0 as a default of 2, not unlimited, so this must be >= 1.
idle_conn_timeout (Idle conn timeout · duration) 90s >= 0s (0s = never expire) Hot-reloads How long an idle upstream connection is kept before closing.
upstream_stream_idle_timeout (Upstream stream idle timeout · duration) 10m > 0s Hot-reloads Cancel an upstream response body after this much silence. Releases the handler, upstream connection and scheduler slot. New response bodies use the reloaded value.
whole_document_read_timeout (Whole-document read timeout · duration) 2m > 0s Hot-reloads Fallback bound for one whole-document upstream read when the caller supplied no deadline. Buffered reads only; streaming SSE is never subject to it.
whole_document_read_max (Whole-document read maximum · duration) 10m > 0s Hot-reloads Ceiling for whole-document upstream reads. Must cover whole_document_read_timeout and models_discovery_timeout.
sse_keepalive_interval (SSE keepalive interval · duration) 15s 1ns..10m Hot-reloads How often a quiet streaming socket gets a : keepalive comment (queue wait, operator hold, mid-stream gaps). Range 1ns..10m. Applies to streams started after reload.
sse_write_progress_timeout (SSE write progress timeout · duration) 1m > 0s Hot-reloads Per-event and keepalive write bound for a client-facing SSE stream. A stalled reader fails that write instead of pinning shutdown. Must not exceed response_write_timeout. Applies to streams started after reload.

Queue & retry

Per-key concurrency, transparent 429/5xx retry and error classification, client error-class disclosure, and quality re-asks.

Setting Default Accepted values Reload Description
max_concurrent (Max concurrent · int) 0 >= 0 (0 = unlimited) Hot-reloads In-flight sends per provider+key. 0 is unlimited.
max_queue_size (Max queue size · int) 1024 >= 0 (0 = unlimited) Hot-reloads Queued waiters per group. 0 is unlimited. It also feeds the default queue cap of a new pause hold.
max_inflight_requests (Max in-flight requests · int) 10000 0..1000000 Hot-reloads Process-wide inference admission bound. Requests above it are refused with 503 and Retry-After instead of buffering their bodies. 0 = unlimited. Hot-reloads.
max_inflight_body_bytes (Max in-flight body bytes · bytes) 256MiB 1MiB..64GiB (0B = unlimited) Hot-reloads Process-wide budget for buffered inbound request bodies. Admission charges the bytes actually read and refuses with 503 when the next read would exceed the budget. A declared Content-Length reserves nothing, so an idle connection cannot hold the budget. Bytes past max_request_bytes are neither charged nor retained: that body is answered 413 by the cap owner. A refused request drains at most its remaining cap window so an over-cap body still answers 413 rather than a retryable 503. 0 = unlimited. Hot-reloads.
max_inflight_response_bytes (Max in-flight response bytes · bytes) 256MiB 1MiB..64GiB (0B = unlimited) Hot-reloads Process-wide budget for buffered upstream response bodies: whole-document translation of non-streaming responses, the non-streaming 200 quality spool (at most 1 MiB plus one 32 KiB chunk), and model discovery. Admission reserves the declared Content-Length when it fits the bounded read cap, otherwise that cap, and refuses with 503 and Retry-After before reading when the budget would be exceeded. The reservation is released once the buffer is written, translated, or abandoned. 0 = unlimited. Hot-reloads.
max_queue_wait (Max queue wait · duration) 10m >= 0s (0s = unlimited) Hot-reloads How long a request may sit in the queue before 429. 0s is unlimited. Operator-hold time is excluded.
max_retries (Max retries · int) 5 0..100 (0 = no retries) Hot-reloads Transparent retries of transient 429 / 5xx / transport failures. Durable quota 429s follow quota_pause_mode instead of this budget.
queue_retry_after (Queue Retry-After · duration) 2s 0s..24h Hot-reloads Retry-After hint sent when our own queue is full or the wait is exceeded. HTTP Retry-After is integer seconds, so the value must be a whole number of seconds.
base_backoff (Base backoff · duration) 1s > 0s Hot-reloads Initial adaptive delay. Doubles each attempt and each consecutive failed request, up to max backoff. A provider Retry-After/reset is a floor (never retry sooner) and does not reset this doubling. Cannot exceed max backoff. Must be > 0.
max_backoff (Max backoff · duration) 2m > 0s Hot-reloads Cap on adaptive backoff. Does not clamp a provider Retry-After / rate-limit-reset (daily limits are often 30–60m). Must be > 0.
quality_retries (Quality retries · int) 1 0..3 (0 = disabled) Hot-reloads Transparent re-attempts of a degenerate 200 (empty completion, empty tool_calls), a truncated stream that relayed no content yet, a retryable in-band provider error dropped before the wire, and Cursor's empty-resume re-ask. 0 disables.
thinking_retries (Thinking retries · int) 1 0..3 (0 = disabled) Hot-reloads Transparent rescue of requests that die or end without an answer during thinking: a stream truncated or reset after reasoning-only output, a cleanly finished reasoning-only completion, a retryable in-band provider error that arrived after reasoning-only output, and a non-streaming reasoning-only body. The fresh attempt appends after the relayed reasoning; answer content and tool calls are never rescued, and finish_reason length never is. Each rescue re-sends the full request (billed). 0 disables.
retryable_error_classes (Retryable error classes · strings) [] - Hot-reloads Operator extensions to the built-in retryable in-band error vocabulary: exact type or code strings (case-insensitive) that a provider reports inside a 200 and that authorize a transparent re-send while no generation content was relayed. Built-ins already cover server_error, the 503 overloaded/unavailable family, api_error, overloaded_error, timeout_error and common gateway spellings; list custom gateway spellings here. Entries add to the built-ins and never move one; a durable quota/billing class is rejected at load, and in-band rate limits, unknown classes and errors after relayed content stay verbatim. Reload applies to new rescue decisions.
rate_limit_error_classes (Rate-limit error classes · strings) [] - Hot-reloads Operator extensions to the built-in transient rate-limit vocabulary (rate_limit_error, rate_limit_exceeded, tier_limit_exceeded, resource_exhausted, rpm/tpm spellings and friends). A gateway that announces its limit under a custom type or code string (case-insensitive) is then retried transparently within the retry budget, counted in the rate-limit series, and classified rate_limit for clients. Unlike a bare 429, a non-429 status carrying the class still counts as a failed request on the health surface. Entries add to the built-ins and never move one; a durable quota/billing spelling is rejected at load. Reload applies to new classifications and re-reads stored history through the same vocabulary.
suppress_client_retries (Suppress client retries · bool) false - Hot-reloads Make the proxy the clients' sole retry authority: stamp x-should-retry: false on every LLM-relay error response. The official OpenAI SDKs (python/node/go/ruby) honor it over their 408/409/429/5xx auto-retry and over Retry-After, including relayed provider hints; the proxy's own retry ladder has already run by then. Cannot suppress transport-level failures, and clients that ignore the header (the Vercel AI SDK / opencode stack) keep their own policy.
client_error_class (Client error class · string) "off" off, header Hot-reloads Negotiate millivolt's canonical error class to clients on the LLM relay surface. off leaves relayed responses exactly as the provider sent them. header stamps X-Proxy-Error-Class (rate_limit, quota, unavailable, unreachable, timeout, content, client_error, proxy, canceled, upstream) and a class-derived X-Should-Retry: true|false on every error status before the status line commits, so a client can tell a rate limit announced by a 400 from a client fault and a durable quota block from transient backpressure without parsing provider envelopes. The body is never rewritten. suppress_client_retries still wins while it is on; mid-stream SSE errors cannot carry new headers. Reload applies to new responses.

Error storm protection

Bounded recovery queues for affected providers or exact model names. Hot-reloads; policy changes clear observations and wake waiters. Banner visibility changes do not reset protection.

Setting Default Accepted values Reload Description
storm_enabled (Enable protection · bool) false - Hot-reloads Detect elevated upstream failure rates and queue affected sends for bounded recovery probes. Opt-in; requests can still fail on cancellation, exhausted retry budgets, queue limits or permanent errors. Retries can repeat upstream execution and billing.
storm_provider_enabled (Provider protection · bool) true - Hot-reloads Gate every model on a provider only when all active models individually meet the sample and failure thresholds. Active means a request in the same detection window; idle models expire.
storm_model_enabled (Model protection · bool) true - Hot-reloads Gate an affected exact recorded model name within its provider. Dashboard model grouping does not broaden this scope.
storm_banner_enabled (Show incident banner · bool) true - Hot-reloads Show affected providers/models, safe error categories, failure percentages, queue state and recovery timing at the top of the dashboard. Changing visibility does not reset protection.
storm_window (Detection window · duration) 1m 1s..1h Hot-reloads Rolling time range for successful and eligible failed upstream attempts. Models with no requests in this window expire from provider-wide checks.
storm_min_samples (Minimum samples · int) 20 1..1000000 Hot-reloads Minimum successful or eligible failed upstream attempts in the detection window before a model qualifies. Provider-wide protection requires every active model to qualify. Retries count as separate attempts; this is not a distinct-request percentage.
storm_error_percent (Failure threshold · int (percent)) 50 1..100 Hot-reloads Minimum percentage of sampled upstream attempts with a selected failure. Applied to each exact model after the minimum sample count; all active models must qualify for provider-wide protection.
storm_initial_backoff (Initial recovery delay · duration) 5s 1ms..1h Hot-reloads Initial storm cooldown before a recovery probe. Cannot exceed maximum recovery delay. Valid upstream Retry-After/reset hints still apply independently.
storm_max_backoff (Maximum recovery delay · duration) 2m 1ms..1h Hot-reloads Cap on exponentially increased storm cooldowns. Must be at least initial recovery delay. Does not shorten upstream Retry-After/reset hints.
storm_backoff_multiplier (Recovery delay multiplier · int) 2 2..10 Hot-reloads Multiply the storm cooldown after a failed recovery probe, up to maximum recovery delay.
storm_jitter_percent (Recovery delay jitter · int (percent)) 20 0..100 (0 = no jitter) Hot-reloads Random spread as a percentage of the storm cooldown to separate recovery attempts. 0 disables jitter.
storm_recovery_successes (Recovery successes · int) 2 1..100 Hot-reloads Consecutive successful recovery probes required before queued traffic resumes normally.
storm_max_queue (Maximum queued requests · int) 1000 1..100000 Hot-reloads Maximum requests waiting for storm recovery. Excess work is rejected locally; requests and their bounded bodies stay in process memory while waiting.
storm_max_wait (Maximum queue wait · duration) 5m 1ms..24h Hot-reloads Maximum duration of each storm gate wait, not total request duration across all retries and other queues. Caller cancellation may end the wait earlier.
storm_max_scopes (Maximum tracked scopes · int) 2048 2..100000 Hot-reloads Bound on provider and exact-model detector scopes retained in process memory. Increasing it does not establish a total process memory ceiling.
storm_max_retries (Additional storm retries · int) 20 0..1000 (0 = no additional retries) Hot-reloads Additional retries beyond max_retries only while a relevant storm is active and the failure matches a selected eligible transient condition. Does not replay meaningful emitted upstream content. 0 keeps the ordinary retry budget.
storm_status_codes (HTTP failure statuses · strings) - "500" - "502" - "503" - "504" - Hot-reloads Exact status strings selected for detection and additional storm retries: 429 or 500..599, without duplicates. Empty disables HTTP status triggers. 429 is opt-in because limits may be credential-specific; durable quota/billing 429s remain excluded from storm evidence (quota_pause_mode owns their reaction).
storm_transport_errors (Transient transport failures · bool) true - Hot-reloads Include transport failures already eligible for the shared retry policy. Caller cancellation and permanent transport errors are excluded.
storm_stream_errors (Response and stream failures · bool) true - Hot-reloads Include recorded in-band, stream and quality failures to protect future sends. Reports a generic response error without retaining error bodies for the banner. Never replays meaningful emitted upstream content.

Quota pause

Provider-wide reaction to a 429 carrying a durable quota/billing error (insufficient_quota/credits, spend limits): surface it immediately, park the provider behind a recovery gate, or hold it until an operator resumes. OpenAI-compatible wire only; Anthropic and Cursor upstreams keep the off behavior.

Setting Default Accepted values Reload Description
quota_error_classes (Quota error classes · strings) [] - Hot-reloads Operator extensions to the built-in durable quota/billing vocabulary (insufficient_quota/credits, spend and hard limits, and the numeric business codes of some gateways). A matching type or code string (case-insensitive) is a durable account condition: the ordinary retry ladder never absorbs it, quota_pause_mode owns the reaction, and it counts as an error rather than flow control. Entries add to the built-ins and never move one; a spelling already built into another class is rejected at load. Reload applies to new classifications and re-reads stored history through the same vocabulary.
quota_pause_mode (Mode · string) "off" off, retry, manual Hot-reloads Reaction to a 429 carrying a durable quota/billing error (insufficient_quota/credits, spend limits; in-band occurrences included): off surfaces it immediately; retry opens a provider-wide recovery gate - the triggering request absorbs the 429 and re-sends per the storm recovery delays, new requests park on the gate (bounded by storm_max_queue/storm_max_wait), and the gate closes after quota_pause_recovery_successes consecutive successful probes (the default 1 closes on the first 2xx); manual creates an indefinite provider pause that only an operator resumes. The built-in classification is fixed (quota_error_classes may add unknown spellings); only the reaction is configurable. OpenAI-compatible wire only; Anthropic and Cursor upstreams keep the off behavior.
quota_pause_recovery_successes (Recovery successes · int) 1 1..100 Hot-reloads Consecutive successful recovery probes that close a retry-mode quota gate. 1 releases the queue on the first 2xx. Meaningful only in retry mode.

Conversations

How the request log groups turns into conversations.

Setting Default Accepted values Reload Description
conversation_idle_gap (Idle gap · duration) 30m > 0s Hot-reloads A gap longer than this starts a new conversation for the same client+key. Generous so thinking pauses inside one task don't split it. Must be > 0.
conversation_max_open (Max open · int) 64 1..100000 Hot-reloads Cap on concurrently-open conversations per client+key. Oldest is evicted past it.
sub_conversations (Sub-conversation tracking · sub_conversations) [] - Hot-reloads Per-client tracking of a request body field that carries a sub-conversation identity, exemplified by opencode's promptCacheKey. Each entry names one classified client exactly (no wildcard, no case folding) plus 1..4 exact JSON field names matched only at the request body's top level; the first present field decides (a duplicate top-level key drops the identity): a present value that is not a JSON string value counts as absent and the next field is checked, while a present string that cannot be decoded or fails the identity rules (non-empty after trimming, valid UTF-8, free of control bytes, at most 512 bytes) drops the identity for the request - no later field is consulted - and a dropped identity is never a client error. The drop is conservative: a value whose raw spelling exceeds the bounded scan window is dropped even when its trimmed form would pass. The value groups the request under a k: conversation identity, after the X-Proxy-Session header (s:) and before automatic client+key grouping; X-Proxy-Parent-Session still owns parent linkage. Requests translated to the Anthropic wire gain a top-level prompt_cache_key carrying the value. strip: true removes the entry's configured fields from the top level of the client body before relay or translation, including a Responses client body; the default false forwards the body unchanged. Unlike the request-overrides body fields, strip is not limited to Chat or Anthropic inbound. Param names are exact printable-ASCII JSON key segments up to 64 bytes, no quotes or backslashes; duplicate clients and duplicate params within an entry are rejected. Caps: 16 entries, 4 params per entry. Default: an empty list, tracking fully off. Hot-reloads.

Format translation

Cross-protocol translation policy, streamed tool-call bound, Anthropic defaults and Cursor agent.v1 bridging.

Setting Default Accepted values Reload Description
anthropic_default_max_tokens (Anthropic default max tokens · int (tokens)) 4096 1..1000000 Hot-reloads Injected into translated Anthropic requests when the client sent none of max_tokens, max_completion_tokens or max_output_tokens.
stream_tool_calls_max (Max streamed tool calls · int) 1024 1..10000 Hot-reloads Most tool calls one translated response stream may carry: concurrent open blocks for Anthropic and Responses, distinct calls for Chat (its stream has no per-call close event). A stream past the bound ends with a translation_error. Raise it only for providers that emit unusually large parallel tool batches.
translation_loss (Translation loss policy · string) "warn" warn, reject Hot-reloads What a cross-protocol translation does when the target cannot represent a client field. warn (default) drops optional fields and reports them in X-Proxy-Warnings; reject fails the request with 400 naming the fields before any upstream send and refuses every recorded loss, including a documented approximation. Structural features are always rejected when they cannot be honored. A response-side loss is never fatal: a non-streaming response reports it in X-Proxy-Warnings, and a streaming response drops it without a header. Hot-reloads.
cursor_default_context_window (Cursor default context window · int (tokens)) 0 >= 0 (0 = omit (client default)) Hot-reloads Optional context-window hint stamped onto every model in GET /v1/models when X-Proxy-Format is cursor. 0 omits the field.
cursor_park_ttl (Cursor park TTL · duration) 3h > 0s Hot-reloads How long a parked Cursor Run stream stays open waiting for tool results. Must be > 0. Expiry closes the stream; the next continuation cold-starts.
cursor_heartbeat_interval (Cursor heartbeat · duration) 5s 1ns..10m Hot-reloads How often the proxy pings a Cursor agent.v1 run (idle parked runs included). Range 1ns..10m. Applies to runs created after reload.

Storage

Durable SQLite write pipeline. Restart to apply.

Setting Default Accepted values Reload Description
storage_write_chan_cap (Write channel cap · int) 8192 1..1000000 Restart required Buffered records before overflow drops. Restart required.
storage_batch_cap (Batch cap · int) 256 >= 1 Restart required Max records per SQLite transaction. Cannot exceed the write channel cap. Restart required.
storage_flush_interval (Flush interval · duration) 500ms > 0s Restart required Partial-batch flush cadence. Must be > 0 (a zero ticker panics). Readiness convicts a wedged writer after one interval plus a progress window of four intervals clamped to 30s..2m. Restart required.
storage_query_timeout (Query timeout · duration) 10s > 0s Restart required Bounds dashboard SELECT queries. Must be > 0. Open and AggAPI bind at start (restart required); pause/throttle persist reads the live snapshot.
storage_query_max_bytes (Query result bytes · bytes) 8MiB 4KiB..64MiB Restart required Maximum JSON bytes returned by /metrics/query; also bounds SQLite string/blob values. Oversized results fail with HTTP 413, never truncate. Does not cap all SQLite working memory. Restart required.
storage_query_max_rows (Query result rows · int) 10000 1..100000 Restart required Maximum rows returned by /metrics/query. Oversized results fail with HTTP 413; use SQL filters or LIMIT. Does not limit dashboard aggregates or exports. Restart required.

Backup

Download or restore a self-checked archive of the saved config, the SQLite history, or both. Restore inspects first and the apply is confirmed. Merge keeps live defaults and live request ids; replace loads and validates the backup, writes canonical YAML, and preserves CLI-overridden listen and db_path.

Setting Default Accepted values Reload Description
backup_max_bytes (Backup size cap · bytes) 1GiB 1MiB..4GiB Hot-reloads Maximum size of one Settings backup download or restore upload. Larger archives are refused. Downloads and restores stream through private staging files; this is the admit/reject ceiling, not a heap guarantee.

MCP

The operator-gated /mcp LLM tool endpoint: mcp_enabled registers the route or keeps it reserved (404, never inference), and the limits bound what its tools return to a model. Restart to apply; the route table and the endpoint's services are built once at boot.

Setting Default Accepted values Reload Description
mcp_enabled (Enable MCP endpoint · bool) true - Restart required Register the streamable HTTP MCP endpoint at /mcp for LLM tool clients, operator-gated like /metrics and Bearer-only: a session cookie alone is refused. Off keeps the route reserved (404, never inference); the /mcp/ look-alike reservation and the operator gate stay either way. Restart required.
mcp_page_size (Page size · int) 50 10..500 Restart required Default page for the records, values and audit capture listings the endpoint serves to a model. Restart required.
mcp_query_max_rows (Query result rows · int) 200 1..100000 Restart required Default client-side row limit when a query call omits max_rows. An explicit max_rows may raise it up to this setting's maximum; the proxy's own storage_query_max_rows still answers 413 first. Restart required.
mcp_query_max_bytes (Query result bytes · bytes) 128KiB 1B..64MiB Restart required Client-side cap on the encoded size of one query tool result. Keeps whole rows, truncating with an explicit marker rather than silently. Restart required.
mcp_capture_max_bytes (Capture max bytes · bytes) 256KiB 1B..64MiB Restart required Size above which a stored capture document is withheld whole from the audit_capture_get tool; a document that large is more than a model should read in one call. Restart required.
mcp_query_timeout (Query timeout · duration) 2m > 0s Restart required Bounds one proxy read the endpoint makes that is not the proxy's own bounded SQL reader (a full-history chart or explorer fold can take seconds). Must be > 0. Restart required.
mcp_timeout (Request timeout · duration) 30s > 0s Restart required Bounds every in-process call the endpoint's tools make to the proxy, including mutations. Must be > 0. Restart required.

Dashboard

Live-view cadence and request-log window. Hot-reloads; open dashboards pick up the next tick.

Setting Default Accepted values Reload Description
dash_log_rows (Request log rows · int) 60 10..500 Hot-reloads Page size of the live request log (newest first). Scrolling near the bottom loads more, including older durable history.
dash_poll_interval (Poll interval · duration) 5s > 0s Hot-reloads KPI / pause / SSE-fallback poll cadence. Must be > 0.
dash_chart_refresh (Chart refresh · duration) 15s > 0s Hot-reloads How often the traffic chart re-fetches server-authoritative buckets and period totals. Must be > 0.
dash_explorer_stale (Explorer stale after · duration) 15s > 0s Hot-reloads How old a cached explorer breakdown may be before it is re-fetched. Must be > 0.
dash_background_refresh (Background refresh · bool) false - Hot-reloads Keep the dashboard current while its tab is hidden: the poll runs at the browser's throttled background cadence (about one wake per minute on Chromium) and a Web Lock keeps the SSE feed from being frozen. Desktop Chromium only; Android and iOS still suspend background pages at the OS level. Off keeps the current pause-while-hidden behavior.

Models

Model grouping rules - an ordered rewrite pipeline merging spelling variants of the same model in every grouped surface. Records keep their exact spelling; hot-reloads.

Setting Default Accepted values Reload Description
model_rules (Model grouping rules · model_rules) - {mode: lower, from: "", to: ""} - {mode: pattern, from: "^(?:[a-z0-9][a-z0-9._-]*/)+", to: ""} - {mode: pattern, from: "(?::[a-z0-9._-]+)+$", to: ""} - {mode: pattern, from: "(?:-(?:[a-z]{0,2}fp\\d+\|bf\\d+\|int\\d+\|nf\\d+\|[a-z]?q\\d+(?:_[0-9a-z]+)*\|gptq\|awq\|gguf\|exl2\|mlx\|onnx\|ggml\|fast\|speed\|value\|free\|preview\|latest\|beta\|alpha\|stable\|experimental\|thinking\|reasoning\|xhigh\|high\|medium\|low\|minimal\|(?:\\d{4}\|\\d{8})))+$", to: ""} - {mode: pattern, from: "(\\d)\\.(\\d)", to: "$1-$2"} - Hot-reloads Capped at 64 rules. Ordered rewrite pipeline that merges spelling variants of the same model in every grouped surface (explorer model dimension, scope filters, debug checklist, the Logs download and Settings delete-filter model optgroups, and the request-log leaf’s displayed name). Each rule is one step: exact (whole-string merge old → new), pattern (regex rewrite of all occurrences, $1 capture refs), lower (fold case). Rules apply once in order; the shipped default folds case, strips one or more leading vendor/ namespaces, strips trailing :tag deployments, strips a trailing run of serving decorations (quantization or format such as -fp4, -q4_k_m, -gptq; service tier such as -fast, -speed, -value; release channel such as -preview, -latest; build stamp such as -1017), and unifies . with - between digits. Model tiers such as -flash or -pro, fine-tunes such as -abliterated or -rl, and parameter counts stay distinct. An explicitly empty list groups by the exact stored spelling. Stored values remain unchanged; purge/export match them exactly and request details show the original. Debug matching uses separate native base-model normalization, not these display rules. Hot-reloads.

Providers

Optional per-provider JSON field-name maps: usage/cost response keys, the models metadata endpoint merged into /v1/models, ensured tool names injected into relayed chat bodies, and optional upstream wire headers. Empty = auto-detect.

Setting Default Accepted values Reload Description
providers (Provider field maps · providers) {} - Hot-reloads Per-provider cost/usage JSON field paths, optional models_path/models_keys enrichment, ensured tool names and headers (a string map). Field names only, never static prices. Custom cost_keys must report USD; leave cost_in_usd_ticks to automatic unit conversion. ensure_tools names function tools injected as inert stubs into relayed Chat Completions client bodies on a Chat outbound wire missing them. When the client sent no tools of its own, an absent, null, or auto tool_choice is forced to none; an explicit none stays; a required, named, or malformed choice skips injection entirely. A first-party client's tool-signature gate still recognizes the request. Header values expand {{uuid4}} to a fresh UUID, {{platform}} to the server's Rust-style os; arch pair, and {{opencode-msg-id}}/{{opencode-ses-id}} to fresh opencode-format identifiers; unknown templates stay literal. On HTTP relay requests, configured headers override forwarded/extracted-auth values and X-Proxy-Headers wins. accept-encoding and range are proxy-owned: the X-Proxy-Headers map rejects them with 400, and configured or override values are stripped from the final request. Model discovery and the Cursor bridge construct headers separately and do not apply X-Proxy-Headers. Headers can contain secrets; keep runtime configuration private.
provider_aliases (Provider aliases · aliases) {} - Hot-reloads Map an old provider label to a canonical label. Rewrites existing stored history at boot and on reload; new requests use the canonical label. Existing in-memory records keep their prior label until they age out or the process restarts. Removing a mapping does not restore rewritten history. Chains and self-maps are rejected.

Request overrides

Scoped rewrites of the upstream request: set, replace or remove HTTP headers and set output-token ceilings per client, provider and/or model. Rules match exactly, all matching rules apply in list order, and the later rule wins. Default: an empty list, the feature fully off.

Setting Default Accepted values Reload Description
request_overrides (Request override rules · request_overrides) [] - Hot-reloads Capped at 64 rules. Ordered rules that rewrite the upstream request before relay. A rule matches when every non-empty scope field equals the request's value exactly: client as the classified client, provider as the canonical label after provider_aliases, model as the recorded model id; an empty scope field matches anything. All matching rules apply in list order, and for the same header or body field the later rule wins. headers sets or replaces upstream headers (names are canonicalized, values must be non-empty single-line), remove_headers deletes them, and body sets max_tokens / max_completion_tokens (1..1000000) on a Chat Completions or Anthropic client body. Credential and protocol headers are rejected in an override at load (provider headers may use them; they apply after the credential). Credential class is judged by role, not by a list of vendor spellings: any name carrying authorization/auth, api-key, secret, password, credential, bearer, jwt, signature, cookie, key or token is refused, so x-auth-key, x-amz-credential, x-functions-key and x-subscription-key are covered. Identifier and accounting shapes stay admitted: key-id, cache-key, idempotency-key, and the token counters (x-prompt-tokens, x-token-count) plus the x-ratelimit-* and anthropic-ratelimit-* families. Protocol and framing names (host, content-type, content-length, transfer-encoding and the rest) and the whole x-proxy- control prefix are refused as well. The override applies after provider headers and the client's X-Proxy-Headers map and wins over both, with two enforced exceptions: the target's resolved auth header and Authorization are never set or removed by an override, and Accept-Encoding and Range are stripped after the override merge because the transport owns them. It covers inference sends only, never model discovery or token refresh. Body rewrites apply to Chat Completions or Anthropic client bodies before translation, whatever the target (a Responses client body is not rewritten), and re-stamp the recorded request cap; the Cursor bridge reads no body fields, so body is skipped for cursor targets while header rules still apply. An empty list leaves every request untouched. Hot-reloads.

Proxy CLI flags

The flag set in cmd/proxy owns these rows. Every flag is parsed before any configuration is loaded, so the default below is the registration default, not the effective value after a config file or another override.

Flag Type Default Usage
-config string "proxy.yaml" path to config file (empty for built-in defaults)
-print-config bool false print documented built-in defaults as YAML and exit without loading configuration
-print-example-config bool false print the documented example with enabled provider profiles and exit without loading configuration
-version bool false print application and source build metadata as JSON and exit without loading configuration
-listen string "" override listen address (e.g. 127.0.0.1:8081); keeps instances on distinct ports
-db-path string "" override storage DB path ("" = config value; "none" = disable durable storage)
-pid-file string "" write the process pid to this file after boot
-healthcheck bool false probe GET /healthz on the configured listen address and exit nonzero on failure (Docker HEALTHCHECK); loads config read-only
-readycheck bool false probe GET /readyz on the configured listen address and exit nonzero when storage is not ready (Compose readiness); loads config read-only
search

Search the docs

Search titles, headings, and body text. Press / from anywhere.