self-hosted · cross-protocol · MIT

See every AI request.
Keep your own routing.

A self-hosted, cross-protocol inference proxy with a live dashboard, SQLite history, and controls to pause, limit, or debug traffic. OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages clients can target any configured upstream. Request metadata, timing, tokens, cost, and outcomes; body capture is opt-in.

Run it locally Tour the dashboard

MIT licensedone Go binarySQLite or in-memory

request path self-hosted

client protocols

chat

responses

messages

millivolt translate · route any configured upstream

upstreams

OpenAI

Anthropic

compatible

SQLite / in-memory metadata and metrics

the short version

Route, observe, control.

One request path, one operator surface. The proxy stays out of your provider contracts and the dashboard stays close to the traffic.

01 / route

Keep your own routing.

Choose the upstream base URL, credentials, and path on each request. Constrain accepted prefixes. There is no separate provider registry or key vault to maintain.

Read the protocol guide

02 / observe

See the request, not just the total.

Live rows, durable history, and scoped timelines for clients, providers, models, conversations, tools, status, and keys. Open a row for timing, tokens, cost, and retries.

Tour the dashboard

03 / control

Act before the retry storm.

Pause new work by scope, cap traffic by provider, model, client, key or conversation on a recurring schedule, capture bounded debug bodies, export exactly the records you filtered, and protect against error storms.

Operator workflows

the instrument

One view from total to request.

Global totals stay global; scoped explorer cards, timelines, and the request log share one selected scope. The dashboard is dense because the traffic is.

millivolt dashboard with KPI tiles, provider explorer cards, a traffic timeline, and a request log — Static capture from an authorized redacted history snapshot. The offline indicator reflects the capture session, not service health. Open the visual guide.

KPI band

Requests, tokens, spend, cache hit rate, errors, and speed, with their ranges.

Scoped explorer

Drill into providers, models, clients, conversations, tools, time, status, and keys.

Timelines

Six chart presets share one time window and one selected scope.

Request log

Live rows plus retained history, with a drawer for timing and outcome metadata.

signal

The shape of the traffic.

See requests, tokens, errors, and cost as separate questions. A 429 is a throttle, not an error; the charts keep that distinction.

Traffic chart with request and error timelines — **Traffic.** Requests and errors over the selected window and scope.

Token chart with input, output, reasoning, cached tokens, and cache hit rate — **Tokens.** Input, output, reasoning, cached tokens, and cache hit rate.

Errors chart separating errors from HTTP 429 rate limits — **Errors.** Errors and HTTP 429 throttles stay separate.

Cost chart with spend and request volume — **Cost.** Provider-reported spend beside request volume.

Swipe to compare charts.

operator surface

Control from one console.

One readout says whether anything is in force. One console holds every policy, the export and the restart, and each write leaves a sentence you can undo.

Operator actions, not per-user permissions. Inference stays transparent.

Read operator workflows

operator console paused 1 · captures 1 · limits 3

limit provider cursor.sh · model gpt-5 120 requests / 1m 118 left

pause client claude-code queue cap 1.02K 58m left

added limit for provider cursor.sh · 120 requests / 1m undo

This miniature is the real surface: pick a section. The header reads steady when nothing is in force, otherwise one segment per hold, capture, enforcing limit, off schedule policy, restart or gated backlog, and the overview is that same reading.

run it

Three commands and a browser tab.

Linux with Docker Engine and the Compose plugin. No Go, Node, or source checkout is needed for the container path.

curl -fsSL \
  https://raw.githubusercontent.com/LLM-4-People/millivolt/main/compose.yaml \
  -o compose.yaml
export MILLIVOLT_OPERATOR_TOKEN='replace-with-a-long-random-value'
docker compose up -d

Open http://127.0.0.1:8080/ and sign in with that token.

Fetch the deployment file.The image is published for amd64 and arm64. Named volumes retain history and saved settings.
Set the operator token.One shared credential protects the dashboard and operator plane. Inference is transparent and depends on the network boundary. With it unset, the dashboard denies every request.
Connect your client.Use http://127.0.0.1:8080/v1 as the API base URL. Your upstream key is forwarded; it does not authenticate the dashboard.

Exposure is the boundary.

Keep the listener on loopback, or put an authenticated proxy in front. There are no user accounts. Read security before network exposure.

Connect a client.

Point an OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages client at http://127.0.0.1:8080/v1. Send your upstream base URL in X-Proxy-Base-URL, your upstream key in Authorization, and an optional label in X-Proxy-Client. The proxy does not register providers or keys.

Protocol and client integration · Adapters and compatibility

under the hood

One service, not a stack.

The proxy, dashboard, retained history, and operator API ship together. There is no separate database server, frontend runtime, or monitoring pipeline.

clients

apps

agents

IDE

one Go service

routingper-request upstream URL, credentials, and path

adaptersChat Completions, Responses, Anthropic Messages, Cursor Connect

dashboardembedded assets, live SSE, retained views

operator APIMCP tools and scoped controls

upstreams

OpenAI

Anthropic

compatible

stateSQLite or in-memory ringmetadata, metrics, settings

one binaryGo service with embedded dashboard assets

one credentialdashboard and operator plane; inference is transparent

two storage modesSQLite history or an in-memory ring

MITsource available, no hosted dependency

Architecture and ownership · MCP server

fit and limits

What it is not.

millivolt is built for a single operator or small team that controls the host. The limits are part of the contract, not a footnote.

tenancy

One operator, no user isolation.

A single credential protects the dashboard and operator plane; inference is transparent and depends on the network boundary. There are no per-user accounts or tenant boundaries.

durability

Best-effort accounting.

Sustained overload can drop records even when inference succeeds. The dashboard warns about process-local storage drops.

scope

Observability, not billing.

Cost is provider-reported. This is not an independent invoice, a complete audit log, or a compliance system.

Security boundary · Storage and accounting

measured

Measured, not promised.

Current local measurements of the native process, including a deliberate overload fixture. These are scoped examples, not deployment guarantees.

33.5 MiBamd64 image

26.4 MiBidle RAM, empty history

114.3 MiBpeak RSS at 128 workers

14,166.2requests/s sustained for five minutes at 48 workers

Measured 2026-10-20; sustained capacity at 54d1276, artifact and five-second tables at facb699, on a shared WSL2 workstation. A separate overload fixture records the drop case. See method, tables and scope.

start here

Run it on your machine.

Three commands, one binary, your keys. Keep the traffic where you can see it.

Run it locally Read the docs

See every AI request.Keep your own routing.