Relay

relay is an OpenAI-compatible gateway that fronts the CLIProxyAPI account pool. Clients point their OpenAI SDK at it, authenticate with a relay key, and relay picks a healthy upstream channel, forwards the request (streaming or not), meters the tokens, and bills a credit wallet.

Base URL (dev): http://localhost:3113
API surface: OpenAI-compatible /v1 + /healthz (relay serves these directly — it is not under the /api prefix)
Auth: relay keys (mr_rk_…) in Authorization: Bearer …
Upstream: the CLIProxyAPI account pool, selected per-request

What it does

Every POST /v1/chat/completions flows through the same pipeline:

Authenticate the mr_rk_… relay key (hash lookup, cached in Redis).
Rate-limit the key/user.
Reserve credit on the caller’s wallet (pessimistic hold sized from the request).
Route to a healthy channel and forward to CLIProxyAPI (SSE passthrough when stream: true).
Meter the response usage, price it, settle the wallet reservation, and append a usage-ledger row.
On upstream failure, fail over to the next channel and trip the circuit breaker on the bad one.

API surface

Method	Path	Purpose
`POST`	`/v1/chat/completions`	OpenAI-compatible chat completion (supports `stream: true` SSE)
`GET`	`/v1/models`	List the models the pool can serve
`GET`	`/healthz`	Liveness / readiness probe

Because the shape is OpenAI-compatible, point any OpenAI client at http://localhost:3113/v1 with the relay key as the API key.

Relay keys

Relay authenticates /v1 with relay keys, not Clerk sessions or PATs:

Format mr_rk_…; only a hash is stored (RelayApiKey), and the lookup is cached in Redis for the hot path.
Keys can be scoped/limited and revoked; a revoked or unknown key is rejected fail-closed — no upstream call, no billing.

Relay keys are secrets. They carry spend authority against a wallet — treat them like an API key, rotate on leak, and never log them (relay redacts upstream secrets from errors).

Billing

Relay meters real token usage and charges a credit wallet:

Reserve → settle: before forwarding, relay places a reservation on the CreditWallet; after the response completes it settles to the actual cost and releases the remainder. A failed request releases the full reservation.
Pricing: per-model rates come from the pricing service (kept in sync by keeper); cost is computed from prompt/completion tokens.
Ledger: each settled request appends a UsageLedger / CreditTransaction row for audit and reconciliation.

Insufficient balance fails the request before any upstream call, so the pool is never spent without a covering reservation.

Channels, failover & circuit breaking

The pool is modeled as a set of channels, each mapping to an upstream account/credential in CLIProxyAPI:

Router picks a healthy channel per request (health-aware, avoiding recently-failed ones).
Failover: on a retryable upstream error (including 401/403, which in this pool means the account is exhausted/blocked, and 5xx), relay advances to the next channel rather than surfacing the error.
Circuit breaker: repeated failures on a channel open its breaker so the router skips it; it half-opens after a decay window and recovers on success.

This is why a single bad account degrades gracefully instead of failing user requests.

Relationship to the rest of the system

CLIProxyAPI is the actual upstream account pool; relay talks to it per request over HTTP.
keeper is the background worker that ingests usage, mirrors the pool’s identities, aggregates stats, and keeps model pricing current — relay reads the prices keeper maintains.
platform-api owns the admin console that manages relay keys, channels, and the pool.

Relay owns the RelayApiKey and RelayChannel models; see Architecture & conventions for cross-service rules.