XTokenHub

English | 简体中文

AI API aggregation gateway · Unified token management for all your models

Bring together upstream LLM provider APIs as channels, expose standardized protocol endpoints, and track token usage and cache hit rate in real time.

---

Core principle: pass through in the provider's native protocol whenever possible; conversion is only a fallback.

Dashboard	Channel Management
Model Usage	API Keys
macOS Menu Bar Companion

✨ Highlights

• 🔑 Unified token management — Gather all your scattered provider API keys into one panel: channel-based entry, one-click availability probing, enable/disable switching, and balance queries (e.g. DeepSeek). No more digging through config files for keys;
• 🗂 Model grouping — Each channel carries its own model list (one-click pull from upstream), and multiple upstreams are aggregated into a single /v1/models view. Route by priority / weighted random with automatic failover across channels serving the same model;
• 📊 Instant usage insight — The dashboard shows request count, token usage, cache hit rate, and average latency in real time, aggregated by model / channel / caller key, with a GitHub-style token activity heatmap and daily trend charts, all pushed live over WebSocket;
• 🔄 Protocol conversion — Exposes OpenAI (/v1/chat/completions, /v1/responses) and Anthropic (/v1/messages) endpoints simultaneously; when inbound and upstream protocols differ, requests are converted automatically. When they match, traffic is passed through natively with zero conversion, preserving tool calls, multimodal payloads, and other full capabilities;
• 💰 Cost tracking & spend forecast — Every request is priced instantly using the upstream's own usage convention (OpenAI and Anthropic bill cache differently), and the dashboard shows today's / this month's spend plus a projected month-end figure. The price table syncs from LiteLLM's public data in one click (~2,900 models) and accepts manual overrides, with optional USD/CNY display;
• 🧾 Gateway keys & per-caller stats — Issue sk-xt-* gateway keys to different clients and aggregate request count and tokens per key, so you always know who consumes the most;
• 🧠 Shared agent memory bank — Cross-session long-term memory: standalone /memory/v1 API + MCP tools, Chinese-friendly full-text search (FTS5 + n-gram, no vectors), namespace classification with one-click LLM classification in the console, plus a relationship graph view;
• 📦 Single-binary self-hosting — The frontend is embedded via go:embed; make build produces one static binary (zero CGO). Copy it to any Linux/macOS machine and run.

Tech stack: Backend Go + Gin + GORM + SQLite (pure-Go driver glebarez/sqlite) + Viper + gorilla/websocket; frontend React 19 + TypeScript + Ant Design 6 + Vite + Apache ECharts. Every module has unit tests — go test ./... -race all green, 87.6% statement coverage.

Architecture

cmd/server/          Entry: config -> DB -> event bus -> WS hub -> router -> graceful shutdown
internal/
  config/            Viper config (defaults <- yaml <- XT_HUB_ env vars)
  database/          GORM + SQLite init and AutoMigrate
  model/             Channel / RequestLog and protocol enums
  repository/        Data access interfaces + GORM impl (interface-based, services mockable)
  service/           Business logic: channel CRUD, protocol probing, logs, stats
  handler/
    admin/           Admin API (channels / logs / stats / WS)
    gateway/         Gateway endpoints (parse inbound request -> executor)
  gateway/           Orchestration: channel selection (native-first) + passthrough/convert + streaming + stats
  provider/          Upstream integration: endpoint resolution, native-protocol prober, converters, usage parsing, token estimation
  eventbus/          In-memory pub/sub bus (decouples business from push)
  ws/                WebSocket hub: connection management, heartbeat keep-alive, async broadcast, slow-consumer eviction
  middleware/        CORS / access log / recovery
  router/            Route registration + frontend SPA hosting (except /api, /v1)
  pkg/               resp unified response / errs error codes / pagination / logger
web/web.go           go:embed frontend bundle (web/dist)
frontend/            React admin source (dashboard / channels / request log / model usage)
internal/tests/      All unit tests (mirrors module layout, external test packages)

Gateway Workflow

1. Create a channel: enter provider BaseURL / API key / model list (or click "pull from upstream" to fetch the real model list). The API style (OpenAI-compatible Bearer / Anthropic-compatible x-api-key) is auto-detected from the BaseURL (hostname or path containing "anthropic" means Anthropic style) and can be overridden. Create one channel per protocol endpoint of the same provider — e.g. DeepSeek: OpenAI face https://api.deepseek.com, Anthropic face https://api.deepseek.com/anthropic (the gateway assembles .../anthropic/v1/messages). BaseURL supports three mounting forms: bare domain (endpoints get /v1/), /v1 suffix (equivalent to bare domain), sub-path mount (e.g. /anthropic, assembles /v1/messages), and versioned mount (e.g. Zhipu https://open.bigmodel.cn/api/paas/v4, assembles .../v4/chat/completions directly without duplicating the version segment). The model list prefers the channel's own .../models; if unavailable (e.g. DeepSeek sub-paths expose no list endpoint), it falls back to the main domain /v1/models;
2. Native protocol probing: send a minimal request (max_tokens=1) to each protocol endpoint; a 2xx marks it a native protocol of the channel (401/404/5xx don't count; native_protocols can be corrected manually). Bare-domain BaseURLs probe all three protocols (to detect multi-protocol relays); sub-path BaseURLs (like DeepSeek's https://api.deepseek.com/anthropic) are treated as protocol mounts and only probe that provider's protocol family. If no probe model is specified, a real one is picked automatically: upstream's own model list -> channel config models -> main-domain fallback list -> built-in fallback, avoiding false verdicts on nonexistent models;
3. Request routing: filter enabled channels by model -> prefer native channels (zero-conversion passthrough) -> only fall back to protocol conversion if no native channel exists -> priority ascending + weighted random by weight -> automatic failover on failure (network errors / 401 / 403 / 408 / 429 / 5xx);
4. Metering & stats: parse upstream usage first (stream_options.include_usage is added automatically for OpenAI streaming; Anthropic uses message_start/message_delta events); when upstream doesn't report usage, a local heuristic estimation kicks in. Cache hit rate is computed from cached_tokens (OpenAI) / cache_read_input_tokens (Anthropic); every request is written to request_logs and pushed to the frontend in real time via WebSocket.

Protocol conversion matrix (fallback path; v1 focuses on text chat + common sampling params; tool calls / multimodal are only guaranteed on the passthrough path):

Inbound \ Upstream	openai_compatible	anthropic
chat_completions	native passthrough	convert
responses	convert	convert
messages	convert	native passthrough

Converting upstreams don't support the responses protocol (the OpenAI Responses API is semantically heavy; native channels take priority).

WebSocket Real-time Push

• Endpoint: GET /api/v1/ws; message envelope {type, payload, ts}
• Events: request.completed (every gateway request), stats.updated, channel.probe_result, channel.status_changed, channel.balance_updated
• Business logic only publishes to the eventbus; the WS hub subscribes and broadcasts — business and push are fully decoupled; server-side ping/pong keep-alive, automatic eviction of slow consumers
• Frontend src/api/ws.ts: exponential backoff reconnection (starting at 1s, capped at 30s), subscribe by type, connection state drives the header breathing light

Dashboard

• Today scope: top stat cards count requests/tokens/cache hit rate for "today" (from local midnight, via the since parameter) rather than a rolling 24h window;
• Lifetime cards: total tokens, peak tokens (single-day high), longest chat duration (single successful request), current/longest consecutive usage days (GitHub-streak style, derived from per-day aggregation);
• Token activity heatmap: GitHub-style calendar (last 26 weeks, aggregated by local day), with daily / weekly / cumulative coloring modes, shown alongside "Top callers · 30 days" (requests and tokens aggregated by gateway key);
• Daily token trend: switch between last 7 / 30 days, smoothed multi-model curves (day × model aggregation, top 6 + others);
• Model usage donut: same data source and palette as the trend chart, total tokens for the range in the center, legend with share and usage;
• Chart interactions: hovering the trend chart shows crosshairs and per-model breakdowns for the day; hovering the donut boldens the sector, switches the center to its share, and highlights the legend; hovering the heatmap shows that day's tokens;
• Real-time request stream: rows are merged into sessions by caller (X-Session-Id inbound header, stored as session_id), summarizing request count / tokens / cache hit / latency; click a row's arrow to expand request-level details. Callers without a session header stay on a single row; each row has a "view headers" action showing the full inbound request headers (request_headers).

Cost Tracking & Spend Forecast

• Per-request cost: every request is priced the moment the upstream response arrives and stored with the log row (cost_usd), visible in both the request log and the dashboard. Historical rows are never rewritten when prices change; use "recompute historical cost" in Settings when you want them recalculated.
• Spend cards: the dashboard shows today's spend, this month's spend, and this month's projected spend; the model and caller leaderboards carry cost too.
• End-of-period forecast: with ≥ 3 complete days of history, the remaining period is projected at the trailing 7-day daily average; otherwise it falls back to linear extrapolation from this period's burn rate. When the period just started (under 10 minutes) or nothing has been spent yet, no forecast is shown and the reason is stated instead of guessing from a tiny sample. Setting a monthly budget additionally yields a projected date of hitting it.
• Price table: search, add, edit and delete per-model rates in Settings, or sync the LiteLLM public price table in one click (~2,900 models, USD per token, including cache read/write and long-context tiers). Rows you edit are marked "manual" and survive later syncs.
• Unpriced warning: models with usage but no price are listed separately (their cost counts as 0); add a rate and hit "recompute historical cost" to backfill.
• Multi-currency: each price row is denominated in either CNY or USD. CNY rows are converted to USD at your configured rate before being recorded, so cost_usd stays a single USD figure; the display currency can be switched to CNY.
• Off-peak (time-of-day) rates: a price row can declare its peak window, and requests outside it are billed at the row's off-peak rates — so the "half price during off-peak hours" pricing common among Chinese model providers (e.g. DeepSeek) is recorded faithfully.

Pricing semantics

Input tokens are split according to the upstream protocol's convention. The two conventions must not be mixed, or requests will be double- or under-billed:

• OpenAI-style (chat_completions / responses): prompt_tokens already includes cache hits, so uncached input = prompt − cached, and cache writes are not billed separately.
• Anthropic-style (messages): input_tokens excludes cache reads and writes, so cache reads and cache writes are billed on their own.

This is why cost is computed and persisted at gateway write time: the stored row only carries the inbound protocol, so after a protocol conversion (e.g. a chat inbound request served by a messages upstream) the upstream convention can no longer be recovered. Two columns record what was used: usage_style (the convention) and price_period (peak / off_peak), both for auditing and recompute.

Rates: public tables quote per million tokens while the database stores per single token; the Settings form takes per-million values. A Chinese provider's "cache hit" price is this project's cache read rate (e.g. DeepSeek: ¥0.04/M peak, ¥0.02/M off-peak). Leaving cache read at 0 falls back to the input rate (matching the public table's own behaviour), which markedly overestimates cost when the cache hit rate is high — cached tokens get billed at the much pricier input rate. The Settings editor warns about this explicitly.

Currency and off-peak rates

Every price row carries a currency: USD (always the case for synced rows) or CNY. A CNY row is converted to USD using the USD→CNY rate under "Billing & display" before being folded into cost_usd, so all aggregation, forecasting and leaderboards remain single-currency USD. When no rate is configured (0), CNY rows cost 0 (treated as unpriced, rather than producing a number in the wrong unit); Settings warns about this, and "recompute historical cost" backfills once a rate is set.

Off-peak pricing is expressed as the row's peak window (everything else is off-peak) and is configured per model:

<weekdays>;<span>[,<span>...]

• Weekdays: 1=Monday … 7=Sunday, supporting ranges and lists — 1-5, 1,3,5, 6-7. Two holiday tokens join them (case-insensitive): !H — dates inside a Chinese statutory-holiday block (the whole State Council rest period, shifted rest days included) are off-peak all day; W — shifted workdays (weekend days worked to make up a holiday) count as matching the weekday set.
• Spans: HH:MM-HH:MM, comma-separated; the end must be later than the start and must not wrap past midnight (off-peak is expressed by listing the peak windows, so wrapping is never needed).
• Evaluation is always in Beijing time (UTC+8), independent of the server timezone.

DeepSeek's published rule (weekdays 09:00–12:00 and 14:00–18:00 are peak; holidays, weekends included, are off-peak all day) is:

1-5,!H;09:00-12:00,14:00-18:00

The !H / W tokens are judged against a holiday calendar synced from a public Chinese holiday API (each year's holiday/shifted-workday dates, kept in the holiday_dates table and refreshed on a timer — see holiday.* below). Rules without the tokens behave exactly as before; when the calendar holds no data yet, the tokens fall back to plain weekday judging.

Settings' price editor has an "apply DeepSeek template" button for this. Leaving an off-peak rate at 0 falls back to the peak rate (the period is still recorded).

Known limitations: long-context pricing supports a single tier (once a threshold is set, a prompt above it reprices the whole request at the above-threshold rates; multi-tier upstream pricing is approximated by its first tier), and off-peak rates apply only to the four base rates, not to the above-threshold tier; the public price table has USD fixed prices only and no time-of-day data, so off-peak and CNY pricing can only come from manually configured rows; requests whose token counts were locally estimated because the upstream reported no usage carry an estimated cost too; models absent from the price table cost 0.

API

Admin endpoints (/api/v1, no login auth yet):

GET    /api/v1/ws                     WebSocket real-time push
GET    /api/v1/channels               Channel list (paginated)
POST   /api/v1/channels               Create channel
POST   /api/v1/channels/lookup-models Pull upstream model list {provider, base_url, api_key}
GET    /api/v1/channels/balances      Batch channel balance query (provider detected by BaseURL host;
                                      currently DeepSeek only, 5-minute TTL cache + WS push)
GET    /api/v1/channels/:id
PUT    /api/v1/channels/:id
DELETE /api/v1/channels/:id
POST   /api/v1/channels/:id/probe     Trigger native protocol probe {model?} (auto-picked from upstream model list by default)
GET    /api/v1/keys                   Gateway key list (paginated)
POST   /api/v1/keys                   Create key (server generates sk-xt-*; response contains the plaintext once)
PUT    /api/v1/keys/:id               Rename / enable-disable / note (key body is immutable)
DELETE /api/v1/keys/:id
GET    /api/v1/logs                   Request logs (protocol/forward_mode/channel_id/key_id/model/stream/hours/error_only filters)
GET    /api/v1/logs/cleanup           Log cleanup status (enabled/retention days/last run info)
POST   /api/v1/logs/cleanup           Trigger a cleanup manually (returns deleted rows/duration/cutoff)
GET    /api/v1/stats/summary?hours=24 Summary (requests/tokens/cache hit rate/avg latency/passthrough share;
                                      pass since=<unix seconds> for an explicit window start; the frontend
                                      passes local midnight for the "today" scope)
GET    /api/v1/stats/trend?days=7     Daily trend (days up to 366, for heatmap/long ranges)
GET    /api/v1/stats/by-model?hours=24
GET    /api/v1/stats/by-channel?hours=24
GET    /api/v1/stats/by-key?hours=24  Aggregate by caller key (requests/tokens/cache hit rate)
GET    /api/v1/stats/lifetime         Lifetime totals (total tokens/peak day/longest single duration/consecutive days)
GET    /api/v1/stats/trend-by-model?days=7 Day × model token usage (multi-model trend lines)
GET    /api/v1/stats/cost/forecast?period=today|month Spend forecast (spent / projected /
                                      basis / confidence; adds projected_exceeded_date
                                      when a monthly budget is configured)
GET    /api/v1/stats/cost/unpriced?hours=720 Models with usage but no price entry
POST   /api/v1/stats/cost/recompute   Recompute historical cost {from?,to?,only_missing?}
GET    /api/v1/settings/prices        Price table (paginated, q fuzzy search, used_only filter;
                                      returns sync status / unpriced models / billing settings)
POST   /api/v1/settings/prices        Add a manual price (rates are USD per single token)
POST   /api/v1/settings/prices/sync   Sync the public price table now (manual rows are kept)
PUT    /api/v1/settings/prices/:id    Edit a price (marks the row manual so sync stops overwriting it)
DELETE /api/v1/settings/prices/:id
GET    /api/v1/settings/billing       Billing display settings (currency / rate / monthly budget)
PUT    /api/v1/settings/billing
GET    /api/v1/settings/holidays      Holiday calendar for a year (?year=YYYY) + sync status
POST   /api/v1/settings/holidays/sync Sync the holiday calendar now
GET    /healthz

Gateway endpoints (OpenAI / Anthropic clients connect directly, API key enforced by default):

GET  /v1/models            Aggregated model list of all enabled channels (OpenAI spec format,
                           plus Anthropic's type/display_name fields for dev-tool recognition)
POST /v1/chat/completions
POST /v1/responses
POST /v1/messages

Gateway auth & per-caller stats

• Create keys on the "API Keys" page (sk-xt- + 32 random hex chars); auto-copied to clipboard on creation; supports enable/disable and delete (historical logs keep the key-name snapshot)
• Caller auth: Authorization: Bearer sk-xt-... (OpenAI-compatible) or x-api-key: sk-xt-... (Anthropic-compatible); missing/wrong/disabled keys return 401 (formatted per the inbound protocol)
• Every request log records key_id/key_name; /stats/by-key and the log page's "caller" filter let you aggregate token usage per key
• Toggle: gateway.require_key (yaml) or XT_HUB_GATEWAY__REQUIRE_KEY=false to disable enforced auth (anonymous calls are accounted under no key)

# Point an OpenAI SDK at the gateway
curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer sk-xt-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Hello"}]}'

# Point an Anthropic SDK at the gateway (x-api-key)
curl http://localhost:8080/v1/messages \
  -H "x-api-key: sk-xt-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-3-5-sonnet","max_tokens":64,"messages":[{"role":"user","content":"Hello"}]}'

Channel Balance Queries

• The balance column on the "Channel Management" page: the provider is detected from the channel's BaseURL host; a known provider is queried with the channel's own API key (GET /api/v1/channels/balances returns in batch);
• Currently supports DeepSeek only (official GET /user/balance, showing total/grant/top-up breakdown, CNY/USD); other channels show "—";
• Detection uses only the host (e.g. api.deepseek.com) — paths are ignored to avoid misjudging relays whose path mentions a provider name; BaseURLs with /v1 or /anthropic mount segments are stripped to the main domain root for the query;
• Results carry a 5-minute TTL cache (short 1-minute cache on failure); successful fetches refresh live via WS channel.balance_updated; updating/deleting a channel invalidates the cache;
• Zhipu Coding plans (window percentage), Xiaomi MiMo (Cookie auth only), and OpenCode Go/Zen (no public API) are not yet supported — see "Known Limitations".

Agent Memory Bank

A shared long-term memory for every AI agent talking to the gateway: write down conclusions, decisions, conventions and lessons worth keeping across development/execution tasks — any agent can recall them later. Zero vector models; plain SQLite (FTS5 + an app-side CJK n-gram channel).

• Standalone memory API (/memory/v1/*, same auth headers as the gateway): add / search / update / soft-delete / relation links / graph / stats; same-content writes within a namespace are deduplicated (duplicated: true); supersedes retires old entries without deleting them;
• Namespace classification: writers pick the namespace explicitly (auto-created when new), or leave it uncategorized; the console's one-click classifier routes uncategorized entries into namespaces using a configurable channel + model (LLM judgment, no embeddings);
• Chinese-friendly full-text search: FTS5 plus an app-side n-gram channel (overlapping CJK bigrams + ASCII words); 2-character Chinese queries match reliably, single characters fall back to LIKE; ranking blends BM25 relevance × recency × importance × access boost (hits raise their own heat); results carry per-namespace hit facets;
• MCP endpoint: POST /mcp (Streamable HTTP, stateless) exposing memory_add / memory_search / memory_get / memory_update / memory_forget / memory_link / memory_namespaces;
• Console "Memories" page: list (search / filters / batch-move / pin / delete) + relationship graph (SVG force-directed, nodes colored by namespace, edges typed related/supersedes/conflicts/derived_from) + namespace management (rename / merge / delete / describe).

Writing and searching

# Add (explicit namespace; duplicate content is idempotent with duplicated=true)
curl -s $BASE/memory/v1/memories \
  -H "Authorization: Bearer sk-xt-..." -H 'Content-Type: application/json' \
  -d '{"content":"Payment callbacks must be idempotent; replayed notices return success.",
       "namespace":"payment","kind":"convention","tags":["payment"],"importance":4}'

# Search (CJK substrings match; returns a blended score plus namespace facets)
curl -s "$BASE/memory/v1/memories/search?q=idempotent" -H "Authorization: Bearer sk-xt-..."

Mount the MCP in Claude Code

claude mcp add --transport http xtokenhub-memory http://127.0.0.1:9192/mcp \
  --header "Authorization: Bearer sk-xt-..."

See memory.* in configs/config.yaml (switch, auth, result limits, classifier batch); pick the classifier channel + model on the console's "Memories → Classification Settings" (channels speaking chat_completions or messages).

Quick Start

# Backend + frontend in one build (single binary with embedded frontend)
make build          # -> bin/xtokenhub
./bin/xtokenhub -config configs/config.yaml

# Dev mode (backend 8080 + Vite 5173 proxy)
make dev

# Build only the frontend bundle into web/dist
make web

Config precedence: XT_HUB_* env vars > configs/config.yaml > built-in defaults (e.g. XT_HUB_SERVER__PORT=9090).

SQLite data lands in data/xtokenhub.db (WAL mode, single writer connection).

Deploying to a Linux device (systemd, start on boot)

deploy/ ships a systemd unit, an environment-file example, and an installer for running the service as a boot-enabled daemon instead of nohup. The artifact is a fully static single binary with no runtime dependencies.

# One shot: cross-compile linux/arm64 + upload + install unit + enable at boot
make deploy-pmos XT_HOST=root@192.168.1.110

# Password-auth devices (sshpass), and write the outbound proxy along the way
make deploy-pmos XT_HOST=root@192.168.1.110 XT_SSHPASS=xxx XT_PROXY=http://127.0.0.1:7890

Manual install works too (the script is idempotent — re-running it is an upgrade):

make dist-pmos                     # -> dist/xtokenhub-pmos-aarch64.tar.gz
scp dist/xtokenhub-pmos-aarch64.tar.gz deploy/{xtokenhub.service,xtokenhub.env.example,install.sh} <device>:/tmp/
# On the device:
mkdir -p /tmp/d && tar xzf /tmp/xtokenhub-pmos-aarch64.tar.gz -C /tmp/d
XT_ROOT=/path/to/XTokenHub XT_BIN=/usr/local/bin/xtokenhub XT_PROXY=http://127.0.0.1:7890 \
  sh /tmp/install.sh /tmp/d/xtokenhub-pmos-aarch64

The installer stops the old nohup process to free the port (matching only processes whose exe points at xtokenhub), atomically replaces the binary, then installs and enables the unit. If a local mihomo service exists, it adds a startup ordering dependency on it.

Highlights of deploy/xtokenhub.service:

Setting	Why
WorkingDirectory	Points at the deploy root; configs/config.yaml and database.path are relative paths, so this must be correct
EnvironmentFile=-/etc/default/xtokenhub	Injects the outbound proxy and XT_HUB_* overrides; the - prefix means a missing file is not an error
Restart=on-failure	Auto-restarts on abnormal exit (systemctl stop counts as a clean exit and does not)
StandardOutput=journal	Logs go to journald instead of an ever-growing nohup.out
ProtectSystem=strict + ReadWritePaths=.../data	Read-only filesystem except data/; the program only writes SQLite

Price-table sync runs once on first start (when the table is empty). If the device cannot reach GitHub directly and needs a proxy, check reachability first (a failed sync does not block startup, but costs stay at 0):

curl -sS -o /dev/null -w '%{http_code}\n' https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json

Common operations:

systemctl status xtokenhub            # status
journalctl -u xtokenhub -f            # live logs (replaces tail -f nohup.out)
systemctl restart xtokenhub           # restart
systemctl is-enabled xtokenhub        # start on boot?

Log retention cleanup

request_logs is the only table that grows forever (one row per request). The service cleans up every 24 hours by default, deleting logs older than max_days (default 90). You can also click "Clean expired logs" on the "Request Logs" page, or trigger manually via POST /api/v1/logs/cleanup (returns "cleanup in progress" if one is already running).

Setting	Default	Description
retention.enabled	true	Automatic background cleanup; manual trigger still works when off
retention.max_days	90	Keep the most recent N days of logs (>0)
retention.interval_hours	24	Cleanup cycle (hours)
retention.batch_size	1000	Rows deleted per batch (>=100, batched to avoid holding the single writer connection)
retention.vacuum	false	Run VACUUM after cleanup to reclaim disk (exclusive lock; enable at off-peak hours)

Env override example: XT_HUB_RETENTION__MAX_DAYS=30.

Note: the SQLite file doesn't shrink automatically after cleanup (deletion only frees pages); enable vacuum or run VACUUM manually. Lifetime dashboard stats are bounded by the retention period; the heatmap/daily trends show history within their display ranges. Cost recompute is bounded the same way (90 days by default).

Cost-related configuration

Key	Default	Description
pricing.enabled	true	Enable cost tracking and forecasting; when off, no price table is fetched and cost stays 0
pricing.auto_sync	true	Sync the public price table on a timer
pricing.sync_interval_hours	24	Sync interval in hours (>=1)
pricing.source_url	empty	Price table URL; empty uses the built-in LiteLLM public table
pricing.timeout_seconds	20	Price table fetch timeout in seconds (>=1)
holiday.enabled	true	Enable the holiday calendar (the !H / W peak-window tokens judge against it)
holiday.auto_sync	true	Sync the holiday calendar on a timer
holiday.sync_interval_hours	168	Sync interval in hours (>=1); new yearly arrangements are picked up within a week of publication
holiday.source_url	empty	Holiday API URL with a {year} placeholder (appended as a path segment when absent); empty uses the built-in holiday-cn calendar (GitHub raw, sourced from State Council notices; timor.tech format is also parsed)
holiday.timeout_seconds	20	Holiday calendar fetch timeout in seconds (>=1)
billing.display_currency	USD	Default display currency, USD / CNY (seeded on first init only; Settings wins afterwards)
billing.usd_cny_rate	0	USD→CNY rate, maintained by hand (required to display CNY, and to convert CNY-priced model rows)
billing.monthly_budget_usd	0	Monthly budget; 0 = none. Only drives the forecast overrun hint, never blocks requests

The price table is synced once on first start (when empty). A failed sync is only logged and startup continues with cost at 0; hit "sync now" in Settings to retry.

macOS Menu Bar Companion (XTokenHubMenuBar)

XTokenHubMenuBar/ is a standalone SwiftUI app (iStat Menus-style, built on macOS 26 Liquid Glass) that shows gateway token usage and the live request stream in the system menu bar. It only consumes this project's admin API and WebSocket — zero backend changes:

brew install xcodegen
cd XTokenHubMenuBar && make run

Testing

Tests live in internal/tests/<module>/ (external test packages, exported APIs only), decoupled from the implementation directory:

Module	Approach
pkg / config	Table-driven + yaml/env loading
database / repository	Real :memory: SQLite (CRUD/pagination/aggregation/concurrency)
eventbus	Multiple subscribers, unsubscribe, panic isolation
ws	Real gorilla dials: broadcast, heartbeat, slow-consumer eviction
provider	httptest fake upstreams: prober verdicts, model-list fetching (tolerant parsing), conversion matrix, streaming usage parsing
gateway	Channel selection (native-first/weight/failover), passthrough non-rewriting, conversion streams, stats persistence
service / handler / router	httptest end-to-end: CRUD, probing, gateway 400/503/502, SPA fallback

make test       # go test ./... -race
make cover      # coverage HTML (currently 87.6%)
cd frontend && pnpm test:run   # Vitest (WS reconnect/backoff, formatting, data transforms)

Known Limitations

• The conversion path supports text chat only (tool calls, multimodal, cache_control and other complex payloads require native passthrough channels);
• Balance queries currently support DeepSeek only (official API); Zhipu Coding plan usage is an undocumented API (returns window percentage), Xiaomi MiMo balance requires web-cookie auth that expires in about a day, and OpenCode Go/Zen expose no public balance API — none are integrated;
• Local token estimation is heuristic (CJK ≈ 1.5 chars/token, Latin ≈ 4 chars/token), only as a fallback when upstreams don't report usage;
• Gateway API keys are supported (auth + per-caller stats, see above); the admin /api/v1 still has no login auth — for self-hosted intranet use, apply your own network isolation. Keys are stored in plaintext (same as provider keys);
• Cache hit rate and per-key aggregation are based on RequestLog snapshots; after a key is deleted, its historical usage remains under its name;
• Cost is a local estimate from published list prices, based on the gateway's own metering; it may differ from your upstream bill (cache billing conventions, tiered pricing, batch discounts, granted credits, etc.). When a price row has no cache-hit rate, cached tokens are billed at the input rate, overestimating cost when the cache hit rate is high. Long-context pricing supports a single tier only; CNY rows depend on a hand-maintained FX rate and cost 0 when none is set; off-peak windows are evaluated in Beijing time and cannot wrap past midnight; requests whose token counts were locally estimated carry an estimated cost too.

License

Released under the MIT License.