Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@

### Fixed

- **`session-budget-breaker` now accrues streamed turns, so its ceilings can fire on real traffic.** `onStreamEvent` read the session id from `ctx.headers`, but the stream ctx built in `proxy/stream.mjs` carries `{event, meta, telemetry, responseHeaders, drop}` and no request headers, so no streamed request ever accrued: the token, cost and rate levers all stayed at zero, and Claude Code streams every turn. It now reads the id `onRequest` stashed on the shared `meta`, as `onResponse` already did for `stream:false`. The suite had passed only because every stream fixture handed in `headers`; they now model the real ctx, and one test drives the shipped `streamResponse` end to end.

- **`claude-opus-5-5` is now priced, tracked for divergence, and no longer billed as `claude-opus-5`; `claude-sonnet-5-5` and `claude-haiku-5-5` are now priced too.** The model tables (`tools/rates.json`, `tools/update-rates.mjs`, `proxy/model-families.mjs`) had no entry for it, so `tools/cost-report.mjs`'s `lookupRates()` matched the first `rates.json` key in file order where either id prefixed the other — `claude-opus-5`'s rates, 25% too high on input/output and off by more than 2x on the cache-read discount the pricing page documents for this model — and bare `claude-opus-5` / `claude-sonnet-5` read as `"unknown"` in `proxy/model-families.mjs`, so a request served by a different family only latched `model_divergence_sticky` on the third divergent turn instead of the first. `claude-sonnet-5-5` (Claude Code's default Sonnet since 2.1.284) had no row in `tools/rates.json` or `tools/update-rates.mjs` either, so a refresh never wrote it and the session-budget-breaker's exact-id lookup counted no cost for it; both carry it now, and `tools/probe-tool-addition.mjs` probes it in place of `claude-sonnet-5`. `claude-haiku-5-5` (Claude Code 2.1.293's default Haiku) had no row either, and bare `claude-haiku-5` read as `"unknown"` in `proxy/model-families.mjs`. It is priced at the lower tier of the page's two ($0.10 input / $0.50 output per MTok for prompts up to 100,000 tokens), hand-written in `tools/rates.json` because the tiered row does not parse and a refresh keeps entries the parse does not find, so prompts over 100,000 tokens (which the page prices at $0.50 / $2.50) are under-counted. `claude-sonnet-5-5` cache reads are priced at $0.10 per MTok (0.05x input, as `claude-opus-5-5`'s are), not $0.20: Claude Code 2.1.296 and the pricing page as of 2026-10-10 both say so. `tools/update-rates.mjs` accepts that ratio for it from that page's date on, so a refresh no longer refuses the live row, while an older page's $0.20 still parses at its own date.

- **A refused fd-3 handover no longer makes the proxy claim it handed the socket on.** `inheritedSocket` was computed from "handover was attempted", not "handover succeeded", so a proxy that was refused fd 3 and fell back to binding its own port still advertised an inherited socket. On `SIGTERM` it then spawned a successor pointed at the same unservable descriptor and exited `75` — telling the supervisor a successor holds the socket — while the port it actually served was released with nobody on it. Exits `0` now, spawns nothing, and leaves no orphan.

- **A dead log reader no longer kills the supervisor or the relay.** The proxy gained an EPIPE swallower after a measured 27-minute outage; the launcher's holder and the gap relay share the same pipe and never got one, so the same killed reader took down the process whose whole job is to put the proxy back. Both now install stream `'error'` listeners, as does the proxy in reverse mode, where its own guard had been attached only when forward mode was active. The interactive wrapper deliberately keeps the old behaviour — a foreground producer whose consumer dies should end.
Expand Down
5 changes: 3 additions & 2 deletions proxy/extensions/session-budget-breaker.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -346,8 +346,9 @@ export default {
if (!ev || ev.type !== "message_start") return;
const split = usageInputTokens(ev.message && ev.message.usage);
if (split === null) return; // unparseable/missing → don't update (fail-open)
let sid;
try { sid = resolveSessionId(ctx.headers); } catch { return; }
// The stream ctx (proxy/stream.mjs) carries no request headers, so read
// the sid onRequest stashed on the shared meta, as onResponse does.
const sid = ctx.meta && ctx.meta._sbbSessionId;
if (!sid) return;
accrue(sid, split, ev.message && ev.message.model, ctx.responseHeaders);
} catch { /* fail-open: never let accounting throw affect the request */ }
Expand Down
3 changes: 3 additions & 0 deletions proxy/model-families.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,12 @@ export const MODEL_FAMILIES = [
{ root: "mythos", family: "mythos" },
{ root: "claude-opus-4-7", family: "opus" },
{ root: "claude-opus-4-8", family: "opus" },
{ root: "claude-opus-5", family: "opus" },
{ root: "claude-sonnet-4-6", family: "sonnet" },
{ root: "claude-sonnet-4-7", family: "sonnet" },
{ root: "claude-sonnet-5", family: "sonnet" },
{ root: "claude-haiku-4-5", family: "haiku" },
{ root: "claude-haiku-5", family: "haiku" },
];

// Family classification — substring-matched against the lowercased model
Expand Down
40 changes: 40 additions & 0 deletions test/cost-report.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";

// cost-report.mjs runs main() at import and does not export lookupRates(), so
// the only way to exercise its rates lookup end-to-end is to run it as a
// subprocess with usage telemetry piped on stdin.

const __dirname = dirname(fileURLToPath(import.meta.url));
const SCRIPT = join(__dirname, "..", "tools", "cost-report.mjs");

// Cost of 1M input + 1M output + 1M cache-read tokens on `model`, from the bundled rates.json.
function costOf(model) {
const usage = JSON.stringify({
model,
input_tokens: 1_000_000,
output_tokens: 1_000_000,
cache_read_input_tokens: 1_000_000,
}) + "\n";

const result = spawnSync(process.execPath, [SCRIPT, "--format", "json"], {
input: usage,
encoding: "utf8",
});

assert.equal(result.status, 0, `cost-report exited ${result.status}: ${result.stderr}`);
return JSON.parse(result.stdout).calls[0].cost;
}

test("prices a claude-opus-5-5 call from the bundled rates.json", () => {
assert.equal(costOf("claude-opus-5-5"), 24.2,
"1M input + 1M output + 1M cache-read at Opus 5.5's own rates (4/20/0.2 $ per MTok)");
});

test("prices a claude-sonnet-5-5 call from the bundled rates.json", () => {
assert.equal(costOf("claude-sonnet-5-5"), 12.1,
"1M input + 1M output + 1M cache-read at Sonnet 5.5's own rates (2/10/0.1 $ per MTok)");
});
2 changes: 1 addition & 1 deletion test/fixtures/pricing-page-2026-07-27.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions test/fixtures/pricing-page-2026-09-28.html

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions test/fixtures/pricing-page-2026-10-10.html

Large diffs are not rendered by default.

16 changes: 9 additions & 7 deletions test/gap-relay-chain.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,15 @@ import net from "node:net";
import { spawn } from "node:child_process";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
import { freePort } from "./proc-helpers.mjs";

const relayPath = join(dirname(fileURLToPath(import.meta.url)), "..", "bin", "gap-relay.mjs");

// A DEAD HOP IS A PORT NOBODY CAN BIND, not a freed one: the kernel can hand that
// number to the next call or the relay's carrier, and the relay folds a repeated
// hop into one and drops its own address, so two refusals traced as one (1 !== 2
// on CI). Below 1024 an unprivileged runner cannot bind; connecting refuses at once.
const DEAD1 = 1, DEAD2 = 2;

// An endpoint that records being reached and answers a CONNECT. Every one of
// them records, so a failure names which was touched instead of leaving an
// empty set — an assertion that fires on `[]` has already discarded the
Expand Down Expand Up @@ -144,9 +149,8 @@ const connectThrough = (port, target) => new Promise((resolve) => {
test("refuses rather than dialling direct when CACHE_FIX_REQUIRE_HOP=1", async () => {
const touched = [];
const origin = await endpoint("ORIGIN", touched);
const dead = await freePort();
try {
await withRelay(`http://127.0.0.1:${dead}`, async ({ port, stderr }) => {
await withRelay(`http://127.0.0.1:${DEAD1}`, async ({ port, stderr }) => {
const reply = await connectThrough(port, `127.0.0.1:${origin.port}`);
await new Promise((r) => setTimeout(r, 400));
assert.deepEqual(touched, [],
Expand Down Expand Up @@ -259,9 +263,8 @@ test("a refused first hop falls to the SECOND, not straight to a direct dial", a
const touched = [];
const origin = await endpoint("ORIGIN", touched);
const hop2 = await endpoint("HOP2", touched);
const dead = await freePort();
try {
await withRelay(`http://127.0.0.1:${dead},http://127.0.0.1:${hop2.port}`, async ({ port, stderr }) => {
await withRelay(`http://127.0.0.1:${DEAD1},http://127.0.0.1:${hop2.port}`, async ({ port, stderr }) => {
const reply = await connectThrough(port, `127.0.0.1:${origin.port}`);
await new Promise((r) => setTimeout(r, 300));
// Carrying VIA A HOP: the hop's own reply is piped straight back, so the
Expand All @@ -284,9 +287,8 @@ test("a refused first hop falls to the SECOND, not straight to a direct dial", a
test("direct is the LAST resort, reached only when no hop will carry", async () => {
const touched = [];
const origin = await endpoint("ORIGIN", touched);
const dead1 = await freePort(), dead2 = await freePort();
try {
await withRelay(`http://127.0.0.1:${dead1},http://127.0.0.1:${dead2}`, async ({ port, stderr }) => {
await withRelay(`http://127.0.0.1:${DEAD1},http://127.0.0.1:${DEAD2}`, async ({ port, stderr }) => {
const reply = await connectThrough(port, `127.0.0.1:${origin.port}`);
await new Promise((r) => setTimeout(r, 300));
// AND ON THE DIRECT PATH TOO, where the 200 is ours to write rather than
Expand Down
9 changes: 9 additions & 0 deletions test/model-families.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,15 @@ test("modelFamily: returns 'unknown' for unmatched / empty / non-string", () =>
assert.equal(modelFamily(123), "unknown");
});

test("modelFamily: bare Opus 5, Sonnet 5 and Haiku 5 ids, and the 5.5 point release, classify correctly", () => {
assert.equal(modelFamily("claude-opus-5"), "opus");
assert.equal(modelFamily("claude-opus-5-5"), "opus");
assert.equal(modelFamily("claude-sonnet-5"), "sonnet");
assert.equal(modelFamily("claude-sonnet-5-5"), "sonnet");
assert.equal(modelFamily("claude-haiku-5"), "haiku");
assert.equal(modelFamily("claude-haiku-5-5"), "haiku");
});

// --- Back-compat: cache-telemetry re-exports modelFamily ---

test("back-compat: `modelFamily` is also exported from cache-telemetry.mjs (one indirection)", async () => {
Expand Down
47 changes: 41 additions & 6 deletions test/proxy-session-budget-breaker.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@ import assert from "node:assert/strict";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { readFileSync, rmSync } from "node:fs";
import { Readable, Writable } from "node:stream";
import ext from "../proxy/extensions/session-budget-breaker.mjs";
import { streamResponse, createTelemetryRecord } from "../proxy/stream.mjs";

// Directive: docs/directives/proxy-subagent-budget-circuit-breaker.md
// Highest-value class = the fail-open table. A block requires gate `on` + a
Expand All @@ -12,9 +14,12 @@ import ext from "../proxy/extensions/session-budget-breaker.mjs";
const SID = "sess-abc";
const H = (sid = SID) => ({ "x-claude-code-session-id": sid });
// A message_start stream event carrying input+cache_creation tokens. `model` is
// optional (cost lever prices per-model; token/rate levers ignore it).
// optional (cost lever prices per-model; token/rate levers ignore it). The ctx
// mirrors proxy/stream.mjs processLine, {event, meta, telemetry, responseHeaders,
// drop}: NO request headers, so the session id rides on meta, where onRequest
// stashed it.
const start = (inp, cc = 0, model = undefined, sid = SID, responseHeaders = undefined) => ({
headers: H(sid),
meta: { _sbbSessionId: sid },
responseHeaders,
event: { type: "message_start", message: { model, usage: { input_tokens: inp, cache_creation_input_tokens: cc, output_tokens: 0, cache_read_input_tokens: 0 } } },
});
Expand Down Expand Up @@ -92,7 +97,7 @@ test("gate on, ceiling set, first request (no tally yet) → forward", async ()
test("unparseable usage does not update tally (metric-local fail-open)", async () => {
process.env.CACHE_FIX_SESSION_BUDGET = "on";
process.env.CACHE_FIX_SESSION_BUDGET_TOKENS = "10";
await ext.onStreamEvent({ headers: H(), event: { type: "message_start", message: { usage: { input_tokens: "not-a-number" } } } });
await ext.onStreamEvent({ meta: { _sbbSessionId: SID }, event: { type: "message_start", message: { usage: { input_tokens: "not-a-number" } } } });
const r = await ext.onRequest(req());
assert.equal(r, undefined, "unparseable input_tokens → tally not updated → forward");
assert.equal(ext.__testOnly.tally(SID), undefined);
Expand All @@ -101,7 +106,7 @@ test("unparseable usage does not update tally (metric-local fail-open)", async (
test("missing usage block → no update, forward", async () => {
process.env.CACHE_FIX_SESSION_BUDGET = "on";
process.env.CACHE_FIX_SESSION_BUDGET_TOKENS = "10";
await ext.onStreamEvent({ headers: H(), event: { type: "message_start", message: {} } });
await ext.onStreamEvent({ meta: { _sbbSessionId: SID }, event: { type: "message_start", message: {} } });
const r = await ext.onRequest(req());
assert.equal(r, undefined);
});
Expand Down Expand Up @@ -151,7 +156,7 @@ test("input + cache_creation both count toward the tally", async () => {
test("tally is per-session: session B is unaffected by session A's burn", async () => {
process.env.CACHE_FIX_SESSION_BUDGET = "on";
process.env.CACHE_FIX_SESSION_BUDGET_TOKENS = "100";
await ext.onStreamEvent({ headers: H("A"), event: { type: "message_start", message: { usage: { input_tokens: 500 } } } });
await ext.onStreamEvent({ meta: { _sbbSessionId: "A" }, event: { type: "message_start", message: { usage: { input_tokens: 500 } } } });
const rA = await ext.onRequest(req("A"));
const rB = await ext.onRequest(req("B"));
assert.ok(rA && rA.skip === true, "A over → blocked");
Expand Down Expand Up @@ -204,6 +209,16 @@ test("cost lever prices cache_creation at the higher cache_write_1h rate", async
assert.ok(Math.abs(ext.__testOnly.costUsd(SID) - 1.0) < 1e-9, `expected ~$1.00, got ${ext.__testOnly.costUsd(SID)}`);
});

// 100k input tokens at the 5.5 model's $/M (Sonnet $2, Haiku $0.10); an unpriced id would read $0.
for (const [model, usd] of [["claude-sonnet-5-5", 0.2], ["claude-haiku-5-5", 0.01]]) {
test(`cost lever prices ${model}, a default model, instead of failing open`, async () => {
process.env.CACHE_FIX_SESSION_BUDGET = "on";
process.env.CACHE_FIX_SESSION_BUDGET_COST_USD = "1";
await ext.onStreamEvent(start(100000, 0, model));
assert.ok(Math.abs(ext.__testOnly.costUsd(SID) - usd) < 1e-9, `expected ~$${usd}, got ${ext.__testOnly.costUsd(SID)}`);
});
}

test("cost lever UNDER the ceiling → forward", async () => {
process.env.CACHE_FIX_SESSION_BUDGET = "on";
process.env.CACHE_FIX_SESSION_BUDGET_COST_USD = "1";
Expand Down Expand Up @@ -311,11 +326,31 @@ test("a throw in onStreamEvent never propagates (fail-open)", async () => {
process.env.CACHE_FIX_SESSION_BUDGET = "on";
process.env.CACHE_FIX_SESSION_BUDGET_TOKENS = "100";
// event undefined → internal guards return; must not throw
await assert.doesNotReject(() => ext.onStreamEvent({ headers: H(), event: undefined }));
await assert.doesNotReject(() => ext.onStreamEvent({ meta: { _sbbSessionId: SID }, event: undefined }));
const r = await ext.onRequest(req());
assert.equal(r, undefined);
});

// --- Streaming accrual through the REAL stream.mjs ctx ---
// A hand-spelled onStreamEvent ctx once carried request `headers` that the real
// one never does, so every streamed turn accrued nothing while this suite stayed
// green. Drive the shipped streamResponse so the ctx shape is stream.mjs's own.

test("a streamed response accrues through the real stream ctx and trips the ceiling", async () => {
process.env.CACHE_FIX_SESSION_BUDGET = "on";
process.env.CACHE_FIX_SESSION_BUDGET_TOKENS = "100";
process.env.CACHE_FIX_SESSION_BUDGET_COST_USD = "1000";
const meta = { route: "messages" };
assert.equal(await ext.onRequest(req(SID, true, meta)), undefined);
const sse = `data: ${JSON.stringify(start(150, 0, "claude-opus-4-6").event)}\n\n`;
const sink = new Writable({ write(_c, _e, cb) { cb(); } });
await streamResponse(Readable.from([Buffer.from(sse)]), sink, createTelemetryRecord(), [ext], meta, {});
assert.equal(ext.__testOnly.tally(SID).tokens, 150, "token lever accrues");
assert.ok(ext.__testOnly.costUsd(SID) > 0, "cost lever accrues");
const r = await ext.onRequest(req(SID, true, { route: "messages" }));
assert.ok(r && r.skip === true, "the next streamed request must block");
});

// --- Non-streaming (stream:false) accrual: onResponse path ---
// Codex review r1 blocker: server.mjs routes non-streaming /v1/messages
// responses through onResponse only, so a breaker that learns spend from
Expand Down
Loading
Loading