Skip to content

clientd: embeddings and video generation routes (operator-enabled models) - #21

Open
Legatia wants to merge 4 commits into
ethereum:mainfrom
Legatia:clientd-media-endpoints
Open

Legatia wants to merge 4 commits into
ethereum:mainfrom
Legatia:clientd-media-endpoints

Conversation

@Legatia

@Legatia Legatia commented Oct 4, 2026 •

Copy link
Copy Markdown

Client side of #20.

Problem

zkapi-clientd exposes only chat completions and the model list. OpenRouter
also offers embeddings and an asynchronous video API, and clients of the
daemon need both. Video differs from chat: a job runs for minutes, and
polling and downloading need the key that submitted it.

Change

New local routes, mirroring OpenRouter's shapes:

  • POST /v1/embeddings, GET /v1/embeddings/models
  • POST /v1/videos (202, local job ID), GET /v1/videos/{id},
    GET /v1/videos/{id}/content?index=N, DELETE /v1/videos/{id},
    GET /v1/videos/models

Model gate: a model is usable on an endpoint only if the issuer's policy
gives it an explicit reviewed tier and OpenRouter's public catalog for that
endpoint lists it. There is no untiered fallback and no :online
normalization. The current policy enables none, so every request returns
400 model_not_enabled and the model lists are empty.

Structure:

  • internal/zkapi/client.go: Complete becomes forward(endpoint), shared
    with embeddings. waitForLease takes the endpoint for its budget lookup;
    fresh endpoints never reuse or cache a key.
  • internal/zkapi/media.go: endpoint table, endpoint-scoped budget,
    MediaModels, Embeddings.
  • internal/zkapi/videos.go: video job lifecycle.
  • internal/server/media.go: optional MediaBackend interface, routing,
    field allowlists. server.go splits chat forwarding into post and relay,
    which the new routes reuse; chat behaviour is unchanged.
  • cmd/zkapi-clientd/media.go: zkInference media methods and safe error
    mapping.

Requests keep chat's rules: keyless loopback or local API key, browser
origins rejected, 16 MiB bodies, field allowlists (drops user,
session_id, trace, routing preferences; video also drops callback_url
and previous_job_id, keeps provider.options), the 402 funding_required
and wallet error mapping, no retries, and fixed log route labels
(/v1/videos/{id}) without IDs.

Lease handling

  • Embeddings follow chat exactly, including the reuse window: a compatible
    cached key may serve both, as documented for chat.
  • A video submission takes a fresh lease (a cached chat key is retired
    first). The job then owns the wallet's single request slot until it fails,
    is cancelled or expires upstream, all outputs are downloaded completely,
    the finished job is deleted, or the key's usable lifetime ends (a timer).
  • On release, the reuse group ends now and background settlement retires
    the key through the existing /wallet/settle path.
  • Chat and embeddings wait for the slot as they do behind a stream; a video
    submission made while another job is active or being submitted returns
    409 video_job_active. A submission claims that state under the same lock
    as the active-job check, so two concurrent submissions start one job.
  • A submission whose local caller disconnects is still read to completion
    and tracked, so a created job is not orphaned. Submission is never retried.
  • The daemon never follows provider-supplied unsigned_urls. Content is
    fetched only from {inference base}/videos/{job}/content on the
    configured origin.
  • Job records live in memory only (kept for one hour, at most 64). They hold
    no prompt or video, and the provider job ID and key never leave the
    client. The Go side persists no lease state today; companion recovery
    (Recover interrupted OA leases in server and CLI #12) still covers a restart mid-job.

Client-only vs operator-dependent

Client-only (this PR): the routes, model gate, lease hold and release,
error mapping, docs.

Operator or server, tracked in the proposal issue:

  • policy entries, ideally with a kind field;
  • tiers above $6 (client, companion and org ceiling);
  • lease lifetime for long jobs (TTL is server-wide, 5 min);
  • settlement grace for cost posted after key expiry;
  • confirmation that OpenRouter enforces the key limit against a job's full
    price;
  • no workspace-level callback_url on the org account.

Embeddings need only policy entries.

Tests

internal/zkapi/media_test.go (fake companion and OpenRouter, synctest):

  • embeddings success with lease allowance and key;
  • model_not_enabled for a chat model, unpriced, disabled and uncatalogued
    models, with no lease and no provider call;
  • 402 before any provider call;
  • media model lists;
  • video submit, poll and content: the same key on every job call, chat
    blocked meanwhile, no settlement until the download completes, then
    settlement and a fresh chat key;
  • failure releases;
  • expiry releases at the lease deadline;
  • incomplete download holds until expiry, then 410;
  • one active job and the delete rules;
  • two concurrent submissions start one job, and the other gets 409;
  • a fresh key rather than the reusable chat key;
  • a rejected submission releases its key;
  • no prompt or model crosses the wallet bridge.

internal/server/media_test.go:

  • field allowlists;
  • malformed requests and index bounds never reach the backend;
  • routing, 405 and 404;
  • non-video content rejected;
  • safe upstream error mapping with Retry-After;
  • backend without media support returns 404;
  • keyless loopback access and browser-origin rejection;
  • log labels without job IDs or bodies.

cmd/zkapi-clientd/media_test.go, media_e2e_test.go: error mapping
(invalid_model names the media model lists), plus
an end-to-end run through the real local API, mapping and client (funding
required, model gate, full video lifecycle, logs without prompts, IDs or
keys).

Verification

In zkapi-clientd, the same commands as CI:

go build ./...        ok
go vet ./...          ok (also GOOS=linux)
gofmt -l .            no output
go test -race ./...   all packages ok

The new tests passed 10 consecutive -race runs. A trial merge with #19
(clientd-scriptable-funding) has no conflicts, and build and tests pass on
the merged tree. No live provider or wallet calls were made.

Out of scope

  • server, companion or policy changes;
  • tiers above $6;
  • longer or renewable leases;
  • concurrent video jobs;
  • webhooks (callback_url);
  • job continuation (previous_job_id);
  • filtering media models out of GET /v1/models;
  • other OpenRouter endpoints (rerank, speech, images).

A generic /v1/* passthrough was considered and not chosen: each endpoint
needs its own field allowlist, catalog scope and response type, and async
endpoints need lease ownership. The synchronous path is shared instead
(forward(endpoint)), so another synchronous endpoint is a table entry plus
an allowlist.

Follow-up after #19 merges: zkInference.Complete should use walletError,
and walletError should take #19's updated funding_required /
withdrawal_pending messages (until then the media routes still say "Run
zkapi-clientd config to add funding"). This PR keeps its own copy so it stays
independent of #19.

Merge notes: main + #19 + #21 merge cleanly in either order. With #16 (Tor
timeouts) there is one trivial conflict in model_budget.go: this PR moves
publicModelIDs' timeout into catalogIDs, so #16's 60 s applies there.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JZTNAFcTA7BQNUu6U83Gdc

Tobd33 and others added 2 commits October 4, 2026 23:23
Add local routes for OpenRouter embeddings and asynchronous video jobs:
POST /v1/embeddings, POST /v1/videos, GET /v1/videos/{id},
GET /v1/videos/{id}/content, DELETE /v1/videos/{id}, and
GET /v1/{embeddings,videos}/models.

Models must have an explicit tier in the issuer's model policy and appear
in OpenRouter's catalog for that endpoint; there is no untiered fallback.
The live policy enables none, so the routes stay inert until the operator
enables models.

Embeddings share chat's lease, reuse and no-retry rules. A video job takes a
fresh lease and owns the wallet's single request slot until the job fails,
its outputs are downloaded, it is deleted, or the key's usable lifetime
ends; settlement then retires the key as usual. Job state is in memory
only, holds no prompt, and the provider job ID and key never leave the
client. callback_url, previous_job_id and identity fields are dropped, and
provider-supplied content URLs are never followed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JZTNAFcTA7BQNUu6U83Gdc
Describe the new routes, that they need operator-enabled models, how a
video job holds the wallet's lease, and the provider's retention of
generated video.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JZTNAFcTA7BQNUu6U83Gdc
Legatia and others added 2 commits October 8, 2026 07:04
Two submissions that both arrived while the request slot was busy passed
the active-job check, and the second then started its own paid job as soon
as the first job released the lease. A submission now claims a flag under
videoMu together with that check, so a concurrent one gets 409
video_job_active. The docs also list cancelled and upstream-expired jobs
among the release conditions.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JZTNAFcTA7BQNUu6U83Gdc
invalid_model on /v1/embeddings and /v1/videos now names
/v1/embeddings/models and /v1/videos/models instead of /v1/models, and the
404 fallback lists the new routes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JZTNAFcTA7BQNUu6U83Gdc
Legatia added a commit to pt-evm/zkapi that referenced this pull request Oct 8, 2026
@Legatia
Legatia marked this pull request as ready for review October 8, 2026 05:11

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants