Skip to content

fix(proxy): preserve caller prompt cache directives - #633

Open
dpearson2699 wants to merge 1 commit into
automazeio:mainfrom
dpearson2699:fix/preserve-prompt-cache-directives
Open

dpearson2699 wants to merge 1 commit into
automazeio:mainfrom
dpearson2699:fix/preserve-prompt-cache-directives

Conversation

@dpearson2699

Copy link
Copy Markdown

Summary

Cache-enabled Claude Code/SDK requests lose their explicit prompt-cache directives in the front proxy, even without a thinking suffix. Preserve caller-supplied directives by removing the blanket stripping call and its unused helper.

 POST provider request
   rewrite model alias / thinking suffix
-  delete block-level cache_control
   select CLIProxyAPI or Vercel
   forward request

Add real-listener forwarding regressions with a loopback capture backend. Defaulted listener/backend port parameters keep those tests isolated from running services; production defaults remain 8317/8318. Coverage includes system/tool/message/nested-content directives, thinking and Copilot alias rewrites, Amp provider path rewriting, UTF-8 Content-Length, response relay, and preservation of native OpenAI/Gemini cache fields on eligible routes.

Fixes #632.

Evidence

  • Before: swift test --filter ThinkingProxyTests failed 9 cache-preservation assertions across plain Claude, thinking-suffix, and Copilot-alias requests. The production path logged Stripped cache_control fields from request body; the loopback backend received the fields with their directives removed. Native OpenAI/Gemini field controls passed before the fix.
  • After: the same regression passes. make build, make test (42 tests), and git diff --check pass. Deliberately removing OpenAI's prompt_cache_key and Gemini's cachedContent in a temporary production-path mutation caused both control fixtures to fail; the mutation was removed.
  • Astra independently confirmed the diagnosis and reviewed the final patch without actionable findings.

All forwarding checks use local ephemeral loopback ports and synthetic bodies. They do not contact model providers, consume inference quota, or establish provider cache hits or a precise subscription-quota saving. Native-field controls verify transport preservation, not validity of a complete provider integration. Build output includes an existing unrelated AppDelegate.windowDidClose SDK warning.

Merge Danger

Door: Two-way

The change is reversible and does not migrate data or change default ports.

Blast Radius: Provider-requests

Every eligible provider request now retains its caller-supplied block-level cache_control fields, including Vercel Claude requests. The old traversal already left OpenAI's prompt_cache_key/prompt_cache_retention and Gemini's cachedContent/contents untouched; no caching failure for those providers is claimed.

This removes the historical workaround from #248. A large Droid OAuth payload reportedly failed with markers and succeeded after stripping, while a minimal marked payload succeeded. That payload-dependent 400 may recur; its precise cause and relevance to the current backend remain unverified. Live provider caching and that historical compatibility case require separate checks when quota is available.

Remove the blanket cache_control traversal that discarded caller directives before provider dispatch. Preserve model alias and thinking transformations.

Add real-listener regression coverage and native OpenAI/Gemini field preservation controls. Allow listener/backend port injection, retaining production defaults, so quota-free forwarding checks use isolated loopback ports.

Validation: regression failed before deletion, then passed; make build; make test (42 tests); git diff --check. Temporary native-field deletion mutations were detected and removed. Astra reviewed the patch with no actionable findings.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-11T13:16:01.415240Z 660c1c1 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

VibeProxy strips caller cache_control on provider routes (v1.8.358)

1 participant