Repository navigation
fix(proxy): preserve caller prompt cache directives - #633
Open
dpearson2699 wants to merge 1 commit into
Open
dpearson2699 wants to merge 1 commit into
dpearson2699 wants to merge 1 commit into
Conversation
Remove the blanket cache_control traversal that discarded caller directives before provider dispatch. Preserve model alias and thinking transformations. Add real-listener regression coverage and native OpenAI/Gemini field preservation controls. Allow listener/backend port injection, retaining production defaults, so quota-free forwarding checks use isolated loopback ports. Validation: regression failed before deletion, then passed; make build; make test (42 tests); git diff --check. Temporary native-field deletion mutations were detected and removed. Astra reviewed the patch with no actionable findings.
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cache-enabled Claude Code/SDK requests lose their explicit prompt-cache directives in the front proxy, even without a thinking suffix. Preserve caller-supplied directives by removing the blanket stripping call and its unused helper.
POST provider request rewrite model alias / thinking suffix - delete block-level cache_control select CLIProxyAPI or Vercel forward requestAdd real-listener forwarding regressions with a loopback capture backend. Defaulted listener/backend port parameters keep those tests isolated from running services; production defaults remain 8317/8318. Coverage includes system/tool/message/nested-content directives, thinking and Copilot alias rewrites, Amp provider path rewriting, UTF-8 Content-Length, response relay, and preservation of native OpenAI/Gemini cache fields on eligible routes.
Fixes #632.
Evidence
swift test --filter ThinkingProxyTestsfailed 9 cache-preservation assertions across plain Claude, thinking-suffix, and Copilot-alias requests. The production path loggedStripped cache_control fields from request body; the loopback backend received the fields with their directives removed. Native OpenAI/Gemini field controls passed before the fix.make build,make test(42 tests), andgit diff --checkpass. Deliberately removing OpenAI'sprompt_cache_keyand Gemini'scachedContentin a temporary production-path mutation caused both control fixtures to fail; the mutation was removed.All forwarding checks use local ephemeral loopback ports and synthetic bodies. They do not contact model providers, consume inference quota, or establish provider cache hits or a precise subscription-quota saving. Native-field controls verify transport preservation, not validity of a complete provider integration. Build output includes an existing unrelated
AppDelegate.windowDidCloseSDK warning.Merge Danger
Door: Two-way
The change is reversible and does not migrate data or change default ports.
Blast Radius: Provider-requests
Every eligible provider request now retains its caller-supplied block-level
cache_controlfields, including Vercel Claude requests. The old traversal already left OpenAI'sprompt_cache_key/prompt_cache_retentionand Gemini'scachedContent/contentsuntouched; no caching failure for those providers is claimed.This removes the historical workaround from #248. A large Droid OAuth payload reportedly failed with markers and succeeded after stripping, while a minimal marked payload succeeded. That payload-dependent 400 may recur; its precise cause and relevance to the current backend remain unverified. Live provider caching and that historical compatibility case require separate checks when quota is available.