Add Anthropic caching support to OpenRouter LLM implementation #7492
+354
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Resolves #7479
What Happens Now
When you use OpenRouter with Anthropic models (like
anthropic/claude-sonnet-4
,anthropic/claude-opus-4
, etc.) and caching is enabled:System Message Caching: If
cacheSystemMessage
is true, the system message will be sent withcache_control: { type: "ephemeral" }
Conversation Caching: If
cacheConversation
is true, the last two user messages in the conversation will be cachedAutomatic Processing: OpenRouter automatically handles the caching headers internally - no additional headers needed
Example Request Body
When sending a request to OpenRouter with an Anthropic model, the modified body will look like:
Non-Anthropic Models
When using non-Anthropic models (like GPT-4, Llama, etc.) through OpenRouter, the caching fields are NOT added, ensuring compatibility with all model providers.
Checklist
Tests
OpenRouter.vitest.ts is added.
Summary by cubic
Add Anthropic caching support to OpenRouter for Claude models. When caching is enabled, we annotate the system and last two user messages with cache_control so OpenRouter handles caching; other models are unchanged.