Context Caching -- How To Use With [get_fast_api_app]

[dandolf81]

With the introduction of context caching feature, what is the correct way to utilize this feature when we are using the get_fast_api_app deployment strategy that is referenced here: https://google.github.io/adk-docs/deploy/gke/

If I'm not mistaken, the initialization of the FastAPI application leads down to this execution stream in the AdkWebServer (https://github.com/google/adk-python/blob/main/src/google/adk/cli/adk_web_server.py#L450-L454)

if isinstance(agent_or_app, BaseAgent):
  agentic_app = App(
      name=app_name,
      root_agent=agent_or_app,
      plugins=extra_plugins_instances,
  )

There does not appear to be any opportunity of injecting the context_cache_config here

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: To enable context caching with get_fast_api_app, you need to export a pre-configured App object from your agent.py file. The get_fast_api_app function will automatically discover and use your App instance, including its ContextCacheConfig.

---

Hello! That's an excellent question, and your observation is correct. The get_fast_api_app function and the underlying AdkWebServer logic don't directly expose a parameter to inject the ContextCacheConfig.

This is because context caching is configured at the App level. When the deployment tools find only a root_agent exported from your agent.py, they create a default App wrapper for you, which doesn't have context caching enabled.

The correct way to enable this feature is to construct and export the App object yourself within your agent.py.

Here is an example of how you can structure your agent.py:

from google.adk.agents import Agent, App, ContextCacheConfig

# 1. Define your agent as usual
my_agent = Agent(
    # ... your agent configuration
)

# 2. Define your context cache configuration
my_cache_config = ContextCacheConfig(
    min_tokens=4096,      # Example: Only cache if context is larger than 4096 tokens
    ttl_seconds=600,      # Example: Cache expires after 10 minutes
    cache_intervals=3     # Example: Refresh cache after 3 invocations
)

# 3. Create an App instance, passing in your agent and the cache config
my_app = App(
    name="my_fastapi_app",
    root_agent=my_agent,
    context_cache_config=my_cache_config,
)

# 4. Export the App object. The deployment tools will discover this.
# For backward compatibility, you can also export root_agent.
app = my_app
root_agent = my_agent

By exporting app = my_app, the ADK deployment tools will detect and use your fully configured App instance, including the ContextCacheConfig, instead of creating a default one. This gives you full control over the App's configuration when using get_fast_api_app.

This approach is demonstrated in the cache_analysis sample agent [1]. You can also find more details on the configuration options in the ContextCacheConfig source file [2].

---

[1] https://github.com/google/adk-python/blob/main/contributing/samples/cache_analysis/agent.py
[2] https://github.com/google/adk-python/blob/main/src/google/adk/agents/context_cache_config.py