Is Runner re-usable across requests from frontend?

[nimanthadilz]

I am building a multi-agent app that needs to be exposed via an API. I am using a FastAPI app (not the one provided by get_fast_api_app from google.adk.cli.fast_api. Because I don't want to expose all the endpoints created by ADK and have a requirement to get custom fields in the request body from frontend.)

To achieve above, I have a route in my FastAPI app that uses Runner class to instantiate a runner object for every request.

@orchestrator_router.post("/chat", response_model=QueryResponse)
 async def call_orchestrator(req: QueryRequest):
     try:
         logger.info(f"Received query: {req.user_query}")
 
         # Session handling
         session_service = SessionService.get_instance()
         session = None
 
         if not req.session_id:
             req.session_id = str(uuid.uuid4())
         else:
             session = await session_service.get_session(
                 user_id=req.user_id, app_name=APP_NAME, session_id=req.session_id
             )
             if not session:
                 logger.info(
                     f"Session ID {req.session_id} not found. Creating new session."
                 )
                 req.session_id = str(uuid.uuid4())
 
         if not session:
             session = await session_service.create_session(
                 app_name=APP_NAME, user_id=req.user_id, session_id=req.session_id
             )
             logger.info(f"Created new session: {session.id} for user: {req.user_id}")
 
         # Runner
         runner = Runner(
             agent=orchestrator_agent, app_name=APP_NAME, session_service=session_service
         )
 
         user_message = types.Content(
             role="user", parts=[types.Part(text=req.user_query)]
         )
 
         # We need to exhaust the generator to ensure proper cleanup,
         # even if we find the response early.
         agent_response: str = FALLBACK_RESPONSE_WHEN_NO_RESPONSE_FROM_AGENT
         async for event in runner.run_async(
             user_id=req.user_id, session_id=session.id, new_message=user_message
         ):
             if event.is_final_response():
                 if event.content and event.content.parts:
                     agent_response = event.content.parts[0].text
 
         logger.info(f"Final response: {agent_response}")
         return QueryResponse(
             data=QueryResponseData(
                 responses=Responses(agent_response=agent_response),
                 meta_data={SESSION_ID: session.id},
             ),
             success=True,
             error=None,
         )
 
     except Exception as e:
         logger.error(f"Error processing query: {e}", exc_info=True)
         return QueryResponse(
             data=QueryResponseData(
                 responses=Responses(
                     agent_response=FALLBACK_RESPONSE_WHEN_UNEXPECTED_ERROR_EXECUTING_AGENT,
                 ),
                 meta_data={SESSION_ID: req.session_id} if req.session_id else {},
             ),
             success=False,
             error=QueryResponseError(message=str(e)),
         )

Is it correct to instantiate a runner object for each request?
Or can it be reused across multiple requests from frontend?

I didn't find explicit instructions about this in the ADK documentation. But when I checked Python code of ADK. I found in ADKWebServer class that it actually reuses runner objects by caching them (Link)

async def get_runner_async(self, app_name: str) -> Runner:
    """Returns the cached runner for the given app."""
    # Handle cleanup
    if app_name in self.runners_to_clean:
      self.runners_to_clean.remove(app_name)
      runner = self.runner_dict.pop(app_name, None)
      await cleanup.close_runners(list([runner]))

    # Return cached runner if exists
    if app_name in self.runner_dict:
      return self.runner_dict[app_name]

Please help me to understand the correct use of this Runner object.

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: Yes, the Runner object is designed to be reusable across multiple requests. You should not create a new one for each request.

---

You've made a great observation by looking into the ADKWebServer source code! Your analysis is spot on.

The Runner is intended to be a long-lived object that you create once when your application starts. It holds the configuration for your agent and its associated services (like SessionService, ArtifactService, etc.).

Creating a new Runner for every incoming API request is inefficient because it involves repeated object creation and setup, which can add unnecessary latency to your responses. The intended pattern is to "create once, use many times."

For a FastAPI application, the recommended approach is to initialize the Runner during the application's startup. You can achieve this using a lifespan event handler:

from contextlib import asynccontextmanager
from fastapi import FastAPI
from google.adk.runners import Runner
from your_project import orchestrator_agent, APP_NAME, session_service

# Global variable to hold the runner
adk_runner = None

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Code to run on startup
    global adk_runner
    print("Initializing ADK Runner...")
    adk_runner = Runner(
        agent=orchestrator_agent, 
        app_name=APP_NAME, 
        session_service=session_service
    )
    # The application is now ready to accept requests
    yield
    # Code to run on shutdown (optional)
    print("Shutting down...")


app = FastAPI(lifespan=lifespan)


@app.post("/chat")
async def call_orchestrator(req: QueryRequest):
    # Now, you can reuse the global adk_runner object here
    # instead of creating a new one.
    
    # ... your existing logic for session handling ...

    # We need to exhaust the generator to ensure proper cleanup,
    # even if we find the response early.
    agent_response: str = FALLBACK_RESPONSE_WHEN_NO_RESPONSE_FROM_AGENT
    async for event in adk_runner.run_async(
        user_id=req.user_id, session_id=session.id, new_message=user_message
    ):
        if event.is_final_response():
            if event.content and event.content.parts:
                agent_response = event.content.parts[0].text
    
    # ... rest of your response logic ...

This approach ensures that the Runner is initialized only once, making your application more performant and aligned with the design of the ADK, as seen in the ADKWebServer implementation you referenced [1].

[1] https://github.com/google/adk-python/blob/main/src/google/adk/cli/adk_web_server.py