Artifacts

[pavan-moe]

I have a multi-agent system set up, but the tool outputs are quite large. They are being sent with every subsequent request, and I want to change this. Ideally, I want to send the tool output immediately to the first LLM and then store it in an artifact. My agent should be able to refer to that artifact on demand when needed.

I have already set up the artifact and can store and load it, but the tool output is still being sent every time. I considered returning a summary of the tool output along with the artifact URL. However, the agent calling the tool should summarize the output first, ensuring that the summary stays in the session context for the next call. If the detailed output is truly needed, the agent can then use the load_artifacts tool.

I want to explore ways to manage the context window effectively since the agent needs to call multiple tools, with each tool output consuming at least 10-12k tokens. Previous conversations and outputs are quickly pushing the limits of the context window. Is there a better, more scalable, and modular approach to ensure that I always stay within the context window?

[pavan-moe]

please suggest any idiomatic way to do this .

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: You can effectively manage large tool outputs and stay within the context window by using an after_tool_callback to intercept the tool's result. Inside the callback, you can summarize the output, save the original full output to an artifact, and then return only the summary to the agent.

---

You've identified a common challenge in multi-agent systems, and your approach is on the right track. Here’s a more detailed, scalable, and modular way to manage large tool outputs in ADK, building on your idea of using artifacts.

The key is to use an after_tool_callback. This allows you to intercept the output of a tool before it's sent back to the LLM and included in the conversation history.

Here’s how you can implement this pattern:

1. Define an after_tool_callback function: This function will be triggered after any tool in your agent is executed.

2. Inside the callback:

   • Check for large output: Identify which tools produce large outputs that you want to manage.
   • Summarize the output: Create a summary of the large tool result.
   • Save the full output to an artifact: Use the tool_context to access the artifact service and save the original, large tool output. This will give you an artifact URI.
   • Return a modified result: Instead of the original large output, return a dictionary containing the summary and the artifact URI.
   • Prevent redundant summarization: Set tool_context.actions.skip_summarization = True. This is a crucial step that tells the ADK to not send the tool output to the LLM for summarization, which prevents the large data from entering the context window [1].

Here is a conceptual example of what your after_tool_callback might look like:

from google.adk.tools import ToolContext, BaseTool
import json

async def summarize_and_save_to_artifact(
    *,
    tool: BaseTool,
    tool_args: dict,
    tool_context: ToolContext,
    result: dict,
) -> dict:
    # Check if the tool is one that produces large outputs
    if tool.name == "my_large_output_tool":
        # 1. Summarize the large result
        summary = f"Summary of the tool output: {result['summary']}" # Or generate a summary

        # 2. Save the full result to an artifact
        artifact_name = f"tool_output_{tool_context.function_call_id}.json"
        artifact_uri = await tool_context.save_artifact(
            name=artifact_name,
            data=json.dumps(result).encode("utf-8"),
            mime_type="application/json",
        )

        # 3. Modify the result to include only the summary and artifact URI
        modified_result = {
            "summary": summary,
            "artifact_uri": artifact_uri,
        }

        # 4. Tell the agent to not summarize this new result
        tool_context.actions.skip_summarization = True

        return modified_result

    # For other tools, return the result as is
    return result

# You would then add this callback to your agent definition:
my_agent = LlmAgent(
    # ... other agent parameters
    callbacks=[after_tool_callback(summarize_and_save_to_artifact)],
)

By using this approach, only the small summary and the artifact URI are added to the conversation history, keeping your context window lean. If a subsequent agent or tool needs the full details, it can use the artifact_uri to load the complete data using a load_artifacts tool.

This method is scalable and modular because it separates the concern of context management from the tool's primary function. You can apply this logic selectively to tools that need it without altering the tools themselves.

For more information, you can refer to the ADK documentation on Callbacks, ToolContext, and Artifacts.

[1] https://google.github.io/adk-docs/tools/