Move Agent abortSignal to stream/generate options, instead of Agent constructor options

[jamiter]

I'm glad to see that the Agent class is no longer marked as experimental!

One of the reasons to use it is to "Reuse configurations - Same model settings, tools, and prompts across different parts of your application"

While using it though, I noticed that the abortSignal is passed directly to the Agent constructor. In my opinion it would make more sense to have it available at the function calling level though, like agent.stream. Because the agent isn't reusable currently.

import { Agent } from 'ai';

const agent = new Agent({
  model: 'openai/gpt-4o',
  system: 'You are a helpful assistant.',
  // abortSignal needs to be passed here
});

const { text } = await agent.generate({
  prompt: 'What is the weather in NYC?',
  // I expect to pass abortSignal here
});

Example

To clarify, I would like to define some agents which can later be used in an api route. The route handler will have the request.signal available to pass to the agent.stream call. Currently I cannot define these agents before the route has been called.

This is what I would like to do:

// file: agents/weather-agent.ts
export const weatherAgent = new Agent({
  model: 'openai/gpt-4o',
  system: 'You are a weather expert',
  tools: {
    // some weather tools
  }
});

// file: api/chat/route

import { weatherAgent } from './agents/weather-agent.ts';

export async function POST(request) {
  const messages = getCurrentMessages(); // Get the messages from database or request

  const result = await weatherAgent.stream({
    messages,
    abortSignal: request.signal, // <-- NOT POSSIBLE, but only now I have the signal available to pass to the agent
  })
  
  return result.toUIMessageStreamResponse({});
}

We have multiple agents and depending on the question the right agent will answer. It would be create to be able to define these agents and than pick the correct one.

Workaround

Currently I need an agent factory to pass the abortSignal to the agent on creation. So instead of defining and exporting each agent directly, each agent needs a wrapper.

// file: get-weather-agent.ts

export function getWeatherAgent({
  abortSignal
}) {
  return new Agent({
    model: 'openai/gpt-4o',
    system: 'You are a weather expert',
    abortSignal, // There we go
    tools: {
      // some weather tools
    }
}

// file: api/chat/route

import { getWeatherAgent } from './agents/get-weather-agent.ts';

export async function POST(request) {
  const messages = getCurrentMessages(); // Get the messages from database or request

  const weatherAgent = getWeatherAgent({ abortSignal: request.signal });

  const result = await weatherAgent.stream({
    messages,
  })
  
  return result.toUIMessageStreamResponse({});
}

Other considerations

abortSignal might not be the only one available on request time. experimental_context is another.

[github-actions[bot]]

This discussion was automatically locked because it has not been updated in over 30 days. If you still have questions about this topic, please ask us at community.vercel.com/ai-sdk