Guidance on persisting messages

[mattpocock]

Persisting messages as it stands feels full of pitfalls. The one's I'm encountering are:

• The UI messages and Core messages are very different shapes. Throw in a third shape - the way you store it in the DB (for now, let's call it DBMessage) and you've got a boatload of glue code you need to write. I would appreciate a guide on how to store messages in the database.
• The current Chatbot Message Persistence doc uses an unhelpful setup where the entire history is saved in JSON. A setup where only the latest messages are saved would be appreciated.
• This glue code can be prone to errors, especially since some parts of the glue are inside the AI SDK library itself. I am personally seeing tool results not being saved in my implementation (largely copied from Vercel AI Chatbot). There is likely something wrong with my implementation - but I'm not sure what it is. Guidance in docs would be appreciated.

[nicoalbanese]

Hey! Pulled together an example with Postgres and Drizzle that stores messages atomically rather than loading and saving full chats on each new response.

https://github.com/nicoalbanese/ai-sdk-persistence-db

Quick heads up: we are making major improvements to useChat over the next 6-8 weeks which might impact content that's created for the current useChat.

[hewliyang]

Was attempting to implement persistence on a custom /chat completions endpoint on my Python server (using x-vercel-ai-data-stream) in hopes I could make use of useChat on the client side.

After I saw the 3 different types of Message across the ai-chatbot repo, and the number of hoops and transforms were needed to transform across all 3, I figured writing my own SSE parser would be the lesser task!

It's a poor model to follow in TS, and an even bigger pain if your completions endpoint is not written in it.

[zbeyens]

It has been challenging to do optimistic updates - avoiding invalidating chat query on finish. Especially when the response has many messages:

• streamText onFinish has a list response.messages while useChat has only one response message, that's a merged version of response.messages.
• useChat response message id is not predictable: while streaming multiple parts, the last useChat message dynamically changes its id to the last part one. @nicoalbanese is this expected? Could we keep using the first part one?
• To avoid saving multiple assistant messages in a row, I've modified sanitizeResponseMessages to return a single message:

export function sanitizeResponseMessages({
  messages,
  reasoning,
}: {
  messages: ResponseMessage[];
  reasoning?: string;
}): Message {
  const parts: Message['parts'] = [];
  const toolResultIds: string[] = [];
  
  // A better solution would be to store the first message 
  const id = messages.at(-1)!.id;

  // First pass: collect all tool result IDs
  for (const message of messages) {
    if (message.role === 'tool') {
      for (const content of message.content) {
        if (content.type === 'tool-result') {
          toolResultIds.push(content.toolCallId);
        }
      }
    }
  }

  // Second pass: process all messages into parts
  for (const message of messages) {
    if (message.role === 'assistant') {
      if (typeof message.content === 'string') {
        if (message.content.trim().length > 0) {
          parts.push({
            text: message.content,
            type: 'text',
          });
        }
      } else {
        // Filter and add valid content parts
        message.content.forEach((content) => {
          if (content.type === 'tool-call') {
            if (toolResultIds.includes(content.toolCallId)) {
              parts.push({
                toolInvocation: {
                  args: content.args,
                  state: 'call',
                  toolCallId: content.toolCallId,
                  toolName: content.toolName,
                },
                type: 'tool-invocation',
              });
            }
          } else if (
            content.type === 'text' &&
            content.text.trim().length > 0
          ) {
            parts.push({
              text: content.text,
              type: 'text',
            });
          }
        });
      }
    } else if (message.role === 'tool') {
      message.content.forEach((content) => {
        if (content.type === 'tool-result') {
          // Find the matching tool invocation part and update it
          const invocationIndex = parts.findIndex(
            (part) =>
              part.type === 'tool-invocation' &&
              part.toolInvocation.toolCallId === content.toolCallId
          );

          if (invocationIndex !== -1) {
            const part = parts[invocationIndex];

            if (part.type === 'tool-invocation') {
              parts[invocationIndex] = {
                toolInvocation: {
                  ...part.toolInvocation,
                  result: content.result,
                  state: 'result',
                },
                type: 'tool-invocation',
              };
            }
          }
        }
      });
    }
  }

  // Add reasoning if provided
  if (reasoning) {
    parts.unshift({
      details: [],
      reasoning,
      type: 'reasoning',
    });
  }

  // Return a single message with all parts
  return {
    id,
    content: '',
    parts,
    role: 'assistant',
  };
}

Looking forward an unified structure around parts.

[beamercola]

@nicoalbanese Is there anything you can say about what will be changing in the sdk? Trying to decide if we should put more efforts in now or wait. Thanks! Very exciting

[zhm]

useChat response message id is not predictable: while streaming multiple parts, the last useChat message dynamically changes its id to the last part one. @nicoalbanese is this expected? Could we keep using the first part one?

I'm seeing this happen when a tool call transitions from call -> result in the client, its id changes in the client. But the server onFinish's with a different id for that same message after appendResponseMessages. The result is the client messages doesn't match the database. It doesn't seem ideal that the client ends up with a different id for the same message. It seems like the logic for reconciling messages in appendResponseMessages doesn't match what useChat is doing. It appears appendResponseMessages is consolidating in the first message id, but useChat is using the latest message id.

[zhm]

Actually I misspoke, it looks like the client changes the message id when it receives the step-start part after the tool call when it's receiving the follow-on LLM response.

[aether6430]

Has there been a proper way to manage persistence using vercel-ai yet? Including tool-calls and type safety

[dir]

I genuinely don't think anyone (publicly) has managed to figure it out in a relational, type-safe manner. Even when I've viewed the outputs of popular open source wrappers around Vercel AI SDK like Mastra, they just slam all of the data into a JSON field and call it a day.

Would love to properly store the tool calls, results, messages (actual text messages, lol), and reasoning.

[annilq]

the Message has toolInvocations and parts property,they are useful in [Generative User Interfaces] section(https://sdk.vercel.ai/docs/ai-sdk-ui/generative-user-interfaces) ,if it is necessary to save these info db for persisting？

[mattpocock]

AFAIK, parts is all you need

[ElectricCodeGuy]

This here is how I store it:

Database Structure

I use two main tables with a one-to-many relationship:

1. chat_sessions

   CREATE TABLE chat_sessions (
     id STRING PRIMARY KEY,
     user_id STRING,
     chat_title STRING,
     public BOOLEAN,
     created_at TIMESTAMP,
     updated_at TIMESTAMP
   );

2. chat_messages

   CREATE TABLE chat_messages (
     id STRING PRIMARY KEY,
     chat_session_id STRING REFERENCES chat_sessions(id),
     content STRING,
     is_user_message BOOLEAN,
     created_at TIMESTAMP,
     reasoning JSONB,
     sources JSONB,
     tool_invocations JSONB,
     attachments JSONB,
   );

Instead of storing the entire history as a JSON blob, I store individual messages with their metadata. The complex parts (sources, tool invocations, etc.) can either be:

1. Stored as JSONB columns (as shown above)
2. Split into their own tables with foreign keys to chat_messages
3. You could also go even further and make a new table for each of your tools where you have args and return stored and a forgin key to your chat_messages table

When retrieving messages, I use SQL JOINs to reconstruct the complete conversation and then pass it through a message parser function that converts the database records to the proper Message types expected by the chat interface.

works super well and is (quite) easy to setup :P

I made example repo here : https://github.com/ElectricCodeGuy/SupabaseAuthWithSSR

Note: that the attachment is stored as a data string, but this might not be optimal if the data string is 20MB. A bucket would be a better option but parsing it back and forth between client and server becomes a bit tricky.

[mattpocock]

I would strongly recommend storing it as parts instead.

Tool invocations, sources, and reasoning are all deprecated. Parts is the future of how to persist the AI SDK.

(I'm not an official source, just recommending what I generally recommend)

[ElectricCodeGuy]

Everything is stored and parsed out using the new parts :) Perhaps i should write it more clearly in my msg

[gauravvgat]

Why would your persistence be ties to the AI-SDK library? As long as you are able to reconstruct the history it should be fine.

[grmkris]

recently i participated on some hackathon where i wanted to build a simple "template" app with chat history well integrated using drizzle... i quite like the result:

https://github.com/grmkris/seoul-2025-buildai-hackathon/blob/main/apps/api/src/db/schema/chat/chat.db.ts

each message is stored as jsonb import type { Message } from "ai";

[miketromba]

I'm also storing as jsonb. It works. But I fear breaking SDK changes in the future - I really dread having to write some complex migration on all of the JSON entries in my DB to transform to new structures.

[gruni1992]

Looking at the Vercel AI Chatbot that Matt linked, this is how I transform the onFinish result.response.messages to a Message which is usable by the fronted. Seems like a dirty workaround, but works so far:

import { appendResponseMessages, CoreAssistantMessage, CoreToolMessage, Message, streamText } from "ai";

type ResponseMessage = (CoreAssistantMessage | CoreToolMessage) & { id: string }

const responseMessagesToMessage = (responseMessages: ResponseMessage[]): Message =>
  appendResponseMessages({
    messages: [{ id: 'unused', role: 'user', content: 'unused' }],
    responseMessages,
  })[1]

streamText({
    model: azureChatModel,
    messages,
    tools,
    maxSteps: 10,
    onFinish: async (result) => await insertMessage(sessionId, responseMessagesToMessage(result.response.messages))
  })

[nicoalbanese]

Hey folks - I've updated this template to store parts rather than the raw message in JSONB. In future AI SDK versions, we will most likely add IDs to each part, which would allow us to move them out to a separate table. I've also added delete message functionality that showcases how to work with generated IDs.

This is the recommended approach at the moment but very open to feedback and improvements here!

[zachrip]

Parts having ids would be wonderful for react keys, please add this!

[hoanginc144]

I'm a noob at this but JSONB doesn't preserve key order, considering there could be many steps in a message it'd look odd if they're out of order. So I think storing the parts as JSON is better? Thank you for the template though, as I managed to solve my own problem trying to save the parts to the db by running your code!

[MatheusDubin]

@hoanginc144 ordering in most cases should be handled by whatever layer you have between retrieving the stored data and wherever it is going on to, be it an abstraction such as a ORM or just code, mostly because then you can centralize your sorting logic based on specific properties such as date time ones (createdAt)

[michaelv-greenlite]

@nicoalbanese appreciate the updates here. Can you provide clarification on an issue I'm having following your example?

My messages containing SourceUIPart are streamed to and rendered correctly on the client. However, the server-side onFinish callback of streamText seems to only contain source information within StepResult.sources, not the within the response message itself. This means persisting the result of appendResponseMessages to the db doesn't save all of the user's stream data (source parts and potentially others).

Does this align with the behavior you'd expect?

 const messages = appendClientMessage({
    messages: dbMessages,
    message
  })

  return createDataStreamResponse({
    execute: (dataStream) => {
      const result = streamText({
        model: perplexity('sonar-pro'),
        messages,
        toolCallStreaming: true,
        maxSteps: 5,
        experimental_generateMessageId: createIdGenerator({
          prefix: 'msgs',
          size: 16
        }),
        async onFinish({ response }) {
          const newMessage = appendResponseMessages({
            messages,
            responseMessages: response.messages
          }).at(-1)!

          await insertMessage({
            supabase,
            message: {
              // ...
              parts: newMessage.parts as Json[], // does not contain source parts
            }
          })
        }
      })

      result.mergeIntoDataStream(dataStream, {
        sendSources: true,
        sendReasoning: true
      })
    },
    onError: (error) => {
      return error instanceof Error ? error.message : String(error)
    }
  })

[michaelv-greenlite]

As far as I can tell, this is the most straightforward way to persist in a way that actually saves all the parts:
https://github.com/ibelick/zola/blob/3d879fa77d50e48ba8278b196099a9a5b4e2d10c/app/api/chat/db.ts#L31

[miketromba]

Persistence in the onFinish callback is an incomplete/insufficient approach for multi-step agents.

There's an issue with more complex agents in real-world use-cases that have complex tools with side-effects. The issue is that sometimes a stream may error mid-way through. If the agent is 5 tool calls deep and has already made significant mutations to various entities in the system, your example doesn't persist the partial generation (the parts/final partial part leading up to the error). This sucks from a UX perspective because when the user goes to interact with the agent again, a) the message history is corrupted - even though the agent made mutations, the user cannot see in the chat history anymore - even though the entities have changed. and b) The agent doesn't remember which mutations it has already made since they were never persisted in the chat history... leading it to start repeating it's previous instructions from the user and duplicate all of it's efforts.

Right now my workaround is having the client handle this partial errored state persistence by hitting a POST /chat/partial endpoint which sends the partial message that was streamed down to the user back up to the server for persistence (a very unnecessary round-trip with additional potential points of failure, in my view).

Open to any thoughts on this!

[miketromba]

Follow-up, I have begun experimenting with a single-step architecture where I set maxSteps: 1 and let the client control it's recursive LLM calling logic.

So in the client-side onFinish, I check if finish reason was a tool call, and if so, and my local steps counter has not exceeded a client-defined maximum, I append a new user message with id: CONTINUE which I:
a) exclude from my chat history rendering by detecting that id, and
b) detect server-side as a continuation request and handle it accordingly.

The id: CONTINUE messages also get excluded from my persistence logic server-side.

The benefit of this is that I am able to persist in the onFinish callback more atomically - on every step. It also gives me more granular control over the context payload I'm sending on each individual step instead of letting the SDK just keep naively appending to it, bloating the tokens being sent as input.

Still experimenting and not sure how practical this is yet but it looks promising.

[dskoda1]

this shouldn't really be a problem if the api route is allowed to just finish all the way through even on a client disconnect right?

[gauravvgat]

This works with generateText, but I have not been able to make it work with streamText.

[mattpocock]

You need to consumeStream() to make it always reach onFinish even if the client disconnects

[miketromba]

@dskoda1 In my case, I have an agent that may take 10 mins and call 10 different tools to complete execution... it would be a bad UX to let the agent keep doing it's thing- the thread has to be locked (concurrency would break my system), meaning the user wouldn't be able to work on their project until the agent's loop completes, and the user might be surprised/disappointed by the agent's mutations to their project - they won't be able to interrupt the agent and course-correct it if it starts going in the wrong direction as they would if they were monitoring the stream in real-time on the client. So it's important to be able to immediately kill the loop and persist the partial state.

[grmkris]

That is very strong idea. Initially i liked vercel ai sdk because it was just a simple interface for different llm providers. But with multiple steps options it is turning into a mini framework which i wanted to avoid in first place. I might explore this as well.

…

On Sat, 24 May 2025 at 03:59, Michael Tromba ***@***.***> wrote: Follow-up, I have begun experimenting with a single-step architecture where I set maxSteps: 1 and let the client control it's recursive LLM calling logic. So in the client-side onFinish, I check if finish reason was a tool call, and if so, and my local steps counter has not exceed a client-defined maximum, I append a new user message with id: CONTINUE which I: a) exclude from my chat history rendering by detecting that id, and b) detect server-side as a continuation request and handle it accordingly. The id: CONTINUE messages also get excluded from my persistence logic server-side. The benefit of this is that I am able to persist in the onFinish callback more atomically - on every step. It also gives me more granular control over the context window I'm providing on each step instead of letting the SDK just keep naively appending to it, bloating the tokens being sent as input. Still experimenting and not sure how practical this is yet but it looks promising. — Reply to this email directly, view it on GitHub <#4845 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADWTG22UPKKLFZXHJFDARVD277HARAVCNFSM6AAAAABW7KRS32VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGMRVGM2DANA> . You are receiving this because you commented.Message ID: ***@***.***>

[dskoda1]

Related to some other commenters, my biggest challenge seems to be handling the sse stream being cancelled due to client disconnect. Ideally when the client calls the backend there's someway to separate the actual stream text call so that it stays running until it completes, with appropriate message persistence, and then if the client disconnects, only the layer above the stream text call ends up being cancelled. Anyone have suggestions on how this could be done? I'm not super keen on breaking the stream text call out into a separate architectural component outside of say a nextjs api route because then it feels like we are doing more work than necessary to stand up say a message queue or redis stream or something, and surely there is a better / simpler solution to this?

[dskoda1]

No it wouldn't be too bad - assuming the api route that was responsible for streaming the text from the llm and persisting into the db is allowed to finish, when the user reloads the page they would have the full completion visible in the UI.

Putting aside some long running task such as a deep research, most chat responses aren't super long.

What I'm trying to say is consumeStream alone isn't preventing the nextjs api route from being aborted as soon as the client disconnects however, preventing the database persistence from happening, and that is where the experience ends up feeling broken. Because whatever the llm was doing just gets cut short then. I'm wondering if there is any nextjs level config needed to allow the api route to continue.

Also thank you for your time and responses in this thread @mattpocock its highly appreciated!

[dskoda1]

its possible theres an issue with v5 but I can't prove it because even though im using the new toUIMessageStreamResponse i am still just getting a StreamTextResult which has consumeStream available. I'll try to create a minimal reproduction of it with v5.

[dskoda1]

here is the basic implementation of a chat ui, not even dealing with persistence, but demonstrating that clients disconnecting kills the api route even with consumeStream: https://github.com/dskoda1/consume-stream-bug-repro

this is not even using the v5 sdk so its definitely not related to the version, but it does mean that this example is wrong and or theres some other config needed to allow consumeStream to work? https://ai-sdk.dev/docs/ai-sdk-ui/chatbot-message-persistence#handling-client-disconnects

If somewhere else is better for me to post this perhaps as a bug / issue im happy to do so.

[apferrarone]

@dskoda1 Is abortSignal: req.signal telling the llm to stop streaming as soon as the client disconnects in your example? If you remove that do you still see this issue? I am not seeing this issue myself when using consumeStream - it appears to be working as far as I can tell. When I send a message, close the tab, then come back I see the full assistant message was persisted to my db. I also have a higher maxDuration value set - although the default of 10s should be enough for a 4o response.

[dskoda1]

yes I was trying too many things and combining them; removing the abortSignal did indeed fix the consume stream issue. thanks @apferrarone

[zhm]

Is there any value in persisting step-start parts? Are they needed to reconstruct anything within the internals or can they be safely discarded for persistence?

[robertlong]

I'd also like to know this. It seems like they are used to reconstruct the model messages from ui messages in v5. Particularly when a ToolInvocationPart (now UIToolPart) is processed. But, still I'd like to know the significance of signaling the start of a step. Why is the model message's tool-call and tool-result part types not enough?

[lazakrisz]

It's required for multi step agents, see here: https://ai-sdk.dev/docs/ai-sdk-ui/chatbot-tool-usage#step-start-parts

[TheSlavant]

Would be amazing to have a guide on persisting messages in v5. Feels like it's still an open question that almost everyone runs into at some point.

[TheSlavant]

@notflip I'm considering it because we have a new project. We already built it in v4 and are close to launch, but I ran into the issues above and started looking into v5 for prod.

[TheSlavant]

@nicoalbanese any way we should replicate in v5 what experimental_generateMessageId used to do?

The v5 guide (thanks for that!) includes it, but looks like it's been deprecated.

[nicoalbanese]

no problem! if you don't want to use generateId, you can use createIdGenerator:

import { createIdGenerator } from 'ai';

const generateIdCustom = createIdGenerator({ prefix: 'msgc', separator: '-' });
const myId = generateIdCustom();

does this solve it for you?

[TheSlavant]

Thank you! We ended up staying on v4 after all. When we migrated to v5, we started running into an openai error.

[lgrammel]

@TheSlavant should be fixed now #7099

[nicoalbanese]

Just an update here that this was a big focus of v5 and we will have an updated template with best practices, incorporating a lot of your feedback here!

Importantly, we do not store things as JSONB anymore! Stay tuned.

[qantrepreneur]

Looking forward to this new template, when can we expect it?

[nicoalbanese]

Next few days, ironing out the last few bugs

[qantrepreneur]

Next few days, ironing out the last few bugs

That's great! Thank you for the swift response.

[jmreidy]

Thanks so much @nicoalbanese ! 🙏

[nicoalbanese]

Hey! This has been shipped, more info here: #4845 (comment)

[nicoalbanese]

Thanks for bearing with me here! This template has now been updated to AI SDK 5 and now uses a much more robust and scalable persistence pattern. I have done my best to optimise performance where possible. Please do let me know feedback and how we can make this better.

For a TLDR of the new pattern, please see the README.

---

So what changed?

Previously, we were storing chats and messages. This was simple, but parts were stored as a jsonb() column. This obviously presented data integrity and migration issues.

Prefix-based part storage

To resolve this, we've moved to a prefix-based approach for storing message parts directly in the database schema. Instead of using a flexible but problematic JSONB column, we now have dedicated columns for each message part type with specific prefixes:

• text_*: Text content parts
• reasoning_*: Reasoning/thinking parts
• file_*: File attachments
• source_url_*: URL sources
• source_document_*: Document sources
• tool_[toolName]_*: Tool calls (e.g., tool_getWeatherInformation_*)
• data_[dataType]_*: Custom data parts (e.g., data_weather_*)

So where before our message schema looked like this:

export const chats = pgTable("chats", {
  id: varchar()
    .primaryKey()
    .$defaultFn(() => nanoid()),
});

export const roleEnum = pgEnum("role", ["user", "assistant", "system", "data"]);

export const messages = pgTable("messages", {
  id: varchar()
    .primaryKey()
    .$defaultFn(() => nanoid()),
  chatId: varchar()
    .references(() => chats.id, { onDelete: "cascade" })
    .notNull(),
  createdAt: timestamp().defaultNow().notNull(),
  parts: jsonb().$type<UIMessage["parts"]>().notNull(), // BAD
  role: roleEnum().notNull(),
});

Now, it looks like this:

export const chats = pgTable("chats", {
  id: varchar()
    .primaryKey()
    .$defaultFn(() => generateId()),
});

export const messages = pgTable(
  "messages",
  {
    id: varchar()
      .primaryKey()
      .$defaultFn(() => generateId()),
    chatId: varchar()
      .references(() => chats.id, { onDelete: "cascade" })
      .notNull(),
    createdAt: timestamp().defaultNow().notNull(),
    role: varchar().$type<MyUIMessage["role"]>().notNull(),
  },
  (table) => [
    index("messages_chat_id_idx").on(table.chatId),
    index("messages_chat_id_created_at_idx").on(table.chatId, table.createdAt),
  ],
);

export const parts = pgTable(
  "parts",
  {
    id: varchar()
      .primaryKey()
      .$defaultFn(() => generateId()),
    messageId: varchar()
      .references(() => messages.id, { onDelete: "cascade" })
      .notNull(),
    type: varchar().$type<MyUIMessage["parts"][0]["type"]>().notNull(),
    createdAt: timestamp().defaultNow().notNull(),
    order: integer().notNull().default(0),

    // Text fields
    text_text: text(),

    // Reasoning fields
    reasoning_text: text(),

    // File fields
    file_mediaType: varchar(),
    file_filename: varchar(), // optional
    file_url: varchar(),

    // Source url fields
    source_url_sourceId: varchar(),
    source_url_url: varchar(),
    source_url_title: varchar(), // optional

    // Source document fields
    source_document_sourceId: varchar(),
    source_document_mediaType: varchar(),
    source_document_title: varchar(),
    source_document_filename: varchar(), // optional

    // tools are stored in separate cols
    tool_getWeatherInformation_toolCallId: varchar(),
    tool_getWeatherInformation_state: varchar().$type<ToolUIPart["state"]>(),
    tool_getWeatherInformation_input:
      jsonb().$type<getWeatherInformationInput>(),
    tool_getWeatherInformation_output:
      jsonb().$type<getWeatherInformationOutput>(),
    tool_getWeatherInformation_errorText: varchar(),

    // Data parts
    data_weather_id: varchar().$defaultFn(() => generateId()),
    data_weather_location: varchar().$type<MyDataPart["weather"]["location"]>(),
    data_weather_weather: varchar().$type<MyDataPart["weather"]["weather"]>(),
    data_weather_temperature:
      real().$type<MyDataPart["weather"]["temperature"]>(),

    providerMetadata: jsonb().$type<MyProviderMetadata>(),
  },
  (t) => [
    // Indexes for performance optimisation
    index("parts_message_id_idx").on(t.messageId),
    index("parts_message_id_order_idx").on(t.messageId, t.order),

    // Other constraints
  ],
);

Full implementation

This prefix-based column naming convention provides several key advantages including type safety with strongly-typed columns, better query performance through direct column access, database-level data integrity constraints, migration-friendly schema changes, and efficient indexing.

Simplified message persistence workflow

The other big change that comes thanks to AI SDK 5 is where and how we are saving messages.

Our suggestion has always been to persist messages in the UIMessage (prev. Message) format. This is because UIMessage contains the full message history, while ModelMessage (previously CoreMessage) is a lossy, stripped-down format optimized for sending to language models.

However, saving UIMessages was complicated with the conversion from ModelMessage (the format generated by streamText), requiring the appendResponseMessages utility function:

const result = streamText({
  model: openai("gpt-4o-mini"),
  messages: convertToModelMessages(messages),
  maxSteps: 5,
  tools,
});

return result.toDataStreamReponse({
  onFinish: async ({ response }) => {
    const newMessage = appendResponseMessages({
      messages,
      responseMessages: response.messages,
    }).at(-1)!;

    await upsertMessage({
      id: newMessage.id,
      chatId: chatId,
      message: newMessage as UIMessage,
    });
  },
});

That is why we've made changes to the onFinish method on toUIMessageStreamResponse to expose the generated messages in UIMessage format. This means saving new messages can be done like this:

const result = streamText({
  model: openai("gpt-4o-mini"),
  messages: convertToModelMessages(messages),
  stopWhen: stepCountIs(5),
  tools,
});

return result.toUIMessageStreamResponse({
  originalMessages: messages, // pass in all previous messages
  onFinish: async ({ responseMessage, messages }) => {
    // save just most recent assistant message with responseMessage
    await saveMessage({
      chatId,
      id: responseMessage.id,
      message: responseMessage,
    });
    // or, save full message history with messages
    await saveChat({
      chatId,
      messages,
    });
  },
});

Full implementation

---

There are obviously many more changes but these are the two central changes that really improve the overall process of persisting your AI SDK application state.

What's missing?

This template isn't perfect and is still being improved. Notably, it's missing persistence of partial state. We will be working on this soon but wanted to get this template out so folks could comment and improve where necessary.

[ardblok]

Just wanted to say a big thanks for putting together this template, it's been super helpful!
I've been wracking my brain over stopping streams and partial state persistence, so I'm excited to hear you're picking this up soon.

[niels-bosman]

While this might work well for some cases, in our case we would like to be able to persist these messages from the front-end (through server actions, ai back-end is hosted elsewhere), but this is not really possible since onFinish() does not always include all chat messages.

[robertlong]

While I think the single jsonb column storing all of your message parts for a message was bad (we did this too until recently) the polymorphism handled by the many columns approach isn't scalable enough for some peoples applications.

Instead I would recommend a message part table with a type column and a jsonb content column. Where type is the discriminator for the contents of the jsonb column. You would then use something like Zod to ensure that the contents of that column are valid for the given schema when inserted.

There is still some room for error here because the db doesn't enforce the adherence to the content schema, but if your database layer has strict runtime checks for inserting into that column and you continue to update that schema in a additive manner (new types and adding columns only, no renames or deletes) then you will have a much better time scaling to many tool types without an incredibly wide table schema. Both are valid IMO but they each have tradeoffs.

[bestickley]

@Rohit-Singh-Rawat and @robertlong, I too at first didn't like specifying 5 columns per tool and 2+ columns per data part but this convinced me it's good idea from template README here

These constraints prevent partial or corrupted message parts from being stored, ensuring reliable message reconstruction during retrieval.

[grmkris]

but this way i cannot have dynamic tools :(