Expo 52 working in web, but on mobile throws: "AI_APICallError: Failed to process successful response"

[nordwestt]

The problem

I have streaming working on mobile using the new expo/fetch api (without using the vercel/ai library, just calling endpoints directly)

Then, I implemented my streaming using the vercel/ai library - and this works completely fine in the web context.

For mobile, I had to polyfill TransformStream, since this was marked as missing. But after this, it seemed to almost be working, except I get the error "AI_APICallError: Failed to process successful response".

Debug info

Using the docs, I casted the error to the APICallError type, and printed out some info.

url: http://192.168.1.50:11434/api/chat
requestBodyValues: {"format": undefined, "messages": [{"content": "Hello, how are you?", "role": "user"}], "model": "llama3.2:latest", "options": {"temperature": 0}, "tools": undefined}
statusCode: 200
responseBody: undefined
data: undefined

I've been banging my head against the wall on this issue for a few days - does anyone have any input that might help?
Been testing out combinations of various solutions suggested in existing discussions but no luck: #655 (comment)

Would love to use the vercel/ai library in my React Native project - and I'll make the solution available for anyone to clone.

Thank you SO much in advance - I appreciate any help on this topic

[lgrammel]

Can you provide more details about the polyfill you are using and the backend that sends the response? Is the error reproducible on web when using the polyfill only?

[lgrammel]

The issue is an incompatibility between the type of the ReadableStream that is returned by fetch and the polyfill. I tried to fix this in various ways for an hour to no avail.

Either Hermes needs to implement TransformStream, or you can try turning off Hermes: https://reactnative.dev/docs/hermes

[lgrammel]

Also, have you seen our expo guide? https://sdk.vercel.ai/docs/getting-started/expo (you need to customize fetch a bit)

[nordwestt]

Thanks for looking into it - could you point me towards the code section where you found the incompatibility?

Will try to disable hermes and use JavaScriptCore instead and cross my fingers that it's not also missing the necessary functions

[nordwestt]

Yep, I looked at the guide as one of the first things. What's strange is that it reads like it's supposed to work when using Expo 52 with the expo/fetch implementation. I'm curious to get in touch with the Author of the page - do you know who it is?

[lgrammel]

Here's some of the stuff I tried for reference:

import 'web-streams-polyfill/polyfill';

import { fetch as undiciFetch } from 'undici';
import { createReadableStreamWrapper } from '@mattiasbuelens/web-streams-adapter';
import { ReadableStream as PolyfillReadableStream } from 'web-streams-polyfill';
const toPolyfillReadable = createReadableStreamWrapper(
  PolyfillReadableStream as any,
);

import { createOpenAI } from '@ai-sdk/openai';
import { streamText } from 'ai';
import 'dotenv/config';

const openai = createOpenAI({
  fetch: async (url, options) => {
    const response = await undiciFetch(url as any, options as any);
    const polyfillBody = toPolyfillReadable(response.body!);

    console.log(polyfillBody);
    console.log(typeIsObject(polyfillBody));
    console.log(
      Object.prototype.hasOwnProperty.call(
        polyfillBody,
        '_readableStreamController',
      ),
    );
    console.log(isReadableStream2(polyfillBody));

    return new Proxy(response, {
      get(target, prop) {
        if (prop === 'body') {
          return polyfillBody;
        }
        return Reflect.get(target, prop);
      },
    }) as any;
  },
});

async function main() {
  const result = streamText({
    model: openai('gpt-3.5-turbo'),
    maxTokens: 512,
    temperature: 0.3,
    maxRetries: 5,
    prompt: 'Invent a new holiday and describe its traditions.',
  });

  for await (const textPart of result.textStream) {
    process.stdout.write(textPart);
  }

  console.log();
  console.log('Token usage:', await result.usage);
  console.log('Finish reason:', await result.finishReason);
}

main().catch(console.error);

export function typeIsObject(x: any): x is object | Function {
  return (typeof x === 'object' && x !== null) || typeof x === 'function';
}

export function isReadableStream(readable: any) {
  if (!typeIsObject(readable)) {
    return false;
  }
  if (typeof (readable as any).getReader !== 'function') {
    return false;
  }
  return true;
}

export function isReadableStream2(x: unknown): x is ReadableStream {
  if (!typeIsObject(x)) {
    return false;
  }

  if (!Object.prototype.hasOwnProperty.call(x, '_readableStreamController')) {
    return false;
  }

  return x instanceof PolyfillReadableStream;
}

[lgrammel]

Trying to understand this whole issue better. When you say "For mobile, I had to polyfill TransformStream", what do you mean exactly? what are you targeting and what are the library versions? we have seen people do this successful on Expo for mobile so this is confusing to us, we want to learn what's different

[nordwestt]

Mobile in my case means Android (google pixel running Android 15).
I get a runtime error when running called "Property 'TransformStream' doesn't exist", which I found you've already dealt with in this issue: #4083 - it essentially shows up as a big bright error page when loading the app, that won't let you continue.

This happens both when running the APK I build and also when using Expo Go (a development app that can load the project)
And when I polyfill with TransformStream from "web-streams-polyfill" then the app loads normally, and that's as far as I've gotten, where I get the "Failed to process successful response" when sending a message.

Here are some hopefully relevant library versions from my package.json:
"ai": "^4.1.4",
"expo": "^52.0.27",
"text-encoding": "^0.7.0",
"react": "^18.2.0",
"react-native": "^0.76.6",
"web-streams-polyfill": "^4.0.0"

[nordwestt]

Also, since more info is always useful, here is a link to the specific commit so you can see the code in context.
The path to where I'm using vercel/ai library is in: "/src/services/chat/providers/OllamaProvider.ts"

[lgrammel]

for testing purposes, can you try our official openai provider with openai instead of the ollama provider? i want to rule out that it's something ollama provider specific

[nordwestt]

Just tried - and I'm getting the same error

It works via web just fine, but on mobile it says "AI_APICallError: Failed to process successful response"

url: https://api.openai.com/v1/chat/completions
requestBodyValues: {"frequency_penalty": undefined, "function_call": undefined, "functions": undefined, "logit_bias": undefined, "logprobs": undefined, "max_completion_tokens": undefined, "max_tokens": undefined, "messages": [{"content": "Hey, how are you?", "role": "user"}], "metadata": undefined, "model": "gpt-4o", "parallel_tool_calls": undefined, "prediction": undefined, "presence_penalty": undefined, "reasoning_effort": undefined, "response_format": undefined, "seed": undefined, "stop": undefined, "store": undefined, "stream": true, "stream_options": undefined, "temperature": 0, "tool_choice": undefined, "tools": undefined, "top_logprobs": undefined, "top_p": undefined, "user": undefined}
statusCode: 200
responseBody: undefined
data: undefined

[jbreite]

Any updates on this? Not sure my problem is exactly the same, but I'm also using the SDK with an expo-52 project. Everything was working great until I upgraded my version to ^4.1.11 from 4.0.23. I needed to do this so I can add more providers.

This led me to this error:

 (NOBRIDGE) ERROR  [Error: Failed to parse stream string. Invalid code f.] ERROR,  Failed to parse stream string. Invalid code f.

I tried the solution from here. I put that on my useChat on the client, which then led to this error.

 (NOBRIDGE) ERROR  [ReferenceError: Property 'TextDecoderStream' doesn't exist] ERROR,  Property 'TextDecoderStream' doesn't exist

Which makes me think I need to add polyfills to the client to support this. I'm not sure which ones to add though.

Happy to clarify anything that could help

[jbreite]

I haven't tried that yet, but possibly my issue could help (if I figure out how to solve) it since it seems that the client now specifying streamProtocol to text

[jbreite]

Okay so I was able to get it work with this polyfill. This Reddit user helped me out: https://www.reddit.com/r/expo/comments/1idzgjo/upgraded_the_vercel_ai_sdk_to_the_latest_version/

import { Platform } from "react-native";

if (Platform.OS !== "web") {
  const setupPolyfills = async () => {
    const { polyfillGlobal } = await import(
      "react-native/Libraries/Utilities/PolyfillFunctions"
    );

    const { TextEncoderStream, TextDecoderStream } = await import(
      "@stardazed/streams-text-encoding"
    );

    polyfillGlobal("TextEncoderStream", () => TextEncoderStream);
    polyfillGlobal("TextDecoderStream", () => TextDecoderStream);
  };

  setupPolyfills();
}

export {};

I then import that at the top of my polyfill like this at the top of my _layout.tsx file

import "@/polyfills";

The fact that this works is a plus, but now that I am using streamProtocol: "text", I can't get the stream of tool calls that I am using. Is there a solve for this error a different way?

 (NOBRIDGE) ERROR  [Error: Failed to parse stream string. Invalid code f.] ERROR,  Failed to parse stream string. Invalid code f.

[nordwestt]

Tried with the suggested solution. I get "Cannot read property 'ReadableStream' of undefined"
When I polyfill with web-streams-polyfill, then I end up with an error saying: "First parameter has member 'readable' that is not a ReadableStream."

Might look into creating an expo-compatible ollama provider one of these days - but for now I'll just stick with my legacy code that I know works fine :)

[toonverbeek]

@nordwestt I'm curious, did you ever get this to work? Even now, 5 months later on Expo SDK 53 + latest AI SDK I can't get this to reliably work. It works on my emulator, but as soon as I build it and run the app on a physical device it starts throwing an error about TextEncoder not found. Thanks!

[nordwestt]

Unfortunately not - my conclusion is that React Native's implementation of the spec is incomplete. It was not fixed by passing expo's fetch implementation.
So currently the solution is a re-write of the different provider implementations in vercel ai - which would be incredibly time-consuming.
So I've just shelved it for the time being sadly

[longseespace]

If you're stuck on AI SDK 4 like me, I was able to get it to work with polyfills:

1. Versions:

• Expo 53
• AI SDK v4
• Latest @ai-sdk/* v1 (for AI SDK 4)

2. Polyfill

Other than standard polyfills recommended by Expo, I need to polyfill ReadableStream with async iterator support and Response with ReadableStream as body. Both are crucial.

import { Platform } from "react-native";
import structuredClone from "@ungap/structured-clone";
import { ReadableStream } from "web-streams-polyfill";

const FileList = function() {}

if (Platform.OS !== "web") {
  const setupPolyfills = async () => {
    const { polyfillGlobal } = await import(
      "react-native/Libraries/Utilities/PolyfillFunctions"
    );

    const { TextEncoderStream, TextDecoderStream } = await import(
      "@stardazed/streams-text-encoding"
    );

    if (!("structuredClone" in global)) {
      polyfillGlobal("structuredClone", () => structuredClone);
    }

    polyfillGlobal("TextEncoderStream", () => TextEncoderStream);
    polyfillGlobal("TextDecoderStream", () => TextDecoderStream);
    polyfillGlobal("FileList", () => FileList);
    
    // Polyfill ReadableStream with async iterator support
    polyfillGlobal("ReadableStream", () => {
      // Add async iterator support to ReadableStream prototype
      if (!ReadableStream.prototype[Symbol.asyncIterator]) {
        ReadableStream.prototype.values ??= function({ preventCancel = false } = {}) {
          const reader = this.getReader();
          return {
            async next() {
              try {
                const result = await reader.read();
                if (result.done) {
                  reader.releaseLock();
                }
                return result;
              } catch (e) {
                reader.releaseLock();
                throw e;
              }
            },
            async return(value) {
              if (!preventCancel) {
                const cancelPromise = reader.cancel(value);
                reader.releaseLock();
                await cancelPromise;
              } else {
                reader.releaseLock();
              }
              return { done: true, value };
            },
            [Symbol.asyncIterator]() {
              return this;
            }
          };
        };

        ReadableStream.prototype[Symbol.asyncIterator] ??= ReadableStream.prototype.values;
      }
      
      return ReadableStream;
    });
    
    // Polyfill Response to properly handle ReadableStream as body
    const OriginalResponse = global.Response;
    
    polyfillGlobal("Response", () => {
      class ResponseWithStream extends OriginalResponse {
        constructor(body, init) {
          // If body is a ReadableStream, we need to handle it specially
          if (body && typeof body === 'object' && typeof body.getReader === 'function') {
            // Create a dummy response first
            super(null, init);
            
            // Store the stream as a property
            Object.defineProperty(this, '_bodyStream', {
              value: body,
              writable: false,
              enumerable: false,
              configurable: false
            });
            
            // Override the body getter to return our stream
            Object.defineProperty(this, 'body', {
              get() {
                return this._bodyStream;
              },
              enumerable: true,
              configurable: true
            });
            
            // Override bodyUsed to track if stream has been consumed
            let bodyUsed = false;
            Object.defineProperty(this, 'bodyUsed', {
              get() {
                return bodyUsed;
              },
              set(value) {
                bodyUsed = value;
              },
              enumerable: true,
              configurable: true
            });
            
            console.log('[Response Polyfill] Created Response with ReadableStream body');
          } else {
            // For non-stream bodies, use original behavior
            super(body, init);
          }
        }
        
        // Override methods that consume the body to mark bodyUsed
        async text() {
          if (this._bodyStream) {
            this.bodyUsed = true;
            const reader = this._bodyStream.getReader();
            const chunks = [];
            const decoder = new TextDecoder();
            
            try {
              while (true) {
                const { done, value } = await reader.read();
                if (done) break;
                chunks.push(decoder.decode(value, { stream: true }));
              }
              return chunks.join('');
            } finally {
              reader.releaseLock();
            }
          }
          return super.text();
        }
        
        async json() {
          if (this._bodyStream) {
            const text = await this.text();
            return JSON.parse(text);
          }
          return super.json();
        }
        
        async arrayBuffer() {
          if (this._bodyStream) {
            this.bodyUsed = true;
            const reader = this._bodyStream.getReader();
            const chunks = [];
            
            try {
              while (true) {
                const { done, value } = await reader.read();
                if (done) break;
                chunks.push(value);
              }
              
              // Combine chunks into single ArrayBuffer
              const totalLength = chunks.reduce((acc, chunk) => acc + chunk.byteLength, 0);
              const result = new Uint8Array(totalLength);
              let offset = 0;
              for (const chunk of chunks) {
                result.set(new Uint8Array(chunk), offset);
                offset += chunk.byteLength;
              }
              return result.buffer;
            } finally {
              reader.releaseLock();
            }
          }
          return super.arrayBuffer();
        }
      }
      
      return ResponseWithStream;
    });
  };

  setupPolyfills();
}

export {};