-
Notifications
You must be signed in to change notification settings - Fork 4
Docs: Update Tool Error Handling #438
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
40d5aaf
e68081c
5f34885
e793a11
99c35ee
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,78 +1,252 @@ | ||
--- | ||
title: "Tool Errors" | ||
description: "Documentation for the different types of tool errors in the Arcade Tool SDK" | ||
title: "Tool Error Handling" | ||
description: "Learn how to handle errors when building tools with Arcade's Tool Development Kit (TDK)" | ||
--- | ||
|
||
# Tool errors in Arcade | ||
# Tool Error Handling | ||
|
||
When working with Arcade's Tool SDK, you may encounter various types of errors. This guide will help you understand and handle these errors effectively. | ||
When building tools with Arcade's Tool Development Kit (TDK), understanding error handling is crucial for creating robust and reliable tools. This guide covers everything you need to know about handling errors from a tool developer's perspective. | ||
|
||
## Handling errors in your tools | ||
## Error handling philosophy | ||
|
||
In most cases, you don't need to raise errors explicitly in your tools. The `@tool` decorator takes care of proper error propagation and handling. When an unexpected error occurs during tool execution, Arcade's `ToolExecutor` and `@tool` decorator will raise a `ToolExecutionError` with the necessary context and traceback information. | ||
Arcade's error handling is designed to minimize boilerplate code while providing rich error information. In most cases, you don't need to explicitly handle errors in your tools because the `@tool` decorator automatically adapts common exceptions into appropriate Arcade errors. | ||
|
||
However, if you want to retry the tool call with additional content to improve the tool call's input parameters, you can raise a [RetryableToolError](/home/build-tools/retry-tools-with-improved-prompt) within the tool. | ||
## Error hierarchy | ||
|
||
## Common error scenarios | ||
Arcade uses a structured error hierarchy to categorize different types of errors: | ||
|
||
``` | ||
ToolkitError # (Abstract base class) | ||
├── ToolkitLoadError # Occurs during toolkit import | ||
└── ToolError # (Abstract) | ||
├── ToolDefinitionError # Detected when tool is added to catalog | ||
│ ├── ToolInputSchemaError # Invalid input parameter types/annotations | ||
│ └── ToolOutputSchemaError # Invalid return type annotations | ||
└── ToolRuntimeError # Errors during tool execution | ||
├── ToolSerializationError # (Abstract) | ||
│ ├── ToolInputError # JSON to Python conversion fails | ||
│ └── ToolOutputError # Python to JSON conversion fails | ||
└── ToolExecutionError # Errors during tool execution | ||
├── RetryableToolError # Tool can be retried with extra context | ||
├── ContextRequiredToolError # Additional context needed before retry | ||
├── FatalToolError # Unhandled bugs in the tool implementation | ||
└── UpstreamError # HTTP/API errors from external services | ||
└── UpstreamRateLimitError # Rate limiting errors from external services | ||
``` | ||
|
||
## Error adapters | ||
|
||
Let's explore some common error scenarios you might encounter: | ||
Error adapters automatically translate common exceptions (from `httpx`, `requests`, SDKs, etc.) into appropriate Arcade errors. This means zero boilerplate error handling code for you. To see which SDKs have error adapters, see [arcade_tdk/error_adapters/__init__.py](https://github.com/ArcadeAI/arcade-ai/blob/main/libs/arcade-tdk/arcade_tdk/error_adapters/__init__.py) | ||
|
||
### 1. Output type mismatch | ||
### Automatic error adaptation | ||
|
||
This occurs when the expected output type of a tool does not match the actual output type when executed. | ||
For tools using `httpx` or `requests`, error adaptation happens automatically: | ||
|
||
```python | ||
from typing import Annotated | ||
from arcade_tdk import tool | ||
import httpx | ||
|
||
```json | ||
{ | ||
"name": "tool_call_error", | ||
"message": "tool call failed - Example.Hello failed: Failed to | ||
serialize tool output" | ||
} | ||
@tool | ||
def fetch_data( | ||
url: Annotated[str, "The URL to fetch data from"], | ||
) -> Annotated[dict, "The data fetched from the API endpoint"]: | ||
"""Fetch data from an API endpoint.""" | ||
# No need to wrap in try/catch - Arcade handles HTTP errors automatically | ||
response = httpx.get(url) | ||
response.raise_for_status() # This will be adapted to UpstreamError if it raises | ||
return response.json() | ||
``` | ||
|
||
For example, the following tool will raise the above error because the tool's definition specifies that the output should be a string, but the tool returns a list: | ||
### Explicit error adapters | ||
|
||
For tools using specific SDKs, you can specify error adapters explicitly: | ||
|
||
```python | ||
import googleapiclient | ||
from typing import Annotated | ||
from arcade_tdk import tool | ||
from arcade_tdk.error_adapters import GoogleErrorAdapter | ||
|
||
@tool( | ||
requires_auth=Google(scopes=["https://www.googleapis.com/auth/gmail.readonly"]), | ||
error_adapters=[GoogleErrorAdapter] | ||
) | ||
def send_email( | ||
num_emails: Annotated[int, "The number of emails to send"], | ||
) -> Annotated[dict, "The emails sent using the Gmail API"]: | ||
"""Send an email using the Gmail API.""" | ||
# Google API Client errors will be automatically adapted to Upstream Arcade errors for you | ||
service = _build_gmail_service(context) | ||
emails = service.users.messages().get( | ||
userId="me", | ||
id=num_emails | ||
).execute() # This will be adapted to UpstreamError if it raises | ||
parsed_emails = _parse_emails(emails) | ||
return parsed_emails | ||
``` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Bug: Email Sending Example Fails: Retrieves Emails InsteadThe |
||
|
||
## When to raise errors explicitly | ||
|
||
While Arcade handles most errors automatically, there are specific cases where you should raise errors explicitly: | ||
|
||
### RetryableToolError | ||
|
||
Use when the LLM can retry the tool call with more context to improve the tool call's input parameters: | ||
|
||
```python | ||
from typing import Annotated | ||
from arcade_tdk import tool | ||
from arcade_tdk.errors import RetryableToolError | ||
|
||
@tool(requires_auth=Reddit(scopes=["read"])) | ||
def search_posts( | ||
subreddit: Annotated[str, "The subreddit to search in"], | ||
query: Annotated[str, "The query to search for"], | ||
) -> Annotated[list[dict], "The posts found in the subreddit"]: | ||
"""Search for posts in a subreddit.""" | ||
if is_invalid_subreddit(subreddit): | ||
# additional_prompt_content should be provided back to the LLM | ||
raise RetryableToolError( | ||
"Please specify a subreddit name, such as 'python' or 'programming'", | ||
additional_prompt_content=f"{subreddit} is an invalid subreddit name. Please specify a valid subreddit name" | ||
) | ||
# ... rest of implementation | ||
``` | ||
|
||
### ContextRequiredToolError | ||
|
||
Use when additional context from the user or orchestrator is needed before the tool call can be retried by an LLM: | ||
|
||
```python | ||
from os import path | ||
from typing import Annotated | ||
from arcade_tdk import tool | ||
from arcade_tdk.errors import ContextRequiredToolError | ||
|
||
@tool | ||
def hello(name: Annotated[str, "The name of the friend to greet"]) -> str: | ||
""" | ||
Says hello to a friend | ||
""" | ||
return ["hello", name] | ||
def delete_file(filename: Annotated[str, "The filename to delete"]) -> Annotated[str, "The filename that was deleted"]: | ||
"""Delete a file from the system.""" | ||
if not os.path.exists(filename): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
raise ContextRequiredToolError( | ||
"File with provided filename does not exist", | ||
additional_prompt_content=f"{filename} does not exist. Did you mean one of these: {get_valid_filenames()}", | ||
) | ||
# ... deletion logic | ||
``` | ||
|
||
### 2. Input parameter type error | ||
### ToolExecutionError | ||
|
||
This occurs when the input parameter of a tool is of an unsupported type. | ||
Use for unrecoverable, but known, errors when you want to provide specific error context: | ||
|
||
```python | ||
from typing import Annotated | ||
from arcade_tdk import tool | ||
from arcade_tdk.errors import ToolExecutionError | ||
|
||
@tool | ||
def process_data(data_id: Annotated[str, "The ID of the data to process"]) -> Annotated[dict, "The processed data"]: | ||
"""Process data by ID.""" | ||
try: | ||
data = get_data_from_database(data_id) | ||
except Exception as e: | ||
raise ToolExecutionError("Database connection failed.") from e | ||
# ... processing logic | ||
``` | ||
Type error encountered while adding tool list_org_repositories from | ||
arcade_github.tools.repositories. Reason: issubclass() arg 1 must be a class | ||
|
||
### UpstreamError | ||
|
||
Use for custom handling of upstream service errors: | ||
|
||
```python | ||
from arcade_tdk import tool | ||
from arcade_tdk.errors import UpstreamError | ||
import httpx | ||
|
||
@tool | ||
def create_issue(title: str, description: str) -> dict: | ||
"""Create a GitHub issue.""" | ||
try: | ||
response = httpx.post("/repos/owner/repo/issues", json={ | ||
"title": title, | ||
"body": description | ||
}) | ||
response.raise_for_status() | ||
except httpx.HTTPStatusError as e: | ||
if e.response.status_code == 422: | ||
raise UpstreamError( | ||
"Invalid issue data provided. Check title and description.", | ||
status_code=422 | ||
) from e | ||
# Let other HTTP errors be handled automatically | ||
raise | ||
|
||
return response.json() | ||
``` | ||
|
||
## Common error scenarios | ||
|
||
### Tool definition errors | ||
|
||
These errors occur when your tool has invalid definitions and are caught when the tool is loaded: | ||
|
||
#### Invalid input parameter types | ||
|
||
```python | ||
from arcade_tdk import tool | ||
|
||
@tool | ||
def invalid_tool(data: tuple[str, str, str]) -> str: # ❌ Tuples not supported | ||
"""This will raise a ToolInputSchemaError.""" | ||
return f"Hello {data[0]}" | ||
``` | ||
|
||
#### Missing return type annotation | ||
|
||
```python | ||
from arcade_tdk import tool | ||
|
||
@tool | ||
def invalid_tool(name: str): # ❌ Missing return type | ||
"""This will raise a ToolOutputSchemaError.""" | ||
return f"Hello {name}" | ||
``` | ||
|
||
For example, the following tool will raise the above error because the tool input parameter is of an unsupported type: | ||
#### Invalid parameter annotations | ||
|
||
```python | ||
from typing import Annotated | ||
from arcade_tdk import tool | ||
|
||
@tool | ||
def hello(names: Annotated[tuple[str, str, str], "The names of the friends to greet"]) -> str: | ||
""" | ||
Says hello to a list of friends | ||
""" | ||
return f"Hello, {names[0]}, {names[1]}, and {names[2]}!" | ||
def invalid_tool(name: Annotated[str, "desc1", "desc2", "extra"]) -> str: # ❌ Too many annotations | ||
"""This will raise a ToolInputSchemaError.""" | ||
return f"Hello {name}" | ||
``` | ||
|
||
### 3. Unexpected HTTP error during tool execution | ||
### Runtime errors | ||
|
||
This occurs when the tool makes an HTTP request and receives a non-2xx response. Specifically for the example below, the authenticated user did not have permission to access the private organization's repositories. | ||
These errors occur during tool execution: | ||
|
||
```json | ||
{ | ||
"name": "tool_call_error", | ||
"message": "tool call failed - Github.ListOrgRepositories failed: Error accessing 'https://api.github.com/orgs/a-private-org/repos': Failed to process request. Status code: 401" | ||
} | ||
#### Output type mismatch | ||
|
||
```python | ||
from typing import Annotated | ||
from arcade_tdk import tool | ||
|
||
@tool | ||
def invalid_output(name: Annotated[str, "Name to greet"]) -> str: | ||
"""Says hello to a friend.""" | ||
return ["hello", name] # ❌ Returns list instead of string | ||
``` | ||
|
||
This will raise a `ToolOutputError` because the return type doesn't match the annotation. | ||
|
||
## Best practices | ||
|
||
1. **Let Arcade handle most errors**: There's no need to wrap your tool logic in try/catch blocks unless you need custom error handling. | ||
|
||
2. **Use specific error types**: When you do need to raise errors explicitly, use the most specific error type available. | ||
|
||
3. **Include additional context**: For `RetryableToolError` and `ContextRequiredToolError`, use the `additional_prompt_content` parameter to guide the LLM or user. | ||
|
Uh oh!
There was an error while loading. Please reload this page.