Skip to content

SDK: Normalize context-window and auth errors; raise typed exceptions when condenser is absent #979

@enyst

Description

@enyst

SDK: Normalize context-window and auth errors; raise typed exceptions when condenser is absent

Summary

Clients currently receive provider/LiteLLM-specific exceptions when the LLM context window is exceeded and no condenser is configured. Although the SDK defines LLMContextWindowExceedError, it is not raised in this scenario; the raw LiteLLM/OpenAI/BadRequest errors bubble up. This makes it hard for SDK clients (e.g., openhands-cli or downstream apps) to handle common error cases consistently.

This issue proposes a consistent, provider‑agnostic error surface and clearer client guidance.


Current behavior (with code references)

  • Agent handles context window exceeded only if a condenser is configured:
    • openhands-sdk/openhands/sdk/agent/agent.py: Agent.step wraps LLM calls. If self.condenser is not None and LLM.is_context_window_exceeded_exception(e) → emit CondensationRequest() and return; otherwise the exception propagates.
  • Detection logic for context-window exceeded is in LLM:
    • openhands-sdk/openhands/sdk/llm/llm.py: LLM.is_context_window_exceeded_exception(e) checks for litellm.ContextWindowExceededError and several provider-specific message patterns.
  • The SDK already defines LLMContextWindowExceedError:
    • openhands-sdk/openhands/sdk/llm/exceptions.py
  • If no condenser is configured and the LLM throws a context-window error, Agent.step re-raises the underlying provider exception. There is no fallback condensation and no SDK-typed error emitted.
  • For reference, the OpenHands server (not the SDK) has a controller shim that catches context-window conditions and either triggers truncation or throws its own LLMContextWindowExceedError. This is app-layer logic, not SDK.

Impact on clients

  • Without a condenser, clients get provider/LiteLLM-specific exceptions (BadRequestError, OpenAIError, etc.). It’s hard to reliably detect:
    • “context too long”
    • “invalid API key”
    • “rate limit”
    • “timeout”

Proposal

A) Raise typed SDK exceptions for common error classes

Define or reuse the following in openhands.sdk.llm.exceptions:

  • LLMContextWindowExceedError (exists)
  • LLMAuthenticationError (invalid API key, unauthorized)
  • LLMRateLimitError
  • LLMTimeoutError
  • LLMServiceUnavailableError
  • LLMBadRequestError (catch‑all 4xx not covered by above)

Include fields like status_code and provider, and set __cause__ to the original exception to preserve details.

B) Central exception mapping in the LLM

Add a single mapper used by both completion() and responses() call sites to translate provider exceptions into SDK exceptions:

  • Context window exceeded (e.g., ContextWindowExceededError, messages like “context length exceeded,” “prompt is too long,” etc.) → LLMContextWindowExceedError
  • 401/403 payloads or messages like “invalid api key,” “unauthorized,” “Missing API key” → LLMAuthenticationError
  • RateLimitErrorLLMRateLimitError
  • TimeoutLLMTimeoutError
  • ServiceUnavailableErrorLLMServiceUnavailableError
  • Remaining BadRequestErrorLLMBadRequestError

Reference existing patterns in LLM.is_context_window_exceeded_exception(e).

C) Agent behavior

  • If a condenser is configured and it handles condensation requests and LLM.is_context_window_exceeded_exception(e) → emit CondensationRequest().
  • Otherwise, let the LLM layer raise the mapped, SDK‑typed exception (preserving __cause__).

D) Documentation

Add an Errors section in the SDK README showing recommended client handling:

  • Context window exceeded: catch LLMContextWindowExceedError → suggest enabling a condenser or shortening inputs.
  • Authentication: catch LLMAuthenticationError → prompt user to set a valid API key.
  • Rate‑limit/Timeout: catch LLMRateLimitError/LLMTimeoutError → backoff/retry guidance.

E) Backward compatibility

  • Success paths are unchanged; only failure paths are normalized.
  • Preserve the original exception via chaining (__cause__) so logs/debugging retain provider details.

F) Test plan

  • With condenser: simulate context-window exceed → Agent emits CondensationRequest (no exception).
  • Without condenser: simulate context-window exceed → mapped LLMContextWindowExceedError raised by LLM.
  • Simulate 401/403 → LLMAuthenticationError.
  • Simulate RateLimitErrorLLMRateLimitError.
  • Simulate timeout → LLMTimeoutError.
  • Verify __cause__ points to the original provider exception.

G) Notes on defaults and CLI

  • The OpenHands server often enables a default condenser. The SDK should remain unopinionated, but we can add a simple helper to construct a reasonable default condenser (e.g., LLMSummarizingCondenser or PipelineCondenser).
  • As a follow‑up, add minimal handling to openhands-cli to catch LLMContextWindowExceedError and display a helpful prompt such as: “Enable condenser in settings to continue long sessions.”

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions