-
Notifications
You must be signed in to change notification settings - Fork 39
Description
SDK: Normalize context-window and auth errors; raise typed exceptions when condenser is absent
Summary
Clients currently receive provider/LiteLLM-specific exceptions when the LLM context window is exceeded and no condenser is configured. Although the SDK defines LLMContextWindowExceedError, it is not raised in this scenario; the raw LiteLLM/OpenAI/BadRequest errors bubble up. This makes it hard for SDK clients (e.g., openhands-cli or downstream apps) to handle common error cases consistently.
This issue proposes a consistent, provider‑agnostic error surface and clearer client guidance.
Current behavior (with code references)
- Agent handles context window exceeded only if a condenser is configured:
openhands-sdk/openhands/sdk/agent/agent.py:Agent.stepwraps LLM calls. Ifself.condenser is not NoneandLLM.is_context_window_exceeded_exception(e)→ emitCondensationRequest()and return; otherwise the exception propagates.
- Detection logic for context-window exceeded is in LLM:
openhands-sdk/openhands/sdk/llm/llm.py:LLM.is_context_window_exceeded_exception(e)checks forlitellm.ContextWindowExceededErrorand several provider-specific message patterns.
- The SDK already defines
LLMContextWindowExceedError:openhands-sdk/openhands/sdk/llm/exceptions.py
- If no condenser is configured and the LLM throws a context-window error,
Agent.stepre-raises the underlying provider exception. There is no fallback condensation and no SDK-typed error emitted. - For reference, the OpenHands server (not the SDK) has a controller shim that catches context-window conditions and either triggers truncation or throws its own
LLMContextWindowExceedError. This is app-layer logic, not SDK.
Impact on clients
- Without a condenser, clients get provider/LiteLLM-specific exceptions (
BadRequestError,OpenAIError, etc.). It’s hard to reliably detect:- “context too long”
- “invalid API key”
- “rate limit”
- “timeout”
Proposal
A) Raise typed SDK exceptions for common error classes
Define or reuse the following in openhands.sdk.llm.exceptions:
LLMContextWindowExceedError(exists)LLMAuthenticationError(invalid API key, unauthorized)LLMRateLimitErrorLLMTimeoutErrorLLMServiceUnavailableErrorLLMBadRequestError(catch‑all 4xx not covered by above)
Include fields like status_code and provider, and set __cause__ to the original exception to preserve details.
B) Central exception mapping in the LLM
Add a single mapper used by both completion() and responses() call sites to translate provider exceptions into SDK exceptions:
- Context window exceeded (e.g.,
ContextWindowExceededError, messages like “context length exceeded,” “prompt is too long,” etc.) →LLMContextWindowExceedError - 401/403 payloads or messages like “invalid api key,” “unauthorized,” “Missing API key” →
LLMAuthenticationError RateLimitError→LLMRateLimitErrorTimeout→LLMTimeoutErrorServiceUnavailableError→LLMServiceUnavailableError- Remaining
BadRequestError→LLMBadRequestError
Reference existing patterns in LLM.is_context_window_exceeded_exception(e).
C) Agent behavior
- If a condenser is configured and it handles condensation requests and
LLM.is_context_window_exceeded_exception(e)→ emitCondensationRequest(). - Otherwise, let the LLM layer raise the mapped, SDK‑typed exception (preserving
__cause__).
D) Documentation
Add an Errors section in the SDK README showing recommended client handling:
- Context window exceeded: catch
LLMContextWindowExceedError→ suggest enabling a condenser or shortening inputs. - Authentication: catch
LLMAuthenticationError→ prompt user to set a valid API key. - Rate‑limit/Timeout: catch
LLMRateLimitError/LLMTimeoutError→ backoff/retry guidance.
E) Backward compatibility
- Success paths are unchanged; only failure paths are normalized.
- Preserve the original exception via chaining (
__cause__) so logs/debugging retain provider details.
F) Test plan
- With condenser: simulate context-window exceed → Agent emits
CondensationRequest(no exception). - Without condenser: simulate context-window exceed → mapped
LLMContextWindowExceedErrorraised by LLM. - Simulate 401/403 →
LLMAuthenticationError. - Simulate
RateLimitError→LLMRateLimitError. - Simulate timeout →
LLMTimeoutError. - Verify
__cause__points to the original provider exception.
G) Notes on defaults and CLI
- The OpenHands server often enables a default condenser. The SDK should remain unopinionated, but we can add a simple helper to construct a reasonable default condenser (e.g.,
LLMSummarizingCondenserorPipelineCondenser). - As a follow‑up, add minimal handling to
openhands-clito catchLLMContextWindowExceedErrorand display a helpful prompt such as: “Enable condenser in settings to continue long sessions.”