[Openhands] we should add test to test this init_state will actually modify state in-place #794
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR implements issue #757 by creating a comprehensive example that demonstrates how to use the OpenHands SDK to automatically scan a codebase for # TODO(openhands) comments and create pull requests to implement them. Features: - todo_scanner.py: Scans codebase for TODO(openhands) comments with filtering - todo_agent.py: Uses OpenHands to implement individual TODOs in feature branches - workflow.yml: GitHub Actions workflow for automation - Comprehensive README with setup and usage instructions - Complete test suite with 10 test cases The implementation follows the same patterns as examples/github_workflows/01_basic_action and provides practical automation for self-improving codebase capabilities. Co-authored-by: openhands <[email protected]>
Changes TODO(openhands: {pr_url}) to TODO(in progress: {pr_url}) for better clarity
that the TODO is being actively worked on.
Co-authored-by: openhands <[email protected]>
- Add prompt.py file with single PROMPT constant - Remove helper methods from todo_agent.py (create_feature_branch, generate_todo_prompt, create_pull_request) - Simplify TODO scanner to only scan .py, .ts, and .java files - Update update_todo_with_pr_url to verify agent created branch/PR and handle git history smartly - Update tests to reflect simplified scanner behavior - Update README to document new structure Co-authored-by: openhands <[email protected]>
- Reduce todo_scanner.py from ~280 lines to ~80 lines - Remove unnecessary complexity: binary file detection, markdown filtering, context extraction - Keep only essential functionality: scan .py/.ts/.java files for TODO(openhands) comments - Simplify tests to match streamlined implementation - Update README to reflect simplified structure Co-authored-by: openhands <[email protected]>
- Replace get_recent_branches() heuristic with get_pr_info() using GitHub API - Automatically extract head and base branch names from PR URL - Remove guesswork about which branch the agent created - Simplify update_todo_with_pr_url() function signature - Use GITHUB_TOKEN environment variable for API authentication - More reliable branch detection and TODO comment updates Co-authored-by: openhands <[email protected]>
- After agent.run(), detect feature branch by checking current branch - Use find_pr_for_branch() to find PR for the detected branch via GitHub API - Remove complex PR URL parsing from agent responses - Cleaner logic: initial_branch -> agent.run() -> current_branch -> find PR - More reliable than parsing agent text responses for PR URLs - Reduced from 304 to 297 lines with cleaner flow Co-authored-by: openhands <[email protected]>
- Created debug_workflow.py to trigger and monitor the workflow - Provides blocking execution with detailed logs and error reporting - Shows URLs of created PRs or error details - Updated README with debug script documentation Co-authored-by: openhands <[email protected]>
- Enhanced scanner.py with detailed logging and false positive filtering - Added logging to agent.py for better debugging - Updated debug_workflow.py with more verbose output - Added GitHub Actions workflow file to .github/workflows/ - Fixed scanner to exclude test files and documentation references - Scanner now correctly identifies only legitimate TODOs Co-authored-by: openhands <[email protected]>
- Added comprehensive README with setup, usage, and troubleshooting guides - Created test_local.py for local component testing - Documented smart filtering capabilities and debug tools - Added troubleshooting section with common issues and solutions - Included development workflow and contribution guidelines - Documented all features: smart scanning, AI implementation, PR management - Added examples of workflow execution and TODO format requirements The example is now complete and ready for production use. Co-authored-by: openhands <[email protected]>
- Fix line length violations by breaking long lines - Fix import sorting issues - Remove unnecessary f-string prefixes - Fix type annotation issue in test_local.py - All linting and type checking now passes Co-authored-by: openhands <[email protected]>
- Add comprehensive workflow simulation and testing - Create implementation summary documenting all features - Add test files for validation and debugging - All components tested and validated - Ready for production deployment Key features: - Smart TODO scanning with false positive filtering - OpenHands agent integration for TODO implementation - Automatic PR creation and TODO progress tracking - Comprehensive testing and debugging tools - Complete documentation and setup guides Fixes #757 Co-authored-by: openhands <[email protected]>
- Add comprehensive filtering for processed TODOs with PR URLs, progress markers, and GitHub URLs - Improve false positive detection for print statements, docstrings, and quoted strings - Implement robust docstring state tracking to filter TODOs inside multi-line docstrings - Add comprehensive test suite (test_scanner_filtering.py) to validate filtering capabilities - Update README with detailed filtering documentation - Ensure scanner only processes unhandled TODOs to avoid duplicate PRs Co-authored-by: openhands <[email protected]>
- Add pull_request trigger with 'labeled' type - Only run when 'automatic-todo' label is added to PR - Maintains existing workflow_dispatch functionality Co-authored-by: openhands <[email protected]>
- Use litellm_proxy/claude-sonnet-4-5-20250929 model - Use https://llm-proxy.eval.all-hands.dev as base URL - Pass LLM configuration to agent execution environment Co-authored-by: openhands <[email protected]>
- Add rich>=13.0.0 to SDK dependencies (used extensively in codebase) - Add pydantic and rich to tools build requirements - Resolves ModuleNotFoundError during package builds Co-authored-by: openhands <[email protected]>
- Include all SDK dependencies in tools build requirements - Addresses import issues during package build process - Note: Circular import issue with glob module still exists Co-authored-by: openhands <[email protected]>
- Add logic to capture continuation comment lines after TODO(openhands) - Combine multi-line TODO descriptions into single coherent text - Stop at comments that appear to be separate (capital letter heuristic) - Handle empty comment lines gracefully - Preserve full context for better agent understanding Example: # TODO(openhands): we should add test to test this init_state will actually # modify state in-place Now captures: 'we should add test to test this init_state will actually modify state in-place' Co-authored-by: openhands <[email protected]>
Enhanced Features: - Agent now outputs structured result files with TODO processing status - Workflow collects all result artifacts and creates detailed summary - Summary includes list of TODOs with their associated PR URLs - Shows processing status (success/failed/partial) for each TODO - Provides clickable PR links in GitHub Actions summary - Handles error cases and partial completions gracefully Summary Format: ## TODOs and Pull Requests 1. **file.py** line 123: Description of TODO... - 🔗 PR: [https://github.com/repo/pull/456](https://github.com/repo/pull/456) - ✅ Status: success This provides clear visibility into which TODOs were processed and their outcomes. Co-authored-by: openhands <[email protected]>
- Replace forward slashes with underscores in artifact names - GitHub Actions artifact names cannot contain forward slashes - Prevents 'The artifact name is not valid' error Example: 'todo-processing-logs-openhands/sdk/agent/agent.py-88' Becomes: 'todo-processing-logs-openhands_sdk_agent_agent.py-88' Co-authored-by: openhands <[email protected]>
- Added warnings.filterwarnings to suppress UserWarning from pydantic module - Specifically targets PydanticSerializationUnexpectedValue warnings - Improves log readability by removing verbose serialization warnings - Warnings were appearing during LLM interactions and cluttering output Co-authored-by: openhands <[email protected]>
- Replace invalid 'replace' function with proper shell command - Use 'tr' command to sanitize file paths for artifact names - Add separate step to sanitize artifact name before upload - Prevents 'Unrecognized function: replace' workflow validation error Example transformation: 'openhands/sdk/agent/agent.py' -> 'openhands_sdk_agent_agent.py' Co-authored-by: openhands <[email protected]>
…nds-AI/agent-sdk into openhands/todo-management-example
- Removed update_todo_with_pr_url function that pushed to main branch - Agent no longer modifies TODO comments in the main branch - Maintains PR tracking and status reporting without main branch changes - Updated documentation to reflect new behavior - Follows best practice of not pushing directly to main branch The workflow now: 1. Scans TODOs and creates feature branches with PRs 2. Tracks processing status and PR URLs 3. Generates comprehensive summary without modifying main branch Co-authored-by: openhands <[email protected]>
Removed files: - debug_workflow.py (debug tool) - simple_agent_test.py (debug file) - IMPLEMENTATION_SUMMARY.md (extra documentation) - test_*.py (all test files) - workflow.yml (duplicate, actual workflow is in .github/workflows/) - __pycache__/ (cache directory) Updated README.md: - Removed references to deleted debug and test files - Simplified component overview (3 components instead of 4) - Updated features list to focus on core functionality - Streamlined setup and usage instructions - Removed debug/testing sections Remaining essential files: - README.md (comprehensive documentation) - scanner.py (TODO scanner with multi-line support) - agent.py (main TODO processing agent) - prompt.py (agent prompt template) Co-authored-by: openhands <[email protected]>
- Added test_scan_rust_file() to verify .rs file support - Added test_custom_todo_identifier() for configurable identifiers - Added test_custom_identifier_with_special_chars() for regex escaping - Fixed line length compliance for pre-commit checks Co-authored-by: openhands <[email protected]>
- Remove document separators (---) from workflow files - Fix indentation to use consistent 2-space formatting - Align YAML structure properly for better readability Co-authored-by: openhands <[email protected]>
- Remove leading slash from example path filter to match actual paths - Prevents scanner from processing mock TODOs in example files - Fixes workflow failures caused by empty TODO descriptions Co-authored-by: openhands <[email protected]>
- Remove unnecessary git command functions (run_git_command, get_current_branch, find_pr_for_branch) - Remove post-analysis logic after conversation run - Trust the agent to create feature branch and pull request automatically - Simplify result structure by removing pr_url and branch fields - Reduce code complexity and improve maintainability Co-authored-by: openhands <[email protected]>
- Update openhands-sdk subdirectory path from openhands/sdk to openhands-sdk - Update openhands-tools subdirectory path from openhands/tools to openhands-tools - Move TODO management files from examples/github_workflows to examples/03_github_workflows - Update all URLs to reference correct directory structure - Update agent script filename from agent.py to agent_script.py - Add type ignore comments for imports that will be available in workflow environment The agent script is already simplified as requested: - Trusts the agent to handle branch creation and PR management - Removed post-analysis after conversation run - No need for find_pr_for_branch, get_current_branch, or run_git_command functions Co-authored-by: openhands <[email protected]>
- Update exclusion pattern from 'examples/github_workflows/03_todo_management/' to 'examples/03_github_workflows/03_todo_management/' to match new directory structure - Scanner now correctly finds TODO comments in the reorganized codebase Co-authored-by: openhands <[email protected]>
…nds-AI/agent-sdk into openhands/todo-management-example
…path - Update scanner.py path from examples/github_workflows/ to examples/03_github_workflows/ - Update agent_script.py and prompt.py paths to match correct directory structure - Fixes 'No such file or directory' errors in GitHub Actions workflow Co-authored-by: openhands <[email protected]>
- Change openhands/sdk to openhands-sdk (hyphenated) - Change openhands/tools to openhands-tools (hyphenated) - Fixes 'has no subdirectory' error in GitHub Actions workflow - Matches actual directory structure in main branch Co-authored-by: openhands <[email protected]>
- Change from main branch to openhands/todo-management-example branch - Fixes 'No such file or directory' error when copying agent files - TODO management example files only exist on the feature branch - Ensures workflow can access all required files (scanner.py, agent_script.py, prompt.py) Co-authored-by: openhands <[email protected]>
…nds-AI/agent-sdk into openhands/todo-management-example
- Remove complex result structure and JSON file output - Align with basic action script pattern for consistency - Simplify error handling with direct sys.exit() - Update workflow to remove result file checking - Maintain all core functionality without breaking changes Co-authored-by: openhands <[email protected]>
Implements tests for the TODO comment at line 88 in agent.py: - Test that init_state modifies state in-place by adding events - Test that init_state initializes agent tools - Test that init_state doesn't add duplicate SystemPromptEvent - Test that init_state preserves state identity Co-authored-by: openhands <[email protected]>
github-actions
bot
requested review from
malhotra5,
simonrosenberg and
xingyaoww
|
@simonrosenberg is this expected? |
|
|
[Automatic Post]: This PR seems to be currently waiting for review. @xingyaoww, @simonrosenberg, @malhotra5, could you please take a look when you have a chance? |
1 similar comment
|
[Automatic Post]: This PR seems to be currently waiting for review. @xingyaoww, @simonrosenberg, @malhotra5, could you please take a look when you have a chance? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR addresses the TODO comment at line 88 in
openhands-sdk/openhands/sdk/agent/agent.py:Changes
tests/sdk/agent/test_agent_init_state.pywith 4 comprehensive testsAgent.init_stateproperly modifies the conversation state in-placeTests Added
Test Results
All tests pass successfully:
All existing agent tests continue to pass:
Related