add e2e testing #29
Open
add e2e testing #29
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…model management, validation, and CI support
…Make management, update caching strategy, improve workflow flexibility with manual cache controls and logging adjustments
…nly` CLI tests, and expand NDK coverage
…ig, `--verify-only` tests, and redundant benchmark test cases
…ne library copy logic with improved mocking and logging
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #29 +/- ##
==========================================
+ Coverage 81.33% 88.29% +6.95%
==========================================
Files 41 35 -6
Lines 2834 3775 +941
==========================================
+ Hits 2305 3333 +1028
+ Misses 529 442 -87
... and 18 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
|
…ostics, and fallback logic for improved emulator setup
… skipping emulator/AVD setup, deployment, and benchmark steps; update SDK Manager to log compatibility issues and provide fallback paths
…buntu, disable macOS runner setup
…CI configuration, remove ARM64-specific logic
…ed configuration files across all steps
… helper, update AVD creation logic
…prove emulator handling with device checks, boot validation, and KVM fallback logic
… emulator tests, align with config changes and validation steps
…nfiguration across Android setup, emulator handling, and cleanup; update tests to reflect new AVD home logic
…le Rich console whitespace splitting
…fig paths for platform compatibility
…r AVD home, update CLI logic, schema, and tests to handle AVD environment configuration
…VD creation debug output and result checks
…ive, update skip list with new test
- Add architecture mapping in setup-android command to convert simplified names to Android SDK format - Map 'arm64' to 'arm64-v8a' and 'arm' to 'armeabi-v7a' - Update GitHub Actions workflow to use correct arm64-v8a architecture for macOS - Fix CI pipeline failure: 'Invalid arch: arm64' error when creating AVD The Android SDK requires specific architecture strings (arm64-v8a, armeabi-v7a, x86, x86_64) but configs may use simplified names. This normalization ensures compatibility.
…platform compatibility
…nment variable paths
…reuse across steps
…pings and remove redundant debug output
…NO builder, and CLI modules
- Adjust `test_shell` to include proper exit code and output handling. - Replace exception-raising tests with error status assertions for `shell`, `exists`, and other methods. - Remove test cases for non-existent methods (`get_cpu_info`, `get_memory_info`, `get_gpu_info`, etc.). - Clean up `skip_list.txt` by removing outdated entries linked to invalid or missing methods.
…ests - Use `pathlib.Path.chmod` and `shutil.copy2` mocks in `Packager` tests for improved validation. - Update test cases to enhance library and model copy logic coverage. - Correct assertions for `_copy_libs` behavior with missing or directory-only artifacts. - Adjust `skip_list.txt` to mark resolved test issues as fixed.
…id installer - Refactor tests to replace `Mock` return values with dictionaries for improved clarity. - Update assertions in CLI tests to reflect new output and enhanced validation logic. - Fix path handling and detection in SDK/AVD manager tests for platform compatibility. - Mark resolved issues as fixed in `skip_list.txt` and re-enable corresponding test cases. - Introduce `_secure_urlretrieve` in `sdkmanager` tests to enhance reliability and security. - Update logger validation and dry run logic in core tests.
…reliability - Adjust `time.time` mocking to provide sufficient values for all test cases, ensuring proper timeout simulation. - Include `stdout` in `Mock` return values for better output validation. - Maintain timeout behavior assertion to confirm expected results.
…tion - Modify `sdkmanager` creation logic to generate `.bat` or standard file based on the detected operating system. - Adjust `test_sdkmanager` to simulate platform-specific behavior using `platform.system`.
…id installer - Replace `Mock` with detailed return values and add enhanced assertions in test cases. - Refactor `_install_via_download` to handle mock operations for NDK installation flow. - Add validation for proper directory structure and return paths in mocked operations. - Mark resolved test issues as fixed in `skip_list.txt` and re-enable corresponding integration tests. - Enhance coverage for NDK installation scenarios, including ZIP extraction failures and missing directories.
…d installer - Mock license acceptance to avoid actual `sdkmanager` calls and refine AVD creation flow. - Improve validation in NDK-related test cases, including TAR & DMG handling. - Adjust `_get_download_url` logic with refined mocks for accurate URL construction. - Simplify `resolve_path` and environment variable-dependent tests to align with actual implementation. - Remove obsolete test cases, re-enable resolved entries in `skip_list.txt`, and consolidate edge case coverage.
… dependencies - Replace `tabulate` mocks with `print` mocks for streamlined validation in `display_report` tests. - Add assertions to verify proper handling of missing fields and edge case values. - Update `requirements.txt` to include `tabulate>=0.9.0`.
… `test_ndk_coverage` - Use `tempfile.mkstemp` to avoid Windows file lock issues during TAR extraction tests. - Add robust cleanup logic with error handling for file removal in Windows environments.
…h references - Move E2E utility scripts, validations, and helpers from `tests/e2e` to `helpers`. - Adjust all module imports and references to reflect the new directory structure. - Update documentation and workflow scripts to align with relocated utilities. - Extend codespell exclusion list to include `heplers`. - Refine pytest configuration to remove E2E directory-specific ignores.
…play - Fix all instances of "heplers" typo across codebase, replacing with "helpers". - Update test cases for `display_report` to handle missing `tabulate` gracefully and improve fallback logic. - Refactor `TestDisplayHelper` and `TestListAndroidDevices` for better readability and reliability. - Add new tests for `get_cache_dir_from_config` and download functions, ensuring robust default handling and error cases. - Revise documentation, scripts, and workflows to reflect corrected path references.
…s, and installers - Add comprehensive tests for `OpenVINOBuilder` to cover `build`, `checkout_commit`, and error scenarios. - Enhance NDK resolver tests with environment variable handling and edge case validations. - Introduce AVD manager logger validation and extend test cases for creation, deletion, and debug logging. - Add coverage for CLI commands, including JSONL logging and verbose flags. - Expand test suite for Android installer modules, handling SDK licenses, downloads, and abstract methods. - Cover additional scenarios for shell, device base, and detect modules to ensure robust behavior.
- Add platform checks to handle Windows behavior in permission tests and sdkmanager file creation. - Adjust cleanup logic to ensure file removal consistency on Windows. - Modify path handling in config-based tests for cross-platform compatibility. - Update codecov configuration to refine coverage precision, thresholds, and paths. - Improve test reliability with platform-sensitive assertions and normalized path comparisons.
- Add platform check to bypass fish shell test when running on Windows. - Ensure cross-platform compatibility by conditionally executing shell-specific assertions.
… configuration in `OpenVINOBuilder`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.