-
Notifications
You must be signed in to change notification settings - Fork 233
Tests for workflow federated evaluation #1615
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Tests for workflow federated evaluation #1615
Conversation
Signed-off-by: Chaurasiya, Payal <[email protected]>
Signed-off-by: Chaurasiya, Payal <[email protected]>
Signed-off-by: Chaurasiya, Payal <[email protected]>
Signed-off-by: Chaurasiya, Payal <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces federated evaluation tests in the PQ pipeline by adding a new test function and integrating it into the GitHub workflows.
- Added a new helper function, activate_experimental_feature, and a corresponding test case test_federated_evaluation in the end-to-end test suite.
- Introduced a new workflow file (wf_federated_evaluation.yml) for running the federated evaluation tests.
- Updated the PQ pipeline configuration (pq_pipeline.yml) to include the federated evaluation test job.
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
File | Description |
---|---|
tests/end_to_end/test_suites/wf_federated_runtime_tests.py | Added helper for feature activation and a new test function for federated evaluation. |
.github/workflows/wf_federated_evaluation.yml | New workflow to run federated evaluation end-to-end tests. |
.github/workflows/pq_pipeline.yml | Updated pipeline configuration to include the new federated evaluation job. |
), "Notebook run failed" | ||
|
||
# Change the directory back to the original directory | ||
os.chdir(os.getcwd()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Restoring the working directory using os.chdir(os.getcwd()) does not revert to the original directory. Consider storing the original directory (e.g., using original_dir = os.getcwd()) before changing directories, and then restore it with os.chdir(original_dir) after the notebook run.
os.chdir(os.getcwd()) | |
os.chdir(original_dir) |
Copilot uses AI. Check for mistakes.
Signed-off-by: Chaurasiya, Payal <[email protected]>
@tanwarsh Test I added in the PQ pipeline and its failing. You can get the logs - |
This PR can get merge post the fix provided in - #1623 |
Signed-off-by: Chaurasiya, Payal <[email protected]>
…/openfl into wf-federated-evaluation
Bug - #1637 is raised for this test. |
Summary
Add Federated Evaluation tests in PQ pipeline
Type of Change (Mandatory)
Specify the type of change being made.
Description (Mandatory)
Add Federated Evaluation Notebook run in PQ pipeline
Testing
PQ pipeline run - https://github.com/payalcha/openfl/actions/runs/14999204468/job/42142421917