Skip to content

Conversation

payalcha
Copy link
Collaborator

Summary

Add Federated Evaluation tests in PQ pipeline

Type of Change (Mandatory)

Specify the type of change being made.

  • Feature enhancement

Description (Mandatory)

Add Federated Evaluation Notebook run in PQ pipeline

Testing

PQ pipeline run - https://github.com/payalcha/openfl/actions/runs/14999204468/job/42142421917

@payalcha payalcha requested review from Copilot and removed request for tanwarsh and ishaileshpant May 13, 2025 14:50
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces federated evaluation tests in the PQ pipeline by adding a new test function and integrating it into the GitHub workflows.

  • Added a new helper function, activate_experimental_feature, and a corresponding test case test_federated_evaluation in the end-to-end test suite.
  • Introduced a new workflow file (wf_federated_evaluation.yml) for running the federated evaluation tests.
  • Updated the PQ pipeline configuration (pq_pipeline.yml) to include the federated evaluation test job.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
tests/end_to_end/test_suites/wf_federated_runtime_tests.py Added helper for feature activation and a new test function for federated evaluation.
.github/workflows/wf_federated_evaluation.yml New workflow to run federated evaluation end-to-end tests.
.github/workflows/pq_pipeline.yml Updated pipeline configuration to include the new federated evaluation job.

), "Notebook run failed"

# Change the directory back to the original directory
os.chdir(os.getcwd())
Copy link
Preview

Copilot AI May 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restoring the working directory using os.chdir(os.getcwd()) does not revert to the original directory. Consider storing the original directory (e.g., using original_dir = os.getcwd()) before changing directories, and then restore it with os.chdir(original_dir) after the notebook run.

Suggested change
os.chdir(os.getcwd())
os.chdir(original_dir)

Copilot uses AI. Check for mistakes.

@payalcha
Copy link
Collaborator Author

@tanwarsh Test I added in the PQ pipeline and its failing.
https://github.com/payalcha/openfl/actions/runs/14999204468/job/42143395124

You can get the logs -
federated_evaluation_14999204468.zip

@payalcha
Copy link
Collaborator Author

This PR can get merge post the fix provided in - #1623

@payalcha payalcha changed the title Wf federated evaluation Tests for workflow federated evaluation May 14, 2025
@payalcha
Copy link
Collaborator Author

Bug - #1637 is raised for this test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant