Try cypress tests in parallel #252
Conversation
|
Using cypress tests in parallel requires that the tests are written to be run in isolation (testIsolation). The onboarding module tests are not yet updated for this, so we can't take advantage of these faster parallel tests until we can update tests in onboarding (or skip them when running in parallel). Running tests in parallel essentially makes them independent of each other and the state of the site/app. Initially our tests were written sequentially, in that the tests needed to be run in a certain order, or that test 2 depended on the finish state of test 1. When writing for test isolation, the tests can be run in any order. Which is what cypress cloud does when the tests are run in parallel, as the tests are run by whichever container is available to run them, so they must be independent of each other. This also leads to more stable tests, since they are written to test the specific feature, and not rely on other tests. |
* develop: (166 commits) set permalinks for coming soon endpoint call and add timeout for coming soon notice get after enabling coming soon force reload in dashboard test roll back performance to 3.1.6 due to media library issues onboarding 3.0.1 to fix potential logo dead end ensure site is live before running coming soon widget test roll performance update to 3.2.3 bump composer versions and audit fix npm packages 🤖 prep for release - 4.3.10 → 4.4.0 Fix lock conflict Bump onboarding to 3.0 NPM Dev(deps-dev): Bump cypress from 14.5.0 to 14.5.1 Skip onboarding v3 tests Composer(deps): Bump newfold-labs/wp-module-marketplace Delete old-style translation workflow. Support pre and post i18n scripts. Composer(deps): Bump newfold-labs/wp-module-solutions Composer(deps): Bump newfold-labs/wp-module-performance Composer(deps): Bump newfold-labs/wp-module-deactivation Composer(deps): Bump newfold-labs/wp-module-installer Composer(deps): Bump newfold-labs/wp-module-atomic from 1.3.0 to 1.3.1 ...
* develop: (75 commits)
Composer(deps): Bump newfold-labs/wp-module-features from 1.5.0 to 1.5.1
Pass `vars` using `env.
Use the `vars` context outside of `run`.
Don't include `{}`.
Switch to using an Actions variable for name value
Real REAL real repo name.
Ok real real repo name and author.
Use the correct author name.
Delete the Lint YML workflow.
Fix spacing.
Correct `on` event indentation.
Avoid legacy backticks.
Properly indicate unused variable.
Protect against globbing and word splitting.
Avoid the need to run unverified binaries.
undo indentation botch but fix line ordering
Guard against several caching related problems.
Guard against bot spoofing.
Avoid being unintentionally over permissive.
Avoid credentials from `checkout` persisting.
...
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| echo "PACKAGE=bluehost-wordpress-plugin" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Create Distribution Copy | ||
| run: rsync -r --include-from=.distinclude --exclude-from=.distignore . ${{ steps.workflow.outputs.DIST }}/${{ steps.workflow.outputs.PACKAGE }} |
Check failure
Code scanning / octoscan
Expression injection, "steps..outputs." is potentially untrusted. Error
| run: rsync -r --include-from=.distinclude --exclude-from=.distignore . ${{ steps.workflow.outputs.DIST }}/${{ steps.workflow.outputs.PACKAGE }} | ||
|
|
||
| - name: List Distribution Files | ||
| working-directory: ${{ steps.workflow.outputs.DIST }} |
Check failure
Code scanning / octoscan
Expression injection, "steps..outputs." is potentially untrusted. Error
| name: ${{ needs.build.outputs.PACKAGE }} | ||
|
|
||
| - name: Configure WordPress | ||
| run: echo '{"plugins":["${{ steps.workflow.outputs.DIST }}/${{ needs.build.outputs.PACKAGE }}"]}' > .wp-env.override.json |
Check failure
Code scanning / octoscan
Expression injection, "steps..outputs." is potentially untrusted. Error
| group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.head_ref || github.sha }} | ||
| cancel-in-progress: true | ||
| permissions: | ||
| pull-requests: write |
Check failure
Code scanning / zizmor
overly broad permissions Error
| - name: Checkout | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 |
Check warning
Code scanning / zizmor
credential persistence through GitHub Actions artifacts Warning
| echo "PACKAGE=bluehost-wordpress-plugin" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Create Distribution Copy | ||
| run: rsync -r --include-from=.distinclude --exclude-from=.distignore . ${{ steps.workflow.outputs.DIST }}/${{ steps.workflow.outputs.PACKAGE }} |
Check notice
Code scanning / zizmor
code injection via template expansion Note
| echo "PACKAGE=bluehost-wordpress-plugin" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Create Distribution Copy | ||
| run: rsync -r --include-from=.distinclude --exclude-from=.distignore . ${{ steps.workflow.outputs.DIST }}/${{ steps.workflow.outputs.PACKAGE }} |
Check notice
Code scanning / zizmor
code injection via template expansion Note
| - name: Checkout | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 |
Check warning
Code scanning / zizmor
credential persistence through GitHub Actions artifacts Warning
| name: ${{ needs.build.outputs.PACKAGE }} | ||
|
|
||
| - name: Configure WordPress | ||
| run: echo '{"plugins":["${{ steps.workflow.outputs.DIST }}/${{ needs.build.outputs.PACKAGE }}"]}' > .wp-env.override.json |
Check notice
Code scanning / zizmor
code injection via template expansion Note
| name: ${{ needs.build.outputs.PACKAGE }} | ||
|
|
||
| - name: Configure WordPress | ||
| run: echo '{"plugins":["${{ steps.workflow.outputs.DIST }}/${{ needs.build.outputs.PACKAGE }}"]}' > .wp-env.override.json |
Check notice
Code scanning / zizmor
code injection via template expansion Note
Proposed changes
Adding a new workflow to try running Cypress tests in parallel containers. It should make them roughly 4x faster, bringing down the wait from 45-60 minutes to 10-15.
Type of Change
Production
Development
Video
Checklist
Further comments