Skip to content

feat: ACI-4001 add firefox to benchmark #166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: ACI-4001-comp-cache-benchmark
Choose a base branch
from
Open

feat: ACI-4001 add firefox to benchmark #166

Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
e4deda7
feat: ACI-4001 add firefox to benchmark
duplabe Sep 18, 2025
522f423
chore: ACI-4001 fix BENCHMARK_APP_NAMEs
duplabe Sep 18, 2025
553e587
chore: ACI-4001 fix firefox benchmarks
duplabe Sep 18, 2025
37fb199
Merge remote-tracking branch 'origin/ACI-4001-comp-cache-benchmark' i…
duplabe Sep 18, 2025
53271bb
chore: ACI-4001 fix firefox benchmarks
duplabe Sep 18, 2025
29fee3f
chore: ACI-4001 fix firefox benchmarks
duplabe Sep 18, 2025
ff0fc49
chore: ACI-4001 fix firefox benchmarks
duplabe Sep 18, 2025
4b8a7bc
Merge remote-tracking branch 'origin/ACI-4001-comp-cache-benchmark' i…
duplabe Sep 18, 2025
57c3146
chore: ACI-4001 fix firefox benchmarks
duplabe Sep 18, 2025
234a8a6
chore: ACI-4001 fix firefox benchmarks
duplabe Sep 18, 2025
01f7d48
chore: ACI-4001 fix firefox benchmarks
duplabe Sep 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
240 changes: 229 additions & 11 deletions bitrise.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1038,6 +1038,7 @@ workflows:
Tuist
Calendar
IceCubesApp
Firefox
opts:
is_expand: false
benchmark_wordpress_build_baseline:
Expand Down Expand Up @@ -1880,7 +1881,6 @@ workflows:
- USE_COMP_CACHE: $USE_COMP_CACHE
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- script:
title:
inputs:
- content: |-
set -exo pipefail
Expand Down Expand Up @@ -1919,7 +1919,6 @@ workflows:
- USE_COMP_CACHE: $USE_COMP_CACHE
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- script:
title:
inputs:
- content: |-
set -exo pipefail
Expand Down Expand Up @@ -1958,7 +1957,6 @@ workflows:
- USE_COMP_CACHE: $USE_COMP_CACHE
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- script:
title:
inputs:
- content: |-
set -exo pipefail
Expand Down Expand Up @@ -1997,7 +1995,6 @@ workflows:
- USE_COMP_CACHE: $USE_COMP_CACHE
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- script:
title:
inputs:
- content: |-
set -exo pipefail
Expand Down Expand Up @@ -2036,7 +2033,6 @@ workflows:
- USE_COMP_CACHE: $USE_COMP_CACHE
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- script:
title:
inputs:
- content: |-
set -exo pipefail
Expand All @@ -2055,6 +2051,218 @@ workflows:
- USE_DD_CACHE: $USE_DD_CACHE
- USE_SPM: $USE_SPM
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
benchmark_firefox_build_baseline:
envs:
- USE_DD_CACHE: "false"
- USE_SPM: "false"
- USE_COMP_CACHE: "false"
- BENCHMARK_APP_NAME: Firefox
meta:
bitrise.io:
stack: osx-xcode-26.0.x-edge
machine_type_id: g2.mac.large
steps:
- bundle::benchmark-setup:
inputs:
- REPO_URL: [email protected]:mozilla-mobile/firefox-ios.git
- BRANCH: main
- USE_DD_CACHE: $USE_DD_CACHE
- USE_SPM: $USE_SPM
- USE_COMP_CACHE: $USE_COMP_CACHE
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- script:
inputs:
- content: |-
set -exo pipefail

./bootstrap.sh
xcodebuild -downloadComponent MetalToolchain

cd firefox-ios/Client.xcodeproj
sed -i '' 's/"Fennec Development"/"Bitrise Firefox iOS Dev"/' project.pbxproj
sed -i '' 's/Fennec ShareTo Development/Bitrise Firefox iOS Dev - Share To/' project.pbxproj
sed -i '' 's/Fennec WidgetKit Development/Bitrise Firefox iOS Dev - WidgetKit/' project.pbxproj
sed -i '' 's/"XCUITests"/"Bitrise Firefox iOS Dev - XCUI Tests"/' project.pbxproj
sed -i '' 's/Fennec NotificationService Development/Bitrise Firefox iOS Dev - Notification Service/' project.pbxproj
sed -i '' 's/CODE_SIGN_IDENTITY = "iPhone Developer"/CODE_SIGN_IDENTITY = "iPhone Distribution"/' project.pbxproj
cd -

npm install
npm run build
- bundle::benchmark_start_measure:
inputs:
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- xcode-build-for-simulator:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐛 Bug

The Firefox baseline workflow is missing the configuration parameter that other workflows have, which could cause build inconsistencies.

🔄 Suggestion:

Suggested change
- xcode-build-for-simulator:
- xcode-build-for-simulator:
inputs:
- project_path: firefox-ios/Client.xcodeproj
- scheme: Fennec
- configuration: Debug
- destination: 'platform=iOS Simulator,name=iPhone 17,OS=26.0'
- xcodebuild_options: CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO

inputs:
- project_path: firefox-ios/Client.xcodeproj
- scheme: Fennec
- destination: 'platform=iOS Simulator,name=iPhone 17,OS=26.0'
- xcodebuild_options: CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO
- bundle::benchmark_finish:
inputs:
- USE_DD_CACHE: $USE_DD_CACHE
- USE_SPM: $USE_SPM
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
benchmark_firefox_build_cc:
envs:
- USE_DD_CACHE: "false"
- USE_SPM: "false"
- USE_COMP_CACHE: "true"
- BENCHMARK_APP_NAME: Firefox_CC
meta:
bitrise.io:
stack: osx-xcode-26.0.x-edge
machine_type_id: g2.mac.large
steps:
- bundle::benchmark-setup:
inputs:
- REPO_URL: [email protected]:mozilla-mobile/firefox-ios.git
- BRANCH: main
- USE_DD_CACHE: $USE_DD_CACHE
- USE_SPM: $USE_SPM
- USE_COMP_CACHE: $USE_COMP_CACHE
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- script:
inputs:
- content: |-
set -exo pipefail

./bootstrap.sh
xcodebuild -downloadComponent MetalToolchain
- bundle::benchmark_start_measure:
inputs:
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- xcode-build-for-simulator:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐛 Bug

The Firefox CC workflow is missing the configuration parameter that other workflows have, which could cause build inconsistencies.

🔄 Suggestion:

Suggested change
- xcode-build-for-simulator:
- xcode-build-for-simulator:
inputs:
- project_path: firefox-ios/Client.xcodeproj
- scheme: Fennec
- configuration: Debug
- destination: 'platform=iOS Simulator,name=iPhone 17,OS=26.0'
- xcodebuild_options: CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO

inputs:
- project_path: firefox-ios/Client.xcodeproj
- scheme: Fennec
- destination: 'platform=iOS Simulator,name=iPhone 17,OS=26.0'
- xcodebuild_options: CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO
- bundle::benchmark_finish:
inputs:
- USE_DD_CACHE: $USE_DD_CACHE
- USE_SPM: $USE_SPM
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
benchmark_firefox_build_cc_dd:
envs:
- USE_DD_CACHE: "true"
- USE_SPM: "false"
- USE_COMP_CACHE: "true"
- BENCHMARK_APP_NAME: Firefox_CC_DD
meta:
bitrise.io:
stack: osx-xcode-26.0.x-edge
machine_type_id: g2.mac.large
steps:
- bundle::benchmark-setup:
inputs:
- REPO_URL: [email protected]:mozilla-mobile/firefox-ios.git
- BRANCH: main
- USE_DD_CACHE: $USE_DD_CACHE
- USE_SPM: $USE_SPM
- USE_COMP_CACHE: $USE_COMP_CACHE
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- script:
inputs:
- content: |-
set -exo pipefail

./bootstrap.sh
xcodebuild -downloadComponent MetalToolchain
- bundle::benchmark_start_measure:
inputs:
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- xcode-build-for-simulator:
inputs:
- project_path: firefox-ios/Client.xcodeproj
- scheme: Fennec
- destination: 'platform=iOS Simulator,name=iPhone 17,OS=26.0'
- xcodebuild_options: CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO
- bundle::benchmark_finish:
inputs:
- USE_DD_CACHE: $USE_DD_CACHE
- USE_SPM: $USE_SPM
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
benchmark_firefox_build_cc_spm:
envs:
- USE_DD_CACHE: "false"
- USE_SPM: "true"
- USE_COMP_CACHE: "true"
- BENCHMARK_APP_NAME: Firefox_CC_SPM
meta:
bitrise.io:
stack: osx-xcode-26.0.x-edge
machine_type_id: g2.mac.large
steps:
- bundle::benchmark-setup:
inputs:
- REPO_URL: [email protected]:mozilla-mobile/firefox-ios.git
- BRANCH: main
- USE_DD_CACHE: $USE_DD_CACHE
- USE_SPM: $USE_SPM
- USE_COMP_CACHE: $USE_COMP_CACHE
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- script:
inputs:
- content: |-
set -exo pipefail

./bootstrap.sh
xcodebuild -downloadComponent MetalToolchain
- bundle::benchmark_start_measure:
inputs:
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- xcode-build-for-simulator:
inputs:
- project_path: firefox-ios/Client.xcodeproj
- scheme: Fennec
- destination: 'platform=iOS Simulator,name=iPhone 17,OS=26.0'
- xcodebuild_options: CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO
- bundle::benchmark_finish:
inputs:
- USE_DD_CACHE: $USE_DD_CACHE
- USE_SPM: $USE_SPM
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
benchmark_firefox_build_cc_dd_spm:
envs:
- USE_DD_CACHE: "true"
- USE_SPM: "true"
- USE_COMP_CACHE: "true"
- BENCHMARK_APP_NAME: Firefox_CC_DD_SPM
meta:
bitrise.io:
stack: osx-xcode-26.0.x-edge
machine_type_id: g2.mac.large
steps:
- bundle::benchmark-setup:
inputs:
- REPO_URL: [email protected]:mozilla-mobile/firefox-ios.git
- BRANCH: main
- USE_DD_CACHE: $USE_DD_CACHE
- USE_SPM: $USE_SPM
- USE_COMP_CACHE: $USE_COMP_CACHE
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- script:
inputs:
- content: |-
set -exo pipefail

./bootstrap.sh
xcodebuild -downloadComponent MetalToolchain
- bundle::benchmark_start_measure:
inputs:
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
- xcode-build-for-simulator:
inputs:
- project_path: firefox-ios/Client.xcodeproj
- scheme: Fennec
- destination: 'platform=iOS Simulator,name=iPhone 17,OS=26.0'
- xcodebuild_options: CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO
- bundle::benchmark_finish:
inputs:
- USE_DD_CACHE: $USE_DD_CACHE
- USE_SPM: $USE_SPM
- BENCHMARK_APP_NAME: $BENCHMARK_APP_NAME
step_bundles:
generate_gradle_verification_reference:
steps:
Expand Down Expand Up @@ -2303,7 +2511,7 @@ step_bundles:
- [email protected]:
run_if: '{{enveq "USE_SPM" "true"}}'
inputs:
- verbose: true
- verbose: true
benchmark_start_measure:
inputs:
- BENCHMARK_APP_NAME: "$BENCHMARK_APP_NAME"
Expand Down Expand Up @@ -2333,7 +2541,7 @@ step_bundles:
- [email protected]:
run_if: '{{enveq "USE_SPM" "true"}}'
inputs:
- verbose: true
- verbose: true
- git::https://github.com/bitrise-steplib/bitrise-step-save-xcodebuild-cache.git@main:
run_if: '{{enveq "USE_DD_CACHE" "true"}}'
inputs:
Expand Down Expand Up @@ -2386,14 +2594,19 @@ pipelines:
# benchmark_tuist_build_dd_spm_cc: {}
benchmark_calendar_build_baseline: { }
benchmark_calendar_build_cc: { }
# benchmark_calendar_build_cc_dd: {}
# benchmark_calendar_build_cc_dd: {}
benchmark_calendar_build_cc_spm: { }
# benchmark_calendar_build_cc_dd_spm: {}
# benchmark_calendar_build_cc_dd_spm: {}
benchmark_icecubesapp_build_baseline: { }
benchmark_icecubesapp_build_cc: { }
# benchmark_icecubesapp_build_cc_dd: {}
# benchmark_icecubesapp_build_cc_dd: {}
benchmark_icecubesapp_build_cc_spm: { }
# benchmark_icecubesapp_build_cc_dd_spm: {}
# benchmark_icecubesapp_build_cc_dd_spm: {}
benchmark_firefox_build_baseline: { }
benchmark_firefox_build_cc: { }
# benchmark_firefox_build_cc_dd: {}
benchmark_firefox_build_cc_spm: { }
# benchmark_firefox_build_cc_dd_spm: {}
benchmark_results:
depends_on:
- benchmark_wordpress_build_baseline
Expand Down Expand Up @@ -2421,3 +2634,8 @@ pipelines:
# - benchmark_icecubesapp_build_cc_dd
- benchmark_icecubesapp_build_cc_spm
# - benchmark_icecubesapp_build_cc_dd_spm
- benchmark_firefox_build_baseline
- benchmark_firefox_build_cc
# - benchmark_firefox_build_cc_dd
- benchmark_firefox_build_cc_spm
# - benchmark_firefox_build_cc_dd_spm