Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
144 commits
Select commit Hold shift + click to select a range
2634664
Add scripts, configs, and envs for the JAX's new CI
nitins17 Aug 27, 2024
738a7bf
add workflow for running bazel gpu tests locally
nitins17 Aug 27, 2024
e901247
make some tweaks
nitins17 Aug 28, 2024
cab4ae3
Fix runner naming
nitins17 Aug 28, 2024
f169816
check script name with regex match operator
nitins17 Aug 28, 2024
0a5e1af
clean up
nitins17 Aug 28, 2024
8210c6a
fix typos and clean up
nitins17 Aug 28, 2024
6cfb12b
fix path to env
nitins17 Aug 28, 2024
3aaa330
fix path to env (actually)
nitins17 Aug 28, 2024
f2fb445
move wait for connection step to just before running tests
nitins17 Aug 28, 2024
222542d
change docker image for bazel local gpu test workflow
nitins17 Aug 28, 2024
8fe81fb
change image to tag instead of sha
nitins17 Aug 28, 2024
6666472
set default Python version for all builds
nitins17 Aug 28, 2024
f7b47cf
set default Python version for jax-cuda-pjrt and read CLI argument fo…
nitins17 Aug 28, 2024
057b8e3
Create a matrix for building JAX artifacts
nitins17 Aug 28, 2024
843afe3
add default python to matrix for jax-cuda-pjrt
nitins17 Aug 28, 2024
82f0454
exlude pjrt from the python matrix
nitins17 Aug 28, 2024
23fb8c9
add workflow and scripts for running pytests
nitins17 Aug 29, 2024
875a271
make pytest script executable
nitins17 Aug 29, 2024
6384f73
add workflow for pytest
nitins17 Aug 29, 2024
320bf83
change workflow name
nitins17 Aug 29, 2024
204c029
try fixing workflow
nitins17 Aug 29, 2024
cc4e210
fix runner name in pytest workflow
nitins17 Aug 29, 2024
4474934
add connection wait for pytest workflow and fix typo in container fie…
nitins17 Aug 29, 2024
6ae0257
dummy change; push to make github actions detect labels
nitins17 Aug 29, 2024
d7ab9e8
dummy change; push to make github actions detect labels
nitins17 Aug 29, 2024
4858b38
set clang path for all linux ci builds
nitins17 Aug 29, 2024
30a3951
install wheels with system python only for pytests
nitins17 Aug 29, 2024
ff2e5fe
Merge branch 'main' of https://github.com/google-ml-infra/jax-fork in…
nitins17 Aug 30, 2024
60f1231
install missing docker dependencies
nitins17 Aug 30, 2024
e17ca80
install missing docker deps specifically with python3
nitins17 Aug 30, 2024
d9bfc98
fix workflows failing
nitins17 Aug 30, 2024
b1b0953
update docker images in the artifacts workflow
nitins17 Sep 3, 2024
045f4b1
no-op; make github detect label change
nitins17 Sep 3, 2024
d68dcc2
add remaining scripts and workflows for running tests
nitins17 Sep 4, 2024
6105043
update workflow names
nitins17 Sep 4, 2024
22ce7ac
fix typos and add missing env
nitins17 Sep 4, 2024
b1a6ace
disable artifact build workflow
nitins17 Sep 4, 2024
ea3cba3
pass hermetic python env correctly in bazel cpu workflow
nitins17 Sep 4, 2024
077d76b
update gpu workfow runner name
nitins17 Sep 4, 2024
5981ec9
rework some envs
nitins17 Sep 4, 2024
b0b3171
rename JAXCI_USE_RBE to JAXCI_BUILD_ARTIFACT_WITH_RBE
nitins17 Sep 4, 2024
6cba3bd
Fix typo
nitins17 Sep 4, 2024
4c8afb9
add windows runner to build artifacts workflow and disable gpu workflows
nitins17 Sep 4, 2024
6630dc0
clean up bazel gpu tests
nitins17 Sep 4, 2024
4c6090e
rework how envs are passed to docker container inside setup_docker.sh
nitins17 Sep 4, 2024
850fa06
disable running in container to test windows builds
nitins17 Sep 4, 2024
958010d
enable bazel gpu jobs
nitins17 Sep 4, 2024
1f91951
tweak some setup settings
nitins17 Sep 4, 2024
1de1480
change how envs are passed to the container in setup_docker.sh
nitins17 Sep 5, 2024
3be81b6
add wait for connection to gpu workflow for testing and fix small iss…
nitins17 Sep 5, 2024
ff0062d
install wheels locally for bazel test local
nitins17 Sep 5, 2024
f57f3fb
disable cpu test workflows and move wait for connection in bazel gpu …
nitins17 Sep 5, 2024
6544ca9
make github detect label change
nitins17 Sep 7, 2024
cbdcbbf
Explictly make the shell as bash for script to run on Windows
nitins17 Sep 7, 2024
50d3380
fix issue with scripts when running on windows runners
nitins17 Sep 8, 2024
6674827
build without rbe to test windows runners
nitins17 Sep 8, 2024
dbeca71
Merge branch 'main' of https://github.com/google-ml-infra/jax-fork in…
nitins17 Sep 8, 2024
f4cb21e
remove root .bazelrc to avoid conflicts
nitins17 Sep 8, 2024
007356d
fixes for building a windows wheel
nitins17 Sep 8, 2024
662ddf6
fix wheel build command for windows
nitins17 Sep 8, 2024
f7d503a
enable artifact build on Linux runners
nitins17 Sep 8, 2024
59a0efa
apply container only for linux jobs
nitins17 Sep 8, 2024
1d01b3b
comment out container settings to fix artifact workflow and make gith…
nitins17 Sep 9, 2024
7171258
comment out linux runners
nitins17 Sep 9, 2024
0ed41d7
remove path to bash
nitins17 Sep 9, 2024
4e551a0
update condition to select rbe config
nitins17 Sep 9, 2024
042f6c5
make github detect label change
nitins17 Sep 9, 2024
46a9b30
enable rbe for artifact build
nitins17 Sep 9, 2024
f1fb463
make github detect label change
nitins17 Sep 12, 2024
44c7ad2
Merge branch 'main' into srnitin/task-jax-ci-rework
nitins17 Sep 12, 2024
5d766c9
Add more functionality to the build CLI
nitins17 Sep 13, 2024
cd8f7a9
Merge branch 'srnitin/task-jax-ci-rework' of https://github.com/googl…
nitins17 Sep 13, 2024
64a17eb
clean up scripts
nitins17 Sep 13, 2024
d2c899d
indent image field in build artifacts workflow
nitins17 Sep 13, 2024
57c1bf1
fix syntax error
nitins17 Sep 13, 2024
3dcf44f
another try at fixing syntax error
nitins17 Sep 13, 2024
4fb642d
fix envs
nitins17 Sep 13, 2024
8c69f79
test cpu test workflows
nitins17 Sep 14, 2024
83140e3
fix env in workflow
nitins17 Sep 14, 2024
7cc5bfc
some fixes
nitins17 Sep 14, 2024
92cecea
fix syntax error
nitins17 Sep 14, 2024
0f41ab4
explictly set shell to bash
nitins17 Sep 14, 2024
31ddd2d
convert msys paths to windows paths
nitins17 Sep 14, 2024
e2f4e63
make github detect label change
nitins17 Sep 14, 2024
6ca9a8a
Change how mysys paths are converted to windows-like paths
nitins17 Sep 14, 2024
f4c6761
minor fix
nitins17 Sep 14, 2024
f83da93
fix error when args are None
nitins17 Sep 14, 2024
bfc334d
make github detect label change
nitins17 Sep 14, 2024
9db6fda
remove quotes around local xla path to fix windows issue
nitins17 Sep 14, 2024
13fe6f9
fix path issue on windows when installing wheels locally
nitins17 Sep 15, 2024
ae65260
clean up
nitins17 Sep 16, 2024
e7e7dca
update scripts
nitins17 Sep 16, 2024
515c397
update env var name
nitins17 Sep 16, 2024
87d277a
update default env file
nitins17 Sep 16, 2024
fa21235
update env var
nitins17 Sep 16, 2024
877b12a
update scripts
nitins17 Sep 17, 2024
88ebac3
update .bazelrc
nitins17 Sep 17, 2024
1ba0bf6
update envs
nitins17 Sep 17, 2024
faa98f3
update default env
nitins17 Sep 17, 2024
0bfc868
Merge branch 'main' of https://github.com/google-ml-infra/jax-fork in…
nitins17 Sep 17, 2024
36d04de
update scripts
nitins17 Sep 19, 2024
3b55033
update scripts
nitins17 Sep 19, 2024
5627330
update pytest workflow
nitins17 Sep 19, 2024
31a6767
make github detect label change
nitins17 Sep 19, 2024
3a910bd
update bazel gpu local workflow
nitins17 Sep 19, 2024
766738f
enable bazel gpu local workflow
nitins17 Sep 19, 2024
5f3ee2c
fix syntax error
nitins17 Sep 19, 2024
12590a8
update env var
nitins17 Sep 19, 2024
534bc1a
update env
nitins17 Sep 19, 2024
7ba4a37
update linux arm64 container image
nitins17 Sep 19, 2024
7a2cc1c
disable import jax command in bazel gpu local script
nitins17 Sep 19, 2024
8b65c70
add env var setting to pytest cpu env file
nitins17 Sep 19, 2024
dc11f12
enable pytest gpu workflow
nitins17 Sep 20, 2024
ab6f3ff
update bazel cpu workflow
nitins17 Sep 20, 2024
e58035e
fix error in pytest gpu script
nitins17 Sep 20, 2024
637b0a5
fix installing wheels correctly
nitins17 Sep 20, 2024
460cb11
fix issue when installing wheels
nitins17 Sep 20, 2024
c65c2b2
modify scripts to run tpu tests
nitins17 Sep 20, 2024
c448ea6
set jaxci python env var
nitins17 Sep 20, 2024
5b450af
set the tpu cores in the env file
nitins17 Sep 20, 2024
1418ba8
enable bazel gpu rbe tests
nitins17 Sep 20, 2024
9f739ab
make github detect label change
nitins17 Sep 20, 2024
391a913
change to python 3.10
nitins17 Sep 23, 2024
3b98f9b
install the right version of python deps for pytest cpu
nitins17 Sep 23, 2024
dd41e8d
fix syntax error
nitins17 Sep 23, 2024
ed31404
Debug GPU failures
nitins17 Sep 23, 2024
04537e7
debug gpu vms getting killed
nitins17 Sep 23, 2024
b880ed5
debug gpu vms getting killed (2)
nitins17 Sep 23, 2024
a8716ce
set cuda compute arg correctly
nitins17 Sep 24, 2024
de602b5
set cuda compute arg correctly (2)
nitins17 Sep 24, 2024
3783c84
Merge branch 'main' of https://github.com/google-ml-infra/jax-fork in…
nitins17 Sep 24, 2024
bc6d9cb
do not override xla repository when running gpu tests locally
nitins17 Sep 24, 2024
b8978b5
bump device memory limit when running tests locally on single gpu
nitins17 Sep 25, 2024
26b1f59
change bazel build to bazel run
nitins17 Sep 26, 2024
7b63e0c
increase test timeout
nitins17 Sep 26, 2024
a7663fa
update runner for bazel local gpu workflow
nitins17 Sep 27, 2024
e25bbe8
Change config back to stable driver
MichaelHudgins Sep 27, 2024
e11b465
Update build.py
MichaelHudgins Sep 27, 2024
061ad8a
Update build_artifacts.sh
MichaelHudgins Sep 27, 2024
51792c6
Update run_bazel_test.sh
MichaelHudgins Sep 27, 2024
a798aa0
Update run_pytest.sh
MichaelHudgins Sep 27, 2024
9970cec
Update pytest_gpu.yml
MichaelHudgins Oct 1, 2024
3b167e8
Update bazel_gpu_local.yml
MichaelHudgins Oct 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
328 changes: 0 additions & 328 deletions .bazelrc

This file was deleted.

31 changes: 31 additions & 0 deletions .github/workflows/bazel_cpu.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: Run Bazel CPU tests with RBE

on:
pull_request:
branches:
- main

jobs:
build:
continue-on-error: true
strategy:
matrix:
runner: ["linux-x86-n2-64", "linux-arm64-t2a-48"]

runs-on: ${{ matrix.runner }}
container: ${{ (contains(matrix.runner, 'linux-x86') && 'us-central1-docker.pkg.dev/tensorflow-sigs/tensorflow/build:670606426-python3.12') ||
(contains(matrix.runner, 'linux-arm64') && 'us-central1-docker.pkg.dev/tensorflow-sigs/tensorflow/linux-arm64-arc-container:latest') }}

env:
# Disable running `run_docker_container.sh`. JAX self-hosted runners already run in a Docker
# container.
JAXCI_RUN_DOCKER_CONTAINER: 0
JAXCI_HERMETIC_PYTHON_VERSION: "3.12"

steps:
- uses: actions/checkout@v3
# Halt for testing
- name: Wait For Connection
uses: ./actions/ci_connection/
- name: Run Bazel CPU Tests
run: ./ci/run_bazel_test.sh "ci/envs/run_tests/bazel_cpu"
Loading
Loading