Run Torchtitan ROCm workflow on cron schedule & push to Main branch only #2016
Conversation
…ent to run_tests.py.
akashveramd
requested review from
fegin,
tianyu-l,
wconstab and
wwwjn
as code owners
meta-cla
bot
added
the
CLA Signed
tianyu-l
left a comment
tianyu-l
left a comment
There was a problem hiding this comment.
SGTM. I guess we can't verify ROCm continues to work because this PR removes the per-PR test. Let's merge & wait to see if things are OK.
Context: We are making ROCm test to run only for (1) merging commits to main, and (2) periodically testing every 6 hours, but stopping it for (3) PR submission/updates. The reasons are
- ROCm CI resource has limited capacity #1786 (comment)
- For some permission reasons, PRs from outside repo / contributors without write access can't run ROCm tests (whereas they can run CUDA).
|
@tianyu-l: There was some ongoing work on .github/workflows/integration_test_8gpu_features.yaml to run ROCm workflow on condition. Because currently the workflow uses 'matrix' at a wrong location, which is an error and due to which the workflow didn't ran for both CUDA & ROCm. Once I fix that, the workflow will run on condition. We'll have to undo the merge for now. |
|
@akashveramd hmm OK, I thought the cuda test passed, so I merged it. Could you help submit a revert PR so I can stamp? Thanks! |
…& push to Main branch only" (#2017) Reverts PR: #2016 Addressing following issues in this PR- - Running Torchtitan ROCm workflow on cron schedule & only when push to Main branch. CUDA workflow will run as is. - Refactor Torchtitan test run to address older PR comment #1786 (comment) Co-authored-by: tianyu-l <[email protected]>
Addressing following issues in this PR-