gh-783: Write PR comment if regression test workflow fails #787
Conversation
| f=$REGRESSION_TEST_OUTPUT_FILE; start=$(grep -n -m1 'Performance has | ||
| regressed:' "$f" | cut -d: -f1) | ||
| || { echo "No match"; exit 1; }; | ||
| s_dash=$(awk -v n=$start 'NR<=n && /^-+$/ {l=NR} END{print l}' "$f"); | ||
| e_dash=$(awk -v n=$start 'NR>=n && /^-+$/ {print NR; exit}' "$f"); | ||
| PR_COMMENT_MSG=$(sed -n "${s_dash},${e_dash}p" "$f"); echo | ||
| "PR_COMMENT_MSG=$PR_COMMENT_MSG" >> $GITHUB_OUTPUT; |
There was a problem hiding this comment.
Good job working this out 😆 just wonder how maintainable this is?
There was a problem hiding this comment.
TBH it was a ChatGPT job, with some tweaks
There was a problem hiding this comment.
Do you want some extra comments so we can at least try to reproduce in a different way in the future, if needed?
There was a problem hiding this comment.
I'm starting to go off the idea really, based on today's meeting
There was a problem hiding this comment.
Does the PR comment not give us good traceability if we discover a significant performance regression in the future?
There was a problem hiding this comment.
Yes, but should we be merging those anyway?
There was a problem hiding this comment.
I am thinking of the scenario were we've deemed it acceptable but then in the future we change our mind. Or to catch the case where our benchmarks weren't thorough enough to show the actual extent of the problem.
paddyroddy
added
the
needs-2-reviewers
connoraird
force-pushed
the
connor/issue-783
branch
from
81f3f3b to
75a8e34
Compare
Description
To allow writing a comment on a PR if the regression tests fail, this PR extracts the relevant lines from the regression test failure message to then post in a PR comment.
Closes: #783
Changelog entry
Added: PR comment from failing regression test
Checks