Skip to content

Conversation

yangw-dev
Copy link
Contributor

@yangw-dev yangw-dev commented Sep 4, 2025

Stack from ghstack (oldest at bottom):

Add github notification settings

currently we only create github comment if and only a regression status is detected.
later we can add if we want to add suspicious too,

Prerequest

currently user must:

  1. create a github issue first, and put it in the Policy section to make this work
  2. Each issue should be associated with a butterfly rule to link to internal workplace/ oncall emails

later improvement

as you see, currently if a regression does not resolved, it will send notification to github everyday.
since we have those report in db, we can later do exponential notification based on previous report status

[ghstack-poisoned]
[ghstack-poisoned]
Copy link

vercel bot commented Sep 4, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Preview Updated (UTC)
torchci Ignored Ignored Preview Sep 10, 2025 3:20am

yangw-dev added a commit that referenced this pull request Sep 4, 2025
ghstack-source-id: d7ee0a4
Pull-Request: #7096
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 4, 2025
yangw-dev added a commit that referenced this pull request Sep 4, 2025
ghstack-source-id: d7ee0a4
Pull-Request: #7096
[ghstack-poisoned]
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 4, 2025
ghstack-source-id: d275444
Pull-Request: #7096
[ghstack-poisoned]
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 4, 2025
ghstack-source-id: c0e39cf
Pull-Request: #7096
[ghstack-poisoned]
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 4, 2025
ghstack-source-id: 4d8c673
Pull-Request: #7096
[ghstack-poisoned]
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 4, 2025
ghstack-source-id: c06d14d
Pull-Request: #7096
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 4, 2025
ghstack-source-id: 07998ec
Pull-Request: #7096
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 4, 2025
ghstack-source-id: a445d60
Pull-Request: #7096
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 4, 2025
ghstack-source-id: d2e6db6
Pull-Request: #7096
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 4, 2025
ghstack-source-id: 65127a4
Pull-Request: #7096
[ghstack-poisoned]
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 5, 2025
ghstack-source-id: 87b959c
Pull-Request: #7096
yangw-dev added a commit that referenced this pull request Sep 5, 2025
ghstack-source-id: 87b959c
Pull-Request: #7096
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 5, 2025
ghstack-source-id: 813fc6d
Pull-Request: #7096
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 5, 2025
ghstack-source-id: 22b1d9d
Pull-Request: #7096
@yangw-dev yangw-dev mentioned this pull request Sep 6, 2025
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 9, 2025
…7092)

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* #7112
* #7096
* #7095
* #7094
* __->__ #7092

setup base for notification pipeline.
add config to provide guidance for each pipepline how to process data
and generate benchmark regression report
[ghstack-poisoned]
[ghstack-poisoned]
yangw-dev added a commit that referenced this pull request Sep 9, 2025
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* #7112
* #7096
* #7095
* __->__ #7094

main class to detect regression
1. use baseline data to generate baseline for regression detection,
currently support (max,min, latest, earliest) within the baseline data
window
2. decide wether the target data list under the threshold for regression
detection based on the baseline value.
3. if detected more than 2 continuous dp from current timestamp, mark as
regression
[ghstack-poisoned]
Copy link
Contributor

@huydhn huydhn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stamp to unblock

@yangw-dev yangw-dev changed the base branch from gh/yangw-dev/5/base to main September 10, 2025 02:48
yangw-dev added a commit that referenced this pull request Sep 10, 2025
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* #7112
* #7096
* __->__ #7095

add logics to write the report to db table
@yangw-dev yangw-dev merged commit 73a32b8 into main Sep 10, 2025
5 checks passed
@yangw-dev yangw-dev deleted the gh/yangw-dev/5/head branch September 10, 2025 03:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants