Skip to content

Conversation

wenyihu6
Copy link
Contributor

@wenyihu6 wenyihu6 commented Oct 9, 2025

This commit introduces a adds a python script that pipes git diffs to compare
certain metrics sum diff. Generated by claude.

Sample usage:

git diff | python3 pkg/kv/kvserver/asim/tests/cmd/asimdiff.py thrash_pct
git diff head~1 head | python3 pkg/kv/kvserver/asim/tests/cmd/asimdiff.py last  --metric mean

Sample output:

[skewed_cpu_skewed_write.txt] before = 249%,  after = 249%,  diff = +0%
  cpu_util#5: thrash_pct: [s1=40%, s2=76%, s3=49%, s4=30%, s5=33%, s6=21%]  (sum=249%)
...
total diff = -5374%  (total before = 498950%, total after = 493576%)
percent change = -1.1%
avg diff = -22.7%,  max diff = +7824%,  min diff = -5509%
changes: 136 regressions, 99 improvements, 2 unchanged

Epic: CRDB-55052
Release note: none

@wenyihu6 wenyihu6 requested review from a team as code owners October 9, 2025 00:49
@wenyihu6 wenyihu6 requested a review from tbg October 9, 2025 00:50
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@tbg
Copy link
Member

tbg commented Oct 9, 2025

Could you say a few more words about what this does and how one acts on its output?

@wenyihu6
Copy link
Contributor Author

wenyihu6 commented Oct 9, 2025

Did this for the thrashing PR (#154869 (comment)
) since I ran 10 samples and wanted to understand the change without checking each diff manually. It’s probably only useful for the thrashing percentage, though I can imagine it could also be applied to standard deviation.

Copy link
Member

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, let's merge it! I'm not going to review it since it's a little AI-genned tool, but lmk if there's something I should be looking at.

Looks like lint failed, but I imagine this might have been an infra flake (saw many browned-out runners today), but please take a look before bors.

This commit introduces a adds a python script that pipes git diffs to compare
certain metrics sum diff. Generated by claude.

Sample usage:
```
git diff | python3 pkg/kv/kvserver/asim/tests/cmd/asimdiff.py thrash_pct
git diff head~1 head | python3 pkg/kv/kvserver/asim/tests/cmd/asimdiff.py last  --metric mean
```

Sample output:
```
[skewed_cpu_skewed_write.txt] before = 249%,  after = 249%,  diff = +0%
  cpu_util#5: thrash_pct: [s1=40%, s2=76%, s3=49%, s4=30%, s5=33%, s6=21%]  (sum=249%)
...
total diff = -5374%  (total before = 498950%, total after = 493576%)
percent change = -1.1%
avg diff = -22.7%,  max diff = +7824%,  min diff = -5509%
changes: 136 regressions, 99 improvements, 2 unchanged
```
@wenyihu6
Copy link
Contributor Author

Fixed the linter failure. TFTR!

bors r+

@craig
Copy link
Contributor

craig bot commented Oct 10, 2025

@craig craig bot merged commit 44eef4b into cockroachdb:master Oct 10, 2025
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants