Skip to content

Conversation

keks
Copy link
Member

@keks keks commented Aug 21, 2025

No description provided.

@keks
Copy link
Member Author

keks commented Aug 29, 2025

Unfortunately simply disabling the other actions didn't work, because then Github won't put up the benchmarks job. I'll see if there is something better I can do on Monday.

@keks keks force-pushed the keks/better-benchmarks branch from 235d039 to b00db71 Compare September 4, 2025 10:02
Copy link

beep boop, I am the benchmark bot

Comparing PR commit 74f403c against baseline d67a414.

Benchmark Report

Performance Report:

Visualized Results

os: ubuntu-latest_32, keySize: 512

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 512 portable external random ubuntu-latest_32 55,618 ns 55,760 ns -0.26%
Key Generation ML-KEM 512 portable unpacked (external random) ubuntu-latest_32 55,275 ns 55,182 ns +0.17%
Encapsulation ML-KEM 512 portable external random ubuntu-latest_32 59,720 ns 59,653 ns +0.11%
Encapsulation ML-KEM 512 portable unpacked (external random) ubuntu-latest_32 27,253 ns 27,225 ns +0.10%
Decapsulation ML-KEM 512 portable N/A ubuntu-latest_32 66,439 ns 66,076 ns +0.55%
Decapsulation ML-KEM 512 portable unpacked ubuntu-latest_32 43,155 ns 43,172 ns -0.04%
PK Validation ML-KEM 512 portable N/A ubuntu-latest_32 950 ns 931 ns +2.00%

os: ubuntu-latest_32, keySize: 768

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 768 portable external random ubuntu-latest_32 90,936 ns 90,491 ns +0.49%
Key Generation ML-KEM 768 portable unpacked (external random) ubuntu-latest_32 90,573 ns 90,201 ns +0.41%
Encapsulation ML-KEM 768 portable external random ubuntu-latest_32 99,402 ns 99,137 ns +0.27%
Encapsulation ML-KEM 768 portable unpacked (external random) ubuntu-latest_32 36,250 ns 35,704 ns +1.51%
Decapsulation ML-KEM 768 portable N/A ubuntu-latest_32 108,399 ns 108,342 ns +0.05%
Decapsulation ML-KEM 768 portable unpacked ubuntu-latest_32 58,467 ns 58,182 ns +0.49%
PK Validation ML-KEM 768 portable N/A ubuntu-latest_32 1,404 ns 1,384 ns +1.42%

os: ubuntu-latest_32, keySize: 1024

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 1024 portable external random ubuntu-latest_32 144,514 ns 144,205 ns +0.21%
Key Generation ML-KEM 1024 portable unpacked (external random) ubuntu-latest_32 143,959 ns 143,698 ns +0.18%
Encapsulation ML-KEM 1024 portable external random ubuntu-latest_32 155,188 ns 154,360 ns +0.53%
Encapsulation ML-KEM 1024 portable unpacked (external random) ubuntu-latest_32 50,145 ns 49,275 ns +1.73%
Decapsulation ML-KEM 1024 portable N/A ubuntu-latest_32 167,240 ns 166,888 ns +0.21%
Decapsulation ML-KEM 1024 portable unpacked ubuntu-latest_32 80,025 ns 79,191 ns +1.04%
PK Validation ML-KEM 1024 portable N/A ubuntu-latest_32 1,876 ns 1,846 ns +1.60%

os: windows-latest_32, keySize: 512

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 512 portable external random windows-latest_32 57,169 ns 57,126 ns +0.08%
Key Generation ML-KEM 512 portable unpacked (external random) windows-latest_32 56,320 ns 56,397 ns -0.14%
Encapsulation ML-KEM 512 portable external random windows-latest_32 61,074 ns 60,587 ns +0.80%
Encapsulation ML-KEM 512 portable unpacked (external random) windows-latest_32 27,931 ns 27,656 ns +0.98%
Decapsulation ML-KEM 512 portable N/A windows-latest_32 67,215 ns 66,798 ns +0.62%
Decapsulation ML-KEM 512 portable unpacked windows-latest_32 44,115 ns 43,962 ns +0.35%
PK Validation ML-KEM 512 portable N/A windows-latest_32 1,296 ns 1,265 ns +2.39%

os: windows-latest_32, keySize: 768

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 768 portable external random windows-latest_32 94,287 ns 94,201 ns +0.09%
Key Generation ML-KEM 768 portable unpacked (external random) windows-latest_32 93,021 ns 92,932 ns +0.10%
Encapsulation ML-KEM 768 portable external random windows-latest_32 102,594 ns 101,978 ns +0.60%
Encapsulation ML-KEM 768 portable unpacked (external random) windows-latest_32 36,405 ns 36,179 ns +0.62%
Decapsulation ML-KEM 768 portable N/A windows-latest_32 110,658 ns 110,935 ns -0.25%
Decapsulation ML-KEM 768 portable unpacked windows-latest_32 59,903 ns 59,363 ns +0.90%
PK Validation ML-KEM 768 portable N/A windows-latest_32 1,845 ns 1,863 ns -0.98%

os: windows-latest_32, keySize: 1024

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 1024 portable external random windows-latest_32 149,292 ns 149,205 ns +0.06%
Key Generation ML-KEM 1024 portable unpacked (external random) windows-latest_32 147,964 ns 148,094 ns -0.09%
Encapsulation ML-KEM 1024 portable external random windows-latest_32 159,245 ns 157,782 ns +0.92%
Encapsulation ML-KEM 1024 portable unpacked (external random) windows-latest_32 50,332 ns 49,885 ns +0.89%
Decapsulation ML-KEM 1024 portable N/A windows-latest_32 169,331 ns 170,166 ns -0.49%
Decapsulation ML-KEM 1024 portable unpacked windows-latest_32 79,913 ns 80,285 ns -0.47%
PK Validation ML-KEM 1024 portable N/A windows-latest_32 2,386 ns 2,357 ns +1.22%

os: ubuntu-latest_64, keySize: 512

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 512 portable external random ubuntu-latest_64 20,107 ns 20,238 ns -0.65%
Key Generation ML-KEM 512 portable unpacked (external random) ubuntu-latest_64 19,613 ns 19,729 ns -0.59%
Key Generation ML-KEM 512 avx2 external random ubuntu-latest_64 157,896 ns 157,970 ns -0.05%
Key Generation ML-KEM 512 avx2 unpacked (external random) ubuntu-latest_64 157,721 ns 157,597 ns +0.08%
Encapsulation ML-KEM 512 portable external random ubuntu-latest_64 23,501 ns 23,363 ns +0.59%
Encapsulation ML-KEM 512 portable unpacked (external random) ubuntu-latest_64 14,740 ns 14,766 ns -0.18%
Encapsulation ML-KEM 512 avx2 external random ubuntu-latest_64 122,275 ns 122,332 ns -0.05%
Encapsulation ML-KEM 512 avx2 unpacked (external random) ubuntu-latest_64 116,166 ns 116,138 ns +0.02%
Decapsulation ML-KEM 512 portable N/A ubuntu-latest_64 28,761 ns 28,910 ns -0.52%
Decapsulation ML-KEM 512 portable unpacked ubuntu-latest_64 21,899 ns 21,956 ns -0.26%
Decapsulation ML-KEM 512 avx2 N/A ubuntu-latest_64 123,505 ns 123,532 ns -0.02%
Decapsulation ML-KEM 512 avx2 unpacked ubuntu-latest_64 119,683 ns 119,659 ns +0.02%
PK Validation ML-KEM 512 portable N/A ubuntu-latest_64 880 ns 857 ns +2.61%
PK Validation ML-KEM 512 avx2 N/A ubuntu-latest_64 352 ns 353 ns -0.28%

os: ubuntu-latest_64, keySize: 768

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 768 portable external random ubuntu-latest_64 34,072 ns 34,480 ns -1.20%
Key Generation ML-KEM 768 portable unpacked (external random) ubuntu-latest_64 33,723 ns 34,126 ns -1.20%
Key Generation ML-KEM 768 avx2 external random ubuntu-latest_64 89,684 ns 89,675 ns +0.01%
Key Generation ML-KEM 768 avx2 unpacked (external random) ubuntu-latest_64 89,369 ns 89,381 ns -0.01%
Encapsulation ML-KEM 768 portable external random ubuntu-latest_64 39,634 ns 39,414 ns +0.56%
Encapsulation ML-KEM 768 portable unpacked (external random) ubuntu-latest_64 22,005 ns 22,044 ns -0.18%
Encapsulation ML-KEM 768 avx2 external random ubuntu-latest_64 90,653 ns 90,629 ns +0.03%
Encapsulation ML-KEM 768 avx2 unpacked (external random) ubuntu-latest_64 81,155 ns 81,151 ns +0.00%
Decapsulation ML-KEM 768 portable N/A ubuntu-latest_64 46,800 ns 47,027 ns -0.49%
Decapsulation ML-KEM 768 portable unpacked ubuntu-latest_64 32,120 ns 32,311 ns -0.59%
Decapsulation ML-KEM 768 avx2 N/A ubuntu-latest_64 92,326 ns 92,304 ns +0.02%
Decapsulation ML-KEM 768 avx2 unpacked ubuntu-latest_64 86,207 ns 86,374 ns -0.19%
PK Validation ML-KEM 768 portable N/A ubuntu-latest_64 1,275 ns 1,271 ns +0.31%
PK Validation ML-KEM 768 avx2 N/A ubuntu-latest_64 543 ns 539 ns +0.74%

os: ubuntu-latest_64, keySize: 1024

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 1024 portable external random ubuntu-latest_64 53,317 ns 53,855 ns -1.01%
Key Generation ML-KEM 1024 portable unpacked (external random) ubuntu-latest_64 53,016 ns 53,274 ns -0.49%
Key Generation ML-KEM 1024 avx2 external random ubuntu-latest_64 95,591 ns 95,609 ns -0.02%
Key Generation ML-KEM 1024 avx2 unpacked (external random) ubuntu-latest_64 95,298 ns 95,276 ns +0.02%
Encapsulation ML-KEM 1024 portable external random ubuntu-latest_64 60,063 ns 59,773 ns +0.48%
Encapsulation ML-KEM 1024 portable unpacked (external random) ubuntu-latest_64 31,280 ns 31,606 ns -1.04%
Encapsulation ML-KEM 1024 avx2 external random ubuntu-latest_64 97,547 ns 97,389 ns +0.16%
Encapsulation ML-KEM 1024 avx2 unpacked (external random) ubuntu-latest_64 83,865 ns 83,948 ns -0.10%
Decapsulation ML-KEM 1024 portable N/A ubuntu-latest_64 69,376 ns 69,440 ns -0.09%
Decapsulation ML-KEM 1024 portable unpacked ubuntu-latest_64 44,798 ns 45,150 ns -0.79%
Decapsulation ML-KEM 1024 avx2 N/A ubuntu-latest_64 100,193 ns 100,150 ns +0.04%
Decapsulation ML-KEM 1024 avx2 unpacked ubuntu-latest_64 91,139 ns 91,112 ns +0.03%
PK Validation ML-KEM 1024 portable N/A ubuntu-latest_64 1,741 ns 1,678 ns +3.62%
PK Validation ML-KEM 1024 avx2 N/A ubuntu-latest_64 716 ns 709 ns +0.98%

os: windows-latest_64, keySize: 512

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 512 portable external random windows-latest_64 20,244 ns 20,250 ns -0.03%
Key Generation ML-KEM 512 portable unpacked (external random) windows-latest_64 19,747 ns 19,745 ns +0.01%
Key Generation ML-KEM 512 avx2 external random windows-latest_64 152,818 ns 152,775 ns +0.03%
Key Generation ML-KEM 512 avx2 unpacked (external random) windows-latest_64 152,530 ns 152,559 ns -0.02%
Encapsulation ML-KEM 512 portable external random windows-latest_64 23,530 ns 23,501 ns +0.12%
Encapsulation ML-KEM 512 portable unpacked (external random) windows-latest_64 14,865 ns 14,864 ns +0.01%
Encapsulation ML-KEM 512 avx2 external random windows-latest_64 118,390 ns 118,332 ns +0.05%
Encapsulation ML-KEM 512 avx2 unpacked (external random) windows-latest_64 112,260 ns 112,311 ns -0.05%
Decapsulation ML-KEM 512 portable N/A windows-latest_64 28,813 ns 28,798 ns +0.05%
Decapsulation ML-KEM 512 portable unpacked windows-latest_64 22,071 ns 22,070 ns +0.00%
Decapsulation ML-KEM 512 avx2 N/A windows-latest_64 119,451 ns 119,592 ns -0.12%
Decapsulation ML-KEM 512 avx2 unpacked windows-latest_64 115,882 ns 115,844 ns +0.03%
PK Validation ML-KEM 512 portable N/A windows-latest_64 889 ns 890 ns -0.11%
PK Validation ML-KEM 512 avx2 N/A windows-latest_64 366 ns 375 ns -2.46%

os: windows-latest_64, keySize: 768

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 768 portable external random windows-latest_64 34,032 ns 34,042 ns -0.03%
Key Generation ML-KEM 768 portable unpacked (external random) windows-latest_64 33,472 ns 33,362 ns +0.33%
Key Generation ML-KEM 768 avx2 external random windows-latest_64 86,928 ns 86,857 ns +0.08%
Key Generation ML-KEM 768 avx2 unpacked (external random) windows-latest_64 86,490 ns 86,499 ns -0.01%
Encapsulation ML-KEM 768 portable external random windows-latest_64 38,827 ns 38,881 ns -0.14%
Encapsulation ML-KEM 768 portable unpacked (external random) windows-latest_64 22,150 ns 22,151 ns -0.00%
Encapsulation ML-KEM 768 avx2 external random windows-latest_64 88,136 ns 88,642 ns -0.57%
Encapsulation ML-KEM 768 avx2 unpacked (external random) windows-latest_64 78,498 ns 78,446 ns +0.07%
Decapsulation ML-KEM 768 portable N/A windows-latest_64 46,866 ns 46,764 ns +0.22%
Decapsulation ML-KEM 768 portable unpacked windows-latest_64 32,305 ns 32,269 ns +0.11%
Decapsulation ML-KEM 768 avx2 N/A windows-latest_64 89,695 ns 89,601 ns +0.10%
Decapsulation ML-KEM 768 avx2 unpacked windows-latest_64 83,598 ns 83,584 ns +0.02%
PK Validation ML-KEM 768 portable N/A windows-latest_64 1,408 ns 1,417 ns -0.64%
PK Validation ML-KEM 768 avx2 N/A windows-latest_64 544 ns 545 ns -0.18%

os: windows-latest_64, keySize: 1024

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 1024 portable external random windows-latest_64 53,362 ns 53,383 ns -0.04%
Key Generation ML-KEM 1024 portable unpacked (external random) windows-latest_64 52,573 ns 52,544 ns +0.06%
Key Generation ML-KEM 1024 avx2 external random windows-latest_64 92,578 ns 92,440 ns +0.15%
Key Generation ML-KEM 1024 avx2 unpacked (external random) windows-latest_64 92,160 ns 92,328 ns -0.18%
Encapsulation ML-KEM 1024 portable external random windows-latest_64 59,465 ns 59,456 ns +0.02%
Encapsulation ML-KEM 1024 portable unpacked (external random) windows-latest_64 31,737 ns 31,727 ns +0.03%
Encapsulation ML-KEM 1024 avx2 external random windows-latest_64 94,540 ns 94,356 ns +0.19%
Encapsulation ML-KEM 1024 avx2 unpacked (external random) windows-latest_64 81,379 ns 81,342 ns +0.05%
Decapsulation ML-KEM 1024 portable N/A windows-latest_64 69,526 ns 69,659 ns -0.19%
Decapsulation ML-KEM 1024 portable unpacked windows-latest_64 44,548 ns 44,690 ns -0.32%
Decapsulation ML-KEM 1024 avx2 N/A windows-latest_64 97,071 ns 97,013 ns +0.06%
Decapsulation ML-KEM 1024 avx2 unpacked windows-latest_64 88,337 ns 88,387 ns -0.06%
PK Validation ML-KEM 1024 portable N/A windows-latest_64 1,737 ns 1,777 ns -2.30%
PK Validation ML-KEM 1024 avx2 N/A windows-latest_64 722 ns 719 ns +0.42%

os: macos-latest_64, keySize: 512

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 512 portable external random macos-latest_64 10,165 ns 11,963 ns -17.69%
Key Generation ML-KEM 512 portable unpacked (external random) macos-latest_64 9,972 ns 12,482 ns -25.17%
Key Generation ML-KEM 512 neon external random macos-latest_64 5,116 ns 6,413 ns -25.35%
Key Generation ML-KEM 512 neon unpacked (external random) macos-latest_64 4,911 ns 7,388 ns -50.44%
Encapsulation ML-KEM 512 portable external random macos-latest_64 11,337 ns 16,057 ns -41.63%
Encapsulation ML-KEM 512 portable unpacked (external random) macos-latest_64 6,307 ns 9,005 ns -42.78%
Encapsulation ML-KEM 512 neon external random macos-latest_64 5,653 ns 6,932 ns -22.63%
Encapsulation ML-KEM 512 neon unpacked (external random) macos-latest_64 3,085 ns 4,139 ns -34.17%
Decapsulation ML-KEM 512 portable N/A macos-latest_64 13,820 ns 17,186 ns -24.36%
Decapsulation ML-KEM 512 portable unpacked macos-latest_64 9,845 ns 14,230 ns -44.54%
Decapsulation ML-KEM 512 neon N/A macos-latest_64 6,690 ns 7,930 ns -18.54%
Decapsulation ML-KEM 512 neon unpacked macos-latest_64 4,872 ns 5,281 ns -8.39%
PK Validation ML-KEM 512 portable N/A macos-latest_64 407 ns 504 ns -23.83%
PK Validation ML-KEM 512 neon N/A macos-latest_64 392 ns 491 ns -25.26%

os: macos-latest_64, keySize: 768

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 768 portable external random macos-latest_64 16,751 ns 22,147 ns -32.21%
Key Generation ML-KEM 768 portable unpacked (external random) macos-latest_64 16,529 ns 22,447 ns -35.80%
Key Generation ML-KEM 768 neon external random macos-latest_64 9,544 ns 12,997 ns -36.18%
Key Generation ML-KEM 768 neon unpacked (external random) macos-latest_64 9,319 ns 12,338 ns -32.40%
Encapsulation ML-KEM 768 portable external random macos-latest_64 19,502 ns 24,685 ns -26.58%
Encapsulation ML-KEM 768 portable unpacked (external random) macos-latest_64 8,825 ns 11,377 ns -28.92%
Encapsulation ML-KEM 768 neon external random macos-latest_64 10,354 ns 13,981 ns -35.03%
Encapsulation ML-KEM 768 neon unpacked (external random) macos-latest_64 4,117 ns 6,643 ns -61.36%
Decapsulation ML-KEM 768 portable N/A macos-latest_64 21,562 ns 23,954 ns -11.09%
Decapsulation ML-KEM 768 portable unpacked macos-latest_64 13,245 ns 16,600 ns -25.33%
Decapsulation ML-KEM 768 neon N/A macos-latest_64 11,809 ns 13,610 ns -15.25%
Decapsulation ML-KEM 768 neon unpacked macos-latest_64 7,046 ns 8,181 ns -16.11%
PK Validation ML-KEM 768 portable N/A macos-latest_64 669 ns 744 ns -11.21%
PK Validation ML-KEM 768 neon N/A macos-latest_64 626 ns 712 ns -13.74%

os: macos-latest_64, keySize: 1024

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 1024 portable external random macos-latest_64 26,320 ns 34,709 ns -31.87%
Key Generation ML-KEM 1024 portable unpacked (external random) macos-latest_64 26,052 ns 34,954 ns -34.17%
Key Generation ML-KEM 1024 neon external random macos-latest_64 13,713 ns 18,758 ns -36.79%
Key Generation ML-KEM 1024 neon unpacked (external random) macos-latest_64 13,891 ns 18,715 ns -34.73%
Encapsulation ML-KEM 1024 portable external random macos-latest_64 27,859 ns 37,770 ns -35.58%
Encapsulation ML-KEM 1024 portable unpacked (external random) macos-latest_64 11,302 ns 16,948 ns -49.96%
Encapsulation ML-KEM 1024 neon external random macos-latest_64 14,992 ns 21,552 ns -43.76%
Encapsulation ML-KEM 1024 neon unpacked (external random) macos-latest_64 5,625 ns 7,489 ns -33.14%
Decapsulation ML-KEM 1024 portable N/A macos-latest_64 32,640 ns 35,915 ns -10.03%
Decapsulation ML-KEM 1024 portable unpacked macos-latest_64 17,882 ns 23,449 ns -31.13%
Decapsulation ML-KEM 1024 neon N/A macos-latest_64 16,599 ns 19,963 ns -20.27%
Decapsulation ML-KEM 1024 neon unpacked macos-latest_64 8,909 ns 10,773 ns -20.92%
PK Validation ML-KEM 1024 portable N/A macos-latest_64 913 ns 1,032 ns -13.03%
PK Validation ML-KEM 1024 neon N/A macos-latest_64 808 ns 949 ns -17.45%

os: macos-13_64, keySize: 512

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 512 portable external random macos-13_64 25,904 ns N/A Removed 🗑️
Key Generation ML-KEM 512 portable unpacked (external random) macos-13_64 26,377 ns N/A Removed 🗑️
Key Generation ML-KEM 512 avx2 external random macos-13_64 114,939 ns N/A Removed 🗑️
Key Generation ML-KEM 512 avx2 unpacked (external random) macos-13_64 106,809 ns N/A Removed 🗑️
Encapsulation ML-KEM 512 portable external random macos-13_64 29,361 ns N/A Removed 🗑️
Encapsulation ML-KEM 512 portable unpacked (external random) macos-13_64 13,933 ns N/A Removed 🗑️
Encapsulation ML-KEM 512 avx2 external random macos-13_64 83,465 ns N/A Removed 🗑️
Encapsulation ML-KEM 512 avx2 unpacked (external random) macos-13_64 77,045 ns N/A Removed 🗑️
Decapsulation ML-KEM 512 portable N/A macos-13_64 40,739 ns N/A Removed 🗑️
Decapsulation ML-KEM 512 portable unpacked macos-13_64 30,156 ns N/A Removed 🗑️
Decapsulation ML-KEM 512 avx2 N/A macos-13_64 98,716 ns N/A Removed 🗑️
Decapsulation ML-KEM 512 avx2 unpacked macos-13_64 106,427 ns N/A Removed 🗑️
PK Validation ML-KEM 512 portable N/A macos-13_64 1,248 ns N/A Removed 🗑️
PK Validation ML-KEM 512 avx2 N/A macos-13_64 429 ns N/A Removed 🗑️

os: macos-13_64, keySize: 768

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 768 portable external random macos-13_64 38,433 ns N/A Removed 🗑️
Key Generation ML-KEM 768 portable unpacked (external random) macos-13_64 37,858 ns N/A Removed 🗑️
Key Generation ML-KEM 768 avx2 external random macos-13_64 63,416 ns N/A Removed 🗑️
Key Generation ML-KEM 768 avx2 unpacked (external random) macos-13_64 62,890 ns N/A Removed 🗑️
Encapsulation ML-KEM 768 portable external random macos-13_64 47,316 ns N/A Removed 🗑️
Encapsulation ML-KEM 768 portable unpacked (external random) macos-13_64 18,837 ns N/A Removed 🗑️
Encapsulation ML-KEM 768 avx2 external random macos-13_64 64,858 ns N/A Removed 🗑️
Encapsulation ML-KEM 768 avx2 unpacked (external random) macos-13_64 55,141 ns N/A Removed 🗑️
Decapsulation ML-KEM 768 portable N/A macos-13_64 62,336 ns N/A Removed 🗑️
Decapsulation ML-KEM 768 portable unpacked macos-13_64 43,371 ns N/A Removed 🗑️
Decapsulation ML-KEM 768 avx2 N/A macos-13_64 71,145 ns N/A Removed 🗑️
Decapsulation ML-KEM 768 avx2 unpacked macos-13_64 64,344 ns N/A Removed 🗑️
PK Validation ML-KEM 768 portable N/A macos-13_64 1,713 ns N/A Removed 🗑️
PK Validation ML-KEM 768 avx2 N/A macos-13_64 641 ns N/A Removed 🗑️

os: macos-13_64, keySize: 1024

name category keySize platform api os Baseline Current Change
Key Generation ML-KEM 1024 portable external random macos-13_64 59,109 ns N/A Removed 🗑️
Key Generation ML-KEM 1024 portable unpacked (external random) macos-13_64 59,050 ns N/A Removed 🗑️
Key Generation ML-KEM 1024 avx2 external random macos-13_64 70,526 ns N/A Removed 🗑️
Key Generation ML-KEM 1024 avx2 unpacked (external random) macos-13_64 69,335 ns N/A Removed 🗑️
Encapsulation ML-KEM 1024 portable external random macos-13_64 75,595 ns N/A Removed 🗑️
Encapsulation ML-KEM 1024 portable unpacked (external random) macos-13_64 26,757 ns N/A Removed 🗑️
Encapsulation ML-KEM 1024 avx2 external random macos-13_64 74,368 ns N/A Removed 🗑️
Encapsulation ML-KEM 1024 avx2 unpacked (external random) macos-13_64 67,701 ns N/A Removed 🗑️
Decapsulation ML-KEM 1024 portable N/A macos-13_64 87,547 ns N/A Removed 🗑️
Decapsulation ML-KEM 1024 portable unpacked macos-13_64 56,530 ns N/A Removed 🗑️
Decapsulation ML-KEM 1024 avx2 N/A macos-13_64 76,319 ns N/A Removed 🗑️
Decapsulation ML-KEM 1024 avx2 unpacked macos-13_64 68,725 ns N/A Removed 🗑️
PK Validation ML-KEM 1024 portable N/A macos-13_64 2,340 ns N/A Removed 🗑️
PK Validation ML-KEM 1024 avx2 N/A macos-13_64 819 ns N/A Removed 🗑️

Performance Report:

Visualized Results

label: KeyGen, keySize: 44, os: ubuntu-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 KeyGen portable ubuntu-latest_64 78 μs 77 μs +1.28%
libcrux ML-DSA 44 KeyGen avx2 ubuntu-latest_64 24 μs 24 μs 0.00%
pqclean ML-DSA 44 KeyGen auto ubuntu-latest_64 25 μs 25 μs 0.00%

label: Sign, keySize: 44, os: ubuntu-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Sign portable ubuntu-latest_64 339 μs 330 μs +2.65%
libcrux ML-DSA 44 Sign avx2 ubuntu-latest_64 888 μs 891 μs -0.34%
pqclean ML-DSA 44 Sign auto ubuntu-latest_64 77 μs 78 μs -1.30%

label: Verify, keySize: 44, os: ubuntu-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Verify portable ubuntu-latest_64 86 μs 86 μs 0.00%
libcrux ML-DSA 44 Verify avx2 ubuntu-latest_64 34 μs 34 μs 0.00%
pqclean ML-DSA 44 Verify auto ubuntu-latest_64 31 μs 32 μs -3.23%

label: KeyGen, keySize: 65, os: ubuntu-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 KeyGen portable ubuntu-latest_64 152 μs 152 μs 0.00%
libcrux ML-DSA 65 KeyGen avx2 ubuntu-latest_64 43 μs 43 μs 0.00%
pqclean ML-DSA 65 KeyGen auto ubuntu-latest_64 44 μs 44 μs 0.00%

label: Sign, keySize: 65, os: ubuntu-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Sign portable ubuntu-latest_64 535 μs 535 μs 0.00%
libcrux ML-DSA 65 Sign avx2 ubuntu-latest_64 1.07 ms 1.09 ms -1.87%
pqclean ML-DSA 65 Sign auto ubuntu-latest_64 124 μs 123 μs +0.81%

label: Verify, keySize: 65, os: ubuntu-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Verify portable ubuntu-latest_64 145 μs 144 μs +0.69%
libcrux ML-DSA 65 Verify avx2 ubuntu-latest_64 50 μs 51 μs -2.00%
pqclean ML-DSA 65 Verify auto ubuntu-latest_64 49 μs 49 μs 0.00%

label: KeyGen, keySize: 87, os: ubuntu-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 KeyGen portable ubuntu-latest_64 223 μs 223 μs 0.00%
libcrux ML-DSA 87 KeyGen avx2 ubuntu-latest_64 66 μs 69 μs -4.55%
pqclean ML-DSA 87 KeyGen auto ubuntu-latest_64 70 μs 70 μs 0.00%

label: Sign, keySize: 87, os: ubuntu-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Sign portable ubuntu-latest_64 657 μs 657 μs 0.00%
libcrux ML-DSA 87 Sign avx2 ubuntu-latest_64 892 μs 879 μs +1.46%
pqclean ML-DSA 87 Sign auto ubuntu-latest_64 144 μs 143 μs +0.69%

label: Verify, keySize: 87, os: ubuntu-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Verify portable ubuntu-latest_64 233 μs 234 μs -0.43%
libcrux ML-DSA 87 Verify avx2 ubuntu-latest_64 77 μs 77 μs 0.00%
pqclean ML-DSA 87 Verify auto ubuntu-latest_64 74 μs 74 μs 0.00%

label: KeyGen, keySize: 44, os: ubuntu-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 KeyGen portable ubuntu-latest_32 209 μs 209 μs 0.00%
pqclean ML-DSA 44 KeyGen auto ubuntu-latest_32 261 μs 261 μs 0.00%

label: Sign, keySize: 44, os: ubuntu-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Sign portable ubuntu-latest_32 557 μs 559 μs -0.36%
pqclean ML-DSA 44 Sign auto ubuntu-latest_32 815 μs 808 μs +0.86%

label: Verify, keySize: 44, os: ubuntu-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Verify portable ubuntu-latest_32 216 μs 215 μs +0.46%
pqclean ML-DSA 44 Verify auto ubuntu-latest_32 286 μs 286 μs 0.00%

label: KeyGen, keySize: 65, os: ubuntu-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 KeyGen portable ubuntu-latest_32 398 μs 399 μs -0.25%
pqclean ML-DSA 65 KeyGen auto ubuntu-latest_32 476 μs 476 μs 0.00%

label: Sign, keySize: 65, os: ubuntu-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Sign portable ubuntu-latest_32 889 μs 898 μs -1.01%
pqclean ML-DSA 65 Sign auto ubuntu-latest_32 1.3 ms 1.31 ms -0.77%

label: Verify, keySize: 65, os: ubuntu-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Verify portable ubuntu-latest_32 376 μs 377 μs -0.27%
pqclean ML-DSA 65 Verify auto ubuntu-latest_32 475 μs 475 μs 0.00%

label: KeyGen, keySize: 87, os: ubuntu-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 KeyGen portable ubuntu-latest_32 639 μs 636 μs +0.47%
pqclean ML-DSA 87 KeyGen auto ubuntu-latest_32 780 μs 780 μs 0.00%

label: Sign, keySize: 87, os: ubuntu-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Sign portable ubuntu-latest_32 1.15 ms 1.16 ms -0.87%
pqclean ML-DSA 87 Sign auto ubuntu-latest_32 1.66 ms 1.67 ms -0.60%

label: Verify, keySize: 87, os: ubuntu-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Verify portable ubuntu-latest_32 622 μs 623 μs -0.16%
pqclean ML-DSA 87 Verify auto ubuntu-latest_32 800 μs 801 μs -0.13%

label: KeyGen, keySize: 44, os: windows-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 KeyGen portable windows-latest_64 77 μs 77 μs 0.00%
libcrux ML-DSA 44 KeyGen avx2 windows-latest_64 25 μs 25 μs 0.00%
pqclean ML-DSA 44 KeyGen auto windows-latest_64 105 μs 105 μs 0.00%

label: Sign, keySize: 44, os: windows-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Sign portable windows-latest_64 324 μs 326 μs -0.62%
libcrux ML-DSA 44 Sign avx2 windows-latest_64 866 μs 865 μs +0.12%
pqclean ML-DSA 44 Sign auto windows-latest_64 461 μs 459 μs +0.43%

label: Verify, keySize: 44, os: windows-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Verify portable windows-latest_64 86 μs 86 μs 0.00%
libcrux ML-DSA 44 Verify avx2 windows-latest_64 37 μs 34 μs +8.11%
pqclean ML-DSA 44 Verify auto windows-latest_64 121 μs 120 μs +0.83%

label: KeyGen, keySize: 65, os: windows-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 KeyGen portable windows-latest_64 153 μs 152 μs +0.65%
libcrux ML-DSA 65 KeyGen avx2 windows-latest_64 45 μs 52 μs -15.56%
pqclean ML-DSA 65 KeyGen auto windows-latest_64 190 μs 188 μs +1.05%

label: Sign, keySize: 65, os: windows-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Sign portable windows-latest_64 528 μs 536 μs -1.52%
libcrux ML-DSA 65 Sign avx2 windows-latest_64 1.07 ms 1.05 ms +1.87%
pqclean ML-DSA 65 Sign auto windows-latest_64 718 μs 714 μs +0.56%

label: Verify, keySize: 65, os: windows-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Verify portable windows-latest_64 144 μs 144 μs 0.00%
libcrux ML-DSA 65 Verify avx2 windows-latest_64 52 μs 52 μs 0.00%
pqclean ML-DSA 65 Verify auto windows-latest_64 185 μs 185 μs 0.00%

label: KeyGen, keySize: 87, os: windows-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 KeyGen portable windows-latest_64 225 μs 224 μs +0.44%
libcrux ML-DSA 87 KeyGen avx2 windows-latest_64 74 μs 70 μs +5.41%
pqclean ML-DSA 87 KeyGen auto windows-latest_64 299 μs 300 μs -0.33%

label: Sign, keySize: 87, os: windows-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Sign portable windows-latest_64 646 μs 657 μs -1.70%
libcrux ML-DSA 87 Sign avx2 windows-latest_64 872 μs 870 μs +0.23%
pqclean ML-DSA 87 Sign auto windows-latest_64 916 μs 922 μs -0.66%

label: Verify, keySize: 87, os: windows-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Verify portable windows-latest_64 230 μs 230 μs 0.00%
libcrux ML-DSA 87 Verify avx2 windows-latest_64 83 μs 81 μs +2.41%
pqclean ML-DSA 87 Verify auto windows-latest_64 316 μs 316 μs 0.00%

label: KeyGen, keySize: 44, os: windows-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 KeyGen portable windows-latest_32 215 μs 213 μs +0.93%
pqclean ML-DSA 44 KeyGen auto windows-latest_32 222 μs 221 μs +0.45%

label: Sign, keySize: 44, os: windows-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Sign portable windows-latest_32 566 μs 564 μs +0.35%
pqclean ML-DSA 44 Sign auto windows-latest_32 718 μs 704 μs +1.95%

label: Verify, keySize: 44, os: windows-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Verify portable windows-latest_32 221 μs 220 μs +0.45%
pqclean ML-DSA 44 Verify auto windows-latest_32 239 μs 244 μs -2.09%

label: KeyGen, keySize: 65, os: windows-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 KeyGen portable windows-latest_32 405 μs 407 μs -0.49%
pqclean ML-DSA 65 KeyGen auto windows-latest_32 406 μs 407 μs -0.25%

label: Sign, keySize: 65, os: windows-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Sign portable windows-latest_32 915 μs 907 μs +0.87%
pqclean ML-DSA 65 Sign auto windows-latest_32 1.13 ms 1.13 ms 0.00%

label: Verify, keySize: 65, os: windows-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Verify portable windows-latest_32 384 μs 384 μs 0.00%
pqclean ML-DSA 65 Verify auto windows-latest_32 394 μs 395 μs -0.25%

label: KeyGen, keySize: 87, os: windows-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 KeyGen portable windows-latest_32 644 μs 645 μs -0.16%
pqclean ML-DSA 87 KeyGen auto windows-latest_32 662 μs 665 μs -0.45%

label: Sign, keySize: 87, os: windows-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Sign portable windows-latest_32 1.17 ms 1.18 ms -0.85%
pqclean ML-DSA 87 Sign auto windows-latest_32 1.45 ms 1.47 ms -1.38%

label: Verify, keySize: 87, os: windows-latest_32

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Verify portable windows-latest_32 629 μs 631 μs -0.32%
pqclean ML-DSA 87 Verify auto windows-latest_32 669 μs 670 μs -0.15%

label: KeyGen, keySize: 44, os: macos-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 KeyGen portable macos-latest_64 42 μs 46 μs -9.52%
libcrux ML-DSA 44 KeyGen neon macos-latest_64 26 μs 28 μs -7.69%
pqclean ML-DSA 44 KeyGen auto macos-latest_64 21 μs 25 μs -19.05%

label: Sign, keySize: 44, os: macos-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Sign portable macos-latest_64 168 μs 195 μs -16.07%
libcrux ML-DSA 44 Sign neon macos-latest_64 125 μs 139 μs -11.20%
pqclean ML-DSA 44 Sign auto macos-latest_64 72 μs 83 μs -15.28%

label: Verify, keySize: 44, os: macos-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Verify portable macos-latest_64 43 μs 53 μs -23.26%
libcrux ML-DSA 44 Verify neon macos-latest_64 32 μs 37 μs -15.63%
pqclean ML-DSA 44 Verify auto macos-latest_64 26 μs 48 μs -84.62%

label: KeyGen, keySize: 65, os: macos-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 KeyGen portable macos-latest_64 84 μs 113 μs -34.52%
libcrux ML-DSA 65 KeyGen neon macos-latest_64 57 μs 73 μs -28.07%
pqclean ML-DSA 65 KeyGen auto macos-latest_64 48 μs 69 μs -43.75%

label: Sign, keySize: 65, os: macos-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Sign portable macos-latest_64 264 μs 371 μs -40.53%
libcrux ML-DSA 65 Sign neon macos-latest_64 210 μs 327 μs -55.71%
pqclean ML-DSA 65 Sign auto macos-latest_64 115 μs 135 μs -17.39%

label: Verify, keySize: 65, os: macos-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Verify portable macos-latest_64 73 μs 93 μs -27.40%
libcrux ML-DSA 65 Verify neon macos-latest_64 50 μs 74 μs -48.00%
pqclean ML-DSA 65 Verify auto macos-latest_64 39 μs 41 μs -5.13%

label: KeyGen, keySize: 87, os: macos-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 KeyGen portable macos-latest_64 120 μs 132 μs -10.00%
libcrux ML-DSA 87 KeyGen neon macos-latest_64 71 μs 80 μs -12.68%
pqclean ML-DSA 87 KeyGen auto macos-latest_64 57 μs 62 μs -8.77%

label: Sign, keySize: 87, os: macos-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Sign portable macos-latest_64 315 μs 389 μs -23.49%
libcrux ML-DSA 87 Sign neon macos-latest_64 272 μs 367 μs -34.93%
pqclean ML-DSA 87 Sign auto macos-latest_64 130 μs 146 μs -12.31%

label: Verify, keySize: 87, os: macos-latest_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Verify portable macos-latest_64 114 μs 132 μs -15.79%
libcrux ML-DSA 87 Verify neon macos-latest_64 82 μs 87 μs -6.10%
pqclean ML-DSA 87 Verify auto macos-latest_64 55 μs 69 μs -25.45%

label: KeyGen, keySize: 44, os: macos-13_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 KeyGen portable macos-13_64 81 μs N/A Removed 🗑️
libcrux ML-DSA 44 KeyGen avx2 macos-13_64 28 μs N/A Removed 🗑️

label: Sign, keySize: 44, os: macos-13_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Sign portable macos-13_64 278 μs N/A Removed 🗑️
libcrux ML-DSA 44 Sign avx2 macos-13_64 660 μs N/A Removed 🗑️

label: Verify, keySize: 44, os: macos-13_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 44 Verify portable macos-13_64 85 μs N/A Removed 🗑️
libcrux ML-DSA 44 Verify avx2 macos-13_64 36 μs N/A Removed 🗑️

label: KeyGen, keySize: 65, os: macos-13_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 KeyGen portable macos-13_64 159 μs N/A Removed 🗑️
libcrux ML-DSA 65 KeyGen avx2 macos-13_64 49 μs N/A Removed 🗑️

label: Sign, keySize: 65, os: macos-13_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Sign portable macos-13_64 460 μs N/A Removed 🗑️
libcrux ML-DSA 65 Sign avx2 macos-13_64 829 μs N/A Removed 🗑️

label: Verify, keySize: 65, os: macos-13_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 65 Verify portable macos-13_64 145 μs N/A Removed 🗑️
libcrux ML-DSA 65 Verify avx2 macos-13_64 53 μs N/A Removed 🗑️

label: KeyGen, keySize: 87, os: macos-13_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 KeyGen portable macos-13_64 231 μs N/A Removed 🗑️
libcrux ML-DSA 87 KeyGen avx2 macos-13_64 71 μs N/A Removed 🗑️

label: Sign, keySize: 87, os: macos-13_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Sign portable macos-13_64 556 μs N/A Removed 🗑️
libcrux ML-DSA 87 Sign avx2 macos-13_64 720 μs N/A Removed 🗑️

label: Verify, keySize: 87, os: macos-13_64

implementation keySize label hardware os Baseline Current Change
libcrux ML-DSA 87 Verify portable macos-13_64 233 μs N/A Removed 🗑️
libcrux ML-DSA 87 Verify avx2 macos-13_64 84 μs N/A Removed 🗑️

@keks keks force-pushed the keks/better-benchmarks branch from 42e077c to dfa60f9 Compare September 23, 2025 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant