Compress large generated files and decompress them during preloading #154
Description
This repo is currently about 210 MiB (excluding the git data). This makes vendoring the repo into Firefox more difficult. It also puts a strain on users that to just run this benchmark we're expecting them to download about that much data upfront in preloading.
find . -type f -exec du -m {} + | sort -nr | head -100
24 ./wasm/tfjs-model-coco-ssd.js
22 ./wasm/tfjs-model-mobilenet-v1.js
18 ./wasm/tfjs-model-mobilenet-v3.js
18 ./wasm/tfjs-bundle.js
12 ./web-tooling-benchmark/cli.js
12 ./web-tooling-benchmark/browser.js
12 ./RexBench/FlightPlanner/waypoints.js
10 ./Kotlin-compose/build/skiko.wasm
9 ./wasm/tfjs-model-use.js
8 ./wasm/dotnet/build-aot/wwwroot/_framework/dotnet.native.wasm
7 ./SeaMonster/inspector-json-payload.js
6 ./Kotlin-compose/build/compose-benchmarks-benchmarks.wasm
5 ./Octane/mandreel.js
3 ./wasm/dotnet/build-interp/wwwroot/_framework/dotnet.native.wasm
3 ./wasm/dotnet/build-aot/wwwroot/_framework/dotnet.native.js.symbols
3 ./code-load/inspector-payload.js
3 ./code-load/inspector-payload-minified.js
2 ./wasm/dotnet/build-interp/wwwroot/_framework/System.Private.CoreLib.wasm
2 ./wasm/dotnet/build-interp/wwwroot/_framework/icudt_no_CJK.dat
2 ./wasm/dotnet/build-aot/wwwroot/_framework/System.Private.CoreLib.wasm
2 ./wasm/dotnet/build-aot/wwwroot/_framework/icudt_no_CJK.dat
2 ./threejs/three.js
2 ./Octane/typescript-input.js
2 ./Octane/typescript-compiler.js
2 ./Octane/pdfjs.js
2 ./Dart/build/flute.todomvc.dart2wasm.wasm
2 ./Dart/build/flute.complex.dart2wasm.wasm
1 ./WSL/WTypeError.js
... A bunch of WSL fileswasm-tfjs is the largest single contributor at around 91MiB. Now that it's non-default, this is less of an issue (I'll exclude it when vendoring and users won't need to preload it).
But that still leaves a long tail of other wasm files which could really benefit from compression. I think the transformers.js benchmark PR could also benefit from compression.
How would folks feel if we were to compress these files as part of a build script, then decompress them during the preload step on the client side. We could use a DecompressionStream for this in the browser, and a user-space polyfill for that in shell.