Add Ripple

[trueadm]

Adds Ripple. Unfortunately, I'm on my old MacBook M1 Pro and running the benchmarks locally takes forever and is wildly inconsistent each time – so I have no idea how bad Ripple is.

[leeoniya]

you dont need to run them all.

vanillajs-lite is now consistently in first place, so should be enough to just test against that for ballpark numbers.

[trueadm]

I mean, I can't run a single test without it taking about 5mins to each do the first benchmark, not the whole benchmark, just the 01_ create rows benchmark. The scores are terrible too:

It could be because I get a ton of warnings from Webdriver that Chrome is running in emulated mode due to some MacOS emulation feature enabled. FWIW it worked fine 2 weeks ago on my old M3 Max, but I had to give that back to Vercel :)

[leeoniya]

this is what i get on my 2021-era AMD Ryzen7 PRO 5850U, Linux/KDE/Plasma.

[trueadm]

Thanks for checking it for me. :)

[krausest]

I mean, I can't run a single test without it taking about 5mins to each do the first benchmark, not the whole benchmark, just the 01_ create rows benchmark. The scores are terrible too:

Sorry to hear. I also have a MBA with a M1 and sometimes I still use it (e.g. when this benchmark is running 🙄 ) and I still like it, but I see no reason for emulation or any duration close to 5 minutes, so I'd like to look into that.

Did you update and (re-)compile the webdriver-ts folder (cd into webdriver-ts, npm install, npm run compile). Then you might try npm run bench keyed/ripple.
Look for the output it should say "INFO: Using puppeteer benchmark runner". I haven't used webdriver for ages, it was a steady source of annoyance, puppeteer runs way smoother across chrome versions and avoids all x64/arm64 issues - there are no nasty Intel processes shown in the activity monitor for the benchmark.

One maybe important tip. You can specify a specific chrome instance and select frameworks like that:
npm run bench -- --chromeBinary /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --framework keyed/vanillajs-lite keyed/ivi > log.txt
Meanwhile I really think that NOT piping the log into a file is a significant factor for noise and was the main reason incremental runs were worse than full runs (I always piped stdout for full runs and I didn't for incremental runs that were less stable and slower).
If it's still that slow I'd be happy to look at the log.

And of course I'll merge it soon.

[trueadm]

Thanks for the great suggestions. I tried that but it's still very slow. I gave up waiting, but here's a partial log from leaving it for a few minutes: https://gist.github.com/trueadm/cf3a00c7355765503e15ffd94a06b579

[krausest]

Thanks. I compared it against the log on my MBA M1 16 GB. The real difference starts in line 128:

mousedownToClick 123 traces/ivi-v4.0.0-keyed_01_run1k_0.json
**** resultScript = 2.584
duration for ivi and 01_run1k: {"tsStart":420239965643,"tsEnd":420240000548,"duration":34.905,"layouts":1,"raf_long_delay":0,"droppedNonMainProces

The duration for one ivi run 1k iteration on my M1 is 35 msecs on my machine and 1492 msecs on your machine. The duration stays in the range of 1.4 seconds so it's not a warm up effect.
Something must be very wrong here (your M1 is 42 times slower than mine).
It took 28 seconds to complete the whole run 1k benchmark for ivi on my machine.

Is it only for this benchmark that it's that slow?
How much RAM does your Mac have? I see 10 GB used when running the benchmark, so 8 GB total RAM might be a problem.

[trueadm]

It's slow in every benchmark too, not just the first one. The ones where the CPU throttled take 15+ minutes to run.

I have 16GB on my M1 Pro. TBH things got bad after upgrading Sequoia. What MacOS version do you have?

[trueadm]

Anyway, I've pushed an updated package version and kind of given up with this MacBook. Maybe it's just a hardware issue.

[leeoniya]

and kind of given up with this MacBook. Maybe it's just a hardware issue.

you could try flashing it with Asahi instead of tossing it :)

https://asahilinux.org/docs/platform/feature-support/m1/

[trueadm]

@krausest how did Ripple do?

[krausest]

Here are the results from an incremental run (all frameworks in the screenshot were updated):

Not too bad :)