Chrome is planning on implementing the same privacy and security measures used by the Web Translation API, which uses on-device language packs for translations, to the Web Speech API.

We should consider adding these mitigations to the Web Speech API spec as well: webmachinelearning/writing-assistance-apis#47

TL;DR: Websites must call install() before they can use the on-device speech recognition language packs to prevent fingerprinting.