-
Notifications
You must be signed in to change notification settings - Fork 1
Roadmap Project II
In this project, we have implemented the MathSpeak ruleset in addition to the ChromeVox rule set. While MathSpeak is a robust standard, our work has identified a number of problems. As there is no standardized version of the rule set, various interpretations can be found in the wild, which prove to be inconsistent; in fact, even the official MathSpeak homepage contains contradictions within their various resources. Real world experience should provide data on how to generalize.
An important generalization would be the addition of specialized rules for subject domains. Given the ambiguous nature of mathematics something as simple as (a,b) can describe, e.g., a) 2D coordinates, b) an open interval of real numbers, c) an application of a bilinear form. As soon as the subject area is specified, the correct interpretation can be identified.
A large improvement would be to provide richer speech annotations, including prosody information. Standards such as SSML provide the tools to specify more advanced speech information.
Either of these extensions would require an adjustment and extension of the existing rule sets.
Estimate. Note: all estimates assume a highly qualified developer.
- Additional domain specializations: 100h per domain
- Integration of prosody annotation, e.g. SSML output: 100h
- Adjustment for rule sets: 40h per rule set
The tools developed in this project provide static speech strings. A natural improvement would be to provide speech strings recursively for sub-expressions. This would allow basic forms of navigation, synchronized highlighting etc. inside nested mathematical expressions.
Estimate. 200h
The project focused on providing speech-text for voicing of mathematics via screen readers. Converting to Braille is a natural extension of this work. This would require developing a canonical translation of MathML 3 to existing math Braille formats. It should include sub-expressions and highlighting information.
Estimate. This project would require more research for a realistic estimate. It would also be influenced by plans at ChromeVox for similar tools.
The speech-rule engine is currently not localized. This stems from the fact that the math support in ChromeVox is not localized. A lack of localization blocks use on sites like Wikipedia.
To localize the speech-rule-engine, we need to implement a bespoke localization API that allows for both for the easy localization of rule sets as well as for the selection of localized strings during rule application.
Estimate. ~150h for API provision. This does not include actual localization which could potentially be done on crowd-sourcing platforms (TranslateWiki, Transifex).
The outcome of this project is a nodejs application for server-side use. It would be important to provide a client-side solution in the form of a MathJax extension for on-the-fly speech-text-generation.
To move the results of this project from server to client side, we could follow two approaches.
a) create a webservice that a light-weight MathJax extension could call to generate speech-text b) wrap the entire speech-rule-engine into a (heavy) extension that would generate everything client-side
Both solutions would be an important step. The first one would be relatively lightweight but would require a working internet connection or separate local service. The second one would be heavy (~1.5MB for speech-rule-engine alone) but could be extremely useful in larger applications such as ebook reading systems.
Estimate.
- Light-weight extension: 60h
- Full extension: 150h
The current solution provides speech output. However, to provide this, ChromeVox applies heuristic algorithms that generate semantic information. This information is vital for providing better navigation and exploration of the content as it identifies meaningful structures, break points, nesting levels etc.
A continuation of the project could extract these heuristics and make embed the semantic information to provide standalone information.
Estimate. This project would be very extensive and an estimate requires more research to identify relevant deliverables.
The outcome of this project is already a big step forward for open-source solutions for math accessibility. An important continuation would be the dissemination of the knowledge gained and tools created during the project. A workshop could bring together practitioners from various fields such as AT, authoring, and search, to ensure the results of this work will be integrated into typical content workflows.
Estimate. An estimate depends on the scope of the workshop and support for the participants.