Audio Deja Vu: Audio Formatted Math On The Emacspeak Desktop

1 Overview

This article previews a new feature in the next Emacspeak release — audio-formatted Mathematics using Aural CSS. Volker Sorge worked at Google as a Visiting Scientist from Sep 2012 to August 2013, when we implemented math access in ChromeVox — see this brief overview. Since leaving Google, Volker has refactored and extended his work to create an Open Source Speech-Rule-Engine implemented using NodeJS. This speech-rule-engine can be used in many different environments; Emacspeak leverages that work to enable audio-formatting and interactive browsing of math content.

2 Overview Of Functionality

Math access on the Emacspeak desktop is implemented via module emacspeak-maths.el — see js/node/ in the Emacspeak GitHub repository for setup instructions.

Once loaded, module emacspeak-maths provides a Math Navigator that implements the user interface for sending Math expressions to the Speech-Rule-Engine, and for interactively browsing the resulting structure. At each step of the interaction, Emacspeak receives math expressions that have been annotated with Aural CSS and produces audio-formatted output. The audio-formatted text can itself be navigated in a special Spoken Math emacs buffer.

Module emacspeak-maths.el implements various affordances for dispatching mathematical content to the Speech-Rule-Engine — see usage examples in the next section.

3 Usage Examples

3.1 The Emacspeak Maths Navigator

  • The maths navigator can be invoked by pressing S-SPC (hold down Windows key and press SPC) — this runs the command emacspeak-maths-navigator/body.
  • Once invoked, the /Maths Navigator can be used to enter an expression to read.
  • Pressing SPC again prompts for the LaTeX math expression.
  • Pressing RET guesses the expression to read from the current context.
  • The arrow keys navigate the expression being read.
  • Pressing o switches to the Spoken Math buffer and exits the navigator.

See the relevant chapter in the online Emacspeak manual for details.

3.2 Math Content In LaTeX Documents

  1. Open a LaTeX document containing math content.
  2. Move point to a line containing mathematical markup.
  3. Press S-SPC RET to have that expression audio-formatted.
  4. Use arrow keys to navigate the resulting structure.
  5. Press any other key to exit the navigator.

3.3 Math Content On Wikipedia

  1. Open a Wikipedia page in the Emacs Web Wowser (EWW) that has mathematical content.
  2. Wikipedia displays math as images, with the alt-text giving the LaTeX representation.
  3. Navigate to some math content on the page, then press S-SPC a to speak that content — a is for alt.
  4. As an example, navigate to Wikipedia Math Example, locate math expressions on that page, then press S-SPC a.

3.4 Math Content From The Emacs Calculator

  1. The built-in Emacs Calculator (calc) provides many complex math functions including symbolic algebra.
  2. For my personal calc setup, see tvr/calc-prepare.el in the Emacspeak GitHub repo.
  3. This setting below sets up the Emacs Calculator to output results as LaTeX: (setq calc-language 'tex)
  4. With the above setting in effect, launch the emacs Calculator by pressing M-##.
  5. Press ' — to use algebraic mode — and enter sin(x).
  6. Press a t to get the Taylor series expansion of the above expression, and press x when prompted for the variable.
  7. This displays the Taylor Series expansion up to the desired number of terms — try 7 terms.
  8. Now, with Calc having shown the results as TeX, press S-SPC RET to browse this expression using the Maths Navigator.

4 And The Best Is Yet To Come

This is intentionally called an early preview because there is still much that can be improved:

  1. Enhance the rule engine to infer and convey more semantics.
  2. Improved audio formatting rules to better present the available information.
  3. Update/tune the use of Aural CSS properties to best leverage today's TTS engines.
  4. Integrate math-reading functionality into more usage contexts in addition to the ones enumerated in this article.

5 References

  1. Youtube Video from early 2013 demonstrating Math Access in Chrome
  2. AllThings Digital outlining math access — published June 2013.
  3. Assets 2016 publication describing this work.
  4. js/node/aster-math-examples.tex Collection of math examples in LaTeX from AsTeR. Used to progressively improve speech-rules and the resulting audio-formatted output
  5. Speech-Rule-Engine on github.
  6. Speech-Rule-Engine in action: Accessible Maths in all browsers

Date: 2017-02-08 Wed 00:00

Author: T.V Raman

Created: 2017-02-10 Fri 10:40