Coding by Voice with Open Source Speech Recognition

Presented at The Eleventh HOPE (2016), July 22, 2016, 8 p.m. (60 minutes).

Carpal tunnel and repetitive strain injuries can prevent programmers from typing for months at a time. Fortunately, it is possible to replace the keyboard with speech recognition - David writes Linux systems code by voice. The key is to develop a voice grammar customized for programming. A community has evolved around hacking the commercial Dragon NaturallySpeaking to use custom grammars, but this method suffers from fragmentation, a steep learning curve, and frustrating installation difficulties. In an attempt to make voice coding more accessible, David created a new speech recognition system called Silvius, built on open-source software with free speech models. It can run on cloud servers for ease of setup, or locally for the best latency. He and his collaborators have also prototyped a hardware dongle which types Silvius keystrokes using a fake USB keyboard, and requires no software installation. This talk will include live voice-coding demos with both Dragon and Silvius. The hope is that Silvius will lower the bar for experimentation and innovation, and encourage ordinary programmers to try voice coding, instead of waiting until a crippling injury throws them in at the deep end.


Presenters:

  • David Williams-King
    David Williams-King grew up in the Canadian countryside in a solar-powered house, and is now a computer science PhD student at Columbia University. He researches randomization-based security techniques and maintains an interest in graphics, compilers, and speech recognition. David is privileged to have taught C++ with Bjarne Stroustrup, and proud to have once received an award at the ACM Turing Award ceremony.

Links:

Similar Presentations: