Statistical language modelling and novel parsing techniques for enhanced creation and editing of mathematical e-content using spoken input

The work described in this thesis aims at facilitating the design and im- plementation of web-based editors, driven by speech or natural language input, with a focus on editing mathematics. First, a taxonomy for system architectures of speech-based applications is given. This classification is based on the location of the speech recognition, the speech, and application logic and the resulting flow of data between client and server components. This contribution extends existing system architecture approaches to take into account the characteristics of speech- based systems. We then show, using statistical language modelling techniques, that math- ematics, either spoken or typed, is more predictable than everyday natu- ral languages. We illustrate how these models, in combination with error correction algorithms, can be used to successfully assist the process of cre- ating mathematical expressions within electronic documents using speech. We have successfully implemented systems to demonstrate our findings, which have also been evaluated using standard language modelling evalua- tion techniques. This work is novel as applying statistical language models to the recognition of spoken mathematics has not been evaluated to this extent prior to our work. We create a parsing framework for spoken mathematics, based on mixfix operators, operator precedences and non-deterministic parsing techniques. This framework can significantly improve the design and parsing of spoken command languages such as spoken mathematics. A novel robust error recovery method for an adaptation of the XGLR parsing approach to our operator precedence setting is presented. This greatly enhances the range of spoken or typed mathematics that can be parsed. The novel parsing framework, algorithms and error recovery that we have designed are suitable for more general structured spoken command languages, as well. The algorithms devised in this thesis have been implemented and integrated in a research prototype system called TalkMaths. We evaluate our contri- butions to the new version of this system by comparing the power of our parser with that contained in previous versions, and by conducting a field study where students engage with our system in a real classroom-based environment. We show that using TalkMaths, rather than a conventional mathematics editor, had a positive impact on the learning and understand- ing of mathematical concepts of the participants.