Kapur, A., Sarawgi, U., Wadkins, E., Wu, M., Hollenstein, N., & Maes, P. (2020, April 30). Non-Invasive Silent Speech Recognition in Multiple Sclerosis with Dysphonia. Retrieved July 20, 2020, from http://proceedings.mlr.press/v116/kapur20a.html

Abstract

We present the first non-invasive real-time silent speech system that helps patients with speech disorders to communicate in natural language voicelessly, merely by articulating words or sentences in the mouth without producing any sounds. We collected neuromuscular recordings to build a dataset of 10 trials of 15 sentences from each of 3 multiple sclerosis (MS) patients with dysphonia, spanning a range of severity and subsequently affected speech attributes. We present a pipeline wherein we carefully preprocess the data, develop a convolutional neural architecture and employ personalized machine learning. Our system achieves a mean overall test accuracy of 0.81 at a mean information transfer rate of 203.73 bits per minute averaged over all patients. Our work demonstrates the potential of a reliable and promising human-computer interface that classifies intended sentences from silent speech and hence, paves the path for future work with further speech disorders in conditions such as amyotrophic lateral sclerosis (ALS), stroke, and oral cancer, among others.