Noise-robust speech recognition through auditory feature detection and spike sequence decoding

Phillip B. Schafer, Dezhe Z. Jin

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


Speech recognition in noisy conditions is a major challenge for computer systems, but the human brain performs it routinely and accurately. Automatic speech recognition (ASR) systems that are inspired by neuroscience can potentially bridge the performance gap between humans andmachines.We present a system for noise-robust isolated word recognition that works by decoding sequences of spikes from a population of simulated auditory feature-detecting neurons. Each neuron is trained to respond selectively to a brief spectrotemporal pattern, or feature, drawn from the simulated auditory nerve response to speech. The neural population conveys the time-dependent structure of a sound by its sequence of spikes. We compare two methods for decoding the spike sequences-one using a hidden Markov model-based recognizer, the other using a novel template-based recognition scheme. In the latter case, words are recognized by comparing their spike sequences to template sequences obtained from clean training data, using a similarity measure based on the length of the longest common sub-sequence. Using isolated spoken digits from the AURORA-2 database, we show that our combined system outperforms a state-of-the-art robust speech recognizer at low signal-to-noise ratios. Both the spike-based encoding scheme and the template-based decoding offer gains in noise robustness over traditional speech recognitionmethods.Our system highlights potential advantages of spike-based acoustic coding and provides a biologically motivated framework for robust ASR development.

Original languageEnglish (US)
Pages (from-to)523-556
Number of pages34
JournalNeural computation
Issue number3
StatePublished - 2014

All Science Journal Classification (ASJC) codes

  • Arts and Humanities (miscellaneous)
  • Cognitive Neuroscience


Dive into the research topics of 'Noise-robust speech recognition through auditory feature detection and spike sequence decoding'. Together they form a unique fingerprint.

Cite this