Reverse Engineering the Brain Computations Involved in Speech Processing<-- Return to the list
Start Time: 11:00am
End Time: 12:00pm
Speaker: Nima Mesgarani , Assistant Professor
From: Electrical Engineering Department, Columbia University
Location: 414 CEPSR
Hosted by: Aurel Lazar
Abstract: The brain empowers humans and other animals with remarkable abilities to sense and perceive their acoustic environment in highly degraded conditions. These seemingly trivial tasks for humans have proven extremely difficult to model and implement in machines. One crucial limiting factor has been the need for a deep interaction between two very different disciplines, that of neuroscience and engineering. In this talk, I will present results of an interdisciplinary research effort to address the following fundamental questions: 1) what computation is performed in the brain when we listen to complex sounds? 2) How could this computation be modeled and implemented in computational systems? and 3) how could one build an interface to connect brain signals to machines? I will present results from recent invasive neural recordings in human auditory cortex that show a distributed representation of speech in auditory cortical areas. This representation remains unchanged even when an interfering speaker is added, as if the second voice is filtered out by the brain. In addition, I will show how this knowledge has been successfully incorporated in novel automatic speech processing applications and used by DARPA and other agencies for their superior performance. Finally, I will demonstrate how speech can be read directly from the brain that eventually, can allow for communication by people who have lost their ability to speak. This integrated research approach leads to better scientific understanding of the brain, innovative computational algorithms, and a new generation of Brain-Machine interfaces.
Background and supplementary research information available here.
Speaker Bio: Nima Mesgarani is an assistant professor of Electrical Engineering at Columbia University. He received his Ph.D. in Electrical Engineering from University of Maryland where he worked on neuromorphic speech technologies and neurophysiology of auditory cortex. He was a postdoctoral scholar at Center for Language and Speech Processing at Johns Hopkins University, and subsequently at the Neurosurgery department of University of California San Francisco. His research interests are in human-like information processing of acoustic signals at the interface of engineering and brain science.