Automated speech directionality processing for the hearing impaired

This technology is an auditory device that utilizes deep neural network models to augment the processing and amplification of desired sounds in multi-source environments for individuals with impaired auditory directionality processing.

Unmet Need: Auditory device for amplifying desired sounds and cancelling undesired noise

Currently, those who suffer from central auditory processing disorders (CAPD) that make it difficult to focus on desired sounds in multi-source environments have limited options to augment their hearing. Standard multi-directional hearing aids and beamformers either lack sufficient signal-to-noise augmentation to adequately amplify desired sounds and cancel undesired noise or suffer a tradeoff with correctly identifying the desired sound source. Thus, many CAPD patients struggle to achieve normal speech understanding in day-to-day life.

The Technology: Deep neural network-based method for superior auditory directionality processing and augmented signal-to-noise ratio

This technology utilizes deep neural network models and automatic speech separation algorithms to determine the source of desired sounds in multi-source environments. It further implements a brain-computer interface to decode the user’s neural activity and amplify the signal of the desired sound. It has been successfully utilized to decode the varying attention of human users when presented with multi-speaker audio and is capable of significant signal-to-noise amplification.

Applications:

• Hearing implants and assistive listening devices
• Speech recognition in mobile phones, vehicles, and other hands-free audio recognition devices
• Military, law enforcement, and security personnel auditory augmentation

Advantages:

• High-accuracy user attention decoding
• Fast decoding response time
• Simple, single-microphone hardware
• High signal-to-noise amplification compared to existing systems

Lead Inventor:

Nima Mesgarani, Ph.D.

Patent Information:

Patent Issued (US 11,961,533)

Related Publications:

• Han C, O’Sullivan J, Luo Y, Herrero J, Mehta AD, Mesgarani N. “Speaker-independent auditory attention decoding without access to clean speech sources.” Science Advances. 2019 May 15;5(5)

• O’Sullivan J, Chen Z, Herrero J, McKhann GM, Sheth SA, Mehta AD, et al. “Neural decoding of attentional selection in multi-speaker environments without access to clean sources.” J Neural Eng. 2017 Oct;14(5):056001.

• Mesgarani N, Chang EF. “Selective cortical representation of attended speaker in multi-talker speech perception.” Nature. 2012 May 10;485(7397):10.1038/nature11020.

Tech Ventures Reference:

• IR CU16343, CU17276

• Licensing Contact: Kristin Neuman