Neural Attention-Driven Auditory LLM for Selective Speech Understanding
This technology is an auditory large language model that infers which sound source a person is attending to in complex environments and generates listener-aligned responses for assistive hearing and adaptive voice assistants.
Unmet Need: Listener-intent decoding for selective speech in noisy environments
Current auditory AI and speech-processing solutions primarily optimize source-agnostic transcription and comprehension, treating all concurrent speakers as equally relevant. In multi-talker, noisy environments, this approach cannot determine the user’s intended talker, leading to inaccurate or irrelevant outputs and reduced effectiveness in assistive hearing and voice-interface applications. A critical unmet need is the ability to align system outputs with the listener’s intended auditory focus to improve reliability, usability, and real-world performance.
The Technology: Neural attention-driven auditory LLM for aid in noisy environments
This technology is an auditory large language model (LLM) system that integrates brain signals to infer listener’s auditory attention in noisy environments. Using time-synchronized intracranial EEG (iEEG) recordings, the system identifies the sound source a listener is attending to and uses this information to guide the LLM toward the intended speech stream while suppressing competing background sounds. The platform can also incorporate auxiliary cues such as head orientation and eye gaze, to improve performance in dynamic, multi-speaker environments.
This technology has been validated through real-time testing in human participants.
Applications:
- Speech-processing technologies, including assistive hearing technologies and voice assistants
- Assistive technologies for hearing impairment
- Audio source separation in complex acoustic environments
- Neuroadaptive audio systems for high-noise workplaces
- Clinical neurotechnology for auditory attention monitoring
- Communication systems for aviation, defense and emergency response
- Cognitive neuroscience research on selective auditory attention
- Meeting and conferencing platforms with attention-guided transcription, summarization, and question answering
Advantages:
- Listener-specific speech processing in multi-speaker environments
- Improves speech understanding in noisy environments
- Real-time adaptation to attention shifts
- Integrates multimodal intent cues including eye gaze and head orientation
Lead Inventor:
Patent Information:
Patent pending
Related Publications:
Tech Ventures Reference:
IR CU25368
Licensing Contact: Kristin Neuman