How Edge-AI Chips can Solve the Cocktail Party Problem

Hearing aid technology is severely lagging, and how neural network processors might help save an aging population from dementia.

Welcome to the weekday free-edition of this newsletter that is a small idea, an actionable tip, or a short insight that takes less than 5 minutes to read. On Sundays, I write in-depth technical deep-dives on many aspects of semiconductors and chip design for paid subscribers of this publication. See all the benefits of upgrading your subscription tier!

If you would like to suggest a topic for this newsletter, please use this form.

---

Imagine you’re at a loud party. There's music, laughter, conversation, and clinking of dinnerware. But somehow, you can still have a conversation with your friend standing right next to you. You’re able to focus on their voice and tune out the rest, even though technically, all the sound is hitting your ears at the same time.

That’s because your brain is filtering out background noise so you can pay attention to something important, like someone saying your name, or a conversation you're interested in. This selective hearing is usually referred to as the Cocktail Party Effect. It was first described by Colin Cherry in 1953, who ran experiments asking people to pay attention to one audio channel while ignoring another when both played simultaneously.

Cocktail party impairments

Not withstanding the kinds of impairments I usually encounter at cocktail parties, there are many who cannot separate speech signals from noise. A small percentage of the population with King-Kopetzky Syndrome (KKS) possess normal levels of hearing, but cannot discern speech in noisy environments. KKS is a neurological disorder whose cause is poorly understood. A recent study found that common hearing impairment also leads to the cocktail party effect due to the abnormal blending of the sounds received in each ear. Interestingly, studies also show that the use of hearing aids does not significantly improve the wearer’s ability to discern speech from background sounds.

I can vouch for this observation from my daily interactions with my wife who started wearing hearing aids in her late 30s. Crowded spaces including restaurants, meetings, airports, and even playtime with the children can become exhausting. Not because she can’t hear, but because her brain struggles to pick the sound she wants to hear and stick with it. When all the sounds come in at once, it can get overwhelming to stay with one sound in a blur of voices and reverberation.

Glasses give you back near-normal vision instantly. Hearing aids are nowhere close to restoring hearing to normal levels. Hearing aids amplify and filter sound to compensate. However, fixing hearing loss is more about processing clarity and discrimination rather than amplification and equalization. After prolonged hearing loss, the brain’s auditory pathways weaken or reorganize, so even with amplification, the brain may not interpret sound clearly. This often makes people feel like they can hear louder, but not better, especially in crowded settings like cocktail parties.

Intelligent hearing devices

Improved, intelligent hearing devices are crucial for an aging population, or for a younger generation who already spend too much time with earphones. People with hearing loss are at an increased risk of dementia; hearing loss accounts for 8% of all global dementia cases. Early detection and proper correction can slow the cognitive decline. Hearing loss also leads to increased cognitive load and the resulting societal isolation makes day-to-day living difficult.

Traditionally hearing aids use beamforming with multiple microphones (much like phased array antennas) to increase signal-to-noise ratio. The problem with this approach is that background noise is necessary to identify spatial cues and trying to eliminate them is not ideal.

A promising use of edge-AI is to intelligently process the entire audio spectrum to provide a better hearing experience. Researchers are looking at deep-learning, DSP algorithms, and spiking neural networks to create an intelligent listening (not just hearing) experience. There has been a spate of hearing aid companies like Phonak, Starkey and Oticon who are using Deep Neural Network (DNN) processors in hearing aids in an effort to separate speech signals from background noise.

The other major problem in the hearing aid industry is the sheer cost of these devices. If you thought AirPod ProMax was expensive, you’re in for a world of surprise when you find out that hearing aids often cost thousands of dollars. Given how many people in the world, especially the older generation, need them, there is ample room for much needed disruption in this industry.

A low-priced, AI-enabled hearing aid with extraordinary battery life will be a revolutionary product that would benefit millions — if you’re thinking of building one.