Disrupting audio effects plague conferences with more than 5 people unmuted: Call for New Noise Cancelling AI

Every video conference in which I have invited all participants to be unmuted inevitably gets disrupted by someone’s cat meowing or walking over their keyboard, or a fire truck or a train blares their siren or horn, or someone in the next room drops something, or … In each of these and the thousand other possibilities, having everyone unmuted becomes distracting and the group asks everyone to mute themselves when they are not speaking.

This problem is not specific to Jitsi, but adversely affects ALL modern video conference platforms that I have tried. It is an important reason preventing video conferences from reaching their potential to replace more face-to-face events and save carbon emissions.

The problem is that dynamic conversations require little interjections and the ability to encourage or discourage a speaker with effects like “hmm” or “aoww”. But all this is impossible because computer audio picks up these little background noises and interrupts the flow of the conversation.

Proposed solution: invent a new kind of noise cancellation AI that can filter out unintentional noises but transmit intentional noises. Almost certainly the feature will need to come with a slider to control how aggressively the algorithm works. Sometimes I want to share the tune I’m composing by tapping my pencil on the desk or clicking my fingernails, but most of the time that is a nervous tic that should be filtered out by the AI noise cancellation.

I understand that this is a research project. I suspect it will be very hard to distinguish someone intentionally making a noise and a distracting noise that should be filtered out because it would ruin the dynamic flow of conversation. A dumb but effective option would be to filter out all non-human voices and noises. But a provision to permit someone like Stephen Hawking to join the conference must, of course, be allowed!

I hope someone will be inspired by this feature request to invent the solution. It would make video conferencing much more effective than current technology permits. Even options that help with common noise effects but are not general purpose could be a major step in the right direction.