Voice changing/altering function

Hello!

First of all I want to thank the developers and community for really astonishing work on jitsi!

I am java developer trying to find out if I can add very important (for our project) feature. We need to build a service for anonymous interviewing with voice optional masking/altering.

Can anyone explain where to start research for jitsi platform? Can I theoretically add this functionality to videobridge?

What does that mean, can you give an example?

Could this be done on the client with an AudioContext or something maybe?

I suppose you could do this on the bridge, but the code related to media decoding/encoding is pretty decayed at this point and we’re moving in another direction. So it could still be done, you’d just have to manage the encoding/decoding yourself. Media packets flow through different transformers, so you could add one to ‘transform’ the audio like this.

Our customers want to have ability to change voice while being interviewed. It can be pitch shift or something more sophisticated.

We have to use WebRTC client because of other requirement (collaborative notepad) so we can’t rely on some native android/iOS audio filtering.

I haven’t found yet any abilities to filter voice with WebRTC, so I think I should deep dive into videobridge code.

Did you mean AudioContext in WebAudioAPI? As I can see it could help me a lot. I found some working examples of processing voice with WebAudioAPI before it goes to WebRTC part.

Could you please direct me where to dig in jitsi codebase?

I think that would make a great GSoC project (implementing it in js).

Boris

lib-jitsi-meet’s JitsiMeetJS#createLocalTracks is the general point from which audio tracks are grabbed: https://github.com/jitsi/lib-jitsi-meet/blob/b596bc7d288e1f8f3fa95eace66c6ab13d73a2c3/JitsiMeetJS.js#L303. And I believe this would be the main point from which jitsi-meet calls createLocalTracks: https://github.com/jitsi/jitsi-meet/blob/5c0ae10ccb714511433b3aa0649afbc810ef4dd1/react/features/base/tracks/functions.js#L58. One thing to note is mobile may not have access to AudioContext like the browser does, if mobile is to be a supported use case.