Voice changing/altering function


#1

Hello!

First of all I want to thank the developers and community for really astonishing work on jitsi!

I am java developer trying to find out if I can add very important (for our project) feature. We need to build a service for anonymous interviewing with voice optional masking/altering.

Can anyone explain where to start research for jitsi platform? Can I theoretically add this functionality to videobridge?


#2

What does that mean, can you give an example?


#3

Could this be done on the client with an AudioContext or something maybe?

I suppose you could do this on the bridge, but the code related to media decoding/encoding is pretty decayed at this point and we’re moving in another direction. So it could still be done, you’d just have to manage the encoding/decoding yourself. Media packets flow through different transformers, so you could add one to ‘transform’ the audio like this.


#4

Our customers want to have ability to change voice while being interviewed. It can be pitch shift or something more sophisticated.


#5

We have to use WebRTC client because of other requirement (collaborative notepad) so we can’t rely on some native android/iOS audio filtering.

I haven’t found yet any abilities to filter voice with WebRTC, so I think I should deep dive into videobridge code.


#6

Did you mean AudioContext in WebAudioAPI? As I can see it could help me a lot. I found some working examples of processing voice with WebAudioAPI before it goes to WebRTC part.

Could you please direct me where to dig in jitsi codebase?


#7

I think that would make a great GSoC project (implementing it in js).

Boris


#8

lib-jitsi-meet’s JitsiMeetJS#createLocalTracks is the general point from which audio tracks are grabbed: https://github.com/jitsi/lib-jitsi-meet/blob/b596bc7d288e1f8f3fa95eace66c6ab13d73a2c3/JitsiMeetJS.js#L303. And I believe this would be the main point from which jitsi-meet calls createLocalTracks: https://github.com/jitsi/jitsi-meet/blob/5c0ae10ccb714511433b3aa0649afbc810ef4dd1/react/features/base/tracks/functions.js#L58. One thing to note is mobile may not have access to AudioContext like the browser does, if mobile is to be a supported use case.