Using replaceTrack or setEffect to swap local audio stream -> event listener needed? / questions about stream generation

hi all,

i’ve been trying to swap out the audiostream generated by JitsiMeetJS.createLocalTracks() with another stream from obtained from AudioContext.createMediaStreamDestination().
My goal is to take a stream from the local microphone (which I do now with navigator.mediaDevices.getUserMedia()), then pass it through different webaudio nodes to process the signal, and finally connecting the processed audio to the mediastream destination node, whose stream should then be sent over to the other participant.

I already checked out some threads here in the forum that deal with similiar stuff, but so far I haven’t gotten it to work. If anyone has answers to the following questions, it would help me a lot!

  1. In order to get this working, do I have to modify lib-jitsi-meet? I checked out some of the api’s code on github and I got a bit confused if with the current configuration, jitsi would even take any stream that is created via something else than JitsiMeetJS.createLocalTracks().

  2. In this post the person managed to swap a video track by using replaceTrack() inside an event listener for TRACK_ADDED.
    I also came across this other post where the person is also placing the code inside an event listener for TRACK_ADDED.
    I was wondering why? Because reading the documentation of lib-jitsi-meet gave me the impression that TRACK_ADDED was an event that fired when a remote track is added, not a local one. But I might be wrong?

  3. I am also unsure what JitsiMeetJS.createLocalTracks() actually generates? is it an array of tracks, or is it a stream? or something else?

  4. I also saw that there is a way to achieve something similiar (I think?) with the setEffect() method. Again, I was wondering if it takes a stream or a track? and should it be newTrack.setEffect(oldTrack) or oldTrack.setEffect(newTrack) ? or I am thinking this completely wrong?

I hope I am not asking too much :smiley: thanks for reading this far and have a good weekend!!

so far this is my not-working code. “trackDestination” contains the MediaStreamDestination.

JitsiMeetJS.createLocalTracks({ devices: [ ‘audio’ ] })
.then(onLocalTracks)
.catch(error => {
throw error;
}); };

function onLocalTracks(tracks) {
localTracks = tracks;
for (let i = 0; i < localTracks.length; i++) {

if (isJoined) {
    room.addTrack(localTracks[i]);

    room.on(JitsiMeetJS.events.conference.TRACK_ADDED, function(track) {
    
        if(track.getType() === 'audio' && track.isLocal) {
            room.replaceTrack(track[i], trackDestination[i]);
            $('body').append(
                `<audio autoplay='1' muted='true' id='localAudio${i}' />`);
                track[i].attach($(`#localAudio${i}`)[0]);
        } });};

}};

i get the following errors:

2022-12-16T15:03:45.455Z [modules/RTC/BridgeChannel.js] <7441/_handleChannel/e.onclose>: Channel closed: 1001 endpoint closed Logger.js:154:22

2022-12-16T15:03:51.566Z [modules/statistics/RTPStatsCollector.js] <7441/Ti.prototype.errorCallback>: Get stats error DOMException: An attempt was made to use an object that is not, or is no longer, usable Logger.js:154:22

WebRTC: ICE failed, add a STUN server and see about:webrtc for more details

2022-12-16T15:03:59.230Z [modules/RTC/BridgeChannel.js] <7441/_handleChannel/e.onclose>: Channel closed: 1001 endpoint closed Logger.js:154:22

The description does not look like plan-b interop.js:351:48

2022-12-16T15:04:05.563Z [modules/statistics/RTPStatsCollector.js] <7441/Ti.prototype.errorCallback>: Get stats error DOMException: An attempt was made to use an object that is not, or is no longer, usable Logger.js:154:22

2022-12-16T15:04:09.797Z [modules/xmpp/strophe.util.js] <7441/va/_t.Strophe.log>: Strophe: Disconnect was called because: undefined

2022-12-16T15:04:09.845Z [modules/xmpp/strophe.util.js] <7441/va/_t.Strophe.log>: Strophe: 7441/_onTrackDetach/<@https://my.domain.com/libs/lib-jitsi-meet.min.js:2:635068

2022-12-16T15:04:09.845Z [modules/xmpp/strophe.util.js] <7441/va/_t.Strophe.log>: Strophe: error: undefined https://my.domain.com/libs/lib-jitsi-meet.min.js:2 - TypeError: e.removeEventListener is not a function