Adding tracks to conference sometimes silently fails for the p2p or jvb media session

Developers Jitsi Meet API
#1

Summary

We are using lib-jitsi-meet in our own UI to communicate with a Jitsi Meet architecture behind Docker (thanks to docker-jitsi-meet).

We encountered 2 bugs around which we had to work around when using addTrack() on the JitsiConference. They happen when 2 users join a conference quite simultaneously (I can repro about 80% of the times when opening 2 tabs at the same time, so I guess race conditions are playing a role here).

Bugs

1. Missing audio track on P2P PeerConnection

As a user, this is what happens:

  • User A and User B join a conference almost at the same time.
  • On User B’s side, everything looks fine, but User A can’t hear User B.
  • If User C joins the conference, everything then works perfectly, User A can now hear User B.

More technically, this is what happens:

  • The conference starts with 2 users, using P2P sessions
  • On User B’s side, we notice that myConference.p2pJingleSession.peerconnection.getLocalTracks() shows only one track (for the audio). myConference.jvbJingleSession.peerconnection.getLocalTracks() however returns both the audio and video tracks.
  • When User C enters the conference, the call upgrades and starts using the jvbJingleSession. User B’s tracks are alright on that session so everything works fine now.

This is how we start the call:

  • Run init()
  • Create JitsiConnection
  • Add JitsiConnection listeners
  • Run connection.connect();
  • On events.connection.CONNECTION_ESTABLISHED
    • create myConference with connection.initJitsiConference()
    • add listeners to conference
    • run join() on conference
  • On events.conference.CONFERENCE_JOINED
    • run await myConference.addTrack(track); for the audio and video track
    • run await myConference.setReceiverVideoConstraint() and await conference.setSenderVideoConstraint()
  • We also use other listeners to update the UI but I don’t think that part is relevant.

We currently work around the issue by running the following lines before running myConference.addTrack().

          await myConference.removeTrack(localTracks[i]);
          await myConference.addTrack(localTracks[i]);

But we believe that this issue is due to some race conditions, so such a work around might break for other users with different computer setups, where timings can be different.

Also, that workaround created a second bug, which leads us to…

2. Missing video track on JVB PeerConnection

As a user, this is what happens:

  • User A and User B join a conference almost at the same time
  • On both side, everything looks fine
  • User C joins
  • User A and User C can’t see User B

More technically, this is what happens:

  • The conference starts with 2 users, using P2P sessions.
  • On User B side, we notice that myConference.jvbJingleSession.peerconnection.getLocalTracks() shows only one track Sender (for the audio). It’s not a problem yet since we’re using the p2pJingleSession for the moment, which has both audio and video track senders.
  • When User C enters the conference, the call upgrades and starts using the jvbJingleSession, but User B’s tracks are not updated, and the video track is missing.

We currently work around the issue by force re-adding the track on P2P_CHANGE when we find it’s missing on the active media session. Just adding the missing won’t work because myConference.rtc.getLocalTracks() already returns both tracks so the re-adding of an existing track will not be forced.

  const alreadyAddedTracks = myConference._getActiveMediaSession().peerconnection.getLocalTracks();
  for (let i = 0; i < localTracks.length; i++) {
    if (!alreadyAddedTracks.includes(localTracks[i])) {
      await myConference.removeTrack(localTracks[i]);
      await myConference.addTrack(localTracks[i]);
    }
  }

Questions

Are the bugs above actual bugs, or is it just us not using the API the right way?

Do you have any idea on how to fix them on the API consumer side, without having to force-re-add the tracks or hope that the race conditions won’t happen?

Is there a way to improve lib-jitsi-meet and handle the case? If so, do you have a clue about where I can start investigating in order to potentially submit a pull request?