Change In selectedParticipant Behavior

At CoScreen we have struggled for a while predicting the bridge’s behavior when dealing with selecting/pinning endpoints.

Our use case is that we have multiple users in an audio session, and we have one extremely high priority screen sharing session, that we want to be selected on a per user basis.

Using “selectParticipant” on JitsiConference used to work for us, preventing the bridge from removing the remote track for that particular user.

Suddenly we are seeing the behavior of the bridge continuously firing JitsiMeetJS.events.conference.LAST_N_ENDPOINTS_CHANGED over and over again removing and re-adding the selected participant’s track again and again, sometimes multiple times within the same second.

[2021-06-28 11:53:35.285] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:53:35.685] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:53:35.890] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:53:36.286] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:53:36.592] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:53:37.093] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:54:18.811] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:54:27.219] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:54:27.478] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:54:28.731] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:54:28.980] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:54:31.999] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:54:32.096] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:54:32.756] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:54:33.011] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:54:34.123] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:54:34.671] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:54:34.768] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:54:35.020] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:54:38.030] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:54:38.285] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:54:39.438] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:54:39.691] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:
[2021-06-28 11:54:39.840] [warn] [CaptureBackend] LastN Groups Update: L:, E:26009762
[2021-06-28 11:55:30.517] [warn] [CaptureBackend] LastN Groups Update: L:26009762, E:

Does anyone know of a configuration for lib-jitsi-meet that would meet our requirement above, that we could mark a video track and never allow it to exit the lastN group no matter what happens to the network?

Some updates here, the conference that is experiencing this strange LastN jumping in and out behavior is only sharing a single desktop track. There is no audio and no dominant speaker.

If we have two users streaming desktops, (two videos in the conference), it seems to strangely improve the above behavior of the track swapping in and out of the LastN Groups over and over again.

Any ideas would be greatly appreciated.

Adding some details, we do not observe the error with the following Jitsi version:

ii  jitsi-meet                       2.0.5765-1                         all          WebRTC JavaScript video conferences
ii  jitsi-meet-prosody               1.0.4900-1                         all          Prosody configuration for Jitsi Meet
rc  jitsi-meet-turnserver            1.0.4900-1                         all          Configures coturn to be used with Jitsi Meet
ii  jitsi-meet-web                   1.0.4900-1                         all          WebRTC JavaScript video conferences
ii  jitsi-meet-web-config            1.0.4900-1                         all          Configuration for web serving of Jitsi Meet
ii  jitsi-videobridge2               2.1-478-gc6da57bd-1                all          WebRTC compatible Selective Forwarding Unit (SFU)

Which is a few releases back.

We only observe on 8x8’s infra currently (will check meet.jit.si)

More information

The issue starts for us on the latest

ii  jitsi-meet                       2.0.5963-1                          all          WebRTC JavaScript video conferences
ii  jitsi-meet-prosody               1.0.5056-1                          all          Prosody configuration for Jitsi Meet
ii  jitsi-meet-turnserver            1.0.5056-1                          all          Configures coturn to be used with Jitsi Meet
ii  jitsi-meet-web                   1.0.5056-1                          all          WebRTC JavaScript video conferences
ii  jitsi-meet-web-config            1.0.5056-1                          all          Configuration for web serving of Jitsi Meet
rc  jitsi-videobridge                1126-1                              amd64        WebRTC compatible Selective Forwarding Unit (SFU)
ii  jitsi-videobridge2               2.1-508-gb24f756c-1                 all          WebRTC compatible Selective Forwarding Unit (SFU)

but

ii  jitsi-meet                       2.0.5870-1                          all          WebRTC JavaScript video conferences
ii  jitsi-meet-prosody               1.0.4985-1                          all          Prosody configuration for Jitsi Meet
rc  jitsi-meet-turnserver            1.0.5056-1                          all          Configures coturn to be used with Jitsi Meet
ii  jitsi-meet-web                   1.0.4985-1                          all          WebRTC JavaScript video conferences
ii  jitsi-meet-web-config            1.0.4985-1                          all          Configuration for web serving of Jitsi Meet
rc  jitsi-videobridge                1126-1                              amd64        WebRTC compatible Selective Forwarding Unit (SFU)
ii  jitsi-videobridge2               2.1-492-g5edaf7dd-1                 all          WebRTC compatible Selective Forwarding Unit (SFU)

Does not exhibit any issues.

We have tracked the issue to the latest “stable” release, 2.0.5963, stable/jitsi-meet_5963.

The issue isn’t experienced by the previous stable release: 2.0.5870, stable/jitsi-meet_5870

Looking closer at the source for the latest bridge, we think it probably is something to do with allowOversendOnStage.

What change exactly caused it is hard to discern, this change in BandwidthAllocator.java seems more permissive (Although we are a ‘desktop’ type stream, so I expect we were meeting the previous criteria, and should meet the new criteria):

jvb/src/main/java/org/jitsi/videobridge/cc/allocation/BandwidthAllocator.java:315
-                if (i == 0
-                        && sourceBitrateAllocation.isOnStage()
-                        && BitrateControllerConfig.allowOversendOnStage()
-                        && sourceBitrateAllocation.endpoint.getVideoType() == VideoType.DESKTOP)
+                if (i == 0 && sourceBitrateAllocation.isOnStage() && BitrateControllerConfig.allowOversendOnStage())
                 {

It could be the deployment has a different value for the onstage video suspend configuration parameter in the latest release?

jvb/src/main/kotlin/org/jitsi/videobridge/cc/config/BitrateControllerConfig.kt:82
        /**
         * Whether or not we are allowed to oversend (exceed available bandwidth) for the video of the on-stage
         * participant.
         */
        private val allowOversendOnStage: Boolean by config {
            "org.jitsi.videobridge.ENABLE_ONSTAGE_VIDEO_SUSPEND".from(JitsiConfig.legacyConfig).transformedBy { !it }
            "videobridge.cc.enable-onstage-video-suspend".from(JitsiConfig.newConfig).transformedBy { !it }
            "videobridge.cc.allow-oversend-onstage".from(JitsiConfig.newConfig)
        }

Is this parameter set to true for the current videobridge deployment?

jvb/src/main/java/org/jitsi/videobridge/cc/allocation/BandwidthAllocator.java:319
                if (i == 0 && sourceBitrateAllocation.isOnStage() && BitrateControllerConfig.allowOversendOnStage())
                {
                    oversending |= sourceBitrateAllocation.tryLowestLayer();
                }

It looks like ‘getOnStageEndpoints’ should return our ‘selectedEndpoints’:

jvb/src/main/java/org/jitsi/videobridge/cc/allocation/BandwidthAllocator.java:263
        // On-stage participants are considered selected (with higher prio).
        List<String> selectedEndpoints = new ArrayList<>(allocationSettings.getOnStageEndpoints());
        allocationSettings.getSelectedEndpoints().forEach(selectedEndpoint ->
        {
            if (!selectedEndpoints.contains(selectedEndpoint))
            {
                selectedEndpoints.add(selectedEndpoint);
            }
        });
        return selectedEndpoints;

On the client-side we select only one participant at the time (the currently focussed window) using:

lib/lib-jitsi-meet/modules/qualitycontrol/ReceiveVideoController.js:220
selectEndpoints(ids) {
        this._selectedEndpoints = ids;
        if (this._receiverVideoConstraints) {
            const remoteEndpointIds = ids.filter(id => id !== this._conference.myUserId());
            // Filter out the local endpointId from the list of selected endpoints.
            remoteEndpointIds.length && this._receiverVideoConstraints.updateSelectedEndpoints(remoteEndpointIds);
            this._rtc.setNewReceiverVideoConstraints(this._receiverVideoConstraints.constraints);
            return;
        }
        this._rtc.selectEndpoints(ids);
    }