Participants randomly disconnects from meetings - 2.0.8044

keda82 · May 9, 2023, 6:27am

Understood, are there any prerequisites for the websocketKeepAliveUrl that we need to consider?

we are still on 2.0.8044

NOTE: We do not get this behavior in our test enviornment with 8044, so I dont think it should be the issue

damencho · May 9, 2023, 11:23am

Now you got me confused, what do you mean?

keda82 · May 9, 2023, 7:58pm

I get why you are confused , we suspect it has to do with the difference in what is in front of test vs. prod (fw, reversproxy etc)

Based on this we have been trying to finetune the websocket config options in the reverseproxy. After doing so we can now reproduce the issue at will(not initial plan…) by doing the following:

Client 1 - Edge/Chromium with jwt auth
Client 2 - Edge/Chromium or FF with guest access

As soon as Client 2 joins the room we get the following errors:

CLIENT 1

023-05-09T19:27:33.375Z [JitsiMeetJS.ts] <Object.getGlobalOnErrorHandler>: UnhandledError: Jingle error: {“session”:“JingleSessionPC[session=JVB,initiator=false,sid=3rk9a1sd281f3]”} Script: null Line: null Column: null StackTrace: Error: Jingle error: {“session”:“JingleSessionPC[session=JVB,initiator=false,sid=3rk9a1sd281f3]”}
at https://meetingservice.com/libs/lib-jitsi-meet.min.js?v=6776:2:238302)
at Object.error (https://meetingservice.com/libs/lib-jitsi-meet.min.js?v=6776:2:758494)
at w.Websocket._onClose (https://meetingservice.com/libs/lib-jitsi-meet.min.js?v=6776:2:797306)
r @ Logger.js:154
getGlobalOnEr
Followed by
023-05-09T19:39:12.919Z [modules/xmpp/strophe.util.js] <mt.Strophe.log>: Strophe: Websocket closed unexpectedly
r @ Logger.js:154
mt.Strophe.lo
Followed by
s/xmpp/xmpp.js] <ua.connectionHandler>: XMPP connection dropped!
r @ Logger.js:154
connectionHan

CLIENT 2

2023-05-09T19:54:20.359Z [JitsiMeetJS.ts] : UnhandledError: Strophe: Websocket closed unexpectedly Script: null Line: null Column: null StackTrace: 6885/ua/mt.Strophe.log@https://meetingservice.com/libs/lib-jitsi-meet.min.js?v=6776:2:238302

If we try to reconnect 2-3 times on client 2 we can access the meeting with client 2. I fully understand the issue in on our side, but if you can shed some light on where to look next it would be really helpful. Now that we can reproduce the issue at will maybe it will be easier to find the root cause in our setup. E.g what responses/disconnects should be be looking for in the jitsi websocket traces in the reversproxy?

Setting websocketKeepAliveUrl did not change the behavior.

damencho · May 9, 2023, 8:44pm

When you repro, can you upload logs from client 1 and client 2?

Oh jwt auth and “guest access” I suppose you are talking here about anonymousdomain from config.js. Both things were never tested together and never planned to be used together … so strange things can happen… not that your problem comes from there.

keda82 · May 9, 2023, 8:52pm

Sure, will gather the logs tomorrow!

Regarding jwt and guests, what is the recommended approach? I do not believe we are the only ones that wants to have a scenario were moderators authenticate with jwt and guests can access the meeting directly without authentication.

damencho · May 9, 2023, 8:57pm

For that, you use allow_empty_token = true; with jwt.

keda82 · May 10, 2023, 10:11am

we have most likely found the root cause. It appears the combination of reverseproxy security features and websocket compression in jvb might be causing this. The websocket calls are sometimes being made “corrupt” in the reverseproxy.

Is there a docker value available for disabling compression so we can give it a try?

websockets {
enabled=false
server-id=“default-id”
# Whether to use WebSocket compression (permessage-deflate)
compression = true

damencho · May 10, 2023, 10:50am

Nope, there is not

github.com

jitsi/docker-jitsi-meet/blob/809f63c2d8fc6ab723997d1b5b8d777eb112e402/jvb/rootfs/defaults/jvb.conf#L83


      
              }
          }
          rest {
              shutdown {
                  enabled = {{ $SHUTDOWN_REST_ENABLED }}
              }
          }
          stats {
              enabled = true
          }
          websockets {
              enabled = {{ $ENABLE_COLIBRI_WEBSOCKET }}
              domain = "{{ $WS_DOMAIN }}"
              tls = true
              server-id = "{{ $WS_SERVER_ID }}"
          }
          http-servers {
              private {
                host = 0.0.0.0
              }
              public {

You can disable them with ENABLE_COLIBRI_WEBSOCKET and enable sctp with ENABLE_SCTP.

keda82 · May 26, 2023, 6:10pm

So unfortunately we keep having issues with disconnects. It can occur after 3minutes or 1h so there is no clear pattern

Plan B is to try is to disable websocket for colibri but we have been told this comes at a cost of user experience?

Something we noticed popping up in the webconsole:

Error #1
Manifest: Line: 1, column: 1, Syntax error.

Error #2
MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 conference.userLeft listeners added. Use emitter.setMaxListeners() to increase limit
and

Error #3

[modules/statistics/AudioOutputProblemDetector.js] A potential problem is detected with the audio output for participant 53d9c0c9, local audio levels: [**null,null**], remote audio levels: **undefined**
s @ CallStats.js:470

Could any of these have anything to do with this?