Users disconnected (not ports or lack of turn related)

Dear community and jitsi developers,

Thank you so much for all your hard work and awesome tool. We have a system that are about to release but are having a few issues caused by conferences not being stable.

This is the story:

  1. Users connect correctly to a video bridge that is using port 32102 for the RTP communication:

  1. After 30 minutes, another user is connected to the same conference

  1. And a few minutes later, another one is connected also.

  1. During the time of the conference, logs like these can be seen:

  1. But then, at some point an error like this is seen:

  1. And then, the logs begin showing errors like this:

  1. With now more and more errors like this:

  1. To finally:

However, now this leads to this:

And the user that got disconnected cannot be heard by other people in the conference (or sometimes half of them can hear him but the other half cannot).

This is the log of the yellow user in chrome:

Logger.js:154 2020-09-03T13:01:00.279Z [modules/xmpp/JingleSessionPC.js] ICE connection state:  failed
Logger.js:154 2020-09-03T13:01:00.298Z [modules/xmpp/SdpConsistency.js] <o.makeVideoPrimarySsrcsConsistent>:  TPC[1,p2p:false] sdp-consistency replacing new ssrc657673585 with cached 657673585
Logger.js:154 2020-09-03T13:01:00.313Z [modules/xmpp/JingleSessionPC.js] addRemoteStream - OK, SDPs:  s s
Logger.js:154 2020-09-03T13:01:00.313Z [modules/xmpp/JingleSessionPC.js] <R.notifyMySSRCUpdate>:  removal not necessary
Logger.js:154 2020-09-03T13:01:00.313Z [modules/xmpp/JingleSessionPC.js] <R.notifyMySSRCUpdate>:  addition not necessary
Logger.js:154 2020-09-03T13:01:00.868Z [modules/xmpp/strophe.jingle.js] <g.onJingle>:  on jingle transport-replace from 886@muc.my-website.org/focus <iq xmlns=​"jabber:​client" to=​"nscqswmg0cevvqhf@my-website.org/​sGBUTyoj" type=​"set" from=​"886@muc.my-website.org/​focus" id=​"bnNjcXN3bWcwY2V2dnFoZkBzdGFnaW5nLXN0dWRpby5zdHJlYW0uancub3JnL3NHQlVUeW9qAEpvTUxOLTIxMzEyOQCk2H7JLzOSfSTgMyxklXM5">​…​</iq>​
Logger.js:154 2020-09-03T13:01:00.868Z [modules/xmpp/strophe.jingle.js] <g.onJingle>:  (TIME) Start transport replace:	 493982.7799999621
Logger.js:154 2020-09-03T13:01:00.874Z [modules/xmpp/strophe.jingle.js] <g.onJingle>:  on jingle source-add from 886@muc.my-website.org/focus <iq xmlns=​"jabber:​client" to=​"nscqswmg0cevvqhf@my-website.org/​sGBUTyoj" type=​"set" from=​"886@muc.my-website.org/​focus" id=​"bnNjcXN3bWcwY2V2dnFoZkBzdGFnaW5nLXN0dWRpby5zdHJlYW0uancub3JnL3NHQlVUeW9qAEpvTUxOLTIxMzE1MgCk2H7JLzOSfSTgMyxklXM5">​…​</iq>​
index.js:146 Halt: There are no SSRC groups in the remote description.
Logger.js:154 2020-09-03T13:01:00.906Z [modules/RTC/BridgeChannel.js] <RTCDataChannel.e.onerror>:  Channel error: undefined
o @ Logger.js:154
Logger.js:154 2020-09-03T13:01:00.906Z [modules/RTC/BridgeChannel.js] <RTCDataChannel.e.onclose>:  Channel closed by client
Logger.js:154 2020-09-03T13:01:00.923Z [modules/xmpp/JingleSessionPC.js] <A.peerconnection.oniceconnectionstatechange>:  (TIME) ICE disconnected P2P? false:	 494037.4450000236
Logger.js:154 2020-09-03T13:01:01.752Z [modules/xmpp/JingleSessionPC.js] <R.sendIceCandidate>:  sendIceCandidate: last candidate.
Logger.js:154 2020-09-03T13:01:01.754Z [modules/xmpp/JingleSessionPC.js] <R.sendIceCandidate>:  sendIceCandidate: last candidate.
Logger.js:154 2020-09-03T13:01:01.976Z [modules/xmpp/JingleSessionPC.js] <A.peerconnection.oniceconnectionstatechange>:  (TIME) ICE connected P2P? false:	 495090.3099999996
Logger.js:154 2020-09-03T13:01:01.995Z [modules/xmpp/SdpConsistency.js] <o.makeVideoPrimarySsrcsConsistent>:  TPC[1,p2p:false] sdp-consistency replacing new ssrc657673585 with cached 657673585
Logger.js:154 2020-09-03T13:01:02.040Z [modules/xmpp/JingleSessionPC.js] <R.sendIceCandidate>:  sendIceCandidate: last candidate.
Logger.js:154 2020-09-03T13:01:02.054Z [modules/xmpp/JingleSessionPC.js] <R.sendIceCandidate>:  sendIceCandidate: last candidate.
Logger.js:154 2020-09-03T13:01:02.109Z [modules/xmpp/JingleSessionPC.js] <R.notifyMySSRCUpdate>:  removal not necessary
Logger.js:154 2020-09-03T13:01:02.110Z [modules/xmpp/JingleSessionPC.js] <R.notifyMySSRCUpdate>:  addition not necessary
Logger.js:154 2020-09-03T13:01:02.113Z [modules/xmpp/SDPUtil.js] <Object.candidateToJingle>:  not translating "network-id" = "2"
Logger.js:154 2020-09-03T13:01:02.113Z [modules/xmpp/SDPUtil.js] <Object.candidateToJingle>:  not translating "network-id" = "1"
Logger.js:154 2020-09-03T13:01:02.114Z [modules/xmpp/SDPUtil.js] <Object.candidateToJingle>:  not translating "network-id" = "2"
Logger.js:154 2020-09-03T13:01:02.114Z [modules/xmpp/SDPUtil.js] <Object.candidateToJingle>:  not translating "network-id" = "1"
Logger.js:154 2020-09-03T13:01:02.114Z [modules/xmpp/SDPUtil.js] <Object.candidateToJingle>:  not translating "network-id" = "1"
Logger.js:154 2020-09-03T13:01:02.114Z [modules/xmpp/JingleSessionPC.js] <R.sendTransportAccept>:  Sending transport-accept:  <iq to=​"886@muc.my-website.org/​focus" type=​"set" xmlns=​"jabber:​client" id=​"2ea2a6ba-30af-41c4-94b1-bd12f8ae5963:​sendIQ">​…​</iq>​
Logger.js:154 2020-09-03T13:01:02.128Z [modules/RTC/BridgeChannel.js] <RTCDataChannel.e.onopen>:  datachannel channel opened
Logger.js:154 2020-09-03T13:01:02.129Z [JitsiConferenceEventManager.js] <a.<anonymous>>:  (TIME) data.channel.opened:	 495243.3149999706
Logger.js:154 2020-09-03T13:01:02.130Z [modules/RTC/BridgeChannel.js] <l.sendPinnedEndpointMessage>:  sending pinned changed notification to the bridge for endpoint  null
Logger.js:154 2020-09-03T13:01:02.130Z [modules/RTC/BridgeChannel.js] <l.sendSelectedEndpointsMessage>:  sending selected changed notification to the bridge for endpoints Array(8)
Logger.js:154 2020-09-03T13:01:02.130Z [modules/RTC/BridgeChannel.js] <l.sendReceiverVideoConstraintMessage>:  sending a ReceiverVideoConstraint message with a maxFrameHeight of 360 pixels
Logger.js:154 2020-09-03T13:01:02.131Z [modules/RTC/BridgeChannel.js] <l.sendSetLastNMessage>:  Channel lastN set to: 8
Logger.js:154 2020-09-03T13:01:02.531Z [modules/xmpp/strophe.jingle.js] (TIME) Transport replace success:	 495645.9449999966
Logger.js:154 2020-09-03T13:01:19.545Z [modules/RTC/BridgeChannel.js] <RTCDataChannel.e.onerror>:  Channel error: undefined
o @ Logger.js:154
Logger.js:154 2020-09-03T13:01:19.545Z [modules/RTC/BridgeChannel.js] <RTCDataChannel.e.onclose>:  Channel closed by server
Logger.js:154 2020-09-03T13:01:24.808Z [modules/RTC/BridgeChannel.js] <l._send>:  Bridge Channel send: no opened channel.
o @ Logger.js:154
Logger.js:154 2020-09-03T13:01:24.809Z [JitsiConference.js] <u.sendMessage>:  Failed to send E2E ping request or response. undefined
o @ Logger.js:154
Logger.js:154 2020-09-03T13:01:25.567Z [modules/RTC/BridgeChannel.js] <l._send>:  Bridge Channel send: no opened channel.

We’ve found this pull request https://github.com/jitsi/ice4j/pull/208 however we are already using the latest version of jicofo, prosody and jvb.

This issue in the community has our configuration: FIXED: Healthcheck in Video Bridge not Working anymore

Thank you so much for all the help you can provide.

EDIT:

We are using this in a kubernetes cluster.

@damencho I’m sorry to ping you directly. But, do you have any idea on what can cause this kind of error? We are still seeing it when the users are connected via turn.

No idea, normally ice failures are network issues. So maybe something cut the network between clints and jvb …

1 Like

This happens inside a k8s cluster. It is 4 AWS EC2 On-Demand boxes. The newest has been alive for 10 hours.

Is there anyway we can enable more logging in the jvb? In that way we could try and see if we can find anything. But as it is right now, that is the only thing we can see. The logs in the turn server are also clean (this errors happen only between jvb and turn, users that connect directly to the bridge rarely see errors).

Oh, and btw. Thank you so much for your quick reply!

1 Like