Hello guys,
Recently, I have been facing JVB randomly loosing connectivity with Jicofo. Things went fine for several months and then recently, I am seeing below errors on Jitsi and JVB end and kicks off participants out from the meeting:
JVB version:
ii jitsi-videobridge2 2.1-304-g8488f77d-1 all WebRTC compatible Selective Forwarding Unit (SFU)
Jicofo error logs:
Jicofo logs
Jicofo 2021-04-08 14:35:26.914 SEVERE: [35549] org.jitsi.jicofo.AbstractChannelAllocator.log() jvbbrewery@internal.auth.domain.com/prod2-jvb1 - failed to allocate channels, will consider the bridge faulty: XMPP error:
org.jitsi.protocol.xmpp.colibri.exception.ColibriException: XMPP error:
at org.jitsi.impl.protocol.xmpp.colibri.ColibriConferenceImpl.maybeThrowOperationFailed(ColibriConferenceImpl.java:378)
at org.jitsi.impl.protocol.xmpp.colibri.ColibriConferenceImpl.createColibriChannels(ColibriConferenceImpl.java:282)
at org.jitsi.protocol.xmpp.colibri.ColibriConference.createColibriChannels(ColibriConference.java:112)
at org.jitsi.jicofo.ParticipantChannelAllocator.doAllocateChannels(ParticipantChannelAllocator.java:111)
at org.jitsi.jicofo.AbstractChannelAllocator.allocateChannels(AbstractChannelAllocator.java:271)
at org.jitsi.jicofo.AbstractChannelAllocator.doRun(AbstractChannelAllocator.java:190)
at org.jitsi.jicofo.AbstractChannelAllocator.run(AbstractChannelAllocator.java:150)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Jicofo 2021-04-08 14:35:26.914 SEVERE: [35549] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() One of our bridges failed: jvbbrewery@internal.auth.domain.com/prod2-jvb1
Jicofo 2021-04-08 14:35:26.914 INFO: [35549] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Region info, conference=44099 octo_enabled= false: [[null, null]]
Jicofo 2021-04-08 14:35:26.914 WARNING: [35549] org.jitsi.jicofo.AbstractParticipant.log() Canceling ParticipantChannelAllocator[BridgeSession[id=44099_63c420, bridge=Bridge[jid=jvbbrewery@internal.auth.domain.com/prod2-jvb1, relayId=null, region=null, stress=0.03]]@681790900, Participant[5a836218-7624-11eb-8b8f-02fca26ac9c6@conference.domain.com/633eb26a]@1395261742]@1506962014
Jicofo 2021-04-08 14:35:26.915 INFO: [35549] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Region info, conference=44099 octo_enabled= false: [[null]]
Jicofo 2021-04-08 14:35:26.915 WARNING: [35549] org.jitsi.jicofo.bridge.BridgeSelectionStrategy.log() Failed to select initial bridge for participantRegion=null
Jicofo 2021-04-08 14:35:26.915 SEVERE: [35549] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Can not invite participant β no bridge available.
Jicofo 2021-04-08 14:35:26.915 WARNING: [35549] org.jitsi.jicofo.bridge.BridgeSelectionStrategy.log() Failed to select initial bridge for participantRegion=null
Jicofo 2021-04-08 14:35:26.915 SEVERE: [35549] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Can not invite participant β no bridge available.
Jicofo 2021-04-08 14:35:27.096 INFO: [31] org.jitsi.jicofo.ChatRoomRoleAndPresence.log() Chat room event ChatRoomMemberPresenceChangeEvent[type=MemberLeft sourceRoom=org.jitsi.impl.protocol.xmpp.ChatRoomImpl@18a299a9 member=ChatMember[5a836218-7624-11eb-8b8f-02fca26ac9c6@conference.domain.com/0086944a, jid: 0086944a-ce02-4f16-a24c-aeca514b2246@domain.com/5rd36LQb]@589788786]
JVB error logs:
JVB logs
2021-04-08 14:35:26.628 INFO: [117010] [confId=deb287aafbfcbcc9 gid=278681 stats_id=Augustine-Jjh conf_name=5a836218-7624-11eb-8b8f-02fca26ac9c6 ufrag=60m511f2pc8foh epId=633eb26a local_ufrag=60m511f2pc8foh] Agent.checkListStatesUpdated#1949: ICE state is FAILED
2021-04-08 14:35:26.914 INFO: [116993] [confId=deb287aafbfcbcc9 epId=633eb26a gid=278681 stats_id=Augustine-Jjh conf_name=5a836218-7624-11eb-8b8f-02fca26ac9c6] AbstractEndpoint.expire#233: Expiring.
2021-04-08 14:35:26.914 INFO: [116993] [confId=deb287aafbfcbcc9 epId=633eb26a gid=278681 stats_id=Augustine-Jjh conf_name=5a836218-7624-11eb-8b8f-02fca26ac9c6] Transceiver.teardown#319: Tearing down
2021-04-08 14:35:26.914 INFO: [116993] [confId=deb287aafbfcbcc9 epId=633eb26a gid=278681 stats_id=Augustine-Jjh conf_name=5a836218-7624-11eb-8b8f-02fca26ac9c6] RtpReceiverImpl.tearDown#299: Tearing down
2021-04-08 14:35:26.914 INFO: [116993] [confId=deb287aafbfcbcc9 epId=633eb26a gid=278681 stats_id=Augustine-Jjh conf_name=5a836218-7624-11eb-8b8f-02fca26ac9c6] RtpSenderImpl.tearDown#271: Tearing down
2021-04-08 14:35:26.914 SEVERE: [117036] XmppCommon.handleIQRequest#177: Exception handling IQ request
I checked the ports are open and things are working fine for several months. Also, I have done the NAT configuration on JVB end.
Any idea what might be going wrong? Thanks.