Jitsi on Kubernetes: Address discovery through STUN failed

JohnLukeP · March 21, 2022, 2:56pm

Hi all,

I’m trying to deploy the helm chart provided in the jitsi-contrib section on github on a kubernetes cluster managed by Rancher.
Whether I deploy it using helm (modifying values.yaml with the proper publicURL) o via Rancher interface, adding the Jitsi helm v3 catalog and launching it as an app (providing publicURL as parameter) the JVB pod fails to start. On the pod’sl logs, I can see:

JVB 2022-03-21 15:52:35.722 SEVERE: [23] HealthChecker.run#175: Health check failed in PT0.00013S:
Mon, Mar 21 2022 3:52:35 pm java.lang.Exception: Address discovery through STUN failed
Mon, Mar 21 2022 3:52:35 pm at org.jitsi.videobridge.health.JvbHealthChecker.check(JvbHealthChecker.kt:39)
Mon, Mar 21 2022 3:52:35 pm at org.jitsi.videobridge.health.JvbHealthChecker.access$check(JvbHealthChecker.kt:25)
Mon, Mar 21 2022 3:52:35 pm at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
Mon, Mar 21 2022 3:52:35 pm at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
Mon, Mar 21 2022 3:52:35 pm at org.jitsi.health.HealthChecker.run(HealthChecker.kt:145)
Mon, Mar 21 2022 3:52:35 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor.run(RecurringRunnableExecutor.java:216)
Mon, Mar 21 2022 3:52:35 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor.runInThread(RecurringRunnableExecutor.java:292)
Mon, Mar 21 2022 3:52:35 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor.access$000(RecurringRunnableExecutor.java:36)
Mon, Mar 21 2022 3:52:35 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor$1.run(RecurringRunnableExecutor.java:328)
Mon, Mar 21 2022 3:52:45 pm JVB 2022-03-21 15:52:45.723 SEVERE: [23] HealthChecker.run#175: Health check failed in PT0.00008S:
Mon, Mar 21 2022 3:52:45 pm java.lang.Exception: Address discovery through STUN failed
Mon, Mar 21 2022 3:52:45 pm at org.jitsi.videobridge.health.JvbHealthChecker.check(JvbHealthChecker.kt:39)
Mon, Mar 21 2022 3:52:45 pm at org.jitsi.videobridge.health.JvbHealthChecker.access$check(JvbHealthChecker.kt:25)
Mon, Mar 21 2022 3:52:45 pm at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
Mon, Mar 21 2022 3:52:45 pm at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
Mon, Mar 21 2022 3:52:45 pm at org.jitsi.health.HealthChecker.run(HealthChecker.kt:145)
Mon, Mar 21 2022 3:52:45 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor.run(RecurringRunnableExecutor.java:216)
Mon, Mar 21 2022 3:52:45 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor.runInThread(RecurringRunnableExecutor.java:292)
Mon, Mar 21 2022 3:52:45 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor.access$000(RecurringRunnableExecutor.java:36)
Mon, Mar 21 2022 3:52:45 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor$1.run(RecurringRunnableExecutor.java:328)
Mon, Mar 21 2022 3:52:50 pm [cont-finish.d] executing container finish scripts...
Mon, Mar 21 2022 3:52:50 pm [cont-finish.d] done.
Mon, Mar 21 2022 3:52:50 pm [s6-finish] waiting for services.
Mon, Mar 21 2022 3:52:51 pm [s6-finish] sending all processes the TERM signal.
Mon, Mar 21 2022 3:52:54 pm [s6-finish] sending all processes the KILL signal and exiting.

Apart from setting the public url, which other parameter have to be setup to properly make it work?
The other pods seems to start fine:

JohnLukeP · April 6, 2022, 7:49am

Hi, could somebody help me?

Thanks

Bastien_Dr · April 6, 2022, 8:35am

Hi,

In JVB_STUN_SERVERS env variable do you keep default values ? Did you try with the Jitsi provide one ?

Your k8s cluster can reach Internet without restriction on port and domain ?

Best

JohnLukeP · April 6, 2022, 9:03am

Hi,

I’m using the helm chart as provided, launching it from Rancher. The only variable I did set was the public URL, so everything else has default values.

As per network reachability: if I try getting inside the JVB container and do a “wget www.google.it” , html webpage is correctly downloaded.
Also doing the same from the kubernetes node (the VM) works fine.

As you can see I’m only passing publicURL. What extra settings shoudl be passed to the deployment?

Bastien_Dr · April 6, 2022, 9:05am

Try to put

JVB_STUN_SERVERS=meet-jit-si-turnrelay.jitsi.net:443

To confirm it’s reachable

nc -v meet-jit-si-turnrelay.jitsi.net 443

JohnLukeP · April 6, 2022, 9:33am

Hi, I’ve tried to pass it as parameter but with no avail. I tried getting into the pod shell before the CrashLoopback; the nc command is not available. The wget results in:

root@conference-rios-jitsi-meet-jvb-55fcc5d56f-xbknk:/# wget meet-jit-si-turnrelay.jitsi.net:443
--2022-04-06 11:30:00--  http://meet-jit-si-turnrelay.jitsi.net:443/
Resolving meet-jit-si-turnrelay.jitsi.net (meet-jit-si-turnrelay.jitsi.net)... 193.123.56.236
Connecting to meet-jit-si-turnrelay.jitsi.net (meet-jit-si-turnrelay.jitsi.net)|193.123.56.236|:443... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.

--2022-04-06 11:30:01--  (try: 2)  http://meet-jit-si-turnrelay.jitsi.net:443/
Connecting to meet-jit-si-turnrelay.jitsi.net (meet-jit-si-turnrelay.jitsi.net)|193.123.56.236|:443... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.

However, if I start another pod with Alpine chart on the same namespace of the same cluster, it connects succesfully:

/ # nc -v meet-jit-si-turnrelay.jitsi.net 443
meet-jit-si-turnrelay.jitsi.net (158.101.192.246:443) open

here are the pod’s full logs:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-set-timezone: executing...
[cont-init.d] 01-set-timezone: exited 0.
[cont-init.d] 10-config: executing...
[cont-init.d] 10-config: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
JVB 2022-04-06 12:05:24.515 INFO: [1] JitsiConfig.<clinit>#47: Initialized newConfig: merge of /config/jvb.conf: 1,application.conf @ jar:file:/usr/share/jitsi-videobridge/jitsi-videobridge.jar!/application.conf: 1,system properties,reference.conf @ jar:file:/usr/share/jitsi-videobridge/jitsi-videobridge.jar!/reference.conf: 1,reference.conf @ jar:file:/usr/share/jitsi-videobridge/lib/ice4j-3.0-52-ga9ba80e.jar!/reference.conf: 1,reference.conf @ jar:file:/usr/share/jitsi-videobridge/lib/jitsi-media-transform-1.0-281-gafb453e.jar!/reference.conf: 1
JVB 2022-04-06 12:05:24.534 INFO: [1] ReadOnlyConfigurationService.reloadConfiguration#51: loading config file at path /config/sip-communicator.properties
JVB 2022-04-06 12:05:24.535 INFO: [1] ReadOnlyConfigurationService.reloadConfiguration#56: Error loading config file: java.io.FileNotFoundException: /config/sip-communicator.properties (No such file or directory)
JVB 2022-04-06 12:05:24.536 INFO: [1] JitsiConfig.<clinit>#68: Initialized legacyConfig: sip communicator props (no description provided)
JVB 2022-04-06 12:05:24.536 INFO: [1] JitsiConfig$Companion.reloadNewConfig#94: Reloading the Typesafe config source (previously reloaded 0 times).
JVB 2022-04-06 12:05:24.553 INFO: [1] MainKt.main#90: Starting jitsi-videobridge version 2.1.617-ga8b39c3f
JVB 2022-04-06 12:05:24.878 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.initialize: Using AwsCandidateHarvester.
JVB 2022-04-06 12:05:24.904 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.createStunHarvesters: Using 193.123.34.138:443/udp for StunMappingCandidateHarvester (localAddress=10.42.0.189:0/udp).
JVB 2022-04-06 12:05:25.112 INFO: [19] [hostname=jitsi-conference-prosody.mydomain-it.svc id=shard] MucClient.initializeConnectAndJoin#273: Initializing a new MucClient for [ org.jitsi.xmpp.mucclient.MucClientConfiguration id=shard domain=auth.meet.jitsi hostname=jitsi-conference-prosody.mydomain-it.svc port=null username=jvb mucs=[jvbbrewery@internal-muc.meet.jitsi] mucNickname=jitsi-conference-jitsi-meet-jvb-8446bf5584-2ljgs disableCertificateVerification=true]
JVB 2022-04-06 12:05:25.168 WARNING: [19] MucClient.createXMPPTCPConnectionConfiguration#116: Disabling certificate verification!
JVB 2022-04-06 12:05:25.169 INFO: [1] LastNReducer.<init>#65: LastNReducer with reductionScale: 0.75 recoverScale: 1.25 impactTime: PT1M minLastN: 1 maxEnforcedLastN: 40
JVB 2022-04-06 12:05:25.171 INFO: [1] TaskPools.<clinit>#81: TaskPools detected 1 processors, creating the CPU pool with that many threads
JVB 2022-04-06 12:05:25.204 WARNING: [1] UlimitCheck.printUlimits#111: Running with open files limit 1048576 (hard 1048576), thread limit null (hard null). These values are too low and they will limit the number of participants that the bridge can serve simultaneously.
JVB 2022-04-06 12:05:25.206 INFO: [1] VideobridgeExpireThread.start#88: Starting with 60 second interval.
JVB 2022-04-06 12:05:25.225 INFO: [1] HealthChecker.start#118: Started with interval=10000, timeout=PT30S, maxDuration=PT3S, stickyFailures=false.
JVB 2022-04-06 12:05:25.234 INFO: [1] MainKt.main#119: Not starting CallstatsService, disabled in configuration.
JVB 2022-04-06 12:05:25.236 INFO: [1] MainKt.main#128: Starting public http server
JVB 2022-04-06 12:05:25.249 INFO: [19] [hostname=jitsi-conference-prosody.mydomain-it.svc id=shard] MucClient.initializeConnectAndJoin#332: Dispatching a thread to connect and login.
JVB 2022-04-06 12:05:25.273 INFO: [1] ColibriWebSocketService.<init>#42: WebSockets are not enabled
JVB 2022-04-06 12:05:25.289 INFO: [1] org.eclipse.jetty.util.log.Log.initialized: Logging initialized @1144ms to org.eclipse.jetty.util.log.JavaUtilLog
JVB 2022-04-06 12:05:25.364 INFO: [1] ColibriWebSocketService.registerServlet#86: Disabled, not registering servlet
JVB 2022-04-06 12:05:25.366 INFO: [1] org.eclipse.jetty.server.Server.doStart: jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 11.0.14+9-post-Debian-1deb11u1
JVB 2022-04-06 12:05:25.397 INFO: [1] org.eclipse.jetty.server.handler.ContextHandler.doStart: Started o.e.j.s.ServletContextHandler@64040287{/,null,AVAILABLE}
JVB 2022-04-06 12:05:25.406 INFO: [1] org.eclipse.jetty.server.AbstractConnector.doStart: Started ServerConnector@169da7f2{HTTP/1.1, (http/1.1)}{0.0.0.0:9090}
JVB 2022-04-06 12:05:25.406 INFO: [1] org.eclipse.jetty.server.Server.doStart: Started @1261ms
JVB 2022-04-06 12:05:25.407 INFO: [1] MainKt.main#146: Starting private http server
JVB 2022-04-06 12:05:25.453 INFO: [1] org.eclipse.jetty.server.Server.doStart: jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 11.0.14+9-post-Debian-1deb11u1
JVB 2022-04-06 12:05:25.669 WARNING: [1] org.glassfish.jersey.server.wadl.WadlFeature.configure: JAXBContext implementation could not be found. WADL feature is disabled.
JVB 2022-04-06 12:05:25.750 WARNING: [1] org.glassfish.jersey.internal.inject.Providers.checkProviderRuntime: A provider org.jitsi.rest.Health registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.jitsi.rest.Health will be ignored.
JVB 2022-04-06 12:05:25.750 WARNING: [1] org.glassfish.jersey.internal.inject.Providers.checkProviderRuntime: A provider org.jitsi.rest.Version registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.jitsi.rest.Version will be ignored.
JVB 2022-04-06 12:05:25.966 INFO: [1] org.eclipse.jetty.server.handler.ContextHandler.doStart: Started o.e.j.s.ServletContextHandler@33a55bd8{/,null,AVAILABLE}
JVB 2022-04-06 12:05:25.967 INFO: [1] org.eclipse.jetty.server.AbstractConnector.doStart: Started ServerConnector@38499e48{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
JVB 2022-04-06 12:05:25.967 INFO: [1] org.eclipse.jetty.server.Server.doStart: Started @1822ms
JVB 2022-04-06 12:05:26.289 WARNING: [19] [hostname=jitsi-conference-prosody.mydomain-it.svc id=shard] MucClient.lambda$getConnectAndLoginCallable$7#622: [MucClient id=shard hostname=jitsi-conference-prosody.mydomain-it.svc] error connecting
org.jivesoftware.smack.SmackException$EndpointConnectionException: The following addresses failed: 'RFC 6120 A/AAAA Endpoint + [jitsi-conference-prosody.mydomain-it.svc:5222] (jitsi-conference-prosody.mydomain-it.svc/10.43.161.229:5222)' failed because: java.net.ConnectException: Connection refused (Connection refused)
at org.jivesoftware.smack.SmackException$EndpointConnectionException.from(SmackException.java:334)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:663)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:846)
at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:529)
at org.jitsi.xmpp.mucclient.MucClient.lambda$getConnectAndLoginCallable$7(MucClient.java:617)
at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:167)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
JVB 2022-04-06 12:05:30.240 WARNING: [25] [hostname=jitsi-conference-prosody.mydomain-it.svc id=shard] MucClient.setPresenceExtensions#449: Cannot set presence extension: not connected.
JVB 2022-04-06 12:05:31.418 INFO: [19] [hostname=jitsi-conference-prosody.mydomain-it.svc id=shard] MucClient$2.connected#305: Connected.
JVB 2022-04-06 12:05:31.418 INFO: [19] [hostname=jitsi-conference-prosody.mydomain-it.svc id=shard] MucClient.lambda$getConnectAndLoginCallable$7#628: Logging in.
JVB 2022-04-06 12:05:31.472 INFO: [19] [hostname=jitsi-conference-prosody.mydomain-it.svc id=shard] MucClient$2.authenticated#311: Authenticated, b=false
JVB 2022-04-06 12:05:31.539 INFO: [19] [hostname=jitsi-conference-prosody.mydomain-it.svc id=shard] MucClient$MucWrapper.join#752: Joined MUC: jvbbrewery@internal-muc.meet.jitsi
JVB 2022-04-06 12:05:31.767 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.maybeAdd: Discarding a mapping harvester: org.ice4j.ice.harvest.AwsCandidateHarvester@50ab068b
JVB 2022-04-06 12:05:31.768 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.initialize: Initialized mapping harvesters (delay=7208ms). stunDiscoveryFailed=true
JVB 2022-04-06 12:05:35.225 SEVERE: [23] HealthChecker.run#175: Health check failed in PT0.000137S:
java.lang.Exception: Address discovery through STUN failed
at org.jitsi.videobridge.health.JvbHealthChecker.check(JvbHealthChecker.kt:39)
at org.jitsi.videobridge.health.JvbHealthChecker.access$check(JvbHealthChecker.kt:25)
at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
at org.jitsi.health.HealthChecker.run(HealthChecker.kt:145)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.run(RecurringRunnableExecutor.java:216)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.runInThread(RecurringRunnableExecutor.java:292)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.access$000(RecurringRunnableExecutor.java:36)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor$1.run(RecurringRunnableExecutor.java:328)
JVB 2022-04-06 12:05:45.225 SEVERE: [23] HealthChecker.run#175: Health check failed in PT0.000107S:
java.lang.Exception: Address discovery through STUN failed
at org.jitsi.videobridge.health.JvbHealthChecker.check(JvbHealthChecker.kt:39)
at org.jitsi.videobridge.health.JvbHealthChecker.access$check(JvbHealthChecker.kt:25)
at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
at org.jitsi.health.HealthChecker.run(HealthChecker.kt:145)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.run(RecurringRunnableExecutor.java:216)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.runInThread(RecurringRunnableExecutor.java:292)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.access$000(RecurringRunnableExecutor.java:36)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor$1.run(RecurringRunnableExecutor.java:328)

Bastien_Dr · April 6, 2022, 11:54am

Okay so it’s seems reachable. For now I do not have more idea for digging on this STUN error.
I see you are running on AWS.
Check also you have an error on prosody reachability.

JohnLukeP · April 6, 2022, 12:09pm

on OVH Cloud actually. Prosody doesn’t seem to have problems since the pod start and goes in running mode just fine.
The JVB Pods though, I guess for those errors, fails the health status check and thus marked as crashed and not running.
These are the checks implemented by the kubernetes Jitsi deployment:

Bastien_Dr · April 6, 2022, 12:35pm

Quite strange if you are running on OVH Cloud and JVB say I’m running on AWS.
Try to disable AWS harvester with:

org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true

JohnLukeP · April 6, 2022, 12:56pm

Same error logs:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-set-timezone: executing...
[cont-init.d] 01-set-timezone: exited 0.
[cont-init.d] 10-config: executing...
[cont-init.d] 10-config: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
JVB 2022-04-06 14:53:40.875 INFO: [1] JitsiConfig.<clinit>#47: Initialized newConfig: merge of /config/jvb.conf: 1,application.conf @ jar:file:/usr/share/jitsi-videobridge/jitsi-videobridge.jar!/application.conf: 1,system properties,reference.conf @ jar:file:/usr/share/jitsi-videobridge/jitsi-videobridge.jar!/reference.conf: 1,reference.conf @ jar:file:/usr/share/jitsi-videobridge/lib/ice4j-3.0-52-ga9ba80e.jar!/reference.conf: 1,reference.conf @ jar:file:/usr/share/jitsi-videobridge/lib/jitsi-media-transform-1.0-281-gafb453e.jar!/reference.conf: 1
JVB 2022-04-06 14:53:40.891 INFO: [1] ReadOnlyConfigurationService.reloadConfiguration#51: loading config file at path /config/sip-communicator.properties
JVB 2022-04-06 14:53:40.892 INFO: [1] ReadOnlyConfigurationService.reloadConfiguration#56: Error loading config file: java.io.FileNotFoundException: /config/sip-communicator.properties (No such file or directory)
JVB 2022-04-06 14:53:40.893 INFO: [1] JitsiConfig.<clinit>#68: Initialized legacyConfig: sip communicator props (no description provided)
JVB 2022-04-06 14:53:40.893 INFO: [1] JitsiConfig$Companion.reloadNewConfig#94: Reloading the Typesafe config source (previously reloaded 0 times).
JVB 2022-04-06 14:53:40.909 INFO: [1] MainKt.main#90: Starting jitsi-videobridge version 2.1.617-ga8b39c3f
JVB 2022-04-06 14:53:41.262 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.initialize: Using AwsCandidateHarvester.
JVB 2022-04-06 14:53:41.285 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.createStunHarvesters: Using 193.123.56.238:443/udp for StunMappingCandidateHarvester (localAddress=10.42.0.200:0/udp).
JVB 2022-04-06 14:53:41.482 INFO: [19] [hostname=jitsi-prosody.my-domain-it.svc id=shard] MucClient.initializeConnectAndJoin#273: Initializing a new MucClient for [ org.jitsi.xmpp.mucclient.MucClientConfiguration id=shard domain=auth.meet.jitsi hostname=jitsi-prosody.my-domain-it.svc port=null username=jvb mucs=[jvbbrewery@internal-muc.meet.jitsi] mucNickname=jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj disableCertificateVerification=true]
JVB 2022-04-06 14:53:41.536 INFO: [1] LastNReducer.<init>#65: LastNReducer with reductionScale: 0.75 recoverScale: 1.25 impactTime: PT1M minLastN: 1 maxEnforcedLastN: 40
JVB 2022-04-06 14:53:41.539 INFO: [1] TaskPools.<clinit>#81: TaskPools detected 1 processors, creating the CPU pool with that many threads
JVB 2022-04-06 14:53:41.545 WARNING: [19] MucClient.createXMPPTCPConnectionConfiguration#116: Disabling certificate verification!
JVB 2022-04-06 14:53:41.558 WARNING: [1] UlimitCheck.printUlimits#111: Running with open files limit 1048576 (hard 1048576), thread limit null (hard null). These values are too low and they will limit the number of participants that the bridge can serve simultaneously.
JVB 2022-04-06 14:53:41.560 INFO: [1] VideobridgeExpireThread.start#88: Starting with 60 second interval.
JVB 2022-04-06 14:53:41.578 INFO: [1] HealthChecker.start#118: Started with interval=10000, timeout=PT30S, maxDuration=PT3S, stickyFailures=false.
JVB 2022-04-06 14:53:41.586 INFO: [1] MainKt.main#119: Not starting CallstatsService, disabled in configuration.
JVB 2022-04-06 14:53:41.589 INFO: [1] MainKt.main#128: Starting public http server
JVB 2022-04-06 14:53:41.594 INFO: [19] [hostname=jitsi-prosody.my-domain-it.svc id=shard] MucClient.initializeConnectAndJoin#332: Dispatching a thread to connect and login.
JVB 2022-04-06 14:53:41.624 INFO: [1] ColibriWebSocketService.<init>#42: WebSockets are not enabled
JVB 2022-04-06 14:53:41.642 INFO: [1] org.eclipse.jetty.util.log.Log.initialized: Logging initialized @1137ms to org.eclipse.jetty.util.log.JavaUtilLog
JVB 2022-04-06 14:53:41.728 INFO: [1] ColibriWebSocketService.registerServlet#86: Disabled, not registering servlet
JVB 2022-04-06 14:53:41.730 INFO: [1] org.eclipse.jetty.server.Server.doStart: jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 11.0.14+9-post-Debian-1deb11u1
JVB 2022-04-06 14:53:41.755 INFO: [1] org.eclipse.jetty.server.handler.ContextHandler.doStart: Started o.e.j.s.ServletContextHandler@1c65121{/,null,AVAILABLE}
JVB 2022-04-06 14:53:41.763 INFO: [1] org.eclipse.jetty.server.AbstractConnector.doStart: Started ServerConnector@30c0ccff{HTTP/1.1, (http/1.1)}{0.0.0.0:9090}
JVB 2022-04-06 14:53:41.763 INFO: [1] org.eclipse.jetty.server.Server.doStart: Started @1260ms
JVB 2022-04-06 14:53:41.764 INFO: [1] MainKt.main#146: Starting private http server
JVB 2022-04-06 14:53:41.804 INFO: [1] org.eclipse.jetty.server.Server.doStart: jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 11.0.14+9-post-Debian-1deb11u1
JVB 2022-04-06 14:53:42.025 WARNING: [1] org.glassfish.jersey.server.wadl.WadlFeature.configure: JAXBContext implementation could not be found. WADL feature is disabled.
JVB 2022-04-06 14:53:42.117 WARNING: [1] org.glassfish.jersey.internal.inject.Providers.checkProviderRuntime: A provider org.jitsi.rest.Health registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.jitsi.rest.Health will be ignored.
JVB 2022-04-06 14:53:42.118 WARNING: [1] org.glassfish.jersey.internal.inject.Providers.checkProviderRuntime: A provider org.jitsi.rest.Version registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.jitsi.rest.Version will be ignored.
JVB 2022-04-06 14:53:42.320 INFO: [1] org.eclipse.jetty.server.handler.ContextHandler.doStart: Started o.e.j.s.ServletContextHandler@2a53f215{/,null,AVAILABLE}
JVB 2022-04-06 14:53:42.321 INFO: [1] org.eclipse.jetty.server.AbstractConnector.doStart: Started ServerConnector@1ad926d3{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
JVB 2022-04-06 14:53:42.321 INFO: [1] org.eclipse.jetty.server.Server.doStart: Started @1817ms
JVB 2022-04-06 14:53:42.609 WARNING: [19] [hostname=jitsi-prosody.my-domain-it.svc id=shard] MucClient.lambda$getConnectAndLoginCallable$7#622: [MucClient id=shard hostname=jitsi-prosody.my-domain-it.svc] error connecting
org.jivesoftware.smack.SmackException$EndpointConnectionException: The following addresses failed: 'RFC 6120 A/AAAA Endpoint + [jitsi-prosody.my-domain-it.svc:5222] (jitsi-prosody.my-domain-it.svc/10.43.76.171:5222)' failed because: java.net.ConnectException: Connection refused (Connection refused)
at org.jivesoftware.smack.SmackException$EndpointConnectionException.from(SmackException.java:334)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:663)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:846)
at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:529)
at org.jitsi.xmpp.mucclient.MucClient.lambda$getConnectAndLoginCallable$7(MucClient.java:617)
at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:167)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
JVB 2022-04-06 14:53:46.599 WARNING: [25] [hostname=jitsi-prosody.my-domain-it.svc id=shard] MucClient.setPresenceExtensions#449: Cannot set presence extension: not connected.
JVB 2022-04-06 14:53:47.768 INFO: [19] [hostname=jitsi-prosody.my-domain-it.svc id=shard] MucClient$2.connected#305: Connected.
JVB 2022-04-06 14:53:47.769 INFO: [19] [hostname=jitsi-prosody.my-domain-it.svc id=shard] MucClient.lambda$getConnectAndLoginCallable$7#628: Logging in.
JVB 2022-04-06 14:53:47.822 INFO: [19] [hostname=jitsi-prosody.my-domain-it.svc id=shard] MucClient$2.authenticated#311: Authenticated, b=false
JVB 2022-04-06 14:53:47.893 INFO: [19] [hostname=jitsi-prosody.my-domain-it.svc id=shard] MucClient$MucWrapper.join#752: Joined MUC: jvbbrewery@internal-muc.meet.jitsi
JVB 2022-04-06 14:53:48.144 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.maybeAdd: Discarding a mapping harvester: org.ice4j.ice.harvest.AwsCandidateHarvester@7b7067d4
JVB 2022-04-06 14:53:48.145 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.initialize: Initialized mapping harvesters (delay=7231ms). stunDiscoveryFailed=true
JVB 2022-04-06 14:53:51.578 SEVERE: [23] HealthChecker.run#175: Health check failed in PT0.000138S:
java.lang.Exception: Address discovery through STUN failed
at org.jitsi.videobridge.health.JvbHealthChecker.check(JvbHealthChecker.kt:39)
at org.jitsi.videobridge.health.JvbHealthChecker.access$check(JvbHealthChecker.kt:25)
at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
at org.jitsi.health.HealthChecker.run(HealthChecker.kt:145)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.run(RecurringRunnableExecutor.java:216)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.runInThread(RecurringRunnableExecutor.java:292)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.access$000(RecurringRunnableExecutor.java:36)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor$1.run(RecurringRunnableExecutor.java:328)
JVB 2022-04-06 14:54:01.577 SEVERE: [23] HealthChecker.run#175: Health check failed in PT0.000072S:
java.lang.Exception: Address discovery through STUN failed
at org.jitsi.videobridge.health.JvbHealthChecker.check(JvbHealthChecker.kt:39)
at org.jitsi.videobridge.health.JvbHealthChecker.access$check(JvbHealthChecker.kt:25)
at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
at org.jitsi.health.HealthChecker.run(HealthChecker.kt:145)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.run(RecurringRunnableExecutor.java:216)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.runInThread(RecurringRunnableExecutor.java:292)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor.access$000(RecurringRunnableExecutor.java:36)
at org.jitsi.utils.concurrent.RecurringRunnableExecutor$1.run(RecurringRunnableExecutor.java:328)
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

Seems that also Jicofo can’t connect neither to the jvb nor to prosody:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-set-timezone: executing...
[cont-init.d] 01-set-timezone: exited 0.
[cont-init.d] 10-config: executing...
[cont-init.d] 10-config: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
Jicofo 2022-04-06 14:53:39.677 INFO: [1] Main.main#49: Starting Jicofo.
Jicofo 2022-04-06 14:53:39.881 INFO: [1] JitsiConfig.<clinit>#47: Initialized newConfig: merge of /config/jicofo.conf: 1,system properties,reference.conf @ jar:file:/usr/share/jicofo/jicofo.jar!/reference.conf: 1
Jicofo 2022-04-06 14:53:39.884 INFO: [1] ReadOnlyConfigurationService.reloadConfiguration#40: net.java.sip.communicator.SC_HOME_DIR_LOCATION not set
Jicofo 2022-04-06 14:53:39.886 INFO: [1] JitsiConfig.<clinit>#68: Initialized legacyConfig: sip communicator props (no description provided)
Jicofo 2022-04-06 14:53:39.886 INFO: [1] JitsiConfig$Companion.reloadNewConfig#94: Reloading the Typesafe config source (previously reloaded 0 times).
Jicofo 2022-04-06 14:53:40.515 INFO: [1] FocusManager.start#124: Initialized octoId=1
Jicofo 2022-04-06 14:53:40.542 INFO: [1] JicofoServices.createAuthenticationAuthority#197: Authentication service disabled.
Jicofo 2022-04-06 14:53:40.603 WARNING: [1] [xmpp_connection=client] XmppProviderImpl.createXmppConnection#169: Disabling TLS certificate verification!
Jicofo 2022-04-06 14:53:40.694 INFO: [1] XmppServices.<init>#48: No dedicated Service XMPP connection configured, re-using the client XMPP connection.
Jicofo 2022-04-06 14:53:40.697 INFO: [1] XmppServices.<init>#63: No Jigasi detector configured.
Jicofo 2022-04-06 14:53:40.714 INFO: [1] BridgeSelector.<init>#58: Using org.jitsi.jicofo.bridge.SingleBridgeSelectionStrategy
Jicofo 2022-04-06 14:53:40.724 INFO: [1] [type=bridge brewery=jvbbrewery] BaseBrewery.<init>#100: Initialized with JID=jvbbrewery@internal-muc.meet.jitsi
Jicofo 2022-04-06 14:53:40.727 INFO: [1] JicofoServices.<init>#122: No Jibri detector configured.
Jicofo 2022-04-06 14:53:40.727 INFO: [1] JicofoServices.<init>#128: No SIP Jibri detector configured.
Jicofo 2022-04-06 14:53:40.733 INFO: [1] JicofoServices.<init>#143: Starting HTTP server with config: host=null, port=8888, tlsPort=8843, isTls=false, keyStorePath=null, sendServerVersion=true.
Jicofo 2022-04-06 14:53:40.783 INFO: [1] org.eclipse.jetty.util.log.Log.initialized: Logging initialized @1264ms to org.eclipse.jetty.util.log.Slf4jLog
Jicofo 2022-04-06 14:53:40.855 INFO: [1] org.eclipse.jetty.server.Server.doStart: jetty-9.4.44.v20210927; built: 2021-09-27T23:02:44.612Z; git: 8da83308eeca865e495e53ef315a249d63ba9332; jvm 11.0.14+9-post-Debian-1deb11u1
Jicofo 2022-04-06 14:53:41.138 WARNING: [1] org.glassfish.jersey.server.wadl.WadlFeature.configure: JAXBContext implementation could not be found. WADL feature is disabled.
Jicofo 2022-04-06 14:53:41.206 WARNING: [1] org.glassfish.jersey.internal.inject.Providers.checkProviderRuntime: A provider org.jitsi.rest.Version registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.jitsi.rest.Version will be ignored.
Jicofo 2022-04-06 14:53:41.387 INFO: [1] org.eclipse.jetty.server.handler.ContextHandler.doStart: Started o.e.j.s.ServletContextHandler@49fb0bbd{/,null,AVAILABLE}
Jicofo 2022-04-06 14:53:41.395 INFO: [1] org.eclipse.jetty.server.AbstractConnector.doStart: Started ServerConnector@48535004{HTTP/1.1, (http/1.1)}{0.0.0.0:8888}
Jicofo 2022-04-06 14:53:41.395 INFO: [1] org.eclipse.jetty.server.Server.doStart: Started @1877ms
Jicofo 2022-04-06 14:53:41.745 SEVERE: [15] [xmpp_connection=client] XmppProviderImpl.doConnect#225: Failed to connect/login: The following addresses failed: 'RFC 6120 A/AAAA Endpoint + [jitsi-prosody.conference-rios-net-it.svc:5222] (jitsi-prosody.conference-rios-net-it.svc/10.43.76.171:5222)' failed because: java.net.ConnectException: Connection refused (Connection refused)
org.jivesoftware.smack.SmackException$EndpointConnectionException: The following addresses failed: 'RFC 6120 A/AAAA Endpoint + [jitsi-prosody.conference-rios-net-it.svc:5222] (jitsi-prosody.conference-rios-net-it.svc/10.43.76.171:5222)' failed because: java.net.ConnectException: Connection refused (Connection refused)
at org.jivesoftware.smack.SmackException$EndpointConnectionException.from(SmackException.java:334)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:663)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:846)
at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:529)
at org.jitsi.impl.protocol.xmpp.XmppProviderImpl.doConnect(XmppProviderImpl.java:205)
at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:167)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Jicofo 2022-04-06 14:53:46.884 INFO: [15] [xmpp_connection=client] XmppProviderImpl.doConnect#206: Connected, JID=null
Jicofo 2022-04-06 14:53:46.950 INFO: [15] AvModerationHandler.registrationChanged#120: Discovered av_moderation component at avmoderation.meet.jitsi.
Jicofo 2022-04-06 14:53:46.953 INFO: [15] ConferenceIqHandler.registrationChanged#183: Using breakout room component address: breakout.meet.jitsi
Jicofo 2022-04-06 14:53:46.993 INFO: [15] [type=bridge brewery=jvbbrewery] BaseBrewery.start#171: Joined the room.
Jicofo 2022-04-06 14:53:46.994 INFO: [15] [xmpp_connection=client] XmppProviderImpl.fireRegistrationStateChanged#331: Set replyTimeout=PT15S
Jicofo 2022-04-06 14:53:47.922 INFO: [29] [type=bridge brewery=jvbbrewery] BaseBrewery.addInstance#341: Added brewery instance: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:53:47.928 WARNING: [29] Main$1.warn#154: Key 'jicofo.bridge.average-participant-packet-rate-pps' from source 'typesafe config (reloaded 1 times)' is deprecated: use jicofo.bridge.average-participant-stress
Jicofo 2022-04-06 14:53:47.929 INFO: [29] BridgeSelector.addJvbAddress#95: Added new videobridge: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
Jicofo 2022-04-06 14:53:47.932 INFO: [32] JvbDoctor.bridgeAdded#127: Scheduled health-check task for: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
Jicofo 2022-04-06 14:54:11.053 INFO: [29] [type=bridge brewery=jvbbrewery] BaseBrewery.removeInstance#353: Removed brewery instance: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:54:11.053 INFO: [29] BridgeSelector.removeJvbAddress#112: Removing JVB: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:54:11.054 INFO: [32] JvbDoctor.bridgeRemoved#104: Stopping health-check task for: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
Jicofo 2022-04-06 14:54:21.096 INFO: [30] [type=bridge brewery=jvbbrewery] BaseBrewery.addInstance#341: Added brewery instance: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:54:21.099 INFO: [30] BridgeSelector.addJvbAddress#95: Added new videobridge: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
Jicofo 2022-04-06 14:54:21.101 INFO: [32] JvbDoctor.bridgeAdded#127: Scheduled health-check task for: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
Jicofo 2022-04-06 14:54:41.045 INFO: [16] [type=bridge brewery=jvbbrewery] BaseBrewery.removeInstance#353: Removed brewery instance: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:54:41.045 INFO: [16] BridgeSelector.removeJvbAddress#112: Removing JVB: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:54:41.046 INFO: [32] JvbDoctor.bridgeRemoved#104: Stopping health-check task for: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
Jicofo 2022-04-06 14:54:51.013 INFO: [31] [type=bridge brewery=jvbbrewery] BaseBrewery.addInstance#341: Added brewery instance: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:54:51.014 INFO: [31] BridgeSelector.addJvbAddress#95: Added new videobridge: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
Jicofo 2022-04-06 14:54:51.015 INFO: [32] JvbDoctor.bridgeAdded#127: Scheduled health-check task for: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
Jicofo 2022-04-06 14:55:11.047 INFO: [29] [type=bridge brewery=jvbbrewery] BaseBrewery.removeInstance#353: Removed brewery instance: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:55:11.048 INFO: [29] BridgeSelector.removeJvbAddress#112: Removing JVB: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:55:11.049 INFO: [32] JvbDoctor.bridgeRemoved#104: Stopping health-check task for: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
Jicofo 2022-04-06 14:55:11.049 SEVERE: [36] JvbDoctor$HealthCheckTask.run#148: Error when doing health-check on: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
java.lang.InterruptedException
at java.base/java.lang.Object.wait(Native Method)
at org.jivesoftware.smack.StanzaCollector.nextResult(StanzaCollector.java:206)
at org.jivesoftware.smack.StanzaCollector.nextResult(StanzaCollector.java:179)
at org.jitsi.jicofo.xmpp.UtilKt.sendIqAndGetResponse(Util.kt:81)
at org.jitsi.jicofo.bridge.JvbDoctor$HealthCheckTask.doHealthCheck(JvbDoctor.java:196)
at org.jitsi.jicofo.bridge.JvbDoctor$HealthCheckTask.run(JvbDoctor.java:144)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Jicofo 2022-04-06 14:55:21.061 INFO: [16] [type=bridge brewery=jvbbrewery] BaseBrewery.addInstance#341: Added brewery instance: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:55:21.062 INFO: [16] BridgeSelector.addJvbAddress#95: Added new videobridge: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
Jicofo 2022-04-06 14:55:21.062 INFO: [32] JvbDoctor.bridgeAdded#127: Scheduled health-check task for: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]
Jicofo 2022-04-06 14:55:41.051 INFO: [16] [type=bridge brewery=jvbbrewery] BaseBrewery.removeInstance#353: Removed brewery instance: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:55:41.052 INFO: [16] BridgeSelector.removeJvbAddress#112: Removing JVB: jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj
Jicofo 2022-04-06 14:55:41.052 INFO: [32] JvbDoctor.bridgeRemoved#104: Stopping health-check task for: Bridge[jid=jvbbrewery@internal-muc.meet.jitsi/jitsi-jitsi-meet-jvb-67c5dd7f56-5fxgj, relayId=null, region=null, stress=0.00]

JohnLukeP · April 12, 2022, 1:01pm

Hi,
does anybody have any other idea on what I could try to fix this out?
maybe there’s something else that has to be configured? For example, the jvb service type, what should be the correct configuration for it?
Thanks,
Gianluca

damencho · April 12, 2022, 1:08pm

The error is coming of not able to resolve the address of the hostname where prosody is running to be able to connect.
It is coming from this: docker-jitsi-meet/jvb.conf at b0617c0fe35cdebd5474d8b085d0432c69e708c2 · jitsi/docker-jitsi-meet · GitHub

And if you are running docker you should be using the default value:

github.com

jitsi/docker-jitsi-meet/blob/b0617c0fe35cdebd5474d8b085d0432c69e708c2/env.example#L234

      
        
            
            

            
#
            # Advanced configuration options (you generally don't need to change these)
            #
            
            
# Internal XMPP domain
            XMPP_DOMAIN=meet.jitsi
            
            
# Internal XMPP server
            XMPP_SERVER=xmpp.meet.jitsi
            
            
# Internal XMPP server URL
            XMPP_BOSH_URL_BASE=http://xmpp.meet.jitsi:5280
            
            
# Internal XMPP domain for authenticated services
            XMPP_AUTH_DOMAIN=auth.meet.jitsi
            
            
# XMPP domain for the MUC
            XMPP_MUC_DOMAIN=muc.meet.jitsi

If you change it, you need to make sure it resolves to the address where jvb can connect to prosody on port 5222.

JohnLukeP · April 12, 2022, 1:15pm

Hi @damencho, thanks for the reply. That value “jitsi-prosody.my-domain-it.svc” I think it’s pre-determined by kubernetes logics, since I’m using the Jitsi’s helm chart for kubernetes.

I tried passing they key=vale pair:
XMPP_SERVER=xmpp.meet.jitsi
as rancher’s variable, but seems it is not getting it, as you can see from jvb’s pod configmaps. How the key should be named?

damencho · April 12, 2022, 1:19pm

Not sure where are those coming from, but ask the creator of those for help. Sorry, I’m not familiar with kubernetes, but the error is coming from the bridge not being able to resolve this and to connect to prosody …

JohnLukeP · April 12, 2022, 1:24pm

Thanks for your help. I manage to “inject” the xmpp_server variable as it should be. Unfortunately, it is still not recognized. I fear that this happens because there are several pods, one for each component, instead of a monolithic pod with all the conmponents in different containers:

rg.jivesoftware.smack.SmackException$EndpointConnectionException: Could not lookup the following endpoints: RemoteConnectionEndpointLookupFailure(description='DNS lookup exception for xmpp.meet.jitsi' exception='java.net.UnknownHostException: xmpp.meet.jitsi: Name or service not known')
Tue, Apr 12 2022 3:22:17 pm at org.jivesoftware.smack.SmackException$EndpointConnectionException.from(SmackException.java:334)
Tue, Apr 12 2022 3:22:17 pm at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:663)
Tue, Apr 12 2022 3:22:17 pm at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:846)
Tue, Apr 12 2022 3:22:17 pm at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:529)
Tue, Apr 12 2022 3:22:17 pm at org.jitsi.xmpp.mucclient.MucClient.lambda$getConnectAndLoginCallable$7(MucClient.java:617)
Tue, Apr 12 2022 3:22:17 pm at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:167)
Tue, Apr 12 2022 3:22:17 pm at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
Tue, Apr 12 2022 3:22:17 pm at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
Tue, Apr 12 2022 3:22:17 pm at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
Tue, Apr 12 2022 3:22:17 pm at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Tue, Apr 12 2022 3:22:17 pm at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Tue, Apr 12 2022 3:22:17 pm at java.base/java.lang.Thread.run(Thread.java:829)
Tue, Apr 12 2022 3:22:22 pm JVB 2022-04-12 15:22:22.511 WARNING: [26] [hostname=xmpp.meet.jitsi id=shard] MucClient.setPresenceExtensions#449: Cannot set presence extension: not connected.
Tue, Apr 12 2022 3:22:22 pm JVB 2022-04-12 15:22:22.551 WARNING: [19] [hostname=xmpp.meet.jitsi id=shard] MucClient.lambda$getConnectAndLoginCallable$7#622: [MucClient id=shard hostname=xmpp.meet.jitsi] error connecting
Tue, Apr 12 2022 3:22:22 pm org.jivesoftware.smack.SmackException$EndpointConnectionException: Could not lookup the following endpoints: RemoteConnectionEndpointLookupFailure(description='DNS lookup exception for xmpp.meet.jitsi' exception='java.net.UnknownHostException: xmpp.meet.jitsi')
Tue, Apr 12 2022 3:22:22 pm at org.jivesoftware.smack.SmackException$EndpointConnectionException.from(SmackException.java:334)
Tue, Apr 12 2022 3:22:22 pm at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:663)
Tue, Apr 12 2022 3:22:22 pm at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:846)
Tue, Apr 12 2022 3:22:22 pm at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:529)
Tue, Apr 12 2022 3:22:22 pm at org.jitsi.xmpp.mucclient.MucClient.lambda$getConnectAndLoginCallable$7(MucClient.java:617)
Tue, Apr 12 2022 3:22:22 pm at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:167)
Tue, Apr 12 2022 3:22:22 pm at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
Tue, Apr 12 2022 3:22:22 pm at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
Tue, Apr 12 2022 3:22:22 pm at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
Tue, Apr 12 2022 3:22:22 pm at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Tue, Apr 12 2022 3:22:22 pm at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Tue, Apr 12 2022 3:22:22 pm at java.base/java.lang.Thread.run(Thread.java:829)
Tue, Apr 12 2022 3:22:27 pm JVB 2022-04-12 15:22:27.502 SEVERE: [24] HealthChecker.run#175: Health check failed in PT0.000077S:
Tue, Apr 12 2022 3:22:27 pm java.lang.Exception: Address discovery through STUN failed
Tue, Apr 12 2022 3:22:27 pm at org.jitsi.videobridge.health.JvbHealthChecker.check(JvbHealthChecker.kt:39)
Tue, Apr 12 2022 3:22:27 pm at org.jitsi.videobridge.health.JvbHealthChecker.access$check(JvbHealthChecker.kt:25)
Tue, Apr 12 2022 3:22:27 pm at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
Tue, Apr 12 2022 3:22:27 pm at org.jitsi.videobridge.health.JvbHealthChecker$healthChecker$1.invoke(JvbHealthChecker.kt:31)
Tue, Apr 12 2022 3:22:27 pm at org.jitsi.health.HealthChecker.run(HealthChecker.kt:145)
Tue, Apr 12 2022 3:22:27 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor.run(RecurringRunnableExecutor.java:216)
Tue, Apr 12 2022 3:22:27 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor.runInThread(RecurringRunnableExecutor.java:292)
Tue, Apr 12 2022 3:22:27 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor.access$000(RecurringRunnableExecutor.java:36)
Tue, Apr 12 2022 3:22:27 pm at org.jitsi.utils.concurrent.RecurringRunnableExecutor$1.run(RecurringRunnableExecutor.java:328)
Tue, Apr 12 2022 3:22:27 pm JVB 2022-04-12 15:22:27.510 WARNING: [26] [hostname=xmpp.meet.jitsi id=shard] MucClient.setPresenceExtensions#449: Cannot set presence extension: not connected.
Tue, Apr 12 2022 3:22:27 pm JVB 2022-04-12 15:22:27.553 WARNING: [19] [hostname=xmpp.meet.jitsi id=shard] MucClient.lambda$getConnectAndLoginCallable$7#622: [MucClient id=shard hostname=xmpp.meet.jitsi] error connecting
Tue, Apr 12 2022 3:22:27 pm org.jivesoftware.smack.SmackException$EndpointConnectionException: Could not lookup the following endpoints: RemoteConnectionEndpointLookupFailure(description='DNS lookup exception for xmpp.meet.jitsi' exception='java.net.UnknownHostException: xmpp.meet.jitsi: Name or service not known')
Tue, Apr 12 2022 3:22:27 pm at org.jivesoftware.smack.SmackException$EndpointConnectionException.from(SmackException.java:334)
Tue, Apr 12 2022 3:22:27 pm at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:663)
Tue, Apr 12 2022 3:22:27 pm at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:846)
Tue, Apr 12 2022 3:22:27 pm at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:529)
Tue, Apr 12 2022 3:22:27 pm at org.jitsi.xmpp.mucclient.MucClient.lambda$getConnectAndLoginCallable$7(MucClient.java:617)
Tue, Apr 12 2022 3:22:27 pm at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:167)
Tue, Apr 12 2022 3:22:27 pm at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
Tue, Apr 12 2022 3:22:27 pm at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
Tue, Apr 12 2022 3:22:27 pm at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
Tue, Apr 12 2022 3:22:27 pm at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Tue, Apr 12 2022 3:22:27 pm at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Tue, Apr 12 2022 3:22:27 pm at java.base/java.lang.Thread.run(Thread.java:829)
Tue, Apr 12 2022 3:22:32 pm JVB 2022-04-12 15:22:32.511 WARNING: [26] [hostname=xmpp.meet.jitsi id=shard] MucClient.setPresenceExtensions#449: Cannot set presence extension: not connected.
Tue, Apr 12 2022 3:22:32 pm JVB 2022-04-12 15:22:32.554 WARNING: [19] [hostname=xmpp.meet.jitsi id=shard] MucClient.lambda$getConnectAndLoginCallable$7#622: [MucClient id=shard hostname=xmpp.meet.jitsi] error connecting
Tue, Apr 12 2022 3:22:32 pm org.jivesoftware.smack.SmackException$EndpointConnectionException: Could not lookup the following endpoints: RemoteConnectionEndpointLookupFailure(description='DNS lookup exception for xmpp.meet.jitsi' exception='java.net.UnknownHostException: xmpp.meet.jitsi')
Tue, Apr 12 2022 3:22:32 pm at org.jivesoftware.smack.SmackException$EndpointConnectionException.from(SmackException.java:334)
Tue, Apr 12 2022 3:22:32 pm at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:663)
Tue, Apr 12 2022 3:22:32 pm at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:846)
Tue, Apr 12 2022 3:22:32 pm at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:529)
Tue, Apr 12 2022 3:22:32 pm at org.jitsi.xmpp.mucclient.MucClient.lambda$getConnectAndLoginCallable$7(MucClient.java:617)
Tue, Apr 12 2022 3:22:32 pm at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:167)
Tue, Apr 12 2022 3:22:32 pm at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
Tue, Apr 12 2022 3:22:32 pm at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
Tue, Apr 12 2022 3:22:32 pm at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
Tue, Apr 12 2022 3:22:32 pm at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Tue, Apr 12 2022 3:22:32 pm at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Tue, Apr 12 2022 3:22:32 pm at java.base/java.lang.Thread.run(Thread.java:829)
Tue, Apr 12 2022 3:22:33 pm [cont-finish.d] executing container finish scripts...
Tue, Apr 12 2022 3:22:33 pm [cont-finish.d] done.
Tue, Apr 12 2022 3:22:33 pm [s6-finish] waiting for services.
Tue, Apr 12 2022 3:22:34 pm [s6-finish] sending all processes the TERM signal.

damencho · April 12, 2022, 1:31pm

I was saying about xmpp.meet.jitsi cause this is what is used in our docker images:

github.com

jitsi/docker-jitsi-meet/blob/b0617c0fe35cdebd5474d8b085d0432c69e708c2/env.example#L234

      
        
            
            

            
#
            # Advanced configuration options (you generally don't need to change these)
            #
            
            
# Internal XMPP domain
            XMPP_DOMAIN=meet.jitsi
            
            
# Internal XMPP server
            XMPP_SERVER=xmpp.meet.jitsi
            
            
# Internal XMPP server URL
            XMPP_BOSH_URL_BASE=http://xmpp.meet.jitsi:5280
            
            
# Internal XMPP domain for authenticated services
            XMPP_AUTH_DOMAIN=auth.meet.jitsi
            
            
# XMPP domain for the MUC
            XMPP_MUC_DOMAIN=muc.meet.jitsi

But how it should work with those helm charts, I have no idea, sorry.

Bastien_Dr · April 14, 2022, 7:49am

Can you share the output of in the namespace you install jitsi ?

kubectl get svc

JohnLukeP · April 29, 2022, 2:19pm

Hi; I do get this:

NAME                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                        AGE
jitsi-jitsi-meet-jvb   ClusterIP   10.43.93.71     <none>        10000/UDP                                      115s
jitsi-jitsi-meet-web   ClusterIP   10.43.215.129   <none>        80/TCP                                         115s
jitsi-prosody          ClusterIP   10.43.221.184   <none>        5280/TCP,5281/TCP,5347/TCP,5222/TCP,5269/TCP   115s

Bastien_Dr · May 5, 2022, 1:27pm

So your XMPP server is behind the address jitsi-prosody and not xmpp.meet.jitsi