Jitsi-videobridge failing Health check

Hi,
i was trying to install a jitsi-meet selfhosted server on Ubuntu Server 18.04.4 LTS. I followed this guide: quick-install
Additionaly I set the hostname to my FQDN.

Everything seems to run fine, but as soon as a second participant joins a meeting i get:
“Unfortunately, something went wrong” and then it tries to reconnect after a few seconds, which also fails.
So i looked in the browser console which tells me
987 Logger.js:154 2020-03-23T07:32:10.133Z [conference.js] <z._onConferenceFailed>: CONFERENCE FAILED: conference.videobridgeNotAvailable
So i looked into the logs of the videobridge under /var/log/jitsi/jvb.log, where is see lots of messages like this:

JVB 2020-03-23 07:31:05.135 SEVERE: [46] org.jitsi.videobridge.health.Health.log() Health check failed in 28ms:
java.lang.NullPointerException
        at org.ice4j.ice.harvest.TcpHarvester.createLocalCandidates(TcpHarvester.java:277)
        at org.ice4j.ice.harvest.TcpHarvester.harvest(TcpHarvester.java:416)
        at org.ice4j.ice.Agent.gatherCandidates(Agent.java:687)
        at org.ice4j.ice.Agent.createComponent(Agent.java:601)
        at org.jitsi.videobridge.IceUdpTransportManager.createIceAgent(IceUdpTransportManager.java:1082)
        at org.jitsi.videobridge.IceUdpTransportManager.<init>(IceUdpTransportManager.java:566)
        at org.jitsi.videobridge.IceUdpTransportManager.<init>(IceUdpTransportManager.java:527)
        at org.jitsi.videobridge.SctpConnection.createTransportManager(SctpConnection.java:337)
        at org.jitsi.videobridge.Channel.initialize(Channel.java:674)
        at org.jitsi.videobridge.Content.createSctpConnection(Content.java:412)
        at org.jitsi.videobridge.health.Health.check(Health.java:184)
        at org.jitsi.videobridge.health.Health.doCheck(Health.java:244)
        at org.jitsi.videobridge.health.Health.doRun(Health.java:495)
        at org.jitsi.utils.concurrent.PeriodicRunnableWithObject.run(PeriodicRunnableWithObject.java:87)
        at org.jitsi.utils.concurrent.RecurringRunnableExecutor.run(RecurringRunnableExecutor.java:216)
        at org.jitsi.utils.concurrent.RecurringRunnableExecutor.runInThread(RecurringRunnableExecutor.java:292)
        at org.jitsi.utils.concurrent.RecurringRunnableExecutor.access$000(RecurringRunnableExecutor.java:36)
        at org.jitsi.utils.concurrent.RecurringRunnableExecutor$1.run(RecurringRunnableExecutor.java:328)

As additional information, I am running this server behind a NAT and i configured /etc/jitsi/videobridge/sip-communicator.properties according to the quick-install guide. Also the portforwards are working, which I checked using nc while the videobridge was not running.
Also I secured my domain following this section from the manual-install guide: https://github.com/jitsi/jicofo#secure-domain

I appreciate any help. Also I can provide further logs if needed.
Thanks,
Markus

Have you tried first restart jvb, then restart jicofo?

thanks for your reply and for having a look at my problem.

tried it, without success, same behaviour and same error in jvb.log

oddly, sometimes (i would guess one out of 10times) things are working as they should, meaning a second person can join the meeting and audio and video just work. But one out of 10 seems to be too little to use the jitsi-meet server in a reasonable manner.

Are you using Firefox? Try with chrome.

I am testing, using chromium and the android app.

so I did some further error hunting with the following results:

from the running services(nginx, jitsi-videobridge, prosody and jicofo) jicofo needs to be started as the last service. Then i can set up a meeting and connect with one other participant (my testsetting is connecting from a laptop running chromium and from an android phone with the jitsi meet app).
This works for one meeting, meaning when i leave the call and end the meeting, i can start another meeting, but as soon as a second participant joins i get the error mentioned in my origin post.

Furthermore when i have a meeting running with two participants as mentioned, i cannot join with a 3rd participant. Same error as always.

Though always monitoring the logfiles in /var/log/jitsi and /var /log/prosody, i see, that the failed health check happens, even if the conference with two participants is working just fine. So this error message might not be the culprit and be there for any other reason (just guessing, does not mean this has to be like that).
Further i found out that the jicofo.log file differs between the successful and the usucessful meetings. Therefore i upload the jicofo.log and jicofo.success.log (I replaced my domain names with example.com)jicofo.log (27.0 KB) jicofo.success.log (28.7 KB)

Those both files let me assume, that the error might be following line
Jicofo 2020-03-24 19:10:48.294 SEVERE: [40] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Can not invite participant -- no bridge available.

I would appreciate if you could have another look at my issues. Let me know if i should provide further information.
Thanks

Have you checked jvb logs for errors?

I could not find significant differences between the jvb.log of a failing meeting and a succesful meeting.
I uoload 4 files: jvb.log and jicogo.og of successfull video conference and jvb.log and jicofo of failing conference, when the second participant joins.

Thanks for taking the time and looking into it.
jvb.success.log (74.8 KB) jicofo.success.log (29.3 KB)
jvb.fail.log (47.3 KB) jicofo.fail.log (28.9 KB)

Your jvb is just not working:

Jicofo 2020-03-25 07:35:25.356 WARNING: [18] org.jitsi.jicofo.JvbDoctor.log() Health check failed on: jitsi-videobridge.example.com error: <error type='cancel'><internal-server-error xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/><text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas' xml:lang='en'>java.lang.NullPointerException</text></error>
Jicofo 2020-03-25 07:35:25.357 INFO: [46] org.jitsi.jicofo.BridgeSelector.log() Removing JVB: jitsi-videobridge.example.com
Jicofo 2020-03-25 07:35:25.357 INFO: [46] org.jitsi.jicofo.JvbDoctor.log() Stopping health-check task for: jitsi-videobridge.example.com
JVB 2020-03-25 07:34:45.754 INFO: [1] org.jitsi.impl.configuration.ConfigurationServiceImpl.log() org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=<192.168.1.3>

JVB 2020-03-25 07:34:45.754 INFO: [1] org.jitsi.impl.configuration.ConfigurationServiceImpl.log() org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=<....>

Have you entered the ip-addresses with < and >. Remove that and restart jvb.

Hm that was the issue. Now everything seems to be running smooth. Thanks for pointing that out.