Cannot see anyone but me and error message in jvb.log on a NATed VM on Debian 10

Hi!

I just followed the installation guide to install Jitsi Meet server on a NATed VM under Debian 10.
By the way, Jitsy Meet works pretty well on official servers and I use Firefox 76.
Moreover I forwarded ports like the doc mentions for NATed networks.

Server is ok when I connect to the URL but when creating a room, I cannot see nor hear anyone but me and there is recurring errors in /var/log/jitsi/jvb.log:

2020-05-22 07:16:49.414 SEVERE: [29] RecurringRunnableExecutor.run#230: The invocation of the method org.jitsi.videobridge.health.Health.run() threw an exception.
java.lang.NoClassDefFoundError: Could not initialize class org.jitsi.videobridge.sctp.SctpManager
        at org.jitsi.videobridge.Endpoint.createSctpConnection(Endpoint.java:868)
        at org.jitsi.videobridge.health.Health.check(Health.java:77)
        at org.jitsi.videobridge.health.Health.performCheck(Health.java:211)
        at org.jitsi.health.AbstractHealthCheckService.run(AbstractHealthCheckService.kt:144)
        at org.jitsi.utils.concurrent.RecurringRunnableExecutor.run(RecurringRunnableExecutor.java:216)
        at org.jitsi.utils.concurrent.RecurringRunnableExecutor.runInThread(RecurringRunnableExecutor.java:292)
        at org.jitsi.utils.concurrent.RecurringRunnableExecutor.access$000(RecurringRunnableExecutor.java:36)
        at org.jitsi.utils.concurrent.RecurringRunnableExecutor$1.run(RecurringRunnableExecutor.java:328)


2020-05-22 07:16:52.015 SEVERE: [35] ComponentMain.lambda$getConnectCallable$0#293: host-unknown, host:localhost, port:5347
org.xmpp.component.ComponentException: host-unknown
        at org.jivesoftware.whack.ExternalComponent.connect(ExternalComponent.java:243)
        at org.jivesoftware.whack.ExternalComponentManager.addComponent(ExternalComponentManager.java:242)
        at org.jivesoftware.whack.ExternalComponentManager.addComponent(ExternalComponentManager.java:222)
        at org.jitsi.meet.ComponentMain.lambda$getConnectCallable$0(ComponentMain.java:285)
        at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:193)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

I tried to set --apis=xmpp,rest in config file but with no result.

I need help!
Thanks!

this could mean that your jvb config is bad… or unfortunately that the jvb debian packaging has room for improvement; jvb systemd config should depend on prosody and it does not, so this could be meaningless and just be a transient message, jvb start to work when prosody is online.

this change has no point IMO

be sure to follow the official up-to-date install guide (on github, NOT youtube) and get the last stable version.

Thanks for replying!
I follow this page : https://jitsi.github.io/handbook/docs/devops-guide/devops-guide-quickstart
There is more up-todate doc?

I scripted all the commands I’ve done in order to automatically (at best) do the process. Here is the script (sorry for comments in french):

 #!/usr/bin/bash

# Nom de domaine du serveur Jitsi...
DN=<mydomainename>

# Récupération de l'IP de la VM...
LOC_IP=`hostname -I |awk '{print $1}'`

# Mise-Ă -jour des paquets...
apt update && apt full-upgrade -y

# Installation de paquets nécessaire à l'installation...
apt -y install gnupg ufw

# Récupération et ajout de la clé du dépôt Jitsi...
wget -qO - https://download.jitsi.org/jitsi-key.gpg.key | apt-key add -

# Ajout du dépôt Jitsi...
echo 'deb https://download.jitsi.org stable/' > /tmp/jitsi-stable.list && mv /tmp/jitsi-stable.list /etc/apt/sources.list.d/

apt update

# Configuration des paquets jitsi-meet pour définir automatiquement le nom d'hôte du serveur
# et le type de certificat au lieu de les remplir interactivement)...
echo "jitsi-meet-prosody jitsi-meet-prosody/jvb-hostname string $DN" | debconf-set-selections
echo "jitsi-meet-turnserver jitsi-meet-turnserver/jvb-hostname string $DN" | debconf-set-selections
echo "jitsi-meet-web-config jitsi-meet/jvb-hostname string $DN" | debconf-set-selections
echo "jitsi-meet-prosody jitsi-videobridge/jvb-hostname string $DN" | debconf-set-selections
echo "jitsi-meet-turnserver jitsi-videobridge/jvb-hostname string $DN" | debconf-set-selections
echo "jitsi-meet-web-config jitsi-videobridge/jvb-hostname string $DN" | debconf-set-selections
echo "jitsi-meet-web-config jitsi-meet/cert-choice select Generate a new self-signed certificate (You will later get a chance to obtain a Let's encrypt certificate)" | debconf-set-selections

# Installation de Jitsi Meet...
apt install -y jitsi-meet

# Ouverture des ports nécessaires dans le pare-feu...
ufw allow 80/tcp
ufw allow 443/tcp
ufw allow 4443/tcp
ufw allow 10000/udp
ufw allow 22/tcp
ufw --force enable

# Adaptation de la configuration Jitsi...
cp /etc/jitsi/videobridge/sip-communicator.properties /tmp
cat /tmp/sip-communicator.properties |grep -v "STUN_MAPPING_HARVESTER_ADDRESSES" >/tmp/prop
echo "org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=$LOC_IP" >> /tmp/prop
echo "org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=$LC_PUB_IP" >> /tmp/prop
mv /tmp/prop /etc/jitsi/videobridge/

# Installation d'un certificat...
/usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh

yes that’s good.
now what is the result ?
is your domain working -> use https://whatsmychaincert.com
is prosody running ? sudo ss -tapnu | grep lua | grep LISTEN -> ports 5222 5347 5280
is videobridge running ? ps fauxww | grep jvb
is jicofo tunning ? ps fauxww | grep jicofo
are jvb and jicofo connected to prosody ? sudo ss -tapnu | grep $(sudo cat /var/run/prosody/prosody.pid) | grep ESTAB -> should return 3 lines with lua, 3 lines with java

1 Like

now what is the result ?

It’s this script I execute to obtain this result.

is your domain working -> use https://whatsmychaincert.com
It’s ok, tells “correct chain”

is prosody running ? sudo ss -tapnu | grep lua | grep LISTEN -> ports 5222 5347 5280

Yep:

tcp   LISTEN     0       128                                      0.0.0.0:5280                                         0.0.0.0:*                                 users:(("lua5.2",pid=479,fd=12))                                               
tcp   LISTEN     0       128                                    127.0.0.1:5347                                         0.0.0.0:*                                 users:(("lua5.2",pid=479,fd=15))                                               
tcp   LISTEN     0       128                                      0.0.0.0:5222                                         0.0.0.0:*                                 users:(("lua5.2",pid=479,fd=5))                                                
tcp   LISTEN     0       128                                      0.0.0.0:5269                                         0.0.0.0:*                                 users:(("lua5.2",pid=479,fd=10))                                               
tcp   LISTEN     0       128                                         [::]:5280                                            [::]:*                                 users:(("lua5.2",pid=479,fd=11))                                               
tcp   LISTEN     0       128                                        [::1]:5347                                            [::]:*                                 users:(("lua5.2",pid=479,fd=14))                                               
tcp   LISTEN     0       128                                         [::]:5222                                            [::]:*                                 users:(("lua5.2",pid=479,fd=4))                                                
tcp   LISTEN     0       128                                         [::]:5269                                            [::]:*                                 users:(("lua5.2",pid=479,fd=9))                                                

is videobridge running ? ps fauxww | grep jvb

Yep!

is jicofo tunning ? ps fauxww | grep jicofo

Yep!

are jvb and jicofo connected to prosody ? sudo ss -tapnu | grep $(sudo cat /var/run/prosody/prosody.pid) | grep ESTAB -> should return 3 lines with lua, 3 lines with java

No java!

tcp   ESTAB      0       0                                      127.0.0.1:5347                                       127.0.0.1:37736                             users:(("lua5.2",pid=479,fd=17))                                               
tcp   ESTAB      0       0                                      127.0.0.1:5222                                       127.0.0.1:38802                             users:(("lua5.2",pid=479,fd=18))                                               
tcp   ESTAB      0       0                                      127.0.0.1:5222                                       127.0.0.1:38794                             users:(("lua5.2",pid=479,fd=16))

yes, my bad, never mind it’s working.

now it seems that jicofo and jvb are running. Could be port 10000 if it’s not NATED.
It’s not strictly necessary if coturn is running but i(s not a given that this thing works and anyway using port 10000 directly is better for performance. So check with tcpdump or tshark if packets are received by your server on port 10000 UDP when another user try to connect.

Effectively, “sudo tcpdump port 10000” prints out nothing when I connect to the conference with my phone :frowning:

However:
sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
80/tcp                     ALLOW IN    Anywhere                  
443/tcp                    ALLOW IN    Anywhere                  
4443/tcp                   ALLOW IN    Anywhere                  
10000/udp                  ALLOW IN    Anywhere                  
22/tcp                     ALLOW IN    Anywhere                  
80/tcp (v6)                ALLOW IN    Anywhere (v6)             
443/tcp (v6)               ALLOW IN    Anywhere (v6)             
4443/tcp (v6)              ALLOW IN    Anywhere (v6)             
10000/udp (v6)             ALLOW IN    Anywhere (v6)             
22/tcp (v6)                ALLOW IN    Anywhere (v6)             

and I forwarded UDP *:10000 to the VM.

well with 2 users it’s in the murky domain of p2p so it may be normal. Try to set p2p to false in the config.js and try again (no need to restart the server) to see if packets are really coming.

Done but nothing change : no udp/10000, same error message, no external video :frowning:

the use of a portable phone is a bit limiting since it’s not possible to do a tcpdump on the originating device to see if packets are startting. There are 2 possibilities at this point: jicofo is failing to start the conference or it is succeeding but the device can’t send udp packets (firewall or whatever). If the device is not even trying to send packets, the conference is not created. It should be in jicofo.log. I don’t know what’s easier to check use a PC instead of a phone and tcpdump’ing on the PC or trying to read the jicofo.log. Also the jvb.log is huge and is only interesting at the point in time when you are really tryng to connect.

I could try with another pc but in the local network. Is it ok?

And what do you mean by starting the conference because the room is set up and users can chat with each other. It’s only video and sound which are not activated.

And with one PC and a phone : on the PC tcpdump prints out no UDP/10000 packets. What is it change to do it between 2 PC ?

Tried with 2 pc and a phone : no udp/10000 on each PC.

After filtering errors, I don’t see other errors than errors posted in the first post in /var/log/jvb.log.

Printing /var/log/jicofo.log there is a timeout error:

 Jicofo 2020-05-22 16:14:15.068 SEVERE: [296] org.jitsi.jicofo.AbstractChannelAllocator.log() jvbbrewery@internal.auth.XXX.com/2917535f-9089-44bb-b515-9845557ffb43 - failed to allocate channels, will consider the bridge faulty: Timed out waiting for a response.
org.jitsi.protocol.xmpp.colibri.exception.TimeoutException: Timed out waiting for a response.
        at org.jitsi.impl.protocol.xmpp.colibri.ColibriConferenceImpl.maybeThrowOperationFailed(ColibriConferenceImpl.java:400)
        at org.jitsi.impl.protocol.xmpp.colibri.ColibriConferenceImpl.createColibriChannels(ColibriConferenceImpl.java:315)
        at org.jitsi.protocol.xmpp.colibri.ColibriConference.createColibriChannels(ColibriConference.java:115)
        at org.jitsi.jicofo.ParticipantChannelAllocator.doAllocateChannels(ParticipantChannelAllocator.java:146)
        at org.jitsi.jicofo.AbstractChannelAllocator.allocateChannels(AbstractChannelAllocator.java:262)
        at org.jitsi.jicofo.AbstractChannelAllocator.doRun(AbstractChannelAllocator.java:181)
        at org.jitsi.jicofo.AbstractChannelAllocator.run(AbstractChannelAllocator.java:149)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

That’s the problem. Jicofo is supposed to orchestrate the videobridge(s), and it is failing to do so. On further reflexion, it’s possible that the change in JVB_OPTS to xmp,rest may be more than useless, it could be actually harmful. Put it back to JVB_OPTS="–apis=rest" and restart the videobridge.

Set “rest” only and execute “sudo systemctl restart jitsi-videobridge2.service; sudo systemctl restart prosody.service; sudo systemctl restart jicofo.service”.

Always no external sound nor video :frowning:

i’d try to restart the whoe system then, redo a test and check if th error message is the same.

What do you mean by restart the all system?

the vm or physical computer if it’s not a vm.

I reboot the VM but same results :frowning:

Getting back to it I’m stunned to think that I forgot to ask you if you checked the results of your scripted install, particularly the sip-communicator.properties, is there something along the lines of

org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true
org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=localhost
org.jitsi.videobridge.xmpp.user.shard.DOMAIN=auth.yoururl.tld
org.jitsi.videobridge.xmpp.user.shard.USERNAME=jvb
org.jitsi.videobridge.xmpp.user.shard.PASSWORD=zaQ5pgBN
org.jitsi.videobridge.xmpp.user.shard.MUC_JIDS=JvbBrewery@internal.auth.yoururl.tld
org.jitsi.videobridge.xmpp.user.shard.MUC_NICKNAME=60f3570f-425b-4871-abc3-88625d0da21a

org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=10.10.10.10
org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=37.37.37.37