Jitsi-videobridge2 gives a Jetty 404 after last nights upgrades

#1

Good day.

I tend to update my machines every night. Yesterday I deployed my first jitsi-videobridge2 test systems, that were working fine. But after last night updates, I am greeted with a jetty 404.

These are my package versions as of this morning:

root@app:/var/log/jitsi# dpkg -l | egrep "jitsi|jdk|prosody|jicofo"  
ii  jicofo                          1.0-549-1                    all          JItsi Meet COnference FOcus
ii  jitsi-meet                      2.0.4468-1                   all          WebRTC JavaScript video conferences
ii  jitsi-meet-prosody              1.0.4025-1                   all          Prosody configuration for Jitsi Meet
ii  jitsi-meet-web                  1.0.4025-1                   all          WebRTC JavaScript video conferences
ii  jitsi-meet-web-config           1.0.4025-1                   all          Configuration for web serving of Jitsi Meet
ii  jitsi-videobridge2              2.1-183-gdbddd169-1          all          WebRTC compatible Selective Forwarding Unit (SFU)
ii  openjdk-11-jre-headless:amd64   11.0.6+10-1~deb10u1          amd64        OpenJDK Java runtime, using Hotspot JIT (headless)
ii  prosody                         0.11.2-1                     amd64        Lightweight Jabber/XMPP server
root@app:/var/log/jitsi# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

Do I need to flush some cache? I restarted the services (prosody|jitsu-videobridge2|jicofo) but that did not help.

Literal error:

## HTTP ERROR 404

Problem accessing /. Reason:

Not Found

[Powered by Jetty:// 9.4.15.v20190215](http://eclipse.org/jetty)

Upgrade logs:

Preparing to unpack .../jitsi-videobridge2_2.1-183-gdbddd169-1_all.deb ...
Unpacking jitsi-videobridge2 (2.1-183-gdbddd169-1) over (2.1-169-ga28eb88e-1) ...
Setting up jitsi-videobridge2 (2.1-183-gdbddd169-1) ...
* Applying /etc/sysctl.d/20-jvb-udp-buffers.conf ...
net.core.rmem_max = 10485760
net.core.netdev_max_backlog = 100000
* Applying /etc/sysctl.d/99-sysctl.conf ...
* Applying /etc/sysctl.d/protect-links.conf ...
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
* Applying /etc/sysctl.conf ...
Preparing to unpack .../jitsi-meet_2.0.4468-1_all.deb ...
Unpacking jitsi-meet (2.0.4468-1) over (2.0.4416-1) ...
Preparing to unpack .../jicofo_1.0-549-1_all.deb ...
Unpacking jicofo (1.0-549-1) over (1.0-544-1) ...
Preparing to unpack .../jitsi-meet-web_1.0.4025-1_all.deb ...
Unpacking jitsi-meet-web (1.0.4025-1) over (1.0.3992-1) ...
Preparing to unpack .../jitsi-meet-web-config_1.0.4025-1_all.deb ...
Unpacking jitsi-meet-web-config (1.0.4025-1) over (1.0.3992-1) ...
Preparing to unpack .../jitsi-meet-prosody_1.0.4025-1_all.deb ...
Unpacking jitsi-meet-prosody (1.0.4025-1) over (1.0.3992-1) ...
Setting up jitsi-meet-prosody (1.0.4025-1) ...
Setting up jitsi-meet-web-config (1.0.4025-1) ...
----------------

You can now switch to a Let’s Encrypt certificate. To do so, execute:
/usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh

----------------
Setting up jicofo (1.0-549-1) ...
Setting up jitsi-meet-web (1.0.4025-1) ...
Setting up jitsi-meet (2.0.4468-1) ...
Processing triggers for systemd (241-7~deb10u3) ...

Maybe there was some interactive question in the package that I obviously did not answer as these updates are installed via a cron job.

Any advise or pointers appreciated.

#2

I’d first try to reboot since sometimes services are linked in non obvious ways and just restarting some may not be as useful.

If rebooting does not cut it, this suggests that jicofo and/or jvb are not running, something that you can confirm by running something like

ps fauxww | grep jetty (jicofo)
and
ps fauxww | grep videobridge (jvb)

if something is not running, you should obviously look at the logs probably under /var/log/jitsi.
FTR there was nothing wrong with my Ubuntu 18.04 LTS but I always reboot after jitsi uprgrade and the not jitsi packages (jdk, prosody, probably nginx) have different versions.

#3

Restarting did not help. I did notice that on a clean install the newly created configuration files differ significantly from my current configs. So it does not seem the config files are migrated when new options arrive / are needed.

The configs I use are based on an older Jitsi release with authentication for room creation added. I will redeploy the machines with a vanilla config and try to reapply the customisations.

Thanks for your response.

#4

The switch to videobridge2 with Debian packaging wrecked my setup indeed. I could get it back on shape only by spinning a new container, doing a clean install on it and checking the differences.

#5

Well yes I did more or less the same on Friday, had it working with videobridge2 coming from my old configs with some modifications. And then my cron job applied automatic updates during the night that broke it all. O well, bleeding edge it is then :slight_smile: