Problem with videobridges and websocket

Install & Config
,
#1

Hello,

few days ago, we deployed jitsi to production public server. Jitsi (jicofo, prosody, nginx) is behind NAT, videobridges too. When I with my co-worker tried to join from local network everything is fine, but when we tried to join to our conference from outside devices, everything crashed.

In developer console prosody cry “WebSocket connection to ‘wss://e-konf.waw.xxx.pl/colibri-ws/default-id/87cde19a6ceb9317/33be6d93?pwd=2rh2e4tcc’ failed: Error during WebSocket handshake: Unexpected response code: 502”. After 1 sec; " <WebSocket.e.onclose>: Channel closed: 1006" and then console is spamming

“[JitsiConference.js] <u.sendMessage>: Failed to send E2E ping request or response. undefined
[modules/RTC/BridgeChannel.js] <l._send>: Bridge Channel send: no opened channel.”

I tried everything, from setting up new Jitsi server to open all ports on Linux firewall and firewall in private network.

Parameters of Jitsi servers:
Jitsi main server - 4vCPU, 16GB RAM, 300/300 Mbps
1st videobridge - 4vCPU, 8GB RAM, 300/300 Mbps
2nd videobridge - 4vCPU, 8GB RAM, 300/300 Mbps
3td videobridge - 4vCPU, 8GB RAM, 300/300 Mbps

#2

Looks like an issue with the configuration of the video bridge proxies… most likely firewall settings… (Error 502 is Bad Gateway, btw)

What does the webserver conf look like on the video bridge?

Maybe this will help? [How to] Why do I see "Unfortunately something went wrong"? Why does my room 'crash'?

#3

Hi, thanks for the reply.

I turned off ufw - linux firewall, and now i’m connecting from private network, not from outside.
Still same problem and same errors in developer console.

Versions of software:

  • Linux dist: Ubuntu 18.04 LTS Server 64-bit
  • ii prosody 0.10.0-1build1 amd64 Lightweight Jabber/XMPP server
    ii jitsi-meet 2.0.5142-1 all WebRTC JavaScript video conferences
    ii jitsi-meet-prosody 1.0.4466-1 all Prosody configuration for Jitsi Meet
    ii jitsi-meet-turnserver 1.0.4466-1 all Configures coturn to be used with Jitsi Meet
    ii jitsi-meet-web 1.0.4466-1 all WebRTC JavaScript video conferences
    ii jitsi-meet-web-config 1.0.4466-1 all Configuration for web serving of Jitsi Meet
    ii jitsi-videobridge2 2.1-376-g9f12bfe2-1 all WebRTC compatible Selective Forwarding Unit (SFU)

Configuration:

Logs:

  • Nginx error log: 1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.29.116.210, server: e-konf.waw.xxxx.pl, request: "GET /colibri-ws/default-id/9ed480535f55051c/43c2f3c4?pwd=2ltn58chrnavjn8f64n2t1r14v HTTP/1.1", upstream: "http://127.0.0.1:9090/colibri-ws/default-id/9ed480535f55051c/43c2f3c4?pwd=2ltn58chrnavjn8f64n2t1r14v", host: "e-konf.waw.xxxxx.pl"
#4

Refresh :frowning_face:

#5

First a general and common issue for Ubuntu 18.04: it’s advised to not keep the system Prosody and setup Prosody 11 via PPA. But that’s not your problem.
From what I see you can’t connect to Jvb with Colibri. First check that jvb is listening on port 9090:
sudo ss -tapnu | grep 9090
you should have a java process (well, jitsi-videobridge2 aka jvb)

#6

I mean 3rd videobridge :wink:

Seems to be used…

Zrzut ekranu (7)
Zrzut ekranu (7)1443×288 25.1 KB

#7

right, the websocket error is an error but it’s not why it does not work. After all video conferencing can work without a data channel to the bridge (only less well), so it’s crashing because of a most basic problem. Check the jvb connectivity, it’s the most common problem

(server)
sudo systemctl stop jitsi-videobridge2
nc -l 10000 -u
(workstation)
echo "123" | nc -u (your public address) 10000
#8

Okey, it seems to firewall policy issue. Fortigate is blocking icmp, but it should pull my request to the server.

Zrzut ekranu (8)
Zrzut ekranu (8)933×226 22.8 KB

^Jitsi server with stopped jitsi-videobridge service

Zrzut ekranu (9)
Zrzut ekranu (9)1223×203 7.53 KB

^ client side

Zrzut ekranu (10)
^ nmap scan from windows

#9

if you have more than 1 JVB, each JVB must have a unique ID. You are using the “default-id” id for all JVBs probably. check the value for server-id in the jvb config:

videobridge {
  websockets {
    enabled = true
    server-id=<unique id per jvb>
  }
}

then in your nginx config, make sure you have a rule for each id:

    # colibri (JVB) websockets for jvb
    location ~ ^/colibri-ws/<unique id>/(.*) {
       proxy_pass http://<ip address of JVB>:9090/colibri-ws/<unique id>/$5$is_args$args;
       proxy_http_version 1.1;
       proxy_set_header Upgrade $http_upgrade;
       proxy_set_header Connection "upgrade";
       tcp_nodelay on;
    }
#10

Thanks, websocket isn’t sending error messages. Still problem with video, I think it’s firewall policy issue. We are waiting for administrator to come back from vacation.

#11

if you can’t change firewall, you can try to setup coturn so that jitsi-meet can use port 443. See here.