2 JVBs are connected in a cluster; Getting WARN: Received request for an nonexistent conference

  1. Hi,

I followed this tutorial: JItsi Meet Multiple Videobridges with Load Balancing – Koffee With Kode to configure 2 JVBs in the same AWS VPC, same AWS Subnet. When i have multiple conferences comingup i see the conference ids getting round-robin between the JVBs. No issues observed so-far on the WEBRTC front.

OS details
OS: Debian 10 buster
Jitsi-meet : latest stable
JVB-1: on the same machine we have prosody, jicofo,
JVB-2: on a new AWS Ec2 machine. Same VPC, Same subnet as JVB-1 machine
Hosted on AWS, EC2. C5.x large

Case 1 :
However when the conference gets started and attached to JVB-2; i see the below error on JVB-1 (/var/log/jitsi/jvb.log )

JVB 2021-12-20 14:08:36.411 WARNING: [38] ColibriWebSocketServlet.createWebSocket#135: Received request for an nonexistent conference: a5f0302a13c62f0b
JVB 2021-12-20 14:09:15.214 WARNING: [32] ColibriWebSocketServlet.createWebSocket#135: Received request for an nonexistent conference: a5f0302a13c62f0b

The above log comes in continuously on JVB-1 for ever. I want to know if this is something i should be worried about? I see the above conference id to be available on JVB-2. Every time JVB-2 hosts the conference the JVB-1 always sends the above warnings

Case 2:
When the conference gets started on JVB-1 ; i dont see the above error in JVB-1 ; or JVB-2

Configurations JVB-1

root@ip-172-31-91-215:/home/admin# cat /etc/jitsi/videobridge/sip-communicator.properties 
org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true
org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=meet-jit-si-turnrelay.jitsi.net:443
org.jitsi.videobridge.ENABLE_STATISTICS=true
org.jitsi.videobridge.STATISTICS_TRANSPORT=muc
org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=localhost
org.jitsi.videobridge.xmpp.user.shard.DOMAIN=auth.example.abc.live
org.jitsi.videobridge.xmpp.user.shard.USERNAME=jvb
org.jitsi.videobridge.xmpp.user.shard.PASSWORD=XXXXLtYj
org.jitsi.videobridge.xmpp.user.shard.MUC_JIDS=JvbBrewery@internal.auth.example.abc.live
org.jitsi.videobridge.xmpp.user.shard.MUC_NICKNAME=jvb1
org.jitsi.videobridge.DISABLE_TCP_HARVESTER=true
org.jitsi.videobridge.xmpp.user.shard.DISABLE_CERTIFICATE_VERIFICATION=true
root@ip-172-31-91-215:/home/admin# cat  /etc/jitsi/videobridge/config 
# Jitsi Videobridge settings

# sets the XMPP domain (default: none)
JVB_HOSTNAME=example.abc.live

# sets the hostname of the XMPP server (default: domain if set, localhost otherwise)
JVB_HOST=

# sets the port of the XMPP server (default: 5275)
JVB_PORT=5347

# sets the shared secret used to authenticate to the XMPP server
JVB_SECRET=XXXXLtYj


# adds java system props that are passed to jvb (default are for home and logging config file)
JAVA_SYS_PROPS="-Dconfig.file=/etc/jitsi/videobridge/jvb.conf -Dnet.java.sip.communicator.SC_HOME_DIR_LOCATION=/etc/jitsi -Dnet.java.sip.communicator.SC_HOME_DIR_NAME=videobridge -Dnet.java.sip.communicator.SC_LOG_DIR_LOCATION=/var/log/jitsi -Djava.util.logging.config.file=/etc/jitsi/videobridge/logging.properties"
root@ip-172-31-91-215:/home/admin# 

Configurations JVB-2

root@ip-172-31-84-154:/home/admin# cat /etc/jitsi/videobridge/sip-communicator.properties 
org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true
org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=meet-jit-si-turnrelay.jitsi.net:443
org.jitsi.videobridge.ENABLE_STATISTICS=true
org.jitsi.videobridge.STATISTICS_TRANSPORT=muc
org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=example.abc.live
org.jitsi.videobridge.xmpp.user.shard.DOMAIN=auth.example.abc.live
org.jitsi.videobridge.xmpp.user.shard.USERNAME=jvb
org.jitsi.videobridge.xmpp.user.shard.PASSWORD=XXXXLtYj
org.jitsi.videobridge.xmpp.user.shard.MUC_JIDS=JvbBrewery@internal.auth.example.abc.live
org.jitsi.videobridge.xmpp.user.shard.MUC_NICKNAME=jvb2
org.jitsi.videobridge.DISABLE_TCP_HARVESTER=true
org.jitsi.videobridge.xmpp.user.shard.DISABLE_CERTIFICATE_VERIFICATION=true
root@ip-172-31-84-154:/home/admin# cat /etc/jitsi/videobridge/config 
# Jitsi Videobridge settings

# sets the XMPP domain (default: none)
JVB_HOSTNAME=example.abc.live

# sets the hostname of the XMPP server (default: domain if set, localhost otherwise)
JVB_HOST=

# sets the port of the XMPP server (default: 5275)
JVB_PORT=5347

# sets the shared secret used to authenticate to the XMPP server
JVB_SECRET=XXXXLtYj


# adds java system props that are passed to jvb (default are for home and logging config file)
JAVA_SYS_PROPS="-Dconfig.file=/etc/jitsi/videobridge/jvb.conf -Dnet.java.sip.communicator.SC_HOME_DIR_LOCATION=/etc/jitsi -Dnet.java.sip.communicator.SC_HOME_DIR_NAME=videobridge -Dnet.java.sip.communicator.SC_LOG_DIR_LOCATION=/var/log/jitsi -Djava.util.logging.config.file=/etc/jitsi/videobridge/logging.properties"
root@ip-172-31-84-154:/home/admin# 

Masked few fields for privacy.

Questions

  1. Should i be worried about the warning i get in JVB-1.
  2. Why are we getting the warnings only on jvb1.
  3. Does it mean if i scale the jvb-s horizontally is it normal to see these warning across?

Because websocket is not configured correctly for JVB2

On jvb-2

i added the given nginx block into : /etc/nginx/available-sites/default ; But i think i am missing colibri-ws folder. should i copy it from somewhere?

On my EC2s, all ports are open to the world.

No, that block should go into your nginx config on your main Jitsi server.

like the following

Hi,

the issue still persists. Below is the nginx config file

on JVB-1 (which has prosody, nginx, jicofo ec)

root@ip-172-31-91-215:/home/admin# cat /etc/nginx/sites-enabled/example.abc.live.conf 
server_names_hash_bucket_size 64;

types {
# nginx's default mime.types doesn't include a mapping for wasm
    application/wasm     wasm;
}
server {
    listen 80;
    listen [::]:80;
    server_name example.abc.live;

    location ^~ /.well-known/acme-challenge/ {
        default_type "text/plain";
        root         /usr/share/jitsi-meet;
    }
    location = /.well-known/acme-challenge/ {
        return 404;
    }
    location / {
        return 301 https://$host$request_uri;
    }
}
server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name example.abc.live;

    # Mozilla Guideline v5.4, nginx 1.17.7, OpenSSL 1.1.1d, intermediate configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
    ssl_prefer_server_ciphers off;

    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:10m;  # about 40000 sessions
    ssl_session_tickets off;

    add_header Strict-Transport-Security "max-age=63072000" always;
    set $prefix "";

    ssl_certificate /etc/letsencrypt/live/example.abc.live/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.abc.live/privkey.pem;

    root /usr/share/jitsi-meet;

    # ssi on with javascript for multidomain variables in config.js
    ssi on;
    ssi_types application/x-javascript application/javascript;

    index index.html index.htm;
    error_page 404 /static/404.html;

    gzip on;
    gzip_types text/plain text/css application/javascript application/json image/x-icon application/octet-stream application/wasm;
    gzip_vary on;
    gzip_proxied no-cache no-store private expired auth;
    gzip_min_length 512;

    location = /config.js {
        alias /etc/jitsi/meet/example.abc.live-config.js;
    }

    location = /external_api.js {
        alias /usr/share/jitsi-meet/libs/external_api.min.js;
    }

    # ensure all static content can always be found first
    location ~ ^/(libs|css|static|images|fonts|lang|sounds|connection_optimization|.well-known)/(.*)$
    {
        add_header 'Access-Control-Allow-Origin' '*';
        alias /usr/share/jitsi-meet/$1/$2;

        # cache all versioned files
        if ($arg_v) {
            expires 1y;
        }
    }

    # BOSH
    location = /http-bind {
        proxy_pass http://127.0.0.1:5280/http-bind?prefix=$prefix&$args;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $http_host;
    }

    # xmpp websockets
    location = /xmpp-websocket {
        proxy_pass http://127.0.0.1:5280/xmpp-websocket?prefix=$prefix&$args;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $http_host;
        tcp_nodelay on;
    }

    # colibri (JVB) websockets for jvb1
    location ~ ^/colibri-ws/default-id/(.*) {
        proxy_pass http://127.0.0.1:9090/colibri-ws/default-id/$1$is_args$args;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        tcp_nodelay on;
    }

location ~ ^/colibri-ws/([0-9.]*)/(.*) {
        proxy_pass http://$1:9090/colibri-ws/$1/$2$is_args$args;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        tcp_nodelay on;
    }

    # load test minimal client, uncomment when used
    #location ~ ^/_load-test/([^/?&:'"]+)$ {
    #    rewrite ^/_load-test/(.*)$ /load-test/index.html break;
    #}
    #location ~ ^/_load-test/libs/(.*)$ {
    #    add_header 'Access-Control-Allow-Origin' '*';
    #    alias /usr/share/jitsi-meet/load-test/libs/$1;
    #}

    location ~ ^/([^/?&:'"]+)$ {
        try_files $uri @root_path;
    }

    location @root_path {
        rewrite ^/(.*)$ / break;
    }

    location ~ ^/([^/?&:'"]+)/config.js$
    {
        set $subdomain "$1.";
        set $subdir "$1/";

        alias /etc/jitsi/meet/example.abc.live-config.js;
    }

    # BOSH for subdomains
    location ~ ^/([^/?&:'"]+)/http-bind {
        set $subdomain "$1.";
        set $subdir "$1/";
        set $prefix "$1";

        rewrite ^/(.*)$ /http-bind;
    }

    # websockets for subdomains
    location ~ ^/([^/?&:'"]+)/xmpp-websocket {
        set $subdomain "$1.";
        set $subdir "$1/";
        set $prefix "$1";

        rewrite ^/(.*)$ /xmpp-websocket;
    }

    # Anything that didn't match above, and isn't a real file, assume it's a room name and redirect to /
    location ~ ^/([^/?&:'"]+)/(.*)$ {
        set $subdomain "$1.";
        set $subdir "$1/";
        rewrite ^/([^/?&:'"]+)/(.*)$ /$2;
    }
}

Did you restart nginx?

Yes. Several times. service nginx restart

did you update /etc/jitsi/videobridge/jvb.conf on JVB2?

Sorry forgot updating jvb.conf. Now things are all good