Problem with coturn on new jvb2 installation on Debian buster

On a fresh install, we encounter problem starting the coturn server and, when started manually, coturn seems to have “authentication problem”.

On a new install, the traffic to the turn service is handled by nginx by default. This is defined in /etc/nginx/modules-enabled/60-jitsi-meet.conf:

# this is jitsi-meet nginx module configuration
# this forward all http traffic to the nginx virtual host port
# and the rest to the turn server

stream {
    upstream web {
        server 127.0.0.1:4444;
    }
    upstream turn {
        server 127.0.0.1:4445;
    }
    # since 1.13.10
    map $ssl_preread_alpn_protocols $upstream {
        "h2"            web;
        "http/1.1"      web;
        "h2,http/1.1"   web;
        default         turn;
    }

    server {
        listen 443;

        # since 1.11.5
        ssl_preread on;
        proxy_pass $upstream;

        # Increase buffer to serve video
        proxy_buffer_size 10m;
    }
}

Problèm 1: the turn server does not start. The port 4445 is not listening:

$ sudo netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:5347          0.0.0.0:*               LISTEN      21850/lua5.2        
tcp        0      0 0.0.0.0:5222            0.0.0.0:*               LISTEN      21850/lua5.2        
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      11279/nginx: master 
tcp        0      0 0.0.0.0:5269            0.0.0.0:*               LISTEN      21850/lua5.2        
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      2790/sshd           
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      11279/nginx: master 
tcp        0      0 0.0.0.0:4444            0.0.0.0:*               LISTEN      11279/nginx: master 
tcp        0      0 0.0.0.0:5280            0.0.0.0:*               LISTEN      21850/lua5.2        
tcp6       0      0 ::1:5347                :::*                    LISTEN      21850/lua5.2        
tcp6       0      0 :::5222                 :::*                    LISTEN      21850/lua5.2        
tcp6       0      0 :::80                   :::*                    LISTEN      11279/nginx: master 
tcp6       0      0 :::5269                 :::*                    LISTEN      21850/lua5.2        
tcp6       0      0 :::22                   :::*                    LISTEN      2790/sshd           
tcp6       0      0 :::8888                 :::*                    LISTEN      5209/java           
tcp6       0      0 :::4444                 :::*                    LISTEN      11279/nginx: master 
tcp6       0      0 :::5280                 :::*                    LISTEN      21850/lua5.2

I adapt the service definition manually for a more verbose logging:

debian@test-visio-443:~$ diff /etc/systemd/system/coturn.service /lib/systemd/system/coturn.service 
12c12
< ExecStart=/usr/bin/turnserver -c /etc/turnserver.conf --pidfile /run/turnserver/turnserver.pid -v
---
> ExecStart=/usr/bin/turnserver --daemon -c /etc/turnserver.conf --pidfile /run/turnserver/turnserver.pid

And I get those message:

Mar 31 15:56:00 test-visio-443 turnserver[24796]: bind: Permission denied
Mar 31 15:56:00 test-visio-443 turnserver[24796]: Cannot bind local socket to addr: Permission denied
  1. Problem when coturn auth

I started coturn manually, as root:

turnserver -c /etc/turnserver.conf --daemon -v

Inspecting the log, I get those message when I connect a client which cannot reach server using udp:

121: IPv4. tcp or tls connected to: 127.0.0.1:45092
121: session 001000000000000002: realm <visio443.xxxx> user <>: incoming packet message processed, error 401: Unauthorized
121: IPv4. Local relay addr: 127.0.0.1:51253
121: session 001000000000000002: new, realm=<visio443xxxx>, username=<1585758001>, lifetime=3600, cipher=TLS_AES_256_GCM_SHA384, method=UNKNOWN
121: session 001000000000000002: realm <visio443.xxxx.be> user <1585758001>: incoming packet ALLOCATE processed, success

And the client can’t send audio or video stream.

What disturbs me is the error 401: Unauthorized which appears in the log. I also get this message when starting turnserver:

CONFIGURATION ALERT: You specified --lt-cred-mech and --use-auth-secret in the same time.
Be aware that you could not mix the username/password and the shared secret based auth methohds. 
Shared secret overrides username/password based auth method. Check your configuration!

and this error:

: ERROR: set_ctx: ERROR: cannot set DH

The turn server config file:

root@test-visio-443:/home/debian# cat /etc/turnserver.conf 
# jitsi-meet coturn config. Do not modify this line
lt-cred-mech
use-auth-secret
keep-address-family
static-auth-secret=.........
realm=visio443.xxxx.be
cert=/etc/letsencrypt/live/visio443.xxxx.be/fullchain.pem
pkey=/etc/letsencrypt/live/visio443.xxxx.be/privkey.pem

no-tcp
listening-port=443
tls-listening-port=4445
external-ip=visio443.xxxxxx.be

I’m not sure it’s related, but upgrading to last stable (4335) breaks JVB. Homepage is no longer responding. Everything was fine in 4101.
I tried a fresh install, same issue. Even install-letsencrypt-cert.sh is faiilling (as it can’t reach port 80).

Take a look into the two interfering definitions for port 443.

The directories below /etc/nginx contain: One definition in directory modules-enabled and the other in sites-enabled.

Thanks for your replys. This is a fresh install, not an upgrade.

It seems to me that definitions does not interferes in nginx. The definition in sites-enabled does listen on port 4444, which is correct in definition in /etc/nginx/modules-enabled/60-jitsi-meet.conf and paste below. It seems that /etc/nginx/modules-enabled/60-jitsi-meet.conf will redirect traffic to port 4444 if needed.

Does jitsi-videobridge2 still embed a TURN server ? should the TURN traffic going to coturn ?

JVB by itself is a relay, but it is not a turn server. All udp traffic goes to jvb, if udp is blocked or the firewall allows only https traffic (port 443) for clients, the connection will fallback to using tcp through Nginx, then true turnserver. This is the same setup we run for meet.jit.si, but on the same machine.
For meet.jit.si we run coturn servers on their own public addresses, here in order to use one DNS and one public address for two services that require port 443 we front them with Nginx and multiplex the traffic.

Thanks. This is what I understood.

But should I see port 4445 to be listening using netstat? Why does coturn seems to remain down, unless it appers started in systemd? Why those error message ?

I also noticed that secrets are not the same in prosody config and coturn config. Is this a problem?

I ran instances of jitsi meet without problem before version 2 (except behind firewall which does notaccepte udp traffic). Since version 2 install seems broken. I am trying to understand and I wish to help (and maybe improve documentation).

If it is about the turnconfig secrets, that will be a problem … hum wonder how did you get in that state …

I did a reinstall from a fresh Debian Buster. The only two commands I run are:

sudo apt install jitsi-meet
sudo /usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh

I am sorry: the secrets are not the same between the file /etc/jitsi/videobridge/config (value JVB_SECRETS) and /etc/turnserver.conf (value static-auth-secret), but are identical between /etc/turnserver.conf (value static-auth-secret) and cat /etc/prosody/conf.avail/visio443.xxx.be.cfg.lua (value turncredentials_secret).

But, on a fresh install on Debian buster:

  • the service coturn is started, but the port 4445 is not LISTENING:
$ netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      14191/nginx: master 
tcp        0      0 0.0.0.0:4444            0.0.0.0:*               LISTEN      14191/nginx: master 
tcp        0      0 0.0.0.0:5280            0.0.0.0:*               LISTEN      14146/lua5.2        
tcp        0      0 127.0.0.1:5347          0.0.0.0:*               LISTEN      14146/lua5.2        
tcp        0      0 0.0.0.0:5222            0.0.0.0:*               LISTEN      14146/lua5.2        
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      14191/nginx: master 
tcp        0      0 0.0.0.0:5269            0.0.0.0:*               LISTEN      14146/lua5.2        
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      9009/sshd           
tcp6       0      0 :::4444                 :::*                    LISTEN      14191/nginx: master 
tcp6       0      0 :::5280                 :::*                    LISTEN      14146/lua5.2        
tcp6       0      0 ::1:5347                :::*                    LISTEN      14146/lua5.2        
tcp6       0      0 :::5222                 :::*                    LISTEN      14146/lua5.2        
tcp6       0      0 :::80                   :::*                    LISTEN      14191/nginx: master 
tcp6       0      0 :::5269                 :::*                    LISTEN      14146/lua5.2        
tcp6       0      0 :::22                   :::*                    LISTEN      9009/sshd           
tcp6       0      0 :::8888                 :::*                    LISTEN      11797/java          
$ sudo systemctl status coturn
● coturn.service - coTURN STUN/TURN Server
   Loaded: loaded (/lib/systemd/system/coturn.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2020-04-01 08:37:41 UTC; 12s ago
     Docs: man:coturn(1)
           man:turnadmin(1)
           man:turnserver(1)
  Process: 16438 ExecStart=/usr/bin/turnserver --daemon -c /etc/turnserver.conf --pidfile /run/turnserver/turnserver.p
  Process: 16440 ExecStartPost=/bin/sleep 2 (code=exited, status=0/SUCCESS)
 Main PID: 16439 (turnserver)
    Tasks: 3 (limit: 4915)
   Memory: 3.9M
   CGroup: /system.slice/coturn.service
           └─16439 /usr/bin/turnserver --daemon -c /etc/turnserver.conf --pidfile /run/turnserver/turnserver.pid
  1. When i start the server manually (turnserver -c /etc/turnserver.conf -v), I get a message Error 401. I do not know if this is an issue. And video/audio does not work:
730: IPv4. tcp or tls connected to: 127.0.0.1:55898
730: session 001000000000000002: realm <visio443.xx.be> user <>: incoming packet message processed, error 401: Unauthorized
730: IPv4. Local relay addr: 127.0.0.1:49753
730: session 001000000000000002: new, realm=<visio443.xxxs.be>, username=<1585817534>, lifetime=3600, cipher=TLS_AES_256_GCM_SHA384, method=UNKNOWN
730: session 001000000000000002: realm <visio443.xxx.be> user <1585817534>: incoming packet ALLOCATE processed, success
1330: session 001000000000000002: TLS/TCP socket disconnected: 127.0.0.1:55898
1330: session 001000000000000002: usage: realm=<visio443.xxx.be>, username=<1585817534>, rp=2, rb=180, sp=2, sb=248

I think something is broken on the installation since jitsi-videobridge2.

I would be happy to help if needed.

Ok, I could get the connection to coturn server working using Chromium. Firefox ESR seems broken when connecting with turnserver (it works when connecting by udp). I do not have the chance to test with the current version.

So, I could identify two bugs:

  • the server coturn did not start because it try to listen on port 443. It works when started “by hand” as root.
  • with firefox ESR, establishing a connection through port 443 and TURN fails (it works when using a connection through port udp 10000

(NOTICE: a previous post does not appears and is blocked by Akismet)

I filled a bug here: https://github.com/jitsi/jitsi-meet/issues/5529