Coturn chronicles

Stage 0
My Coturn chronicles started when I realized that coturn has a problem with Let's Encrypt certificate on Debian Buster. After some tests this issue was solved and I shared the related info on this topic

Stage 1
After a while, my friend said that some participants couldn’t connect to the conference. It seemed that these participants didn’t have permission to access to UDP/10000. but didn’t they connect through the TURN server?

Stage 2
I decided to test the TURN server again. I blocked my outgoing UDP/10000 traffic

iptables -A OUTPUT -p udp --dport 10000 -j DROP

and tried to connect to the conference using three tabs on the browser for the same room. But failed…

Stage 3
As you know, typical Jitsi installation uses TCP/443 for both jitsi-meet and coturn. Nginx checks the incomming packets and directs them to the corresponding upstream according to their types.

I opened the coturn port (TCP/5349) to the public to see if it worked alone. I changed prosody config to publish the new turns port to the clients.

/etc/prosody/conf.avail/meet.mydomain.com.tr.cfg.lua

{ type = "turns", host = "meet.mydomain.com", port = "5349", transport = "tcp" }

This didn’t work on the first try. I had to edit the coturn config.
/etc/turnserver.conf

# external-ip=public-ip-address
allowed-peer-ip=local-ip-address

And finally it worked. This means that Nginx couldn’t direct the packets correctly.

Stage 4
After some research, I saw that it would be better to direct packets by host. Therefore, I needed a second host address (FQDN) and a TLS certificate for the turn server.

  • Added a DNS record for the new FQDN (let’s say turn.mydomain.com)
  • Added this FQDN to the Nginx config (needed for Let’s Encrypt)
    /etc/nginx/sites-enabled/meet.mydomain.com.conf
listen 80;
listen [::]:80;
server_name meet.mydomain.com turn.mydomain.com;
...
...
listen 4444 ssl http2;
listen [::]:4444 ssl http2;
server_name meet.mydomain.com turn.mydomain.com;
  • Created the TLS certificate
certbot certonly --agree-tos --webroot -w /usr/share/jitsi-meet -d turn.mydomain.com
  • Fixed the certificate permissions as described in here

  • Enabled the new certificate on the coturn config.
    /etc/turnserver.conf

cert=/etc/letsencrypt/live/turn.mydomain.com/fullchain.pem
pkey=/etc/letsencrypt/live/turn.mydomain.com/privkey.pem

Stage 5
It was time to edit the Nginx config. I changed the listening IP for turn. Then I commented the map block and added a new one.
/etc/nginx/modules-enabled/60-jitsi-meet.conf

    upstream turn {
        #server 127.0.0.1:5349;
        server private-ip-address:5349;
    }

    #map $ssl_preread_alpn_protocols $upstream {
    #    ~\bh2\b         web;
    #    ~\bhttp/1\.     web;
    #    default         turn;
    #}
    map $ssl_preread_server_name $upstream {
        turn.mydomain.com turn;
        default           web;
    }

coturn didn’t have to listen other than private-ip-address anymore. So I added one more line to the config too. All added/changed lines on /etc/turnserver.conf

cert=/etc/letsencrypt/live/turn.mydomain.com/fullchain.pem
pkey=/etc/letsencrypt/live/turn.mydomain.com/privkey.pem
# external-ip=public-ip-address
listening-ip=private-ip-address
allowed-peer-ip=private-ip-address
no-udp

Stage 6
I changed the prosody config to publish the new settings to the clients.

/etc/prosody/conf.avail/meet.mydomain.com.tr.cfg.lua

{ type = "turns", host = "turn.mydomain.com", port = "443", transport = "tcp" }

Stage 7
Happy end

Stage 8
12 days after the “happy end

When I tried to use the TURN server with an additional JVB, the connection was failed. The issue was fixed after changing the listening IP of the coturn server from 127.0.0.1 to
private-ip-address.

I edited the post according to the new situation.

Stage 9
I commented the external-ip line on /etc/turnserver.conf. It seems that there is no need a hardcoded server IP on the config file.

Stage 10
5 days after “stage 9

stondino00 said that he has an issue with mobile apps on a similar installation. I checked Android app through coturn and it doesn’t work. I don’t know a solution for now.

3 Likes

thank you for sharing it

This is awesome! I was going nuts trying to figure this out.

Hey @emrah

Thanks for sharing your learnings. I tried your solution on Ubuntu 18.04. I had to change the listening-ip from private-ip-address to 127.0.0.1 to make it work.

Also, this is working on Chrome but not on Firefox. Any reason why that might happen? FF complains that my STUN/TURN server has gone offline.

At first, I tried with 127.0.0.1 and it worked but after I added a second videobridge to the cluster, I saw that 127.0.0.1 didn’t work with the external videobridge.

Then I changed the listening-ip to private-ip-address. Don’t forget to change the upstream IP too on /etc/nginx/modules-enabled/60-jitsi-meet.conf

And it’s needed to restart the related services too.

I tested with FF after your post and it didn’t work on my side too. Then I made the same test on meet.jit.si and FF worked this time.

I don’t know yet which differences cause the different results

I added a missing part (stage 6) to the original post.
Setting the turn address on the prosody config.

Firefox-esr is working with my TURN server now but I didn’t change anything. I think that this is related in the restart order of the services. In my previous test, I didn’t restart prosody everytime after coturn changed.

Now I restart the services with the following order

systemctl restart coturn
systemctl restart nginx
systemctl restart prosody
systemctl restart jicofo
systemctl restart jitsi-videobridge2

I commented the external-ip line on /etc/turnserver.conf

It seems that it’s not needed on a simple config and I don’t want a hardcoded server IP in my config files.

Very reasonable. Accordingly I wrote a little shell script to make restarting a bit easier. It also prints out the line showing whether the service is running (or not) after restarting.

#!/bin/sh

print_status() {
    local name=$1
    sudo systemctl restart $name
    printf "%-8s %s\n" $name: "`sudo systemctl status $name | egrep -o 'active .*'`"
}

print_status coturn
print_status nginx
print_status prosody
print_status jicofo
print_status jitsi-videobridge2
1 Like

There is a new issue. Mobile apps don’t work through coturn with this setup. I don’t know a solution for now.

This is brilliant, many thanks for sharing!

For Ubuntu 18.04 I removed listening-ip from turnserver.conf altogether - instance here seems to be listening on 172.xx.xx.xx and 127.xx.xx.xx - defining only 127.xx.xx.xx or private-ip-address caused restricted user to be disconnected (after 10 seconds).

Guess another approach might be to set multiple listening-ip

1 Like

@emrah i was having some Nginx upstream errors so tried your Nginx Jitsi module configuration.
But as i understand, you have Nginx and Coturn in the same VM. Am i right?
In my case they are in different VMs. So set up /etc/nginx/modules-enabled/60-jitsi-meet.conf file as;

stream {
    upstream web {
        server 127.0.0.1:4444;
    }
    upstream turn {
        #server 127.0.0.1:5349;
        server COTURN_EXTERNAL_IP:443;
    }
    # since 1.13.10
    #map $ssl_preread_alpn_protocols $upstream {
    #    ~\bh2\b         web;
    #    ~\bhttp/1\.     web;
    #    default         turn;
    #}

    map $ssl_preread_server_name $upstream {
        coturn.MY_DOMAIN.com turn;
        default             web;
    }


    server {
        listen 443;
        listen [::]:443;

        # since 1.11.5
        ssl_preread on;
        proxy_pass $upstream;

        # Increase buffer to serve video
        proxy_buffer_size 10m;
    }
}

After that i am having following Nginx error.

[crit] 12451#12451: *34 SSL_do_handshake() failed (SSL: error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol) while SSL handshaking, client: 127.0.0.1, server: 0.0.0.0:4444

My coturn VM has it’s own FQDN and certs also. What do you think? What may be the problem?

You don’t need to map the requests according to the protocol or the server name since coturn and jitsi-meet are not on the same machine. All incoming TCP/443 traffic should go to the web application (jitsi-meet) in your case. Then /etc/nginx/modules-enabled/60-jitsi-meet.conf should be like that

stream {
    upstream web {
        server 127.0.0.1:4444;
    }

    server {
        listen 443;
        listen [::]:443;

        proxy_pass web;
        proxy_buffer_size 10m;
    }
}

You only need to set turn address in your /etc/prosody/conf.avail/meet.your-domain.com.cfg.lua

{ type = "turns", host = "coturn.your-domain.com", port = "443", transport = "tcp" }

And static-auth-secret (which is in /etc/turnserver.conf) and turncredentials_secret (which is in /etc/prosody/conf.avail/meet.mydomain.com.tr.cfg.lua) should be same

I assumed that they are their own IPs too. If this is not the case and they share the same IP then the map block will be

    upstream turn {
        server COTURN_VM_IP:5349;
    }

    map $ssl_preread_server_name $upstream {
        coturn.MY_DOMAIN.com turn;
        default             web;
    }

i see. But this config did not work. Because 443 is binded also in nginx conf of jitsi-meet domain.
So there wont be any configuration regarding turn in nginx jitsi module config file also any https binding. Which means this file become useless. Am i right?

Are they sharing the same public IP?
Is coturn listening TCP/5349?

No. They both have their own external ip. They are in different vms. i am trying to use 443 for jitsi-meet instead of default 4444. So i needed to edit nginx conf of my jitsi domain. But adding a server with same port ends up with nginx error for binding the same port which is normal. So i needed to remove server definition in jitsi nginx module config. Which makes this file useless. Right?

You only need to change /etc/nginx/modules-enabled/60-jitsi-meet.conf. Don’t change the site config. And /etc/nginx/modules-enabled/60-jitsi-meet.conf should be like that (this is the whole file, no other lines)

stream {
    upstream web {
        server 127.0.0.1:4444;
    }

    server {
        listen 443;
        listen [::]:443;

        ssl_preread on;
        proxy_pass web;
        proxy_buffer_size 10m;
    }
}

i see. This will also work. By this way what we do is Jitsi meet still listens 4444 any request coming to 443 is proxied to 4444. Right?