I have some doubts that my turnserver is running as TURNS

As subject says: Here my observations so far:

a) In a two-party connection both parties have this iceServerConfig (w.r.t. the server):

There is no relay candidate in the list, however, those are not required so far.

b) The turnserver is started, /etc/turnserver.conf is as it has been generated at installation time

ubuntu@jitsi-meet:~$ ps aux | grep turnserver
turnser+    4832  0.1  0.2 633208  9604 ?        Ssl  13:32   0:00 /usr/bin/turnserver --daemon -c /etc/turnserver.conf --pidfile /run/turnserver/turnserver.pid
ubuntu@jitsi-meet:~$ cat /run/turnserver/turnserver.pid
4832

c) The turnserver.conf has no-tcp specified, what means IMHO, that no TCP listener is started (??)(commenting that doesn’t help either)

d) I can’t see a listener on this port (sudo netstat -tupan) - ok, that can be an NGINX trick :slight_smile:

e) The firewall is open and the port forwarding works. I can capture packets on the jitsi box

e) Once I run the trickle ice test against this URL the TCP connection or jitsi-meet I only see “TCP RESTARTS” on every incoming “SYNC” request…

f) The UDP TURN instance at 3479 works, although it delivers internal addresses as TURN adresses (but this might be the way you are using TURN):

Can somebody explain to me, why the TURNS instance is not working?

Anybody to help please?

Are you running the turnserver with valid certificates?

Thanks for answering.

Using the same as I’m using for NGINX:

From NGINX:

ssl_certificate /etc/letsencrypt/live/jitsi-meet.ddns.net/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/jitsi-meet.ddns.net/privkey.pem;

From turnserver.conf:

cert=/etc/letsencrypt/live/jitsi-meet.ddns.net/fullchain.pem
pkey=/etc/letsencrypt/live/jitsi-meet.ddns.net/privkey.pem

Honestly, I doubt this to be an issue with the cert. The TCP connection not even reaches that state. It is cleared right after the SYNC.

refused connection it seems. If I don’t forget something, could be firewall, application denial (may be logged by coturn ?) or a martian problem. Never forget martians, they are crafty and hide often in convoluted networks. Log them.

Never forget martians, they are crafty and hide often in convoluted networks. Log them.

Hehe… What? :slight_smile:

As said, firewall open, locally and remote. Checked with telnet from the outside. Arrives, but is answered with TCP RST.

application denial (may be logged by coturn ?) o

Well, I tried. /etc/turnserver.conf enables syslog, that is not too chatty. I remember, my COTURN instances jabbered like hell, but there - nothing at all. Tried to log to file to no avail, it seems, it is really not arriving at COTURN.

The fact, that there is nothing listening on port 5349 inside the box isn’t suspicious? How does that come? Also the turnserver.conf setting no-tcp?

Did launch this little test app, once on 8443, once on 5349. Note the different error messages. There is no listener at 5349

import http.server, ssl

server_address = ('localhost', xxxxxx)
httpd = http.server.HTTPServer(server_address, http.server.SimpleHTTPRequestHandler)
httpd.socket = ssl.wrap_socket(httpd.socket,
                               server_side=True,
                               certfile='localhost.pem',
                               ssl_version=ssl.PROTOCOL_TLS)
httpd.serve_forever()

For 8443:

`OSError: [Errno 98] Address already in use``

For 5349:

FileNotFoundError: [Errno 2] No such file or directory

The latter error is for the non-existent pem file

No-tcp is to not have the turn tcp without tls. Is your config using this template jitsi-meet/turnserver.conf at master · jitsi/jitsi-meet · GitHub?
Coturn prints the ports it listens to when starting up. You can check the command line used in the processes and stop and start it from command line to see the logs it prints on startup and why it doesn’t bind to the default tls port

Good idea. Thanks for the explanation. And yes - it is exactly the turnserver.conf you are creating from the quoted template. One minute…

Wow…

==== Show him the instruments, Practical Frost: ====

0: TLS supported
0: DTLS supported
0: DTLS 1.2 supported
0: TURN/STUN ALPN supported
0: Third-party authorization (oAuth) supported
0: GCM (AEAD) supported
0: OpenSSL compile-time version: OpenSSL 1.1.1f  31 Mar 2020 (0x1010106f)
0: 
0: SQLite supported, default database location is /var/lib/turn/turndb
0: Redis supported
0: PostgreSQL supported
0: MySQL supported
0: MongoDB is not supported
0: 
0: Default Net Engine version: 3 (UDP thread per CPU core)

=====================================================

0: Domain name: 
0: Default realm: jitsi-meet.ddns.net
0: 
CONFIG: --no-tcp-relay: TCP relay endpoints are not allowed.
0: ERROR: 
CONFIG ERROR: Empty cli-password, and so telnet cli interface is disabled! Please set a non empty cli-password!
0: WARNING: cannot find certificate file: /etc/letsencrypt/live/jitsi-meet.ddns.net/fullchain.pem (1)
0: WARNING: cannot start TLS and DTLS listeners because certificate file is not set properly
0: WARNING: cannot find private key file: /etc/letsencrypt/live/jitsi-meet.ddns.net/privkey.pem (1)
0: WARNING: cannot start TLS and DTLS listeners because private key file is not set properly
0: NO EXPLICIT LISTENER ADDRESS(ES) ARE CONFIGURED
0: ===========Discovering listener addresses: =========
0: Listener address to use: 127.0.0.1
0: Listener address to use: 192.168.190.25
0: Listener address to use: ::1
0: Listener address to use: 2003:c1:5f3b:4300:204:4bff:feec:39dc
0: =====================================================
0: Total: 2 'real' addresses discovered
0: =====================================================
0: NO EXPLICIT RELAY ADDRESS(ES) ARE CONFIGURED
0: ===========Discovering relay addresses: =============
0: Relay address to use: 192.168.190.25
0: Relay address to use: 2003:c1:5f3b:4300:204:4bff:feec:39dc
0: =====================================================
0: Total: 2 relay addresses discovered
0: =====================================================

I need to ls with sudo, otherwise permission denied…

ubuntu@jitsi-meet:~$ sudo ls -lall /etc/letsencrypt/live/jitsi-meet.ddns.net/fullchain.pem
lrwxrwxrwx 1 root root 48 Aug 27 18:53 /etc/letsencrypt/live/jitsi-meet.ddns.net/fullchain.pem -> ../../archive/jitsi-meet.ddns.net/fullchain1.pem

ubuntu@jitsi-meet:~$ sudo ls -lall /etc/letsencrypt/live/jitsi-meet.ddns.net/privkey.pem
lrwxrwxrwx 1 root root 46 Aug 27 18:53 /etc/letsencrypt/live/jitsi-meet.ddns.net/privkey.pem -> ../../archive/jitsi-meet.ddns.net/privkey1.pem

PS: I have not patched the turnserver.conf in any way…

Now also nginx :slight_smile:

ubuntu@jitsi-meet:~$ nginx -t

nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)

2021/08/30 23:47:07 [warn] 15721#15721: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:1

2021/08/30 23:47:07 [emerg] 15721#15721: cannot load certificate "/etc/letsencrypt/live/jitsi-meet.ddns.net/fullchain.pem": BIO_new_file() failed (SSL: error:0200100D:system library:fopen:Permission denied:fopen('/etc/letsencrypt/live/jitsi-meet.ddns.net/fullchain.pem','r') error:2006D002:BIO routines:BIO_new_file:system lib)

nginx: configuration file /etc/nginx/nginx.conf test failed

Only certbot seems to not have a problem…

ubuntu@jitsi-meet:~$ sudo certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/jitsi-meet.ddns.net.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not yet due for renewal

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

The following certs are not due for renewal yet:
  /etc/letsencrypt/live/jitsi-meet.ddns.net/fullchain.pem expires on 2021-11-25 (skipped)
No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ubuntu@jitsi-meet:~$

Cannot read the certs … There was a problem with the certs and permissions and let’s encrypt … And we did a workaround for it jitsi-meet/coturn-certbot-deploy.sh at eb4fff773b3ea482513eafd21580d5a0b3f8b023 · jitsi/jitsi-meet · GitHub
We are making copy of the certs …

That is strange indeed…
not sure what can cause this …

FYI: I was not using your certificate creation process, since I could not forward port 80 for the ACME check. So I created my own certs and checked them with a DNS record. That worked. The given directory is the initial deployment directory of certbot…

EDIT: … and I provided the location to the jitsi setup

The link points to a relative directory. What is the absolute path? From what directory is COTURN usually started with this config?

Make sure coturn can read the certs … I had hard time figuring that kind of problem with lets encrypt …

journalctl confirms the problems


Aug 30 23:58:23 jitsi-meet.ddns.net turnserver[4650]: 0: WARNING: cannot find certificate file: /etc/letsencrypt/live/jitsi-meet.ddns.net/fullchain.p>
Aug 30 23:58:23 jitsi-meet.ddns.net turnserver[4650]: 0: WARNING: cannot start TLS and DTLS listeners because certificate file is not set properly
Aug 30 23:58:23 jitsi-meet.ddns.net turnserver[4650]: 0: WARNING: cannot find private key file: /etc/letsencrypt/live/jitsi-meet.ddns.net/privkey.pem>


Aug 30 23:58:23 jitsi-meet.ddns.net systemd[1]: Failed to start coTURN STUN/TURN Server.

Your advise leaves me a bit clueless. How could I do that? I first created the certificate, then installed jitsi.

I for sure also did things like nginx -t or nginx -s reload once during the install, and I never noticed the problems I have now, out of the sudden…

ubuntu@jitsi-meet:~$ nginx -s reload
nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied)
2021/08/31 00:04:04 [warn] 6639#6639: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:1
2021/08/31 00:04:04 [emerg] 6639#6639: cannot load certificate "/etc/letsencrypt/live/jitsi-meet.ddns.net/fullchain.pem": BIO_new_file() failed (SSL: error:0200100D:system library:fopen:Permission denied:fopen('/etc/letsencrypt/live/jitsi-meet.ddns.net/fullchain.pem','r') error:2006D002:BIO routines:BIO_new_file:system lib)

I also not wilingly or knowingly started a script accidently which could cause that mess…

Again: What is the absolute path of the directory, which is now symlinked with the original files?

OK, having removed the symlink thing and recreated the certifcate/key in place. It seems to be a permission issue: NGINX is for sure started with sudo access, right? Otherwise you wouldn’t be able to claim port 443. And if I launch the turnserver with sudo everything is fine. Could this be the problem?

Finally:

sudo chmod 755 /etc/letsencrypt/live/

is a very helpful command :slight_smile:

Now I’m also having a listener at 5349.

Thanks for the pointers.