UDP port 10000 blocked behind corporate firewall - possible approaches

I try to set up turnserver from one of your instructions, in sub.domain.cfg.lua, I set:

turncredentials = {
{ type = “stun”, host = “sub.domain”, port = “443” },
{ type = “turn”, host = “sub.domain”, port = “443”, transport = “udp” },
{ type = “turns”, host = “sub.domain”, port = “443”, transport = “tcp” }
};

cross_domain_bosh = false;

In sub.domain.config.js I put useStunTurn: true,

Turnserver is on the same subdomain and the same server as videobridge and other jitsi apps.

Is that correct setup?

is your turnserver up and running?
do you see turn in webrtc-internals?

not sure exactly how to check it?

I found this in webrtc-internals:

https://sub.domain/test, { iceServers: [stun:stun.l.google.com:19302, stun:stun1.l.google.com:19302, stun:stun2.l.google.com:19302], iceTransportPolicy: all, bundlePolicy: max-bundle, rtcpMuxPolicy: require, iceCandidatePoolSize: 0, sdpSemantics: “plan-b” }, {advanced: [{googHighStartBitrate: {exact: 0}}, {googPayloadPadding: {exact: true}}, {googScreencastMinBitrate: {exact: 100}}, {googCpuOveruseDetection: {exact: true}}, {googCpuOveruseEncodeUsage: {exact: true}}, {googCpuUnderuseThreshold: {exact: 55}}, {googCpuOveruseThreshold: {exact: 85}}]}

Did you install a TURN-server? It is not automatically installed with jitsi per default.

in case you installed coturn, you can check the status with “service coturn status” and then you see if it is running

Thanks.

> "I did not install the coturn server.
*> *
> Is it good to install it on the same server or on separate ones?
*> *
> I read somewhere that it is good that coturn server is on port 443, but on this server jitsi and videobridge already use port 443?"

I installed dedicated coturn server on coturn.domain port 443

When I run “service coturn status”, I get active run, everything works fine.

I set the settings for server:

  • lt-cred-mech

  • use-auth-secret

  • static-auth-secret=SomeSecretSharedBetweenProsodyAnbCoturn

  • realm=coturn.domain

  • listening-port=443

  • cert=some_valid_cert.crt

  • pkey=some_valid_cert.key

In prosody I have installed module mod_turncredentials.lua

And in domain.cfg.lua I set:

turncredentials_secret = “secret”;

turncredentials = {

{ type = “stun”, host = “coturn.domain”, port = “443” },

{ type = “turn”, host = “coturn.domain”, port = “443”, transport = “udp” },

{ type = “turns”, host = “coturn.domain”, port = “443”, transport = “tcp” }

};

I still don’t see my turn server in webrtc-internals.

What about the module turncredentials is it loaded? Are you using secure-domain to request username and password for your deployment? If that is the case, make sure you enable the module and under the guest domain.

Thanks,
that’s it.

I put turncredentials in to guest virtual host, and it works now.

https://domain/test, { iceServers: [turns:coturn.domain:443?transport=tcp], iceTransportPolicy: all, bundlePolicy: max-bundle, rtcpMuxPolicy: require, iceCandidatePoolSize: 0, sdpSemantics: “plan-b” }, {advanced: [{googHighStartBitrate: {exact: 0}}, {googPayloadPadding: {exact: true}}, {googScreencastMinBitrate: {exact: 100}}, {googCpuOveruseDetection: {exact: true}}, {googCpuOveruseEncodeUsage: {exact: true}}, {googCpuUnderuseThreshold: {exact: 55}}, {googCpuOveruseThreshold: {exact: 85}}]}

Thank you very much

I have to pick this issue up again.

We have the same problem here: everything works fine except with guests who want to join behind a corporate firewall (only TCP 80/443 outgoing is allowed).

We are running the latest stable version of jitsi-meet and use a simple default installation (everything on one server: meet, jvb, nginx, turnserver). We also use secure domains and added the turncredentials modules to the guest virtual host in the prosody configuration.

chrome://webrtc-internals on the client behind the firewall looks fine (as far as I can tell)

https://host.domain.tld/room, { iceServers: [turns:host.domain.tld:443?transport=tcp], iceTransportPolicy: all, bundlePolicy: max-bundle, rtcpMuxPolicy: require, iceCandidatePoolSize: 0, sdpSemantics: "plan-b" }, {advanced: [{googHighStartBitrate: {exact: 0}}, {googPayloadPadding: {exact: true}}, {googScreencastMinBitrate: {exact: 100}}, {googCpuOveruseDetection: {exact: true}}, {googCpuOveruseEncodeUsage: {exact: true}}, {googCpuUnderuseThreshold: {exact: 55}}, {googCpuOveruseThreshold: {exact: 85}}]}

Nevertheless there is no audio/video from the guest.

Check turrnserver logs did it correctly loaded the certificates?
Have you tried with those clients to use meet.jit.si does it work?

The turnserver logfile doesn’t look too bad to me. As far as I can see there is only one message (Bad configuration format: keep-address-family) which looks like an “error”

Apr 16 04:59:27 jitsi systemd[1]: Starting LSB: coturn TURN Server...
Apr 16 04:59:27 jitsi coturn[26336]:  * Starting coturn  turnserver
Apr 16 04:59:27 jitsi turnserver: 0: Bad configuration format: keep-address-family
Apr 16 04:59:27 jitsi turnserver: 0: Domain name:
Apr 16 04:59:27 jitsi turnserver: 0: Default realm: meet.domain.tld
Apr 16 04:59:27 jitsi turnserver: 0: #012CONFIG ERROR: Empty cli-password, and so telnet cli interface is disabled! Please set a non empty cli-password!
Apr 16 04:59:27 jitsi turnserver: 0: SSL23: Certificate file found: /etc/jitsi/meet/meet.domain.tld.crt
Apr 16 04:59:27 jitsi turnserver: 0: SSL23: Private key file found: /etc/jitsi/meet/meet.domain.tld.key
Apr 16 04:59:27 jitsi turnserver: 0: TLS1.0: Certificate file found: /etc/jitsi/meet/meet.domain.tld.crt
Apr 16 04:59:27 jitsi turnserver: 0: TLS1.0: Private key file found: /etc/jitsi/meet/meet.domain.tld.key
Apr 16 04:59:27 jitsi turnserver: 0: TLS1.1: Certificate file found: /etc/jitsi/meet/meet.domain.tld.crt
Apr 16 04:59:27 jitsi turnserver: 0: TLS1.1: Private key file found: /etc/jitsi/meet/meet.domain.tld.key
Apr 16 04:59:27 jitsi turnserver: 0: TLS1.2: Certificate file found: /etc/jitsi/meet/meet.domain.tld.crt
Apr 16 04:59:27 jitsi turnserver: 0: TLS1.2: Private key file found: /etc/jitsi/meet/meet.domain.tld.key
Apr 16 04:59:27 jitsi turnserver: 0: TLS cipher suite: DEFAULT
Apr 16 04:59:27 jitsi turnserver: 0: DTLS1.2: Certificate file found: /etc/jitsi/meet/meet.domain.tld.crt
Apr 16 04:59:27 jitsi turnserver: 0: DTLS1.2: Private key file found: /etc/jitsi/meet/meet.domain.tld.key
Apr 16 04:59:27 jitsi turnserver: 0: DTLS: Certificate file found: /etc/jitsi/meet/meet.domain.tld.crt
Apr 16 04:59:27 jitsi turnserver: 0: DTLS: Private key file found: /etc/jitsi/meet/meet.domain.tld.key
Apr 16 04:59:27 jitsi turnserver: 0: DTLS cipher suite: DEFAULT
Apr 16 04:59:27 jitsi turnserver: 0: NO EXPLICIT LISTENER ADDRESS(ES) ARE CONFIGURED
Apr 16 04:59:27 jitsi turnserver: 0: ===========Discovering listener addresses: =========
Apr 16 04:59:27 jitsi turnserver: 0: Listener address to use: 127.0.0.1
Apr 16 04:59:27 jitsi turnserver: 0: Listener address to use: <internalIp>
Apr 16 04:59:27 jitsi turnserver: 0: Listener address to use: <dmzIp>
Apr 16 04:59:27 jitsi turnserver: 0: Listener address to use: ::1
Apr 16 04:59:27 jitsi turnserver: 0: =====================================================
Apr 16 04:59:27 jitsi turnserver: 0: Total: 2 'real' addresses discovered
Apr 16 04:59:27 jitsi turnserver: 0: =====================================================
Apr 16 04:59:27 jitsi turnserver: 0: NO EXPLICIT RELAY ADDRESS(ES) ARE CONFIGURED
Apr 16 04:59:27 jitsi turnserver: 0: ===========Discovering relay addresses: =============
Apr 16 04:59:27 jitsi turnserver: 0: Relay address to use: <internalIp>
Apr 16 04:59:27 jitsi turnserver: 0: Relay address to use: <dmzIp>
Apr 16 04:59:27 jitsi turnserver: 0: Relay address to use: ::1
Apr 16 04:59:27 jitsi turnserver: 0: =====================================================
Apr 16 04:59:27 jitsi coturn[26336]:    ...done.
Apr 16 04:59:27 jitsi turnserver: 0: Total: 3 relay addresses discovered
Apr 16 04:59:27 jitsi turnserver: 0: =====================================================
Apr 16 04:59:27 jitsi systemd[1]: Started LSB: coturn TURN Server.
Apr 16 04:59:27 jitsi turnserver: 0: pid file created: /var/run/turnserver.pid
Apr 16 04:59:27 jitsi turnserver: 0: IO method (main listener thread): epoll (with changelist)
Apr 16 04:59:27 jitsi turnserver: 0: WARNING: I cannot support STUN CHANGE_REQUEST functionality because only one IP address is provided
Apr 16 04:59:27 jitsi turnserver: 0: Wait for relay ports initialization...
Apr 16 04:59:27 jitsi turnserver: 0:   relay <internalIp> initialization...
Apr 16 04:59:27 jitsi turnserver: 0:   relay <internalIp> initialization done
Apr 16 04:59:27 jitsi turnserver: 0:   relay <dmzIp> initialization...
Apr 16 04:59:27 jitsi turnserver: 0:   relay <dmzIp> initialization done
Apr 16 04:59:27 jitsi turnserver: 0:   relay ::1 initialization...
Apr 16 04:59:27 jitsi turnserver: 0:   relay ::1 initialization done
Apr 16 04:59:27 jitsi turnserver: 0: Relay ports initialization done
Apr 16 04:59:27 jitsi turnserver: 0: IO method (general relay thread): epoll (with changelist)
Apr 16 04:59:27 jitsi turnserver: 0: IO method (general relay thread): epoll (with changelist)
Apr 16 04:59:27 jitsi turnserver: 0: turn server id=1 created
Apr 16 04:59:27 jitsi turnserver: 0: turn server id=0 created
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS/SCTP listener opened on : 127.0.0.1:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : 127.0.0.1:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : 127.0.0.1:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS/SCTP listener opened on : <internalIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : <internalIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : <internalIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS/SCTP listener opened on : <dmzIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : <dmzIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : <dmzIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv6. TLS/SCTP listener opened on : ::1:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv6. TLS listener opened on : ::1:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv6. TLS listener opened on : ::1:4445
Apr 16 04:59:27 jitsi turnserver: 0: IO method (general relay thread): epoll (with changelist)
Apr 16 04:59:27 jitsi turnserver: 0: turn server id=2 created
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : 127.0.0.1:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : <internalIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : <dmzIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv6. TLS listener opened on : ::1:4445
Apr 16 04:59:27 jitsi turnserver: 0: IO method (general relay thread): epoll (with changelist)
Apr 16 04:59:27 jitsi turnserver: 0: turn server id=3 created
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : 127.0.0.1:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. DTLS/UDP listener opened on: 127.0.0.1:4446
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : <internalIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. TLS listener opened on : <dmzIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. DTLS/UDP listener opened on: 127.0.0.1:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv6. TLS listener opened on : ::1:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. DTLS/UDP listener opened on: <internalIp>:4446
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. DTLS/UDP listener opened on: <internalIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. DTLS/UDP listener opened on: <dmzIp>:4446
Apr 16 04:59:27 jitsi turnserver: 0: IPv4. DTLS/UDP listener opened on: <dmzIp>:4445
Apr 16 04:59:27 jitsi turnserver: 0: IPv6. DTLS/UDP listener opened on: ::1:4446
Apr 16 04:59:27 jitsi turnserver: 0: IPv6. DTLS/UDP listener opened on: ::1:4445
Apr 16 04:59:27 jitsi turnserver: 0: Total General servers: 4
Apr 16 04:59:27 jitsi turnserver: 0: IO method (auth thread): epoll (with changelist)
Apr 16 04:59:27 jitsi turnserver: 0: IO method (auth thread): epoll (with changelist)
Apr 16 04:59:27 jitsi turnserver: 0: IO method (admin thread): epoll (with changelist)
Apr 16 04:59:27 jitsi turnserver: 0: SQLite DB connection success: /var/lib/turn/turndb

I also did a quick test on meet.jit.si and it worked flawless there. So it looks there is something wrong with my configuration.

Hi @damencho, do you have any other ideas what could be wrong?

This is a short summary about what services are running on which port and what is/should be configured. I wrote this to make sure that I set everything up correctly. Hopefully this can help to track down the problem with users behind corporate firewalls where outgoing traffic is only allowed on TCP 80/443.

Services and ports

  • the Jitsi turnserver listens on port 4446 and on 4445 for TLS connections (file: /etc/turnserver.conf)
  • a nginx vhost is running on port 80 for Let’s Encrypt certificate validation (file: /etc/nginx/sites-enabled/<domain>.conf)
  • another nginx vhost is running on port 4444 serving the Jitsi Meet web frontend (file: /etc/nginx/sites-enabled/<domain>.conf)
  • nginx is also listening on port 443. All http traffic which arrives there is forwarded to port 4444 (Jitsi Meet web frontend). Everything that’s non http traffic is forwarded to port 4445 (Jitsi turnserver) (file: /etc/nginx/modules-enabled/60-jitsi-meet.conf)

Firewall

On the firewall in front of my Jitsi server I need the following ports forwarded to the Jitsi server

  • TCP/80
  • TCP/443
  • UDP/4446
  • UDP/10000-20000

Configuration

/etc/jitsi/meet/< domain>-config.js

The STUN/TURN servers must be enabled at two locations. This is already set in the default configuration:

  • useStunTurn: true
  • p2p.useStunTurn: true

Furthermore the STUN servers must be configured (p2p.stunServers). The default configuration is

        stunServers: [
            // { urls: 'stun:<domain>:4446' },
            { urls: 'stun:meet-jit-si-turnrelay.jitsi.net:443' }
        ],

Two questions:

  • Why is the Jitsi turnserver set here by default and not the one from the local instance?
  • If I have clients behind a coorporate firewall the correct url must be 'stun:<domain>:443, doesn’t it?

/etc/jitsi/videobridge/sip-communicator.properties

org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=meet-jit-si-turnrelay.jitsi.net:443

Here is also a default entry to the Jitsi turnserver. The same questions as above

  • Why not the turnserver of the local instance?
  • I have to use org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=<domain>:443, haven’t I?

/etc/prosody/conf.avail/< domain>.cfg.lua

In this file the turncredentials must be set which is done already in the default configuration

turncredentials_secret = "XXXXXXXXX";
turncredentials = {
  { type = "stun", host = "<domain>", port = "4446" },
  { type = "turn", host = "<domain>", port = "4446", transport = "udp" },
  { type = "turns", host = "<domain>", port = "443", transport = "tcp" }
};

This looks good to me and makes sense.

If secure domains are used the VirtualHost for the secure domain must also contain the module for turncredentials

VirtualHost "guest.<domain>"
        authentication = "anonymous"
        modules_enabled = {
            "turncredentials";
        }
        c2s_require_encryption = false

@hirnschmalz

Hi,

I did my config as your. I’m facing the same issues.
My server is directly connected to internet. (no nat)
Small difference :

My turn log remains empty :

0: log file opened: /var/log/turn_1067_2020-04-18.log
0:
RFC 3489/5389/5766/5780/6062/6156 STUN/TURN Server
Version Coturn-4.5.0.7 'dan Eider'
0:
Max number of open files/sockets allowed for this process: 4096
0:
Due to the open files/sockets limitation,
max supported number of TURN Sessions possible is: 2000 (approximately)
0:

==== Show him the instruments, Practical Frost: ====

0: TLS supported
0: DTLS supported
0: DTLS 1.2 supported
0: TURN/STUN ALPN supported
0: Third-party authorization (oAuth) supported
0: GCM (AEAD) supported
0: OpenSSL compile-time version: OpenSSL 1.1.0g  2 Nov 2017 (0x1010007f)
0:
0: SQLite supported, default database location is /var/lib/turn/turndb
0: Redis supported
0: PostgreSQL supported
0: MySQL supported
0: MongoDB is not supported
0:
0: Default Net Engine version: 3 (UDP thread per CPU core)

=====================================================

0: Bad configuration format: keep-address-family
0: -X : Wrong address format: ;; connection timed out; no servers could be reached
~                                                                                    

whereas the command : service coturn status give this :

● coturn.service - LSB: coturn TURN Server
   Loaded: loaded (/etc/init.d/coturn; generated)
   Active: active (running) since Sat 2020-04-18 14:57:36 UTC; 16min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 977 ExecStart=/etc/init.d/coturn start (code=exited, status=0/SUCCESS)
    Tasks: 15 (limit: 4915)
   CGroup: /system.slice/coturn.service
           └─1144 /usr/bin/turnserver -c /etc/turnserver.conf -o -v

Apr 18 15:12:59 ovhgk-jitsi-01 turnserver[1144]: 923: session 004000000000000002: delete: realm=<FQDN>, username=<1587309163>
Apr 18 15:13:02 ovhgk-jitsi-01 turnserver[1144]: 926: session 006000000000000001: usage: realm=<FQDN>, username=<1587309128>, rp=1901, rb=1732569, sp=147, sb=9216
Apr 18 15:13:03 ovhgk-jitsi-01 turnserver[1144]: 927: session 007000000000000001: TLS/TCP socket disconnected: 127.0.0.1:34980
Apr 18 15:13:03 ovhgk-jitsi-01 turnserver[1144]: 927: session 007000000000000001: closed (2nd stage), user <> realm <FQDN> origin <>, local 127.0.0.1:4445, remote 127.0.0.1:34980, reason: TLS/TCP soc
Apr 18 15:13:03 ovhgk-jitsi-01 turnserver[1144]: 927: session 006000000000000001: refreshed, realm=<FQDN>, username=<1587309128>, lifetime=0, cipher=TLS_AES_256_GCM_SHA384, method=UNKNOWN
Apr 18 15:13:03 ovhgk-jitsi-01 turnserver[1144]: 927: session 006000000000000001: realm <FQDN> user <1587309128>: incoming packet REFRESH processed, success
Apr 18 15:13:03 ovhgk-jitsi-01 turnserver[1144]: 927: session 006000000000000001: TLS/TCP socket disconnected: 127.0.0.1:34984
Apr 18 15:13:03 ovhgk-jitsi-01 turnserver[1144]: 927: session 006000000000000001: closed (2nd stage), user <1587309128> realm <FQDN> origin <>, local 127.0.0.1:4445, remote 127.0.0.1:34984, reason: T
Apr 18 15:13:03 ovhgk-jitsi-01 turnserver[1144]: 927: session 006000000000000001: delete: realm=<FQDN>, username=<1587309128>
Apr 18 15:13:03 ovhgk-jitsi-01 turnserver[1144]: 927: session 006000000000000001: peer <Server Pub IP> deleted

other thing error jvb log :

java.lang.Exception: Address discovery through STUN failed 

I even totally reinstalled my server this morrning, in case I was facing a release bug.

Hi @processor,

I also did a clean reinstall with the same results.

Hopefully @damencho can verify our settings and tell if they are correct or if we misunterstood something. Maybe he can provide some information about the configuration of meet-jit-si-turnrelay.jitsi.net (and the differences to our ones).

As a workaround I would be also OK to setup a dedicated STUN/TURN server if the combination of everything on one server don’t work.

That one is using this config https://github.com/jitsi/jitsi-meet/blob/master/doc/debian/jitsi-meet-turn/turnserver.conf it just in a separate pool of machines

Hi @damencho, and what about our configuration changes to use our own STUN/TURN server? Are they correct?

I recommend you theses 2 blog post:

It doesn’t exactly correspond to the current default install with deb packages, with nginx in front, but it helps a lot, especially on how to test a stun/turn server.
Validate with trickle-ice if it’s working, before trying to integrate it with jitsi is the good way.

Hi,

I’m pleased and worried at the same time. It works for us but I did not do anything for this :rofl:
We did tests today and it worked.
Wireshark show us all tcp flow between our client machines and the server.

Now I have few questions:

-If some users have their port 10000 blocked and some not, the whole meeting will use TCP connections or will it be mix with some users in UDP and some other in TCP?

-Does meet-jit-si-turnrelay.jitsi.net play any role in a meeting, except in P2P? if yes when?

-Could meet-jit-si-turnrelay.jitsi.net be replaced by our domain stun? (in this case nginx/coturn)

Sorry for the questions of they look dumb but like we would say in French : “I’m swimming in the mash potato”.

Thanks anyway.

Proc.

Hi

If some users have their port 10000 blocked and some not, the whole meeting will use TCP connections or will it be mix with some users in UDP and some other in TCP?

Mix. Users that are able to reach the JVB on port UDP:10000 keep using this channel. Only firewalled users use the turn server. (and some p2p conferences with only 2 participants too)

-Does meet-jit-si-turnrelay.jitsi.net play any role in a meeting, except in P2P? if yes when?

meet-jit-si-turnrelay.jitsi.net is a standard STUN server. If you provide your own STUN server address instead, meet-jit-si-turnrelay.jitsi.netis not supposed to be used.

Could meet-jit-si-turnrelay.jitsi.net be replaced by our domain stun? (in this case nginx/coturn)

Where do you want to replace it ? The answer is probably yes anyway.
In my case, the only place I still have meet-jit-si-turnrelay.jitsi.net configured is STUN_MAPPING_HARVESTER_ADDRESSES parameter of JVB. This help JVB, in his startup process, to discover his public IPs. I’m not sure if it can work properly if using a STUN server located on the same server…

Please note that I am not part of jitsi team, it is just my understanding of how it seems to work :wink: