Jitsi 5390 + Coturn - P2P Not working

Hi all,
I’m currently having issues with making P2P working.
These are the Jitsi packets I’m using:

jicofo/stable,now 1.0-690-1 all [installed,upgradable to: 1.0-692-hf-1]
jitsi-meet/stable,now 2.0.5390-1 all [installed,upgradable to: 2.0.5390-3]
jitsi-meet-prosody/stable,now 1.0.4628-1 all [installed,automatic]
jitsi-meet-tokens/stable,now 1.0.4628-1 all [installed]
jitsi-meet-turnserver/stable,now 1.0.4628-1 all [installed,automatic]
jitsi-meet-web/stable,now 1.0.4628-1 all [installed,automatic]
jitsi-meet-web-config/stable,now 1.0.4628-1 all [installed,automatic]
jitsi-videobridge2/stable,now 2.1-416-g2f43d1b4-1 all [installed,automatic]

I’ve also enabled JWT authentication and Coturn (on the same machine) for 443 TCP port usage only cor users behindcorporate firewall that prevent using the 10000 and other non-standard port

If I try opening a two-users call with my PC and my smartphone while being on the same network (Wifi) P2P works correctly:
image

If I try using different networks (tried PC on wifi and smartphone on mobile network) it switches back to UDP protocol, thus using Jitsi’s VM bandwith while it shouldn’t:
test1 - PC + Mobile on Mobile network - Copia

How can I fix this situation? The goal is to still keep using coturn and retain the P2P communication in two-partecipants call.

Thanks in advance

Hi, could someone help me?

I don’t understand what you are trying to achieve. If a p2p connection can be established, neither coturn nor the videobridge are involved. If a direct connection fails or there are more than two participants in a call, media will be routed through the videobridge for all participants that can establish UDP connections with the bridge. Participants that cannot connect via UDP to the videobridge will attempt to connect to the TURN server via TCP and the TURN server will relay traffic between the videobridge and the respective participants.

This is how a common setup with coturn is supposed to be working in my understanding. In your test case, it is very likely that your smartphone on the mobile network cannot establish a direct connection with your PC, e.g., due to CGNATs, firewalls of all sorts and so on. Therefore your call is routed through the videobridge as a fallback. Your TURN server is not involved because both your phone and your PC can establis UDP connections to the bridge.

2 Likes

Hi, thanks for your reply.

What I’m trying to achieve is offloading from the Jitsi server all the traffic usage for 2-partecipants calls. Before setting up the TURN server with coturn, as in the guide:

I recall that all the two-partecipants call were switched in P2P and did not pass at all through the Jitsi’s machine, this also when testing from two different connections (my pc on wifi and my smartphone on mobile network). Since I do remember that the P2P function goal was to make the two partecipant directly communicate and send video streams among them witthout using the JVB, I would like to know if this behaviour with the turn server setup is normal or if there could be some misconfiguration.

Thanks!

Ah, now I see, that makes sense of course. So either there is a misconfiguration in your setup preventing a direct connection altogether or the smartphone on mobile net just cannot establish a direct connection. I think you should first make sure that a direct connection can be established. May you can use meet.jit.si for this? If a direct connection succeeds using meet.jit.si or other means but no p2p connection is established when using your server, there might be some misconfiguration. I don’t have an idea what this misconfig could be, though.

Hi, thanks for your reply. Before doing that, I tried again doing a 2 partecipants call with a coworker of mine that uses another connection (no mobile network this time). here’s the connection details, the IP showed is a LAN ip, not public:

image

As you can see it mentions both p2p and turn… but the traffic is still passing through Jitsi:

image

Edit: just tried using meet.jit.si, using mi pc’s wired connection and the mobile phone. Still p2p+turn, so I suppose the traffic is still going through their JVB:
image

Yep, just tested myself using meet.jit.si and two device on my local network. Shows the same p2p (turn) label as in your test and the traffic is definitely routed through a rmeote TURN server. This is not how I expect this to be, my understanding was that TURN would only be used as a fallback solution if a direct p2p connection fails.

However, I don’t know how the setting p2p.useStunTurn in config.js is supposed to behave. Perhaps it will always force using the TURN relay if one is configured? I’d suggest to do some more tests with p2p.useStunTurn=false and see how various different connection scenarios behave.

1 Like

I suppose that you can’t have everything I guess. Maybe it’s not possible to have both P2P (real one) and a coturn server to allow also people behind corporate firewall, maybe it’s a bug, maybe it’s intended to be this way or cannot be accomplished in other ways.
Maybe @damencho could shed some light about this matter

Yep, that is correct …

useStunTurn setting was removed a while ago.

Look how things work, we set the remote description, setting, and the turn server and pass it to webrtc which handles the connection and priorities, it may happen that the turn connection finishes way quicker than direct connection so that is used. To debug this you need to look at webrtc-internals and do wiresharking looking at the packets and understand how ICE works.

1 Like

Some more tests I did:

Scenario: two partecipants call:

  1. me from with my computer (in my home network)
  2. my coworker with his pc from the office (testing both office network and mobile network with thethering)

Conference tests on meet.jit.si:

  • coworker using thethering: P2P
  • coworker using office’s network: P2P (turn)

Conference tests on our domain (a VM hosted on IBM’s cloud):

  • coworker using mobile network (thethering to pc): P2P (turn)
  • coworker using office’s network: P2P (turn)

@damencho considering the Jitsi version I’m using, could you point me to some Jitsi’s config that could be causing this issue and how they should be configured (like on meet.jit.si), so that I can check if they are properly setup?

Thanks,

Gianluca

Thats good news to have a confirmed direct p2p connection via meet.jit.si as this can be used as a reference for testing your own setup.
Can you tell more about your setup? I am afraid the turn setup documentation in the Jitis Handbook linked above is not really up to date…

Do you use mod_turncredentials or mod_external_services in prosody? Do you have entries for both, stun and turn server in your prosody config?

1 Like

Hi @plokta , here are some of the configurations I have (some are standard, some are modified following the TURN guide). Note: Everything (Nginx, Prosody, JVB, Jicofo, Coturn) are hosted on the same machine. We have registered on the DNS my.domain.com and turn-my.domain.conf which points to a public IP and firewall, which forwards all the required ports to Jitsi:

/etc/jitsi/meet/my.domain.com-config.js

...
...
p2p: {
        // Enables peer to peer mode. When enabled the system will try to
        // establish a direct connection when there are exactly 2 participants
        // in the room. If that succeeds the conference will stop sending data
        // through the JVB and use the peer to peer connection instead. When a
        // 3rd participant joins the conference will be moved back to the JVB
        // connection.
        enabled: true,

        // The STUN servers that will be used in the peer to peer connections
        stunServers: [

            // { urls: 'stun:my.domain.com:3478' },
            { urls: 'stun:meet-jit-si-turnrelay.jitsi.net:443' }
        ]
...
...

/etc/jitsi/videobridge/sip-communicator.properties:

org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true
org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=meet-jit-si-turnrelay.jitsi.net:443
org.jitsi.videobridge.ENABLE_STATISTICS=true
org.jitsi.videobridge.STATISTICS_TRANSPORT=muc
org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=localhost
org.jitsi.videobridge.xmpp.user.shard.DOMAIN=auth.my.domain.com
org.jitsi.videobridge.xmpp.user.shard.USERNAME=jvb
org.jitsi.videobridge.xmpp.user.shard.PASSWORD=******
org.jitsi.videobridge.xmpp.user.shard.MUC_JIDS=JvbBrewery@internal.auth.my.domain.com
org.jitsi.videobridge.xmpp.user.shard.MUC_NICKNAME=****************

/etc/prosody/conf.d/my.domain.com.cfg.lua

...
...
turncredentials_secret = "*********";
turncredentials = {
{ type = "stun", host = "my.domain.com", port = "3478" },
{ type = "turn", host = "my.domain.com", port = "3478", transport = "udp" },
{ type = "turns", host = "turn-my.domain.com", port = "443", transport = "tcp" }
};
...
...
VirtualHost "my.domain.com"
-- enabled = false -- Remove this line to enable this host
authentication = "token"
--authentication = "anonymous"
-- Properties below are modified by jitsi-meet-tokens package config
-- and authentication above is switched to "token"
app_id="********"
app_secret="***********"
-- Assign this host a certificate for TLS, otherwise it would use the one
-- set in the global section (if any).
-- Note that old-style SSL on port 5223 only supports one certificate, and will always
-- use the global one.
ssl = {
    key = "/etc/prosody/certs/my.domain.com.key";
    certificate = "/etc/prosody/certs/my.domain.com.crt";
}
speakerstats_component = "speakerstats.my.domain.com"
conference_duration_component = "conferenceduration.my.domain.com"
-- we need bosh
modules_enabled = {
    "bosh";
    "pubsub";
    "ping"; -- Enable mod_ping
    "speakerstats";
    "turncredentials";
    "conference_duration";
    "muc_lobby_rooms";
}
c2s_require_encryption = false
lobby_muc = "lobby.my.domain.com"
main_muc = "my.domain.com"
-- muc_lobby_whitelist = { "recorder.my.domain.com" } -- Here we can whitelist jibri to enter lobby enabled rooms
...
...

/etc/turnserver.conf:

# jitsi-meet coturn config. Do not modify this line
use-auth-secret
keep-address-family
static-auth-secret=***********
realm=my.domain.conf
cert=/etc/certs/star_my.domain.com.crt
pkey=/etc/certs/my.domain.com.key
no-multicast-peers
no-cli
no-loopback-peers
no-tcp-relay
no-tcp
listening-port=3478
tls-listening-port=5349
listening-ip=[jitsi_vm_LAN_IP]
allowed-peer-ip=[jitsi_vm_LAN_IP]
no-udp
no-tlsv1
no-tlsv1_1
# https://ssl-config.mozilla.org/#server=haproxy&version=2.1&config=intermediate&openssl=1.1.0g&guideline=5.4
cipher-list=ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
# jitsi-meet coturn relay disable config. Do not modify this line
denied-peer-ip=0.0.0.0-0.255.255.255
denied-peer-ip=10.0.0.0-10.255.255.255
denied-peer-ip=100.64.0.0-100.127.255.255
denied-peer-ip=127.0.0.0-127.255.255.255
denied-peer-ip=169.254.0.0-169.254.255.255
denied-peer-ip=127.0.0.0-127.255.255.255
denied-peer-ip=172.16.0.0-172.31.255.255
denied-peer-ip=192.0.0.0-192.0.0.255
denied-peer-ip=192.0.2.0-192.0.2.255
denied-peer-ip=192.88.99.0-192.88.99.255
denied-peer-ip=192.168.0.0-192.168.255.255
denied-peer-ip=198.18.0.0-198.19.255.255
denied-peer-ip=198.51.100.0-198.51.100.255
denied-peer-ip=203.0.113.0-203.0.113.255
denied-peer-ip=240.0.0.0-255.255.255.255
syslog

Strange… now I tested again, my pc from my home network and my smartphone on mobile network using meet.jit.si (tried both android app and from chrome browser)… and it works again in p2p (turn). My colleague used the mobile network too, but sharing it to the computer, and in that case he had only p2p…

Anyway, I did other test by myself, with my computer in my home network and my mobile using mobile both network and home network:

Conference tests on meet.jit.si:

using my smartphone as second partecipant - on mobile network (android app): P2P (turn)
using my smartphone as second partecipant - on mobile network (chrome browser): P2P (turn)

using my smartphone as second partecipant - on home network (android app): P2P 
using my smartphone as second partecipant - on home network (chrome browser): P2P 

Conference tests on our domain (a VM hosted on IBM’s cloud):

using my smartphone as second partecipant - on mobile network (android app): P2P (turn)
using my smartphone as second partecipant - on mobile network (chrome browser): P2P (turn)

using my smartphone as second partecipant - on home network (android app): P2P
using my smartphone as second partecipant - on home network (chrome browser): P2P

and at last, same as before, but using both my pcs, one on home network and the other both on home network and mobile network using tethering:

Conference tests on meet.jit.si:

using 2nd pc as second partecipant - on mobile network (tethering) (chrome browser): P2P
using 2nd pc as second partecipant - on home network (chrome browser): P2P

Conference tests on our domain (a VM hosted on IBM’s cloud):

using 2nd pc as second partecipant - on mobile network (tethering) (chrome browser): P2P (turn)
using 2nd pc as second partecipant - on home network (chrome browser): P2P

Apparently, your setup does actually enable direct p2p conferences, although it remains unclear why some direct connection attempts fail and go through the relay. However, this can be caused by a huge number of reasons and may be very cumbersome to debug.
If you still want to try, you will likely need to follow damencho’s advice, learn about all sorts of NAT issues, fully comprehend ICE and understand how candidate priorities are determined by the used implementation.

A great article on all NAT-related issues for direct connectivity can be found at the Tailscale blog: How NAT traversal works · Tailscale Blog

You can simulate ICE candidate gathering using your server over at Trickle ICE - note that if you want to test the full TURN relay, you will need to fetch dynamic credentials from your prosody instance first.

Hi @plokta and @damencho,

thanks both for your reply. Guess I’ll have to talk to my PM and discuss about this matter, since I don’t think I have the necessarily skills to go through all of this investigation.
There’s one thing, though, that I must check.

As stated by @emrah in another thread, where he was explaining to me and another user how the coturn on the same machine works, I noticed that in my infrastructure the coturn server could not “query” itself from the inside.

I’ll make myself more clear: if I, from my computer, do a telnet on turn-my.domain.com on ports 443 (https) and, for example, coturn’s 5349 TLS TCP port, everything works.

If I do the same telnet from the Jitsi (and coturn) machine itself, it won’t work; it appears that a call that starts from the internal network, goes outside (with the registered DNSs) to re-enter inside, is in fact blocked.

In fact, as suggested by emrah, I had to put in my turnserver conf these lines to mitigate the problem:

listening-ip=10.144..
allowed-peer-ip=10.144..
no-udp

Note: 10.144… is Jitsi’s vm LAN address.

Do you think this could be (one of) the cause of my problem or it doesn’t have anything to do with?

You don’t have a non-working TURN issue, on the contrary overworked TURN issue…

Do the same tests after stopped the coturn service to understand if this is a priority issue or a direct connectivity issue

Thanks emrah. For doing so I’d also have to revert the all the other other configs in the guide:

(so reverting nginx from 4444 to 443, change the sturns entry in prosody’s my.domain.com.cfg.lua, disabling turn.conf in nginx/modules enabled…) right?

No, only stop the coturn service

systemctl stop coturn

Ok, I did the test. Tried with PC (home network) and smartphone (mobile network) with coturn service stopped. 10.x is the VM’s lan address, while 172.x is the docker network interface. As you can see it doesn’t show p2p nor p2p (turn):

image

re- Enabled the coturn service, refreshed the page on my browser and it shows p2p (turn)
image

This means that the clients cannot connect to each others directly when using your server. First, you need to solve this issue. The problem isn’t directly related with coturn. coturn is only an option when there is no direct connection possibility and it’s not the cause of the issue.

Are your server and one of the clients in the same network?

1 Like

Sorry, I did the test while my home pc was on the VPN to access the Jitsi’s VM.
Redone the tests by disabling and enabling again coturn, this time disconnected from the VPN at each conference call. Didn’t change much though:

Coturn disabled (169.x is the VM’s public IP, 87.10x my home network public IP)
image

Coturn enabled (10.x is the VM’s LAN address):
image

About your question: no, the Jitsi’s server is a VM hosted on IBM cloud, my pc is connected on my home network and smartphone using mobile network