Using jitsi and turnserver for p2p udp connections bypassing tcp-only/NAT firewall

Install & Config
, ,
#1

Hi all,

I already consumed quite a lot of documentation (see links at the end).

Goal:

  • establish a udp p2p meeting with end2end-encryption using jitsi meeting

Questions:

  • are the assumptions made below about end2end-encryption and turnserver/videoBridge correct?
  • Does the turnserver need any firewall rules to give access from outside for jitsi/prosody?
    (so this is a jitsi and not a turnserver question, therefore it is not in the coturn-forum)
  • did I miss something, e.g. could this work?

Obstacles:

  • At least one peer is placed behind a tcp-only/NAT firewall/proxy
  • due to sometimes very poor tcp performance (lot of packet losses) udp needs to be used
  • the jitsi meeting server (+prosody +jvb) is in a third place, operated by a third party, but config requests can be made

If I understood everything, then

  • jitsi VideoBridge will break end-2-end encryption and for that can’t be used here
  • turnserver does not break end-2-end encryption

Planned Solution:

  • using coturn-Turnserver in the environment of peer A with udp keyhole punching to „open“ Firewalls from „inside“

knowing that

  • this configuration will only work if both Peers can use common codecs for Video/Audio
  • this configuration will not work if more than 2 peers are in the meeting

Signaling path – all communications can use tcp:

Peer A <---> Proxy/FW/NAT <-I-> nginx/jitsi/prosody/STUN <-I-> Proxy/FW/NAT <---> Peer B

<-I-> is standing for Internet here
nginx/jitsi/prosody/STUN are behind firewalls not mentioned here

Video-/Audio-path – all communications use udp:

Peer A <---> FW <---> Turnserver <---> FW/NAT <-I-> FW/NAT <---> Peer B

Due to my opinion it should work this way:

  • Peer A communicates with his „local“ turnserver, which provides peer A with the information about his external NAT-address. This can be checked using trickleICE. Peer A should show a candidate of type udp relay with the external ip of the turnserver. udp connections between peer A and his “local” turnserver are opened from inside by peer A.

  • Peer A informs the prosody signaling server about his relay candidate, which transports this information to peer B.

  • Peer B contacts the STUN to get his own srflx-address and communicates this via prosody signaling server to Peer A, which informs „his“ turn-relay about the srflx-candidate of Peer B. As the turnserver is not reachable from the internet at this moment, peer B won’t provide a relay candidate.

  • peer A tries to reach the srflx-address of peer B using udp, which will be blocked by the firewall

  • the turnserver tries to reach the srflx-address of Peer b using udp, thus opening the FW udp ports (keyhole punching).

  • Peer B tries to reach via udp the relay candidate of peer A, thus opening the udp ports at his local FW. After the turnserver has opened the firewall ports on his side, peer B can connect to the turnserver, using it as relay to communicate with peer A.

  • now the Video-/Audio-communication should be established

Conclusion: I assume the turnserver doesn’t need a direct communication with jitsi/prosody as it is designed to communicate with the „inside“ peer for signaling and with the „external“ peer using keyhole punching for the udp video/audio-sessions, so no firewall rules are needed to open access from the internet to this turnserver.

iceServer-configuration includes a STUN at the jitsi-site to get srflx-address for peer B and TURN for the local site of peer A, nothing else needed.

Remark: If peer B is also behind an tcp-only/NAT-FW, than a second turnserver is needed at his location
Remark: all this is theorie, knowing that theorie not always reflects real life…

Documentation used up to now:

Getting Started | WebRTC
[Build the backend services needed for a WebRTC app: STUN, TURN, and signaling - HTML5 Rocks]
Build the backend services needed for a WebRTC app: STUN, TURN, and signaling - HTML5 Rocks)

RFC5389 STUN

rfc5766 TURN

rfc4787 (ietf.org) udp NAT

RFC5245 ICE

rfc8600 - IETF Tools, rfc6122 - IETF Tools, rfc6121 - IETF Tools XMPP

RFC 4566: SDP: Session Description Protocol

How to set up and configure your own TURN server using Coturn (gabrieltanner.org)

How to Set up Coturn TURN Server for WebRTC | Code for Developer

Jitsi Meet and Firewalls - Meetrix.IO

Settingup a Turn Server for Jitsi Meet - Meetrix.IO

#2

This is not correct. The turnserver can be anywhere on the internet and can be used by any client. And there is nothing special about the turnserver except it needs to have the same shared secret configured as the one on the prosody server to authenticate the clients.
To protect of abuse a short lived passwords are used where the client requests from prosody give me credentials I can use (which are generated with the shared secret) and the client then uses them to connect to the turnserver.

There is no “inside” “external” users from point of view of turnserver. The turnserver just gets connections, authenticate them, and forwards traffic to the bridge public address and port.

1 Like
#3

Thank you for the answer!

So the clients will get the secret from the prosody server and use them to connect to the turnserver, and there is no need for the turnserver to talk to the prosody server directly.

And the jvb will break the end2end-encryption.

The remark regarding peer B is because udp should be used, so peer B needs a bypass for the firewall on his site if the firewall doesn’t allow udp, and using a turnserver for keyhole punching is the only way I know to do this…

The whole setup just serves to achive direct p2p connections with end2end-encryption using udp.

I will report if I achive to get a working config for this design.

#4

Nope, the short lived password is generated on prosody side using the shared secret and is given to the client. And yes, there is no communication between turnserver and prosody.

Stun server is needed for that, not turnserver. But in jitsi-meet case we use coturn which is both.

What do you mean, maybe I’m missing something?
When you install jitsi-meet by default it configures and a turnserver to be used, just run the let’s encrypt script to have valid certificates. So everything is baked already in the default deployment. What’s the difference you are talking about?

#5

The jitsi installation is on a third party site and I have no access to the configuration or the servers. I just communicate the turnserver credentials with them, and they configure the jitsi meeting. So they operate a jitsi environment (with an application in front of jitsi) and I have to develop a setup on our site for bypassing the tcp-only-firewall to get performant udp sessions with end2end encryption. That’s what the “local” coturn-server between two firewalls is for.
As the client on our site gets a udp relay candidate with the external NAT IP of the coturn when testing with trickleICE (using the correct credentials), and the coturn can open udp connections in direction to the Internet on the firewall I believe the configuration on my site is correct. But it’s not working yet. Nearly every system participating (peers, network, proxy, firewall) is managed by another company which makes troubleshooting extremely difficult. So I decided to solve theoretical how it should work (the design described above), which is crucial in this case to understand the different communication pathes needed for webrtc. We see the attempts from trickleICE to open connections from “outside” (peer B, JVB) to peer A, and there are requests to open the firewall in front of the coturn-turnserver for ingoing connections from prosody, JVB and peer b, thought the firewall operating company states that (at least at the moment) there are no incoming requests there.

But thanks to your replies I now know that I don’t have to open firewall ports for prosody at our site. So assuming the firewall operator didn’t made a mistake either my turnserver config is wrong, their jitsi config is wrong, or I missed something else required.

Wait - we have certificates from our company in the turnserver. Maybe jitsi/prosody/webrtc/Browser doesn’t like this, beeing different from the certificates they use in their jitsi setup? But I don’t see eny error messages in this direction in the turnserver log nor edge://webrtc-internals nor about:webrtc.

#6

You need to open chrome net internals and debug it like that finding the sockets. Its very hard to debug this, but not impossible.

The address you use for the turnserver and its certificate needs to match and the certificate needs to be full chain valid one.

#7

The turnserver certificate is a domain certificate, fully chained.

“Its very hard to debug this, but not impossible.” That’s true, without knowing the jitsi config it’s extremely difficult.

Thanks for your help so far! I will be back if I have new informations, might take a while.