Nginx, coturn & port 443

Install & Config
#1

Hi,
I’m a total jitsi newbie. I’ve followed all steps to install jitsi-meet on a cloud vm (Ubuntu 18.04) and things went smooth until I had a conflict with nginx and coturn for port 443.

My understanding is that port 443 is preferred for TURN, but on a jitsi-meet installation, the same port is used by the web server for providing the meet client via HTTPs. coturn should fall back to 4443 but this didn’t happen. Even after restarting the VM, coturn started and grabbed port 443, while nginx failed complaining that could not bind the port.

I’m not sure on how to proceed, since I didn’t touch anything on the standard install and it seems it does not work out of the box.

Second question: assuming I’ll have to scale out, my idea is to have a meet server and multiple videoserver2 servers. I’ve watched the tutorial for this, but it is not clear to me where is coturn positioned in this setup: on the meet server? Or on each videoserver (i.e. multiple turn server running on different hosts)?

Assuming I stop the videoserver on the meet machine, would this solve the contention for port 443?

Thanks a lot for your help!

3 Likes
Https not working on quick install
No live upstreams while connecting to upstream message in nginx log
Jitsi quick install and LXD in Ubuntu 18.04
#2

Do you need NAT traversal? If no, just remove the turnserver.

#3

Hi, thanks for answering. I’m not sure. Most of participants will join from home, so each of them will be NATed behind their routers/modes.

If instead you mean NAT at higher level, I am using a VM on Azure.

#4

So currently, it is nginx which faces internet and gets all http traffic and based on ALPN of http2 traffic we recognize which is for turn and which to just serve web, coturn is listening on port 4444 and internally nginx serving web is listening on 4443.

#5

Honestly I was expecting coturn to listen on 4443, but netstat tells me it is bound to 4445 (not sure why) on some interfaces and 443 on others. No problems on showing my netsat here if needed.

#6

Rethinking at what you said, is there anything that I can do to check nginx is correctly passing traffic over 4444?
I’ve checked nginx config after installint jitsi-meet but I didn’t find related config.

#7

As an addenda, I’ve installed from scratch without installing nginx beforehand. I’ve found out that it’s been installed. So maybe the information on using jetty by default is incorrect. I’ve also found the proxy to 4444 in the module. Not sure my installation is ok, though. I see coturn still binding to port 443 on some interfaces and 4443 on others.

#8

Hey there, maybe you can help me, as it seems I have a similar issue. I just setup Mattermost and Jitsi for my university research group.
Mattermost uses Nginx to forward all incoming traffic for a dedicated hostname on port 443.

I installed Jitsi according to https://github.com/jitsi/jitsi-meet/blob/master/doc/quick-install.md

After a reboot, jitsi always started before nginx. It seems jitsi grabs the port 443 and therefore blocks the starting nginx server.

I found this post: https://github.com/jitsi/jitsi-videobridge/issues/49 and it says that jitsi should automatically free up 443 when nginx is already running. However my systemd order seems to start jitsi before nginx.

Do I have to change something for the “/etc/jitsi/videobridge/sip-communicator.properties”?

Would you have a hint for me @damencho?

Any help is highly appreciated as I am currently super frustrated and need the server, because it also blocks mattermost collaboration.

Thank you!
Tom

root@communication:/etc/nginx/sites-enabled# journalctl -xe
Kernel start-up required 4029835 microseconds.
Initial RAM disk start-up required INITRD_USEC microseconds.
Userspace start-up required 21190533 microseconds.
Mar 31 17:30:23 communication systemd-timesyncd[535]: Timed out waiting for reply from 91.189.91.157:123 (ntp.ubuntu.com).
Mar 31 17:30:23 communication systemd-timesyncd[535]: Synchronized to time server 91.189.94.4:123 (ntp.ubuntu.com).
Mar 31 17:31:32 communication systemd[1]: Starting A high performance web server and a reverse proxy server...
Subject: Unit nginx.service has begun start-up
Defined-By: systemd
Support: http://www.ubuntu.com/support

Unit nginx.service has begun starting up.
Mar 31 17:31:32 communication nginx[2045]: nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)
Mar 31 17:31:33 communication nginx[2045]: nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)
Mar 31 17:31:33 communication nginx[2045]: nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)
Mar 31 17:31:34 communication nginx[2045]: nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)
Mar 31 17:31:34 communication nginx[2045]: nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Address already in use)
Mar 31 17:31:35 communication nginx[2045]: nginx: [emerg] still could not bind()
Mar 31 17:31:35 communication systemd[1]: nginx.service: Control process exited, code=exited status=1
Mar 31 17:31:35 communication systemd[1]: nginx.service: Failed with result 'exit-code'.
Mar 31 17:31:35 communication systemd[1]: Failed to start A high performance web server and a reverse proxy server.
 Subject: Unit nginx.service has failed
 Defined-By: systemd
 Support: http://www.ubuntu.com/support
 Unit nginx.service has failed.
 The result is RESULT.
Is there a way to revert to Jitsi before 30-03-2020?
#9

PS: strangely I dont see any TCP:443 bindings, just this:
netstat -tulpn | grep :443
udp 0 0 10.19.0.1:443 0.0.0.0:* 933/turnserver
udp 0 0 10.19.0.1:443 0.0.0.0:* 933/turnserver
udp 0 0 134.122.123.123:443 0.0.0.0:* 933/turnserver
udp 0 0 134.122.123.123:443 0.0.0.0:* 933/turnserver
udp 0 0 127.0.0.1:443 0.0.0.0:* 933/turnserver
udp 0 0 127.0.0.1:443 0.0.0.0:* 933/turnserver
udp6 0 0 ::1:443 :::* 933/turnserver
udp6 0 0 ::1:443 :::* 933/turnserver

#10

There are two interfering definitions for port 443. Take a look into the directories below /etc/nginx: One definition in directory modules-enabled and the other in sites-enabled.

#11

Hi @gs4711, thank you for the hint.

Sorry for the maybe dumb question. As you said there is /etc/nginx/modules-enabled/60-jitsi-meet.conf
It says (sorry about the >, the comment window doesnt like my formatting):

> stream {
> upstream web {
>     server 127.0.0.1:4444;
> }
> upstream turn {
>     server 127.0.0.1:4445;
> }
> # since 1.13.10
> map $ssl_preread_alpn_protocols $upstream {
>     "h2"            web;
>     "http/1.1"      web;
>     "h2,http/1.1"   web;
>     default         turn;
> }
> 
> server {
>     listen 443;
> 
>     # since 1.11.5
>     ssl_preread on;
>     proxy_pass $upstream;
> 
>     # Increase buffer to serve video
>     proxy_buffer_size 10m;
> }
> }

My default config in sites-enabled start with:
server {

server_name my.server.com;
> 
> listen [::]:443 ssl http2 ipv6only=on; # managed by Certbot
> listen 443 ssl http2; # managed by Certbot

Does this mean I have to change the modules config to specify the jitis FQDN as well? Sorry, I really new to NGINX

#12

Yes, the conflicting definitions prohibit the start of the nginx server. BUT I have no clue how to avoid it as those changes lead to the next problems. Hope that the developers jump into it as there are several threads with the same direction … buggy stable release.

#13

Ah damn, thank you @gs4711 This is super worrying. At least it seems like I am not super dumb

EDIT:
Confirmed, changing port in modules from 443 to for example 4443 lets NGINX start, but obviously Jitsi is broken.

Nginx stream with coturn: all https traffic logs as coming from 127.0.0.1
#14

Where is that? I think I had removed everything jetty related.

So this is the web part, so if you see the web through the browser, it works.

Ooo I stated wrong above, Nginx is 4444 https://github.com/jitsi/jitsi-meet/blob/master/doc/debian/jitsi-meet/jitsi-meet.conf#L6 and coturn is 4445 https://github.com/jitsi/jitsi-meet/blob/master/doc/debian/jitsi-meet/jitsi-meet.conf#L9.

If you want to test turn (make sure you have installed LE certs) then do a simple iptables rule to block udp 10000 port just for you, your public IP where you are running the browser. Start 3 tabs and all your traffic should go through tcp 443 through coturn to reach jvb, you can verify that with Wireshark. So in one case you have video & audio traffic going to port 10000 udp, the other case is 443 tcp.

1 Like
#15

Hello,

after a lot of debugging, erasing all packages and reinstalling them, throwing away the turnserver, the complete setup seems to be calm, BUT the registration of jicofo seems to lack something:

Jicofo 2020-03-31 22:57:38.147 WARNING: [181] org.jivesoftware.smack.AbstractXMPPConnection.callConnectionClosedOnErrorListener() Connection XMPPTCPConnection[not-authenticated] (0) closed with error
org.jivesoftware.smack.XMPPException$StreamErrorException: undefined-condition You can read more about the meaning of this stream error at http://xmpp.org/rfcs/rfc6120.html#streams-error-conditions
<stream:error><undefined-condition xmlns='urn:ietf:params:xml:ns:xmpp-streams'/><text>No stream features to proceed with</text></stream:error>
	at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader.parsePackets(XMPPTCPConnection.java:1064)
	at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader.access$300(XMPPTCPConnection.java:1000)
	at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader$1.run(XMPPTCPConnection.java:1016)
	at java.base/java.lang.Thread.run(Thread.java:834)

Scrutinizing several threads of this forum, it might be an issue with the SSL pairing. Could you please confirm that this might be the reason (which leads to a complete downtime of the overall setup)?

Best regardsm Guenther

#16

So is there anything to reconfigure, as currently the module configuration https://github.com/jitsi/jitsi-meet/blob/master/doc/debian/jitsi-meet/jitsi-meet.conf#L21 blocks NGINX from starting, when there is another site configured in /etc/nginx/sites-enabled, listening on 443 as well?

#17

If I remember correctly, it is cited in the video here: https://jitsi.org/news/new-tutorial-installing-jitsi-meet-on-your-own-linux-server/
Also, this entire tutorial is a bit outdated. It states to open ports 4443/TCP and range 10000-20000/UDP.

Now, assuming everything works with nginx forwarding to coturn on 4445, is it still required to open port 4445/TCP?

#18

I confirm, same problem here. The missing turn server is not the cause. On a clean install, I see:

Jicofo 2020-04-01 09:28:03.423 SEVERE: [17] org.jitsi.impl.protocol.xmpp.XmppProtocolProvider.doConnect().315 Failed to connect/login: The following addresses failed: 'localhost:5222' failed because: localhost/127.0.0.1 exception: java.net.ConnectException: Connection refused (Connection refused)
org.jivesoftware.smack.SmackException$ConnectionException: The following addresses failed: 'localhost:5222' failed because: localhost/127.0.0.1 exception: java.net.ConnectException: Connection refused (Connection refused)
        at org.jivesoftware.smack.SmackException$ConnectionException.from(SmackException.java:278)
        at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:619)
        at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:902)
        at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:383)
        at org.jitsi.impl.protocol.xmpp.XmppProtocolProvider.doConnect(XmppProtocolProvider.java:270)
        at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:193)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Is there a way to revert to Jitsi before 30-03-2020?
Is there a way to revert to Jitsi before 30-03-2020?
#19

Hi,
did you install nginx before installing jitsi? Because I’ve found out that the problem disappears on a clean server if nginx is NOT installed before jitsi-meet. The configuration created by jitsti-meet works.

1 Like
#20

Hi there,

yes, NGINX was installed before. After installing Jigsi it worked flawless. Jigsi simply added a new configuration in /etc/nginx/sites-enabled.

It all changed after updating yesterday. Since then the problem with the occupied port 443 by the modules.conf happened.

I assume the clean config works for you, as there is no other site config with 443.