Can I have Jitsi behind CloudFlare proxy?

I’m trying to run Jitsi using the Docker Compose instructions and I got it up and running at https://meet.status.im/, but no video comes through.

I’m using a CloudFlare proxy for my SSL termination and I’m using one of the available extra ports to expose JVB under port 8443. My understanding is that JVB - which normally listens on 4443 - uses TLS which is why I picked 8443 so it’s HTTPS.

I can’t use 10000 UDP port because CloudFlare does not support proxying UDP ports. Unless there’s some way to tell Jitsi to use the direct IP of my host rather than going through the proxies domain.

I’ve tried setting the suggested options:

org.jitsi.videobridge.SINGLE_PORT_HARVESTER_PORT=10000
org.jitsi.videobridge.DISABLE_TCP_HARVESTER=false
org.jitsi.videobridge.TCP_HARVESTER_PORT=8443
org.jitsi.videobridge.TCP_HARVESTER_MAPPED_PORT=8443
org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=127.0.0.1
org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=174.138.106.181

And I verified the port is available:

 > sudo nmap -Pn -p8443 meet.status.im
Nmap scan report for meet.status.im (104.22.24.181)
Host is up (0.0081s latency).

PORT     STATE SERVICE
8443/tcp open  https-alt

But the video doesn’t seem to work at all.

I checked with netstat -pnt if there are any established connections but nothing comes up.

How can I debug what is wrong with my setup? I don’t see any clear errors in the web browser console when I open a meeting, but the video is clearly not working.

1 Like

I’m seeing these in console logs:


Not sure what to make of those. There’s no info on what actually went wrong.

I actually switched to the recommended LetsEncrypt setup and dropped CloudFlare proxy. I even used the default ports as recommended, exposed 80 and 443 directly from web container and 4443 and 10000 from the jvb container. The web ui works fine but no video or audio is being transmitted.

I tried checking if there was any attempts at connecting to ports TCP 4443 or UDP 10000 with Wireshark but I got absolutely nothing:


It doesn’t seem like it’s trying to establish communication on those at all.
The only port there seems to be any communication is 443.

I tested a meeting at https://meet.jit.si/whatever with the same filter and saw UDP traffic on 10000:


So my issue doesn’t seem to be connectivity, but rather my instance not even trying to connect.

I managed to get it working by reverting to the CloudFlare proxy setup and using an Nginx proxy in front of the web container. I have video and sound when using Chrome(ium) but not Brave. Interestingly enough when I used the default for STUN:

org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=meet-jit-si-turnrelay.jitsi.net:443

It works fine, but when I set the public IP by hand:

org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=127.0.0.1
org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=174.138.106.181

Then suddenly when a 3rd person connects to the meeting video breaks for everyone. Makes no sense.

Also, I still see no traffic on my configured 8443 and 10000 for video bridge, which is very confusing.

Okay, I figured out why it worked with:

org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=meet-jit-si-turnrelay.jitsi.net:443

And not when I set the IP explicitly. When it detects the public IP it uses the IP address of my host, be the IP I was trying to configure with NAT_HARVESTER_PUBLIC_ADDRESS was a floating IP. For some reason it works fine with the host IP but not with the floating IP:


Can anyone explain why it would prefer the host IP? Could it possibly be related to reverse PTR records?

Also, I can’t seem to livestream or record due to this error from jibri:

SEVERE: [82] org.jitsi.jibri.selenium.JibriSelenium.run()
  An error occurred while joining the call: org.openqa.selenium.WebDriverException: <unknown>:
  Failed to read the 'localStorage' property from 'Window': Access is denied for this document.

Buy switching to LetsEncrypt setup and replacing all meet.jitsi values in the .env with my own domain I’ve arrived at a situation in which livestream/recording fails with:

FINE: [63] org.jitsi.jibri.selenium.pageobjects.CallPage.visit() Visiting url https://meet.status.im/test#config.iAmRecorder=true&config.externalConnectUrl=null&config.startWithAudioMuted=true&config.startWithVideoMuted=true&interfaceConfig.APP_NAME="Jibri"&config.analytics.disabled=true&config.p2p.enabled=false
INFO: [63] org.jitsi.jibri.selenium.pageobjects.CallPage.visit() Waited 1154 milliseconds for call page to load
INFO: [63] org.jitsi.jibri.selenium.JibriSelenium.onSeleniumStateChange() Transitioning from state Starting up to Running
INFO: [63] org.jitsi.jibri.service.impl.FileRecordingJibriService.invoke() Selenium joined the call, starting the capturer
INFO: [63] org.jitsi.jibri.util.JibriSubprocess.ffmpeg.launch() Starting ffmpeg with command ffmpeg -y -v info -f x11grab -draw_mouse 0 -r 30 -s 1280x720 -thread_queue_size 4096 -i :0.0+0,0 -f alsa -thread_queue_size 4096 -i plug:cloop -acodec aac -strict -2 -ar 44100 -c:v libx264 -preset veryfast -profile:v main -level 3.1 -pix_fmt yuv420p -r 30 -crf 25 -g 60 -tune zerolatency -f mp4 /config/recordings/kzalpxfezigrignl/test_2020-04-17-17-00-56.mp4 ([ffmpeg, -y, -v, info, -f, x11grab, -draw_mouse, 0, -r, 30, -s, 1280x720, -thread_queue_size, 4096, -i, :0.0+0,0, -f, alsa, -thread_queue_size, 4096, -i, plug:cloop, -acodec, aac, -strict, -2, -ar, 44100, -c:v, libx264, -preset, veryfast, -profile:v, main, -level, 3.1, -pix_fmt, yuv420p, -r, 30, -crf, 25, -g, 60, -tune, zerolatency, -f, mp4, /config/recordings/kzalpxfezigrignl/test_2020-04-17-17-00-56.mp4])
FINE: [46] org.jitsi.jibri.util.ProcessStatePublisher.ffmpeg.invoke() Process ffmpeg hasn't written in 2 seconds, publishing periodic update
INFO: [67] org.jitsi.jibri.capture.ffmpeg.FfmpegCapturer.onFfmpegProcessUpdate() Ffmpeg quit abruptly.  Last output line: plug:cloop: Input/output error
INFO: [67] org.jitsi.jibri.capture.ffmpeg.FfmpegCapturer.onFfmpegStateMachineStateChange() Ffmpeg capturer transitioning from state Starting up to Error: SESSION Ffmpeg failed to start
INFO: [67] org.jitsi.jibri.service.impl.FileRecordingJibriService.onServiceStateChange() File recording service transitioning from state Starting up to Error: SESSION Ffmpeg failed to start
INFO: [67] org.jitsi.jibri.api.xmpp.XmppApi.invoke() Current service had an error, sending error iq <iq to='jibribrewery@internal-muc.meet.status.im/focus' id='35sE2-38' type='set'><jibri xmlns='http://jitsi.org/protocol/jibri' status='off' failure_reason='error'/></iq>
FINE: [67] org.jitsi.jibri.statsd.JibriStatsDClient.incrementCounter() Incrementing statsd counter: stop:recording
INFO: [67] org.jitsi.jibri.JibriManager.stopService() Stopping the current service
INFO: [67] org.jitsi.jibri.service.impl.FileRecordingJibriService.stop() Stopping capturer
INFO: [67] org.jitsi.jibri.util.JibriSubprocess.ffmpeg.stop() Stopping ffmpeg process
INFO: [67] org.jitsi.jibri.util.JibriSubprocess.ffmpeg.stop() ffmpeg exited with value 1
INFO: [67] org.jitsi.jibri.service.impl.FileRecordingJibriService.stop() Quitting selenium
INFO: [67] org.jitsi.jibri.service.impl.FileRecordingJibriService.stop() Participants in this recording: []

Not sure what to make of this as there’s no clear error here. The only thing that seems error-like is:

onFfmpegProcessUpdate() Ffmpeg quit abruptly.
Last output line: plug:cloop: Input/output error

Not sure what I’m supposed to do with Input/output error but at least it seems to indicate that the issue isn’t with the service itself.

It can write to the filesystem since it created recordings/kzalpxfezigrignl/metadata.json:

{"meeting_url":"https://meet.status.im/test","participants":[],"share":true}

Okay, I figured out that the Input/output error from ffmpeg was due to lack of ALSA sound devices in the container:

 % docker exec -it repo_jibri_1 bash
root@857f6bff75b5:/# aplay -l
aplay: device_list:270: no soundcards found...
root@857f6bff75b5:/# arecord -l
arecord: device_list:270: no soundcards found...

And that was happening because of permissions:

root@857f6bff75b5:/# cat /dev/snd/pcmC2D0p
cat: /dev/snd/pcmC2D0p: Permission denied

Which was in turn caused by the fact that I use Docker user remapping to increase security. I had to permanently make sound devices use mode 0666 to fix this:

admin@node-01.do-ams3.jitsi.misc:~ % cat /etc/udev/rules.d/80-sound-permissions.rules    
SUBSYSTEM=="sound", MODE="0666"

And that fixed access to the ALSA loop cards:

root@857f6bff75b5:/# aplay -l | grep '^card'
card 0: Loopback [Loopback], device 0: Loopback PCM [Loopback PCM]
card 0: Loopback [Loopback], device 1: Loopback PCM [Loopback PCM]
card 1: Loopback_1 [Loopback], device 0: Loopback PCM [Loopback PCM]
card 1: Loopback_1 [Loopback], device 1: Loopback PCM [Loopback PCM]
card 2: Loopback_2 [Loopback], device 0: Loopback PCM [Loopback PCM]
card 2: Loopback_2 [Loopback], device 1: Loopback PCM [Loopback PCM]
card 3: Loopback_3 [Loopback], device 0: Loopback PCM [Loopback PCM]
card 3: Loopback_3 [Loopback], device 1: Loopback PCM [Loopback PCM]
card 4: Loopback_4 [Loopback], device 0: Loopback PCM [Loopback PCM]
card 4: Loopback_4 [Loopback], device 1: Loopback PCM [Loopback PCM]

And now recording works!

I’m currently trying to get 10000 to instead be 8443 via CloudFlare as well - were you able to get that bit sorted?

You won’t be able to use UDP 10000 via CloudFlare proxy because it only supports TCP.

You can try changing the TCP fallback of 4443 to 8443 since CloudFlare supports that but I had no luck with it. It seems like jitsi hates it.

Though I haven’t tried setting up org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS to be my domain name. Maybe that would work.

Currently I have it working with LetsEncrypt and I’m afraid to change it in case it breaks.

And now after changing domain I’m getting the same Input/output error from ffmpeg except the ALSA loop devices are configured correctly… wtf is this software…

Okay, I managed to fix the sound permissions again. They were wrong and I verified that using:

while true; do
    PID=$(pgrep ffmpeg)
    if [[ -n "$PID" ]]; then
        echo "PID: $PID"
        sudo strace -f -p "$PID" -o /tmp/strace.log
        break
    fi
done