Jitsi Docker Self Hosting Guide - Scaling Jibris

Install & Config Docker
#1

I have followed Jitsi Docker Self hosting guide and have successfully set it up. My requirement was to have 50 concerrent calls and that is happening successfully. All 4 containers - Jitsi-web, JVB, Prosody and Jicofo are running.

Now my new requirement is to have Multiple Jibri’s on a different server and connect them to this setup and run recording. But I am encountering a lot of issues. I am not getting what should i change in configurations to make it work.

Things that i tried :-

  1. the .env file that we create when doing the self hosting - there i changed the advanced configuration i.e XMPP_DOMAIN, XMPP_AUTH_DOMAIN, etc all related fields from meet.jitsi to mydomain.org or auth.mydomain.org respectively
    Also changed the PUBLIC_URL to mydomain.org
    I made sure all occurences of meet.jitsi in the .env file were changed to their respective but specific mydomain.org

then i changed docker-compose.yml file found meet.jitsi: under network and changed it to mydomain.org:

then i created a new server and basically created docker jibris and changed the respective fields to match both of environment files.

Using the following changes I basically got stuck. Got no results what so ever. Even though all containers started i could not access mydomain.org to start a meeting. It gave connection error.

  1. After above solution did not work I tried to do the other way round. Instead of changing bunch of values in the main server I copied the actual working values from main server to other server where we use the credentails to connect to the main server. I added the main server details and made sure both .env match. All except the PUBLIC_URL was mydomain.org and rest was all that came with .env file was the same in both files across different servers.

I checked if ports was an issue but i have kept all ports open for the moment.

So using these two methods I thought i could scale up Jibris. I need help cuz i dont know where i am going wrong. Also have tried setting up docker jibris with standard installation and they work like a charm. Now need to set those same docker jibris with the docker self hosting guide.

I have also checked by adding alsa loopbacks to main server and then creating jibri containers via the self hosting guide and they work like a charm but multiple jibri containers will just increase the cpu and ram usage and plus there is a limit of 16 jibris per instance. ie 32 alsa loopbacks are possible.

All i want is to seprate those jibri containers from the main server and install them onto different server.
this will reduce the cpu and ram usage and also n/w bandwidth on which JVB heavily relies upon.

So please help me.

#2

Did you get 1 single jibri working?

#3

yes 1 is working but only on the same server. but this creates a limitation of only 16 jibri’s possible ( considering a big server 16/24 core 32/48gb ram ). As alsa loopback for an instance is only from 0,1,2,3…upto, 30,31. Each jibri needs 2 loopbacks. So 16 Jibri’s limit. So need a way to setup these jibri’s on a different machine.

Need 1 call : 1 jibri. So 50 calls : 50 jibri’s

#4

For using different machines you need something like Docker Swarm.

#5

Hey could you please help me with a document that helps me setup docker swarm jitsi meet !

#6

I don’t have such a document sorry. There is a PR you could use as guidance: docker-swarm files by prayagsingh · Pull Request #1134 · jitsi/docker-jitsi-meet · GitHub

#7

i have setup docker swarm and have all containers in the same network.
Instance 1 :- jvb jicofo prosody jitsi-web-front-end
Instance 2 :- One jibri at the moment

Both instances are connected with docker swarm with Instance 1 as master/leader and other as slave

All the containers are in the same external overlay swarm network “jitsi”.

After inspecting network jitsi using docker inspect network jitsi i get the following on instance 1

[
{
“Name”: “jitsi”,
“Id”: “qi9u9dqkoeoxmidvbbsohvds8”,
“Created”: “2022-06-02T07:41:50.995584407Z”,
“Scope”: “swarm”,
“Driver”: “overlay”,
“EnableIPv6”: false,
“IPAM”: {
“Driver”: “default”,
“Options”: null,
“Config”: [
{
“Subnet”: “10.0.2.0/24”,
“Gateway”: “10.0.2.1”
}
]
},
“Internal”: false,
“Attachable”: true,
“Ingress”: false,
“ConfigFrom”: {
“Network”: “”
},
“ConfigOnly”: false,
“Containers”: {
“3e8393169b8401dd09a983cdb18ddcf833a7557cf8e1036f0cd72d187c7a9d5f”: {
“Name”: “jitsi-meet_jvb_1”,
“EndpointID”: “d221d67a6bf198844adb749b75585156d0012ecfa0bdd0c3b8560c5a40d2dfd8”,
“MacAddress”: “02:42:0a:00:02:3f”,
“IPv4Address”: “10.0.2.63/24”,
“IPv6Address”: “”
},
“7a625b692723017cb82b8dcb872b4306c4620758e36e262d5a38318ac7858acd”: {
“Name”: “jitsi-meet_web_1”,
“EndpointID”: “1c7e22fe05d82d31c77a221b5f53f7e05da9640acf929bff1cea5f65e0ca3705”,
“MacAddress”: “02:42:0a:00:02:3d”,
“IPv4Address”: “10.0.2.61/24”,
“IPv6Address”: “”
},
“81807e95bab797d6314edfebeabf072618a9b6ffc5e18e38526e4e25596af11f”: {
“Name”: “jitsi-meet_prosody_1”,
“EndpointID”: “350f5565cb0b2874d5be6540dd5e1b971580b499c51950796b85b7e9f5d64a85”,
“MacAddress”: “02:42:0a:00:02:3b”,
“IPv4Address”: “10.0.2.59/24”,
“IPv6Address”: “”
},
“c2206c59ca9fe7d2830e50e2dc2c3f47dfb8c4c5748fa11adb18a30f6f5d6f21”: {
“Name”: “jitsi-meet_jicofo_1”,
“EndpointID”: “f5bb25ab82ada1d31c45d7f6276109834b7a42794e76f42f8faba41a970be7e7”,
“MacAddress”: “02:42:0a:00:02:3e”,
“IPv4Address”: “10.0.2.62/24”,
“IPv6Address”: “”
},
“lb-jitsi”: {
“Name”: “jitsi-endpoint”,
“EndpointID”: “c0521389f758c326aa99d2a45e479188133875192b89f3da56222b73e8496126”,
“MacAddress”: “02:42:0a:00:02:3c”,
“IPv4Address”: “10.0.2.60/24”,
“IPv6Address”: “”
}
},
“Options”: {
“com.docker.network.driver.overlay.vxlanid_list”: “4098”
},
“Labels”: {},
“Peers”: [
{
“Name”: “e0e2151ac306”,
“IP”: “52.86.18.102”
},
{
“Name”: “176c281f579e”,
“IP”: “172.31.8.228”
}
]
}
]
Error: No such object: network

After inspecting network jitsi using docker inspect network jitsi i get the following on instance 2 aka jibri one
[
{
“Name”: “jitsi”,
“Id”: “qi9u9dqkoeoxmidvbbsohvds8”,
“Created”: “2022-06-02T07:49:53.705691448Z”,
“Scope”: “swarm”,
“Driver”: “overlay”,
“EnableIPv6”: false,
“IPAM”: {
“Driver”: “default”,
“Options”: null,
“Config”: [
{
“Subnet”: “10.0.2.0/24”,
“Gateway”: “10.0.2.1”
}
]
},
“Internal”: false,
“Attachable”: true,
“Ingress”: false,
“ConfigFrom”: {
“Network”: “”
},
“ConfigOnly”: false,
“Containers”: {
“3f44864cf0362aaf12c0f5899b03a9950b054c86497a1a0686a187c0dced98b1”: {
“Name”: “jitsi-meet-2_jibri_1”,
“EndpointID”: “b98482edc46c5d54f184d93643e29f36ef1cb28b3f8b596b6ce7958da59e2efe”,
“MacAddress”: “02:42:0a:00:02:42”,
“IPv4Address”: “10.0.2.66/24”,
“IPv6Address”: “”
},
“lb-jitsi”: {
“Name”: “jitsi-endpoint”,
“EndpointID”: “99eee0cb59950acf59af0fd4d66fecf44cea6f55baa919b11eaf9fc534eba135”,
“MacAddress”: “02:42:0a:00:02:43”,
“IPv4Address”: “10.0.2.67/24”,
“IPv6Address”: “”
}
},
“Options”: {
“com.docker.network.driver.overlay.vxlanid_list”: “4098”
},
“Labels”: {},
“Peers”: [
{
“Name”: “e0e2151ac306”,
“IP”: “52.86.18.102”
},
{
“Name”: “176c281f579e”,
“IP”: “172.31.8.228”
}
]
}
]
Error: No such object: network

So can confirm all containers are in the same network

After doing netstat -tulnp on network of instance 1

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 205836/docker-proxy
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 205859/docker-proxy
tcp 0 0 0.0.0.0:5222 0.0.0.0:* LISTEN 205711/docker-proxy
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd
tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 206668/docker-proxy
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 30630/sshd
tcp6 0 0 :::443 :::* LISTEN 205843/docker-proxy
tcp6 0 0 :::8000 :::* LISTEN 205867/docker-proxy
tcp6 0 0 :::5222 :::* LISTEN 205718/docker-proxy
tcp6 0 0 :::2377 :::* LISTEN 78306/dockerd
tcp6 0 0 :::7946 :::* LISTEN 78306/dockerd
tcp6 0 0 :::111 :::* LISTEN 1/systemd
tcp6 0 0 :::22 :::* LISTEN 30630/sshd
udp 0 0 0.0.0.0:4789 0.0.0.0:* -
udp 0 0 0.0.0.0:111 0.0.0.0:* 1/systemd
udp 0 0 127.0.0.1:323 0.0.0.0:* 922/chronyd
udp 0 0 0.0.0.0:10000 0.0.0.0:* 206642/docker-proxy
udp6 0 0 :::7946 :::* 78306/dockerd
udp6 0 0 :::111 :::* 1/systemd
udp6 0 0 ::1:323 :::* 922/chronyd
udp6 0 0 :::10000 :::* 206648/docker-proxy

After doing netstat -tulnp on network of instance 2 aka jibri

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1229/sshd
tcp 0 0 0.0.0.0:5222 0.0.0.0:* LISTEN 85127/docker-proxy
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd
tcp6 0 0 :::22 :::* LISTEN 1229/sshd
tcp6 0 0 :::5222 :::* LISTEN 85134/docker-proxy
tcp6 0 0 :::2377 :::* LISTEN 1233/dockerd
tcp6 0 0 :::7946 :::* LISTEN 1233/dockerd
tcp6 0 0 :::111 :::* LISTEN 1/systemd
udp 0 0 0.0.0.0:111 0.0.0.0:* 1/systemd
udp 0 0 127.0.0.1:323 0.0.0.0:* 909/chronyd
udp 0 0 0.0.0.0:4789 0.0.0.0:* -
udp6 0 0 :::7946 :::* 1233/dockerd
udp6 0 0 :::111 :::* 1/systemd
udp6 0 0 ::1:323 :::* 909/chronyd

As i see port 5222 is also open but i get the following error when i check logs for jibri container

Jibri 2022-06-02 07:49:55.748 WARNING: [31] MucClient.createXMPPTCPConnectionConfiguration#116: Disabling certificate verification!
Jibri 2022-06-02 07:49:55.767 INFO: [31] [hostname=xmpp.meet.jitsi id=xmpp.meet.jitsi] MucClient.initializeConnectAndJoin#340: Dispatching a thread to connect and login.
Jibri 2022-06-02 07:50:25.785 WARNING: [29] org.jivesoftware.smackx.ping.PingManager.pingServerIfNecessary: XMPPTCPConnection[not-authenticated] (0) was not authenticated
Jibri 2022-06-02 07:50:25.803 WARNING: [31] [hostname=xmpp.meet.jitsi id=xmpp.meet.jitsi] MucClient.lambda$getConnectAndLoginCallable$7#630: Error connecting:
org.jivesoftware.smack.SmackException$EndpointConnectionException: The following addresses failed: ‘RFC 6120 A/AAAA Endpoint + [xmpp.meet.jitsi:5222] (xmpp.meet.jitsi/10.0.2.59:5222)’ failed because: java.net.SocketTimeoutException: connect timed out
at org.jivesoftware.smack.SmackException$EndpointConnectionException.from(SmackException.java:334)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:664)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:849)
at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:526)
at org.jitsi.xmpp.mucclient.MucClient.lambda$getConnectAndLoginCallable$7(MucClient.java:625)
at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:167)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Jibri 2022-06-02 07:50:55.434 FINE: [20] WebhookClient$updateStatus$1.invokeSuspend#107: Updating 0 subscribers of status
Jibri 2022-06-02 07:51:00.807 WARNING: [31] [hostname=xmpp.meet.jitsi id=xmpp.meet.jitsi] MucClient.lambda$getConnectAndLoginCallable$7#630: Error connecting:
org.jivesoftware.smack.SmackException$EndpointConnectionException: The following addresses failed: ‘RFC 6120 A/AAAA Endpoint + [xmpp.meet.jitsi:5222] (xmpp.meet.jitsi/10.0.2.59:5222)’ failed because: java.net.SocketTimeoutException: connect timed out
at org.jivesoftware.smack.SmackException$EndpointConnectionException.from(SmackException.java:334)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:664)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:849)
at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:526)
at org.jitsi.xmpp.mucclient.MucClient.lambda$getConnectAndLoginCallable$7(MucClient.java:625)
at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:167)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)

Now why is the connection getting timed out is my biggest issue.

Please help

#8

That workes for me. Thank you!

#9

Thank you Jitsi Community I have now successfully setup jitsi meet docker image with jibri’s running on docker containers on a different server using docker swarm. Will create a Guide soon !!! Using this we can scale both JVB and Jibri’s. Now I am trying to implement HA Proxy for multiple Jitsi Meets on multiple regions. Thanks Again

1 Like