Octo Cascade Bridges - here's how! FULL GUIDE

Hello everyone,
I just spent about a week and a half on configuring Octo for regional load balancing. Sorry to say the guide on Github is really lacking and not clear so I decided to invest my time on writing this so maybe it’ll spare someone else from going through all the posts out there trying to figure out how to do it right.

Configuration wise - you’ll need one shard (which is a Jitsi Meet instance, Jicofo, Jitsi video bridge = the quick installation) and some video bridges with just the Jitsi Meet front-end instance which we refer to as clients.
The clients will need to have to work on a shared db to share rooms and not create a room on each server - that you’ll need to figure our how to do - if you have questions on this one I can provide answers to the best of my knowledge (our developer set this on up).

First go through the load balancing guide so your secondary bridge can connect and register with Jicofo.
https://github.com/jitsi/jitsi-meet/wiki/jitsi-meet-load-balancing-installation-Ubuntu-18.04-with-MUC-and-JID

Once done open your Jicofo log /var/log/jitsi/jicofo.log with your favorite text editor and search for your JVB’s nickname. It should be there.

Once done go through the Octo guide:
https://github.com/jitsi/jitsi-videobridge/blob/master/doc/octo.md

JVB
Now here’s where things get a bit tricky.
org.jitsi.videobridge.octo.BIND_ADDRESS=10.0.0.1 # the address to bind to locally
org.jitsi.videobridge.octo.PUBLIC_ADDRESS=1.2.3.4 # the address to advertise (in case BIND_ADDRESS is not accessible)
org.jitsi.videobridge.octo.BIND_PORT=4096 # the port to bind to
org.jitsi.videobridge.REGION=region1 # the region that the jitsi-videobridge instance is in

DELETE all the bolded sections in the lines above. They shouldn’t be there they f**k everything up.
I don’t know how whoever wrote that guide didn’t think of stating that.

Then your bind IP is either the server you’re configuring local IP (Attention this is a regional guide meaning two servers won’t be in the same data center meaning they probably won’t be able to connect locally so either connect them using a VPN and use the local IP you’re getting through the VPN) or just use the server your’e configuring Public IP.

Make sure no firewall blocks the port listed or whitelist your server’s IP address.
Jicofo
Add your Octo config to Jicofo - that’s pretty straight forward.

Jitsi Meet
On Jitsi meet you got to enter the lines in the config file exactly as they’re shown with the “testing” and all of it. If you’re using your own instance and connecting via the Jitsi Meet API library add those lines under your conference options and your init options (you probably just need it under the conference options but I did on both just in case).
Then you gotta add your regional information.
Remember, each Jitsi Meet instance needs to be at same area or even on the same server as your regional JVB. So in each region you’ll 1 JVB and 1 jitsi meet instance. The JVB will connect the media stream and the Jitsi Meet will send the region info to Jicofo.

So for your regional info add the following lines to your config file (if using the API just add them next to Octo):
deploymentInfo: {
shard: “shard1”,
region: “RegionA”,
userRegion: “RegionA”
},

What will happen is users will connect to that Jitsi Meet client with it’s URL (let’s say usa.mycompany.com), the Jitsi Meet instance will report it to Jicofo and will connect it with the bridge that is in RegionA. (You gotta match the region name in Jitsi Meet and that region’s JVB. The JVB region is configured on the sip-communicator.properties.

Suggestion - I like to use the Route53 Latency Routing by Amazon cause it checks your user’s latency when typing a single URL and route it to the server that you configured in the policy. So 1 URL meet.mycompany.com can direct to multiple servers around the world based on proximity.

That’s it! now try connecting from two different regions (use a VPN on one of your computers for that matter) and check the Jicofo log. You should see something like this:

Jicofo 2020-05-21 23:29:25.167 INFO: [28] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Region info, conference=ffb214 octo_enabled= true: [[Europe , Europe][US-East, US-East]]

The bolded regions marked above are the JVB regions and the ones following are the clients’ regions.

Troubleshooting:
If you have any issue always track your logs for Jicofo and your JVBs. They helped me a lot.
That and scouring forums and what not trying to figure out what the h*ll

Cheers,
Raviv N

4 Likes

Heads up - make sure the region set on the bridge has no spaces after it.

I had an issue with one of my bridge and couldn’t figure out why it wouldn’t use the region the problem was a space after the region.

This single thing took me nearly a week to figure out!

1 Like

Hi,

I have just one region with one JMS and two JVB, If I have a one room with 30 users, how jicofo distribute this room and his users across two JVB? It will create the room on two JVB? Half users will be distributed on each JVB? How jicofo works?

Thank you!

Because we’re human and we make mistakes (at least I did in this case). It’s fixed now. If you notice other problems with our documentation, feel free to open PRs or at least let us know so we can fix it at the source.

Boris

2 Likes

Hi @Boris_Grozev,

We are thank you for your works in OCTO. It is a great solution.
But i heard from last Jitsi Community Meet in Youtube, there’s a possibilty that an ‘inter-server traffic between shard’ feature (OCTO?) need to be disabled form meet.jit.si for the time being.

My english ear may not be that good, it is true that OCTO will be disabled ? What will be the replacement for it? Thank you.

To my understanding it’ll spread them across the bridges according to load on each bridge. Haven’t tried multiple bridges in one region and one big conference where it was needed but that’s how OCTO region works so stands to reason same region will be the same

Thank you for fixing it Boris.
From issues I’ve read on forum posts I think it’ll be a good idea to include a better and more detailed examples than the one on last section of the OCTO guide.

Is there any comparison on how many users more a OCTO configuration can handle compared to an non OCTO configuration? With everything else being same?

I initially setup a single jitsi-meet server with three video bridges and disabled the videobridge on the meet server. I now hope to use octo so that i can have the videobridges be regional and have a room span bridges.First, hopeful, question: would it be possible to have deploymentInfo1, deploymentInfo2, and deploymentinfo3 in my config.js on my single meet server in order to make my 3 bridges be able to use octo through my single meet server? If the answer to that is no, then do I install both jitsi-meet-web and jitsi-meet-web-config for my videobridges? or just jits-meet-web?

I should mention my meet server is setup for token authentication and the “is moderator,” plugin. I noticed a reference to putting them in config if using api, which i am also doing, so I will try there.

Me and my team are looking into that. From research I’ve done so far the issue is not with the back-end servers JVBs etc… But the load your endpoint can handle. I.e. how many video streams can your Chrome handle at once. I don’t think you can exceed 25 participants in one conference. Even though the load will be scattered on multiple video bridges your computer at some point will exceed it’s CPU capacity which will cause the page to freeze or video loss.

I’ll know more in a week or so so hopefully I can update then.

If you wanna use just one JMS server then you’ll need to figure out how to send Jicofo your region info. I read posts of people using HAproxy and then manipulating the URL info on that is used to open JMS. I tried looking into that but it seemed way too complicated solution than the one I suggested here.
You can just install Jitsi-meet like you did the first time you set it up but just cancel Jicofo.
You’ll need to configure your JMS to connect to your main Jicofo server.

1 Like

I did the full install on my meet/bridge server, but screwed something up connecting it back to my primary jicofo/prosody server. Do I have to keep jicofo on the client and point it at the jicofo of the main?

No. There’s only one Jicofo.
You connect with your client directly to the main one