OCTO for Geo-distributed conferences

Install & Config
#1

Hello, everybody!
In recent Call from 16:22 to 17:20 Emil Ivov said that now OCTO is used mostly (or even only) for large conferences by Jitsi team (I believe at meet.jit.si), so it’s a little bit confusing to me and now I have a couple questions too team.

  1. If OCTO not for geo distributed conferences, so what can be used for that? What Jitsi team advice to use to geographically distribute one conference to 2 shards or 1 shard and bridges in 2 different locations?
  2. Does Jitsi Team advice to use one data center to host all conferences to get best quality and stability of such conferences?
  3. Does RegionBasedBridgeSelectionStrategy makes sense now? Or only IntraRegionBridgeSelectionStrategy makes sense?
  4. Can Team or Emil comment all that in more details?

Thanks for your great product!

2 Likes
Octo Cascade Bridges - here's how! FULL GUIDE
#2

Great questions :slight_smile:

It is used to spread big conferences over several bridges. You cannot spread a conference over two shards … the room must be hosted on same prosody-jicofo.

What do you mean by that? We are running meet.jit.si in 7 regions around the world, so it cannot be same datacenter.

Intra.

What details are you looking for?

#3

Hello, Damian!
And thanks for your quick response!

That is what I mean.
You have 7 regions.
Assume that user from Europe started a call and other users joined that call also from Europe. In that case, I think, all that users should be connected to one or more bridges in Europe region shard.
And now lets assume, that one user from USA joined to that same call.
What bridge will be used for that USA user? Same bridge(s) in Europe or bridge in US region that interconnects with bridge(s) in Europe region?
And if US bridge is used, how it managed? What technology?

Details that OCTO not for spread conference between bridges in different regions but only for spread conference between bridges in same region.

And maybe some details about how to spread big conference between 2 or more bridges in a manageable way. Let say after 30 participants. As I understand, now Jicofo decides to add another bridge to conference based on packet rate.

#4

It will use a bridge from Europe.

We had done measurements for orcto with cross region connections and figure out the end-to-end rtt is even some milliseconds slower compared with the way it is today.
And because cross region needs a mesh of connections and all bridges reporting to all regions the traffic was fine when there was low usage of the service, but when pandamic times came that was not possible to scale, so we switched it off.

There are no details, you just need to use SplitBridgeSelectionStrategy jicofo/SplitBridgeSelectionStrategy.java at 069d876504a4d7ce6456fc51706c43cf2424a6c4 · jitsi/jicofo · GitHub

Yes this is the default, when a bridge starts to be overloaded … You can setup and the other behaviour jicofo/reference.conf at 069d876504a4d7ce6456fc51706c43cf2424a6c4 · jitsi/jicofo · GitHub
This is how we actually currently test big conferences, we let jicofo spread 100 participants per bridge …

2 Likes
Need help in OCTO setup
Maximum number of participants on a meeting on meet.jit.si server
#5

Thanks, Damian for your quick and really detailed answer.
Now I got almost all my OCTO questions answered.

But just to clarify a few last moments…

Can you recommend to use it in production environment? Everywhere in documentations and comments of source code mentions that SplitBridgeSelectionStrategy is only for testing purposes. Now we are using Intra for our single Jitsi shard. And we have plans to implement another shard (or just bridges with Jitsi-Meet-Web and without Prosody+Jicofo) so here is my interest in that topic.

And again, just to clarify that topic - user always start call in his nearest region shard, right? And then if someone connects from different region, he joins that call in not nearest bridge?
Here is example - User in Europe started a call and invited 10 users from US. That call is hosted on some bridge in Europe, but majority of users from US still will connects only to that Europe bridge? Right?

In our case we have 2 big headquarters with a lot of users (Say A and B). And now we have Jitsi shard in one single Data-Center in one of headquarter (Say A). We plan to implement Jitsi shard in second headquarter (Say B)
Out goal is load-balance Jitsi. And fault tolerance Jitsi service if it possible.
So, if one user from B starts a call - does it will be hosted in Jitsi shard on B location?

If all calls of users from B location would be local (I mean not using Jitsi shard from A) that would be good intersite traffic economy and I think quality could be better then now, because bridges in B location are nearer to B location users and E2E RTT would be better.
But if call start user from location B and that call will join 20-30 users only from location A - call experience for such users would be worth then now, I assume. So in that case it can be important who actually starts the call.
Am I right in my minds?

And thanks again whole Jitsi team and specially Damian!

#6

So cross-region works like this: you are the first to join and you get the closes shard, jicofo selects a bridge based on your reported region. Now the call is pinned to that shard, a second participant enters and joins that same call on that shard, but base on the region he gets a new bridge.
So A is first and will use a shard from A and bridge from A, the second participant from B will use the shard A, but will have a bridge from B.

3 Likes
#7

Hi,

I wonder answer of that question. For divide big conferences on the same shard, do we need to use intra or split strategy?

#8

I would recommend to use the IntraRegion strategy. Please have a look at the code to understand the algorithm behind, it’s quite easy to read: jicofo/IntraRegionBridgeSelectionStrategy.java at master · jitsi/jicofo · GitHub

2 Likes
#9

Hi @damencho , how can I test new Secure octo and enable colibri v2? I saw octo configuration updates in recent commits.

thank you,
Milan

#10
#11

But you need latest from unstable. Probably next week we will try to release a new stable with all these changes.

1 Like
#12

@damencho is this supposed to say:

“This implementation is only usable with version 2 of the colibri protocol, which is currently only available over websockets”?

#13

Nope, it is XMPP. It means for those using just the bridge without the rest of the components and using the REST API that they are stuck for now with the old octo. As colibri2 is not available with REST.

#14

Aaaaah, gotcha! :+1:t5:

#15

Thank you @damencho, that is the reason :wink: I’m on latest stable. Colibriv2 will be mandatory with next stable release or the will be some parameter to switch between V1 and V2 version?

Thank you!