OCTO config help

Hi Jitsi community,

We are attempting to set up JVB’s in 4 global AWS datacenters with OCTO and High Availability. We are having a bit of trouble configuring OCTO part.

Would someone be interested to jump on a live call and troubleshoot with us?

Happy to pay for your time.

Thanks!
Daryl

Hi Daryl,

I can’t commit to attend a call right now, but if you describe where you’re at and what the problem looks like I’ll take a look.

Boris

Hi Boris,

I hope you’re doing well.

Basically our biggest challenge is how to get GeoLocation done at HAProxy layer considering that we need a GeoIP Database integration.

Beside that how two different shards can communicate, assuming that first shard is in the US with two servers "Jitsi Meet and JVB installed on a separate machine and a second shard is located on eu-west with similar setup.

If a meeting was created on US shard and then a user is trying to join the same meeting from UK through the EU shard then how HAProxy will detect that this meeting was initiated at the US shard and that this request should be directed to the US shard instead of the nearest shard which is EU ?

Below you can find a screenshot demonstrating what I am was trying to describe above
image

One the above screenshot meeting was created at SA shard then when someone tried to join from the UK then he was directed to the SA shard but he landed into the UK JVB.

One last question, is it possible to override the Backend deployment Info with realtime location information retrieved at client side using any API ? the reason why I am asking is that if this is doable and if this is something you guys might recommend then it will make it easier for us to implement the Geo Location with no need for the extra HAProxy changes you guys have done.

Best Regards,
Mohamed Abada

Hi @Boris_Grozev did you have a chance to think about Mohamed’s response here? Many thanks Daryl

Hi Mohamed,

In our architecture what you are describing is achieved with two separate, independent mechanisms. I recommend you think about and implement them separately as well.

The first mechanism allows conferences to be split among multiple shards (i.e. jicofo instances). We implement this purely on the HTTP level, and this does NOT require clients to know their region. We have a set of HAProxy instances with a shared table mapping a conference name to a backend shard. The proxies are configured to

  1. Route requests to the specified shard if an entry exists in the table

  2. Route requests for new conferences to a specific local backend, and update the table when the conference name is not in the table.

All HTTP requests which are specific for a room (i.e. requests to the BOSH endpoint /http-bind) coming from jitsi-meet clients add the name of the conference as a URL param:
`
Request URL:

https://meet.jit.si/http-bind?room=testconference
`

I think a lot of this is described in Yasen’s tutorial here: https://jitsi.org/news/tutorial-video-how-to-load-balance-jitsi-meet/

The second mechanism allows a single conference to be distributed to multiple videobridge instances according to the clients’ regions. This is where clients’ js code needs to know their local region. There are many different ways to implement this. We do it by adding an additional request header, specifying the region of the HAProxy handling the request (which is also the client region) to requests for /config. The server handling the request then replaces the userRegion value in the server file with the header value (using Server Side Includes, IIRC).

I think it makes sense to abstract this in jitsi-meet to allow other implementations, so contributions are welcome. We opted to do it this way because it avoids adding extra initial delay while sending a request to a separate service.

I hope this helps,

Boris

Thank you Boris, I’ll discuss this with Mohamed.

Thanks a lot for the details shared so far.

We got the second mechanism configured, However for the first Mechanism I am not sure how to get HAProxy servers to share Stick-Table records. Is there any possibility you can share some details about this part of your config ?

Best Regards,
Mohamed Abada

Doesn’t the “peers” section work? https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#3.5
You add sth like

peers myhaproxies
    peer proxy-one [proxy1_ip]:1024
    peer proxy-two [proxy2_ip]:1024

Thanks a lot Yasen.

Is there any port forwarding required from HAPorxy to backend servers ?

I don’t think so, generally no – but it all depends on your specific network setup.
If you use the haproxies to balance over shards with jicofo, as Boris mentioned, the haproxies should be able to contact each other on TCP1024.
Add the peers to the stick-table config in the backends section (see example in the haproxy docs).
Also each haproxy has to connect its backend servers, on the ports you configured – you can have ports different from TCP 443 there, as in any case the HTTPS is terminated on the haproxy, what port it uses to connect to the backend is up to you, could be TCP 443, 80 or any other port, matching the backends webservers config.
So port forwarding to the signalling servers - no.
You always need to have a way to connect to the JVB servers though.

Thanks @Yasen_Pramatarov1 which example HAProxy docs are you referring to?

The one in the link I gave - https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#3.5
It’s the configuration documentation of HAProxy 1.8, Ubuntu 18.04 LTS uses this, but if your version differs, use https://www.haproxy.org/#docs
If you meant if there’s a special documentation for HAProxy on Jitsi platforms, I don’t think there is such.