Scalable JVBs on AWS

Hi. I have a scalable JM setup on AWS with a single JM server (web app, Jicofo, Prosody) and a scalable set of JVB instances. I setup Octo to allow conferences to span multiple JVBs using this ReadMe. The scalable JVBs are setup as follows:

An AWS AutoScale Group that scales the JVB instances up and down based on average CPU utilization across all JVBs. This ASG is configured with a minimum of 1 JVB instance. I set up the scaling policy to monitor the average CPU across all JVBs… if it passes a maximum threshold of 70%, the ASG scales up and initializes a new JVB. If the average CPU falls below a minimum threshold of ~25%, the ASG scales down and decommissions one JVB. I use a startup script to configure each new JVB’s sip-communicator.properties with a unique nickname and the properties needed for Octo. Jicofo’s bridge selection strategy is set to SplitBridgeSelectionStrategy.

This works fine. I tested having multiple users join with audio and video such that the average CPU passes the 70% threshold. This triggered a scale-up and a new JVB was spun up. Incoming users were then connected to the new JVB and they were still able to communicate with everyone else in the conference. So scaling up works fine and I’m confident the JVBs and Octo are configured properly.

But here are two issues that I’m facing:

  1. From what I can tell, Jitsi does not redistribute already present participants to other JVBs to balance the load. Meaning, when my first JVB’s CPU is at 70% (with say 15 users) and a new JVB is spun up, the initial 15 users remain connected to the first JVB thus maintaining its CPU at 70%. It would be best if Jitsi can automatically redistribute users to balance the load (by either # of users, # of packets, CPU%, or whatever other metric) across all present JVBs.

  2. When my ASG scales down and decommissions a JVB instance, the clients connected to that JVB instance freeze up and are not automatically connected to any other active JVB. Instead, they would stay frozen (they don’t send audio/video nor receive any) until they refresh the page or reconnect to the meeting and are then connected to an active JVB.

I was wondering if there are any solutions to these two problems. Did I miss anything in configuration? Can Jitsi Meet, when it detects a new JVB instance, redistribute already active clients to balance them across available JVBs? And, when a JVB instance is taken down, how can clients connected to it automatically get shifted to another available JVB?

Thanks.

This is by design.

You need to gracefully shutdown the jvb. It will stay active till the last participant leaves. You will need to send regularly some aws heartbeats to keep the machine up till it is still in graceful shutdown and jvb is still running.

Well, the ice restart to connect the jvb have a minimal visual cut of audio and video, but anyway doing that is really tricky. Let’s say you we in a pick and several jvbs come up one after another, you risk the same client to have several blips while moved through jvbs … Handling this case is not trivial at all. Same is and for jvb when going down, the best bet there is graceful shutdown.

We may change stuff in the future about migrating participants to different jvbs, we are adding stuff that makes it possible, but nothing is ready or final yet.