Efficiency of Jitsi video conference

Dear all,
I try to self-host jitsi with a local Cloud Service. For a server 8 core CPU, 8GB RAM and 200Mbps Internet bandwitth, it costs about 100USD (I do think that AWS has almost the same price).When I host one class of 35 participants, the CPU utilization is about 40%, the bandwidth is about 120Mbps, It means that I can force one server for two conferences with such configuration (not sure in practice, there won,t be the disconnections in such configuration, but at least it fits theoretically). Then to host 100 class at the same time I have to rent 50 servers, the total cost is about 5k USD.
If I use Zoom to host these 100 classes at the same time, I need 100 licenses of 15USD each. The total cost is 1,5KUSD.
My question is that how can I optimize further the self-host jitsi system? Or naturally the SFU system like JVB consume bandwidth like that and we have to accept these conceptual constraints?
Many thanks

4 cores / 4 GB are OK for an additional JVB in my opinion.

You can create a new instance when load increases and destroy it when load decreases using auto-scaling. No need to keep idle instances up

Surprising bandwidth, it seems that every user is uploading a 1080x720 stream. Did you disable every optimization (simulcast) ? Jitsi-meet is tuned by default to be effective, usually users are immediately optimizing it for ‘perfect video’ and complain that it’s expensive. Duh.

Dear @gpatel-fr,
I already put the following settings (related to Video) in the config.js


// Sets the preferred resolution (height) for local video. Defaults to 720.
resolution: 180,

// w3c spec-compliant video constraints to use for video capture. Currently
// used by browsers that return true from lib-jitsi-meet's
// util#browser#usesNewGumFlow. The constraints are independency from
// this config's resolution value. Defaults to requesting an ideal aspect
// ratio of 16:9 with an ideal resolution of 720.
constraints: {
    video: {
        aspectRatio: 16 / 9,
        height: {
            ideal: 320,
            max: 320,
            min: 180
        }
    }
},

// Enable / disable simulcast support.
disableSimulcast: false,

// Enable / disable layer suspension.  If enabled, endpoints whose HD
// layers are not in use will be suspended (no longer sent) until they
// are requested again.
 enableLayerSuspension: true,

// Default value for the channel "last N" attribute. -1 for unlimited.
channelLastN: -1,
videoQuality: {
    maxBitratesVideo: {
        low: 200000,
        standard: 500000,
        high: 1200000
    },
},


Do I miss something ?
Many thanks

Zoom’s pricing model doesn’t actually work like that; you can only buy up to 9 Pro licenses per account. If you’re buying 50 licenses you need to buy Business licenses, so the cost is around 2k USD. Of course, you could theoretically work around that by splitting the usage up across multiple accounts if you don’t mind breaching their terms.

40% CPU usage for 35 participants seems excessive for an 8 core CPU (is it really 8 cores though, or 8 hyperthreads?), and the bandwidth figure seems a little high too.

seems fine, would be interesting to see how it’s possible.

In your place I’d first use sar (sudo apt install sysstat). Run your conference with 35 users, then run:

sar -u 2 5 -n DEV

then post (or attach if you don’t know how to paste) the output.

then inspect the conference:

curl localhost:8080/colibri/conferences

will give you the conference ID.

Then get all the details about it with

curl localhost:8080/debug/conf-id

(replace ‘conf-id’ by the ID returned by colibri/conferences)

Then save the result into a text file and upload the result (use the up arrow to upload a file, do not paste the result it’s too big and will be unreadable directly)

1 Like

The total bitrate for big calls can grow failrly quickly (quadratically with the number of participants), but you have some tools in your toolbelt to keep things under control, with the most efficient one being LastN (that you can set in config.js/channelLastN).

If you want to get an idea of how bitrate grows I’ve written a small python script here that calculates the total bitrate that goes through the bridge for different call sizes and last-n values.

Cheers

1 Like

Thanks for the info, I have trouble squaring it with this post though.

I think the effective last-n value that is used is min(client last-n, server last-n). Does that make sense?

yes, sure, but the post seems to say that lastN is superseded by pagination.

I am sorry, I am not very familiar yet with the changes introduced by pagination, but either Jaya or Hristo can answer all your questions in that other thread that you linked.

Yep, the UI now requests the video that is in the visible area and the rest - the bridge stops forwarding them.

Going back to the original post, from a rough napkin calcuation (below), the bandwidth seems about right (and note that 200Mbit/s is a relatively low network interface spec for a server these days). Bandwidth needs could be reduced by capping the resolution.

But the CPU usage seems very high for one relatively small conference. I would be checking whether the server specs are accurate, remembering that 8 vCPU != 8 cores, and investigating which process(es) are using so much CPU.

assuming all signalling is set up correctly so layer suspension etc is working, and assuming perfect network conditions so all students are receiving teacher at 720p.

bad signalling = potentially more bw usage
bad connectivity = potentially less bw usage

teacher side:
tile view, assuming all 35 students fit on screen
recv 35x 180p (total 7 Mbit/s)
send 1x 720p (4 Mbit/s)

student side (x 35):
stage view (teacher on stage), assuming 6 tiles fit in the filmstrip
recv (stage) 1x 720p (4 Mbit/s)
recv (filmstrip) 6x 180p (total 1.2 Mbit/s)
send 1x 180p (0.2 Mbit/s)

server side:
recv (teacher) 1x 720p (4 Mbit/s)
recv (students) 35x 180p (total 7 Mbit/s)
send (students → teacher) 35x 180p (total 7 Mbit/s)
send (teacher → students) 35x 720p (total 140 Mbit/s)
send (students → students) 35x 6x 180p (total 42 Mbit/s)

total server recv: 11 Mbit/s
total server send: 189 Mbit/s

1 Like

Dear @jbg ,
Thank you very much for the detailed / clear calculation.
From your explanation, I just have some following arguments / questions:
-Can we reduce the stream 180p less than 200Kbps ?
-At the Student side, jitsi-meet already supports the “paging” feature, i.e. showing (in fact requesting ) only 6 tiles/cameras for given instance ? Can you show me where I can see such implementation in the code please ?
-I donn’t agree that 200Mbps is a relative low network interface nowadays. It is true for the local / national network, but if it is about the International connection, in our country, it is not unusual that the bandwidth is just less than 10Mbps for private connection (students connecting to a JVB server located outside of the country)
Many thanks

Can we reduce the stream 180p less than 200Kbps ?

In theory yes, but this is not the main factor in the bandwidth usage. 74% of the egress bandwidth usage at the server comes from sending the teacher’s 720p video feed to the students. Reducing that by capping the max resolution would have a much bigger impact (and be much easier) than trying to reduce the usage of the 180p streams.

At the Student side, jitsi-meet already supports the “paging” feature, i.e. showing (in fact requesting) only 6 tiles/cameras for given instance ? Can you show me where I can see such implementation in the code please?

Yes, it will signal via the Colibri channel that JVB should not send it streams which it is not displaying. It also signals which resolution it should receive, based on the size that the stream is being displayed at. This is automatic and generally works very well without needing tweaking. Keywords to search for if you want to see the implementation are receivervideoconstraints & sendervideoconstraints, selected / onstage / lastn endpoints, layer suspension and colibri. You would look in the lib-jitsi-meet and jitsi-videobridge codebases to see the client-side and server-side part of the implementations, respectively.

I donn’t agree that 200Mbps is a relative low network interface nowadays. It is true for the local / national network, but if it is about the International connection, in our country, it is not unusual that the bandwidth is just less than 10Mbps for private connection (students connecting to a JVB server located outside of the country)

Yes, but 200Mbit/s is only the bandwidth utilisation at the server side. Each individual participant has much lower bandwidth usage, 11Mbit/s for the teacher and 5.4Mbit/s for each student in the analysis above. Of course, if multiple participants are sharing a connection at some level, I can see that this could become problematic. Capping the maximum resolution so that the teacher is sent in lower resolution would help here too, as well as potentially using servers inside the country.

As Jitsi relays video streams from one user to all other users in multiple bit rates according to available bandwidths and network speeds, most performance related issues would occur on the user machines rather than the Jitsi video bridge it self.

Well, not really: the JVB has to handle a lot more bandwidth for a given conference than any single user machine does. For very large conferences, you do start to see load issues on user machines at high participant numbers though.