Thanks for sharing this, is the problem whatever the virtual background ? I’d assume that a plain (just a color)l background could be easier on the Cpu than a photo. Also, it could be possible that the Cpu load could be lowered by switching back to VP8 (that’s another way of lowering the resoluton too)
I’m not sure - our users usually use images. I can’t tell them just use plain color, sometimes I do tell them to get better computers though lol.
If we limit the sender constraints when vbg is activated it should lower the cpu usage even when on VP9 I assume? I’m not sure there’s an option to stop VP9 and instantly switch to VP8 during a live session.
Truth be told, I’m not sure that using a plain color background would be significant so forget it.
Yes, lowering the sent resolution is a good way to lessen the load.
I forgot to say that there are several features that the browser can and will do without saying much but with great impact;
The traditional advice is to disable speech indicator, but there are also new features involving lots of image handling such as face centering, expression detection, what else. If you have low power computers as client, it’s necessary to follow closely the updates to Jitsi-meet to ensure that these new CPU-hungry features are disabled.
We are also experiencing high cpu load with virtual backgrounds. So bad that we removed the toolbar button for the feature. VP9 and high resolution are more important for us. While not cutting edge, our devices are not low end.
Did you use the Tensor js library? We had the exact same issue when we tried applying VBG before Jitsi added the wasm module. When we switched to wasm everything is much much better but for low-end devices.
Totally agree on the video quality. I can’t stand low-resolution.
you can find (almost) all configurable features under the reference file:
this file holds the current options for the installed software version.
Your config.js under /etc/jitsi/meet is never changed when upgrading Jitsi-meet, but default options are used nonetheless and can be found in the reference config.js. Be aware the most of the time, the displayed default value for a new option is the true default, but problems can happen and it’s not unthinkable that sometimes a mistake is made, the displayed default is false, but the default in the code is true. So it’s better to set it explicitly to false when one don’t want an option.
Maybe try to set
faceLandmarks → everything set to false
disableAddingBackgroundImages: true → will stop users to add heavy images
The only cure is obviously to get a more powerful CPU.
Just had this video lagging again this time on a desktop with i7 10th gen and 64GB ram.
I went into the windows task manager and raised the priority of Chrome’s processes - seemed the rectify the issue.
This user did not use VBG but does have some other processes open. CPU was at 40%.
I wonder if Chrome is being throttled after a while being opened in the background.
I wonder how the user realized that the video was lagging if Chrome was in the background.
Whatever, it’s well known that the process in the foreground (focused) has a higher priority under Windows.
About the 64 G Ram, it’s good to remember that some applications can be extremely hungry, Chrome being a case especially when applications are leaking memory - Jitsi-meet has been pointed at recently in a Github issue IIRC.
Also system configuration problems (exchange file), antivirus, all the usual suspects too.
here all the documentation I have found on lip sync (outside of references to ‘our nasty hack’
* Here we're doing audio + video stream merging in order to take advantage
* of WebRTC's lip-sync feature.
* In Jitsi-meet we obtain separate audio and video streams and send their SSRC
* description to Jicofo which then propagates that to other conference
* participants. Because they are separate stream, the WebRTC stack wil not try
* to synchronize audio and video streams which will often result in bad user
* We are not merging the streams on the client which would result in all
* clients receiving them merged, because of few issues:
* 1. Chrome will not play audio for such merged stream if the user has video
* muted: https://bugs.chromium.org/p/chromium/issues/detail?id=403710
* So we will only merge the stream if the owner has his video unmuted at
* the time when it's being advertised to other participant. This is subject
* to some race conditions and will not always work. That's why this hack is
* not enabled by default.
* 2. We need separate streams for doing video mute and screen streaming. When
* video is being muted it's stream is being removed and added on unmute. And
* when we're doing screen streaming the video stream is replaced with the
* screen one. Now when the stream is merged audio and video are becoming
* tracks of the same stream and on the client side we need to be able to see
* when they change. Unfortunately "track added" and "track removed" events
* are only supported by Chrome. Firefox will display the video fine after
* track change, but there will be no event, Temasys has no events and track
* changes are not supported.
Note that ‘Temasys’ is something lost is the mist of times. If you enable this, you may go where dragon lives. But fortuna juvat audaces like they used to say while burying the pioneers.
We just ran a test on meet.ji.si. It appears there’s a correction Jitsi applied to re-sync the audio and video. The video seems to “jump” back into synchronization with the audio. @damencho@gpatel-fr do you know where this correction is happening? Is it on the JVB or the endpoint device?
See the short recorded sample below from my Dropbox:
Side note - the people shown in the video are NOT professional actors