Virtual backgrounds with Tensorflow BodyPix

So the background blur feature is already using the Tensorflow BodyPix library.

Here you can read a lot more about this BodyPix.

Has anyone tried using image backgrounds instead of the blur?
I’m not a developer myself but I’m thinking of asking my development team to add this option for our users. We’re using Jitsi in Healthcare and it would be great if our users can select from a few professional backgrounds like doctor offices or just a nice living room.

Another question is if anyone tried the ResNet50 architecture instead of the current default MobileNetV1 architecture? It seems that the ResNet50 is newer and more accurate.

virtualBackgrounds++… also video backgrounds!

The challenge will be to make it perform fast enough and not use too much CPU on the client. From my previous tests with the blur it was using a lot of CPU. Hopefully we’ll find ways to optimize this. Think it would be a killer feature.

The blur algorithm is surely more cpu intensive than background replacement. So if CPU is an issue with blur, maybe background replacement is a better way to go.