Facial Recognition Suggestion

The question is not if, but when there will be public discusion concerning Jitsi’s use of facial recognition technology. It will will revolve around security issues.

Jitsi and 8x8 would be wise to update it’s Privacy Policy concerning facial recognition with enough detail to squash potential negative press. It would very unwise for Jitsi and 8x8 to assume otherwise.

Hi there!

Thanks for raising this.

We have indeed been developing some facial analysis features. They are not fully-complete yet, and are not deployed nor they are enabled by default.

Here is what we are doing:

  1. Emotion analysis.

We use facial recognition locally, for each participant using face-api. We use this to detect the perceived “emotion”. This emotion is broadcasted to participants in the meeting. The emotions data is kept as room state, just like the speaker stats and will be deleted once all participants leave the meeting. This feature is off by default and you can test it by enabling it like so: https://github.com/jitsi/jitsi-meet/blob/625a4cf42fd17abd0e49f1a4403952849de9ae1b/config.js#L742-L743

The idea behind this feature is to help those who run a meeting do it more effectively by seeing at a glance the overall feeling during a meeting. For JaaS customers (the hosted Platform as a Service 8x8 offers) we plan to use this as a part of a “meeting summary” which the host can generate.

  1. Face centering.

This one is still work in progress, should be in PR soon. The idea is to broadcast the user’s face coordinates in the viewport to other participants so we can better display the user’s face depending how the recipient’s screen is. Depending on the viewport size, we might go down to a 1/1 aspect ratio, so this put the user’s face in the middle, even though they need not be in the center of the image their camera records.

Hope that helps, happy to answer more questions if you have any.

PS: IANAL, so I can’t comment on the implications towards the privacy policy, but it will surely be checked.

Saghul, thank you for your detailed response - very cool!