E2EE on iOS/Safari with enableEncodedTransformSupport

Hi,

we are using jitsi as an pilot in an iframe integration. Everything worked fine by now. But now we got the legal requirement that our conferences have to be end to end encrypted.

I have implemented a direct activation of e2ee, after the room is joined, which is working fine, but we are facing some issues on safari browsers. We know that enableEncodedTransformSupport is experimental, but unfortunately it is somehow a dealbreaker for our management, that we have to exclude iOS devices completely, since the native sdk has e2ee not included too. The restriction that we only support iOS 15.4+ devices was discussable, since there are now other technical possibilities.

So the issues we have with enableEncodedTransformSupport are following:
The audio stream seems to work fine, but the video stream is only visible in p2p, and even then, veeery laggy (more freezed frames than moving ones) and often it just disables the videostream because of bandwidth.

Is there something that can be configured to improve the performance somehow? Or can someone explain why the performance is so bad in this case?

TBH I only did some lightweight testing on desktop Safari, not iOS. I’m not sure how they handle the insertable streams part, it if needs to touch the main thread that would explain what you see, but that’d be surprising.

Regarding video, what codecs were you using? The current implementation is really only suitable for VP8.

Thanks for the info. I looked into our configs and mentioned a mistake there, so that the preferredCodec is not set properly, so its using the default. We will change it to VP8 and check again, i have no access to the server unfortunately, so i have to wait for the change.
BTW, the same issue appeared in desktop Safari as well.

Just an additional question, do you have an idea when the e2ee in the native SDKs is in a testable state somehow, and is there a possibility to provide help in the development there? The native application would also be possibility to provide the e2ee conferences for older iOS devices, which would be really great for us.

Now we have fixed our config, the videoQuality object is now set properly, but it haven’t helped with the safari issues unfortunately

We have set following configuration, seems very basic to me.

{
  "hosts": {
    "domain": "****",
    "muc": "****"
  },
  "bosh": "//****/http-bind",
  "websocket": "wss://****/xmpp-websocket",
  "testing": {},
  "flags": {},
  "disableReactions": true,
  "enableNoAudioDetection": true,
  "enableNoisyMicDetection": true,
  "enableLayerSuspension": true,
  "startWithVideoMuted": true,
  "channelLastN": -1,
  "videoQuality": {
    "preferredCodec": "VP8",
    "enforcePreferredCodec": true
  },
  "enableEncodedTransformSupport": true,
  "enableWelcomePage": false,
  "enableClosePage": true,
  "defaultLanguage": "de",
  "p2p": {
    "enabled": true,
    "preferredCodec": "VP8",
    "stunServers": [
      {
        "urls": "stun:****:443"
      }
    ]
  },
  "analytics": {},
  "deploymentInfo": {},
  "disableDeepLinking": true,
  "hideConferenceSubject": true,
  "mouseMoveCallbackInterval": 1000,
  "makeJsonParserHappy": "even if last key had a trailing comma"
}

So if someone has ideas what the issue could be, i would appreciate some ideas.

Additional info: We are using JWT for authentication.

Work on the native app is in progress, we’ll have a related announcement soon. Not in a testable state though.

Damn.

This shouldn’t matter.

Thats nice to hear, that you make progress there.

Okay, then i guess it will not be a configuration issue here, but somehow an incompatibility with safari currently :slight_smile: Our QA also tested it, faced the same issues as i, safari video is not working at all in our conferences. I guess since this feature is experimental it will not get so much focus right?

But they had another issue, which i hadn’t in my own testings, they faced some issues in the e2ee with chromiums as well. Sometimes the key rotation seemed not to work properly, and some connections seemed to have the wrong key then, so we had the encrypted video and audio in some streams. Happened most of the times when another user joined or left so i assume its because of the rotation, but one time suddenly inside the call without a join/leave event happening. Are we the only one facing these issues?
If yes, what could be the issue? Unfortunately we are using zscaler in our company network, so we are facing a lot of restrictions regarding ports etc. But our customers have similar issues, so we need to fix it somehow.
Btw in this example picture not everyone saw this participant encrypted, some could decrypt the stream properly, it seemed somehow that the new key was not distributed to all participants, but some.

Am i right that the OLM sessions are p2p connection to each participants? What happens if the peer connection drops inside the call because of network issues, or when the connection could not be established because of blocked ports, is there also a turn server involved then? And what happens if even the ports for the turn servers are blocked. And which turn servers are used for this? Is it the config p2p.stunServers? i see that we only have the stun protocol in there? Could this somehow explain the issue?
If nothing else helps, would be the managed e2ee key mode a way to prevent these kind of issues? (there are 2 commands in the external_api somehow, setMediaEncryptionKey and e2eeKey). I would not like to go this way, but for now i have to find a solution.

sorry for that much questioning, currently the jitsi e2ee topic gains a lot of attention in the company and so does the time pressure then :slight_smile:

(just censored the faces of my colleagues, not sure if they would like to be posted here :slight_smile:)

Update, could reproduce it on my local device with myself right now, error in the console is:

and as i told before, somehow its not for everyone the issue (these are 2 instances of chrome in the same call)

could this help someone for now to identify the problem? will dig a little bit deeper tomorrow in this issues, but i am happy for ideas :slight_smile:

Thanks for taking a look! Yep we are aware of a race condition there and currently working on a fix. @Titus-Andrei_Moldova any updates there?

As for Safari, yeah still experimental and not getting much focus yet.

I implemented the e2ee now with the managed keymode with my own key distribution which works fine now, but i would like to know when this bug is fixed, so i can switch to OLM again. Is there an open github issue for this? I just found Participants joining a call get assigned a different / wrong encryption key · Issue #11556 · jitsi/jitsi-meet · GitHub somehow, is this the one for this known race condition?
Is there also a possibility to gain the focus on the safari implementation somehow? Since iOS 15.4 the functionality is available by default on safari and webkit, so it would be nice to enable this for users. Just as information, audio stream works fine, for iOS user we currently limit the usage to sendAudio only.
Should there be opened an github issue for this?

Feel free to open an issue on GH about the Safari support. I’d be happy to take a patch, but we don’t have the time to look into it anytime soon I’m afraid.

okay i will open an issue, i am not too much into the jitsi code yet, but perhaps i find some time to have a look myself in my spare time and find a solution for this issue.

1 Like

Hey, any related news on this? Would appreciate some news, so i can answer some questions for our manegement. Right now the jitsi topic gets more and more focus and some deep integrations are planned into some of our products, but the missing support of e2ee for iOS devices is right now somehow a blocker.

No updates as of yet, sorry. It’s still marked experimental and behind a flag: jitsi-meet/config.js at 6dd04136dea485beae26966589f05ea8359291f7 · jitsi/jitsi-meet · GitHub

ah sorry, i didn’t meant that, of this i am aware. I meant the native sdk e2ee topic, where you wrote that you are working on it. You wrote that you will have an related announcement soon, but i have not seen it, perhaps i missed it.

Ah that. Yep, we announced this: A stepping stone towards end-to-end encryption on mobile - Jitsi so slowly but surely getting there…

1 Like