Why was LipSyncHack removed from jitsi videobridge

I would like to ask about the LipSyncHack feature that was introduced then later removed from jitsi videobridge.

With my team we work on a videoconference tool that is heavily built on the jitsi stack (jitsi-meet, jitsi-videobridge, jicofo). One of our current major issues is that the audio and video tracks of the participants are not perfectly aligned to each other quite often. (All the participants are using Chrome.)

We tried to tackle down this desynchronization by leveraging the LipSyncHack, but I realized that this was completely removed from the source code of the videobridge (https://github.com/jitsi/jitsi-videobridge/commit/5d609d58b256b05351a61cf68aae5f1e8dce44d2).

My questions would be:

  1. Why was the lipSyncHack nuked?
  2. Is there an other way to eliminate the audio-video desync issues?

Hi @bigyoo,

The lip sync hack was a workaround a bug where the audio wasn’t played out when the following two conditions were met.

  • 1st audio and video were part of the same media stream (i.e. they had the same msid in the remote description)
  • 2nd the video track of the media stream hadn’t “started” (i.e. it hadn’t received any frames).

In order to workaround this bug, the lip-sync hack was injecting a bunch of VP8 keyframes (180p of blackness) at the beginning of the call, in order to trick Chrome that the video track of the media stream had started.

It turns out that recent versions of Chrome don’t have this bug, hence we could get rid of that ugly hack.

Back to you original question, lip sync is broken currently because of the way we rewrite timestamps and I would recommend against activating it (meet.jit.si has it disabled). Unfortunately we don’t have a good solution to that problem, we need to find the time to simplify our timestamp rewriting.

Hopefully this makes sense.

Cheers,
George