Synchronization of audio and video tracks or lipsync issues

We have lipsync issues when we use jitsi stack (on meet.jit.si too). The problem occurs on unstable network connection, when the bandwidth changes. When the connection improves, the audio begins to lag behind the video by 2-3 seconds for ten of seconds.

The webrtc-internals shows, that jitsi uses separate RTCMediaStream for audio and video tracks, by default. How does tracks synchronization work in this case?

I’ve tried to enable LipSyncHack in jicofo (https://github.com/jitsi/jicofo/blob/master/src/main/java/org/jitsi/jicofo/LipSyncHack.java#L34-L62), which merges audio and video tracks into one RTCMediaStream. It works better: the audio lag became smaller and converges faster, but a small constant lag between audio and video remains after network problems (looks like a timestamp shift between tracks).

Do I understand correctly that tracks merging is the right solution to solve lipsync issues but there is a problem with timestamp rewriting on jvb? Do you have more details about the problem? Can we contribute?

@Pawel_Domas @Jonathan_Lennox

@Pawel_Domas @Jonathan_Lennox

Could you help, please?

Also this is a big issue for us.
@saghul, this seems to be the same issue as discussed before, which was not resolved by Chrome83.

Will ping the bridge team.

1 Like

Good morning @ivaxer

Were you able to solve this problem? I’m wondering if “enabling LipSyncHack in jicofo” is the same as setting enableLipSync as true in the config.js file.

We know that these issues are caused by a bad internet connection in the client’s side, but we also notice that even if the connection was improved, this delay remains.

Thank you a lot

Regards, - Alan