The VideoBridge by default seems to split user Audio and Video into two
separate MediaStreams so that they can be independently removed and
I know that you guys have produced the LipSyncHack, and my layman
understanding of this is that it produces blank VP8 video frames and
rewrites the RTP packets and sequence numbers to independently synchronize
the audio and video streams.
For one, I'm interested in exactly 'how' the above effect is actually
achieved, and if there exists any documentation regarding how exactly it
My second question is, will the audio and video frames first arrive from
the published MediaStreams in sync, or is it possible that there is a delay
between both that needs to be resolved?
I.E: I need to assume the Audio and Video streams that are independently
published start in sync, and track changes in their PTS to assure they
remain in sync.
Can I rely on the fact that the begin in sync, given they are being
produced by two separate MediaStreams?
At the moment my process is to obtain the video PTS and the audio PTS, from
each independent stream, zero them out, and to render according to those
Is there any other consideration I should make with RTCP, or with respect
to the LipSyncHack, in order to assure these streams stay in sync?
My third question is, why when the LipSyncHack is enabled, I seem to
receive a couple Audio Frames at the very start of the stream with an RTP
timestamp of '0', followed by what looks like a normal PTS.
Thanks in advance for any help understanding this,