Flagship VP9 Support: Notes and Discussion

@jallamsetty, @Jonathan_Lennox

Jaya & Jonathan, my developers tell me they don’t see those particular calls in 5142. Has the safari workaround been removed? Or is the safari workaround hiding some other place in the code?

Jonathan — we have your PR working with 5142 and it’s already in the hands of my testers. In addition we see no sign yet of the freezing problems that initiated this recent burst of activity! Without further issues, we should have your vp9 PR in the hands of live users by next week!

Those had been added after the stable release.

Hi @Jonathan_Lennox – Just a quick update: Our 5142 build with your PR passed our testing round and is now in the hands of live users! So glad you did this PR and the recent update. VP9 is truly stunning. Thank you again!

Anyone tried it on iPad using chrome browser? Is it working?
@Freddie @Omatic

Sorry, @metadata, our implementation forces all mobile users to use the app to minimize testing scenarios, so we haven’t tested vp9 on mobile chrome.

1 Like

@Jonathan_Lennox

Hi Jonathan. We’re still seeing that weird bug that simulates LastN=1, meaning that only the active speaker’s video is forwarded. And it only happens with VP9. (we’re using 5142 with your latest PR)

Here’s a screen capture demonstrating it: https://vimeo.com/491671555/f6f6418a3e

It doesn’t reproduce very often… and when it does, it seems to be because one of the participants dips below 3mbps estimated bandwidth. But 3mbps should be enough for two simultaneous video feeds, unless the bitrate controller code is incorrectly using the highest quality VP9 level for calculation instead of a medium level – something like that.

Any ideas? Anything you might want from us? Not sure the logs would help in this case.

Thanks, Rick (Omatic)

2 Likes

@Jonathan_Lennox

If we add “org.jitsi.videobridge.TRUST_BWE=false” in /etc/jitsi/videobridge/sip-communicator.properties then the VP9 freezing problem in gallery view (tile view) goes away.

I believe that confirms my suspicion posted above that the VP9 adaptivity logic is computing the required bandwidth incorrectly, causing it to only forward frames for the dominant speaker in gallery view, even when there is low-but-still-sufficient bandwidth. In low-but-still-sufficient bandwidth conditions, the tile-view adaptivity logic should be forwarding the lower-quality layers and it should be using the lower-quality layers’ bandwidth in the calculation.

Without vp9-support branch jar files, I could get vp9 support by adding these lines into

/etc/jitsi/jicofo/sip-communicator.properties

org.jitsi.jicofo.ENABLE_H264=false
org.jitsi.jicofo.ENABLE_VP8=false
org.jitsi.jicofo.ENABLE_VP9=true

In addition it has better resolution. With this vp9 PR, I have lower video quality and some freezing.

I also add this line

org.jitsi.videobridge.TRUST_BWE=false

into

/etc/jitsi/videobridge/sip-communicator.properties

to disable bandwith estimation, but nothing changed.

Is there anything that I missed?

I am using jitsi-meet stable 5142 and jitsi-videobridge vp9-support branch and jitsi-meet-transform vp9-support branch compiled files and libraries which is mentioned here;

Without this PR, I can get the same resolution from the stream sender (jitsi-videobridge stable 2.1-376-g9f12bfe2-1). But with this PR 720p resolution drops down to 192p. I don’t understand why…

Here is the result with PR

Poor quality on receiver side 720p versus 192p

Sender resolution

Receiver resolution

Sender webrtc-internals details

Receiver webrtc-internals details

Here the results from 5142 (jitsi-videobridge stable 2.1-376-g9f12bfe2-1) without compiling (only install from repository and line adding into sip-communicator.properties)

picture quality

sender resolution

receiver resolution

webrtc internals outbound details

webrtc internals inbound details

I have noticed that decoder implementation is different on inbound side (fallback from external decoder) I don’t know why.

What is the logic when a participant joins an existing call that doesn’t support vp9? will all people drop back to an older codec until that person leaves again and then switch back to vp9?

No, that person will stay on VP8 while everyone else is on VP9.

@Jonathan_Lennox,

We’ve been investigating why the bitrate allocation logic is failing. We suspect the code that is computing the bps of the various vp9 layers is producing bad values.

In our logging, we’re seeing values all over the map, with 720p video said to be as low as 68kbps, and 180p video sometimes in the megabits.

I’m including a bit of the log, and the code that created the log (which we placed in singlesourceallocation)

1609248204409 => 604089d0 => 720 => 67.1260 kbps => 30.0
1609248204409 => c1d1b436 => 720 => 2.5338 mbps => 30.0
1609248204672 => 604089d0 => 720 => 68.0420 kbps => 30.0
1609248204672 => c1d1b436 => 720 => 2.5880 mbps => 30.0
1609248205766 => 604089d0 => 720 => 68.3550 kbps => 30.0
1609248205766 => c1d1b436 => 720 => 2.6113 mbps => 30.0
1609248208833 => 604089d0 => 720 => 68.4990 kbps => 30.0
1609248208834 => c1d1b436 => 720 => 2.6181 mbps => 30.0
1609248211301 => fc1e8be9 => 180 => 596.2260 kbps => 30.0
1609248211302 => c1d1b436 => 720 => 2.5419 mbps => 30.0
1609248211485 => fc1e8be9 => 180 => 605.1250 kbps => 30.0
1609248211486 => 604089d0 => 720 => 68.9570 kbps => 30.0

That either ruins the bandwidth savings for the active video sources or breaks with encryption of the streams later because no transcoding of video is possible. Doesn’t sound like a good idea doing it this way.

Yes this is the plan. we are in the process of implementing this in the client. When a participant that doesn’t support VP9 joins a VP9 call, all the participants will switch to VP8 automatically and switch back to VP9 when that participant leaves.

Oops! @jallamsetty I assumed that because you were making an exception for Safari to work in a VP9 environment that meant any client that didn’t support VP9 would just stay on VP8 while others ran in VP9. With this clarification now, I have to ask: will we have the option to enforce only VP9 connections in our environment (beyond merely banning Safari or the like)?

This means that clients that do not support VP9 will not be able to decode video from the clients sending VP9 if everyone doesn’t fallback to VP8. You would see only black screens on the VP8 only client which doesn’t make sense. I haven’t thought about the option for enforcing VP9 only connections, the plan was to make the codec selection logic generic so that we can do the same for any future codecs like AV1.

Having said that, I think I can add an option to config.js for this, something like enforcePreferredCodec
under videoQuality so that the automatic fallback to VP8 doesn’t happen.

3 Likes

I definitely see your point here and I agree with you. However, for most of us who are early adopters of VP9, it’s often for the outweighing benefits that it offers; benefits we won’t want to lose just because one client can’t connect on VP9. It’s easier for us to demand that users connect using VP9-capable clients (for instance, I already enforce a ‘ban’ on Firefox because of its many vagaries and I nudge my users to use the Electron app). Although enforcing VP9 means ios clients can’t connect (for now), that’s a price I’m willing to pay. Having everyone ‘downscaled’ to VP8 on the other hand, no buenos! :grin: So yes, the flag enforcePreferredCodec would be very useful.

Thanks for all you do, @jallamsetty!

1 Like

UPDATE: We fixed our VP9 video-freezing-in-tile-view problem by updating CLIENT-SIDE settings. Namely, simulcast needs to be enabled even though VP9 uses SVC instead of simulcast. If Simulcast is disabled, VP9-encoded video is still sent, but it’s sent in such a way that the server doesn’t decode the available layers, giving it no choice but to forward full resolution of all videos, even in tile view. In low/moderate bandwidth conditions, that results in only the active speaker’s video being forwarded sometimes.

2 Likes