Flagship VP9 Support: Notes and Discussion

One of my main goals recently is to improve Jitsi by adding proper VP9 support across the board. I’m aware I’m mad, yes.

There appear to be several pieces that are not fully assembled, and a few parts I haven’t actually worked out the details on yet that I would appreciate feedback on.

Initial Observations/Thoughts

If you force Jitsi to use VP9, by setting org.jitsi.jicofo.ENABLE_VP9=true in /etc/jitsi/jicofo/sip-communicator.properties (and disabling VP8/H264), it more or less works. But this requires disabling the other codecs, and it doesn’t offer “proper” support.

I’ve added (in my branch, haven’t finished poking so it’s not pushed up yet) “preferVP9” support in the style of preferH264 support, and this seems to work fine for P2P connections. I think getting this working should be fine, though it would still require server configurations to enable (I don’t think enabling VP9 by default is the right option, see below).

Non-simulcast VP9 should be fairly straightforward as well, though I’m not quite there. Simulcast VP9 support will require jvb updates, though there’s a PR for jvb1 that enables most of the support - I’ll figure out how to port this to jvb2’s code, and it looks like some fairly minor changes to properly handle keyframe support. In theory.

Open Issues

However, there are still some issues related to VP9 that I don’t fully understand how to address.

The first is that if a client joins a bridge and doesn’t support VP9, the system should fail back to VP8. I’m unclear as to if this is a theoretical issue in the “Should do at some point…” range, or an actual issue that needs to be addressed for the feature to be at all useful. I know halfway recent Chrome browsers support VP9 (it was added around v48), and experimenting with mobile devices indicates they do support VP9 as well.

The second, (IMO) more important issue is that some client systems, especially lower power ones, simply cannot encode to VP9 at a good resolution/framerate. I’ve got an older Chromebook that can do it, but at a very low framerate - there’s no hardware support and not enough CPU support to encode at a useful quality. For this system, VP9 is actively worse than VP8, because of the reduction in output quality from being CPU limited. It might be possible to detect that a tab is pegging a CPU and drop back to VP8, or it might be possible to identify if a platform has hardware VP9 support. Or allow users to select, but this seems less than helpful.

Any suggestions on either of these would be appreciated. I’ve got time to throw at the project, and am already deep in the related code chunks. Is there anything else obvious I’m missing?

5 Likes

@Arthur_TOUMASSIAN worked with vp9 and libjitsi/videobridge1. Perhaps the two of you can collaborate on this.

Well, i’m looking at it. If i manage to do something, i’ll keep you informed!

@speedy01 @Syonyk

Have you tried to start jicofo with:
org.jitsi.jicofo.ENABLE_H264=false
org.jitsi.jicofo.ENABLE_VP8=false
org.jitsi.jicofo.ENABLE_VP9=true

I’m getting a working plain VP9 (video and screen share). Can you test and inform me?

Yes, that config does seem to work, as long as endpoints all support VP9. I played with that a week or so ago and it seemed to work, though I don’t believe all the mobile clients support VP9 properly (I’m of the impression iOS doesn’t, at least).

I’ve tested the one on app store and it worked actually!!

So you havemodify to mpodify jicofo to put VP9 as first option with vp8 fallback

I’ve been messing around a bit more (I’ve got codec reporting turned on for my test instances based on a change I’ve made), and it looks like iOS does work properly with VP9 - BUT, it will only push up VP8. And… this broadly seems to work. I’ve got some video coming into a node as VP9, some coming in as VP8. Which is fine.

I’m still working on validating simulcast support for VP9, though. I’m not sure if that’s working properly yet.

Actually, it looks like a recent iOS device can do VP9 for P2P - just not JVB. Hm…

The most notable endpoint that cannot do VP9 at this point is the Safari browser. (Safari isn’t fully supported yet in Jitsi-meet, but we’re working on it!). Our current thought is to put the information about what codecs a client can receive into its presence information, and in a bridge call have other clients base their decision about what to send on the information they’ve received (as well as their local CPU capabilities).

In terms of JVB support, I absolutely want to support both simulcast and temporal scalability for VP9, with proper stream rewriting like we have for VP8. Spatial scalability (which as I understand it Chrome sends for screenshares by default) will be somewhat more challenging.

2 Likes

Jonathan - that makes sense, though I would have a mild preference to deploy the option for people as a “Beta, disabled by default, but turn it on if you want to try…” sort of feature earlier if possible. I’ve got it mostly working for some sane set of “working” values in a dev instance, will be pushing those changes up soon for someone to poke at.

The main pain point I’m trying to fix, for my use, at least, is that in many areas in the rural USA, we don’t have good upload bandwidth. A sub-1Mbit upload is still common for many DSL users, and even “good” connections may only be 2-3Mbit upload, which you may not get depending on network congestion. Being able to push VP9 up over such a choked pipe is very definitely a useful option, and in many cases, I think disabling simulcast may make sense too - if someone is only pushing up 500kBit, splitting that into multiple streams doesn’t make a lot of sense (IMO). We’ve already trained people not to use Safari, and it’s not been a fully supported browser.

The temporal scalability features look really exciting to me, and I’d prefer that over simulcast, due to the choked upload reasons - use all the bandwidth for a high quality stream and let the server sort it out. But at least for us, getting “some” VP9 support online soon is better than having nothing for a while.

If either of you happen to be bored, the plumbing I’ve done so far is here:

I’ve added the config options to the config whitelist over in jitsi-meet, but that’s only a minor change.

Experimentally, this seems to work. It appears that jicofo is enabling VP9 support by default - https://github.com/jitsi/jicofo/blob/04e5f27805d9e1aed0d4caca057262c06c7bd1b3/src/main/java/org/jitsi/jicofo/util/JingleOfferFactory.java#L217

With my changes, if I set preferVP9 in the config file sections, clients will use it if able, and will transition back to VP8 if needed (if VP9 isn’t prioritized in the general config section).

I expect there are a few things I’m missing, and I’m not sure where to set the default values for things (it might be good to set the default of disableVP9 to true to maintain existing behavior), but I’d appreciate some feedback on where I am so far!

For testing, showing the codec in the stats is quite useful - I have a few pull requests in for that as well.

5 Likes

looks like I’m able to pass vp9 too…but bandwidth seems higher than that of vp8 and h264.

what are you guys seeing?

What’s the quality? WebRTC generally tries to use the full available bandwidth, but it should look better with VP9 by a large margin.

looks pretty good. just noticed higher bitrate, which I found strange. maybe its more to do with simulcast not working with vp9? I guess I was expecting much lower bitrate than with vp8 and h264. I’ll run some more test :smiley:

Without a way to tell WebRTC to target a quality, it will fill the channel up to some reasonable limit.

On my list of things to figure out is how to specify quality, or at least set a custom bandwidth limit. My connection isn’t exactly amazing, and I’d prefer to be able to use less of the connection.

For now jitsi implements minBitrate only on jicofo side.

I recommend you to modify lib-jitsi-meet SDP management part to add webrtc bandwidth limiters

1 Like

I can confirm, chromium based browsers (chrome, brave, opera, new edge, vivaldi) since version 48 and firefox browser since version 46 support VP9. Safari does not support VP9 and recently added support to VP8. So all OS (GNU/Linux, Windows, macOS and Android) are covered with the exception of iOS that does not support a different browser than safari (all the others are safari browser with a different UI).
Regarding the hardware, practically all the modern SoC and video card have VP9 encoding/decoding capabilities. Moreover, if they do not have encoding capabilities, as common, they could use software implementation via CPU SIMD instructions.

Regarding to bandwidth, VP9 allows to save up 40% of it compared to VP8 at the same quality.

1 Like

Yes, this is my plan. I’d like to be able to have an “advanced” interface that allows one to simply specify parameters - min, max, or “Aim for this bitrate, yes, I know, your estimation says it won’t work, I say it will, just bloody well do it.”

My experience with an older Chromebook is that it struggles with VP9 encoding, but it will do it. I’ve not tested it over low bandwidth links.

Unfortunately, it’s nice out, and this is my rainy day project right now, until I get a few other things finished around the property.

1 Like

Any update on this? Really looking forward to seeing VP9 working in jitsi!

I think it will improve the UX immensely

Thanks!

Hi VP9 enthusiast and @Jonathan_Lennox,

I learnt that Intel’s SVT-VP9 is soo blazing fast and been memory-efficient that they could retire x264 or (has been proven to be retired) google’s libvpx-VP9 decently. They have adopted the encoder in their simple WebRTC project as well here: https://github.com/OpenVisualCloud/Video-Conferencing-Sample . The patch for FFMPEG is available for service here https://github.com/OpenVisualCloud/SVT-VP9/tree/master/ffmpeg_plugin.

All you have to do only by recompile or patch the distro’s FFMPEG and switch current libvpx VP8 or VP9 with libsvt_vp9 or make another fallback with libsvt_vp9 just for test.

I love to see Jitsi can use Intel’s matured VP9 software encoder for RTP payload’s codec and Jibri’s x11grab as well (and maybe someday also with SVT-HEVC, that this still considered too young at the moment)

3 Likes

Weather’s too good lately, I’ve been working on solar installs. Sorry, no updates! Supposed to be some worse weather later this week. But VP9 generally seems to work at a basic level.

I don’t think the encoding system is really something Jitsi can pick, though. Wouldn’t it be up to the browser?