VP9 [with bridge support] soon cometh! Do a happy dance! 🕺🏾

Okay, I’m probably more excited than I should be, but hey, who really has the rights to any meter on excitement? :grin:

WARNING
Let me warn upfront that THIS IS STILL EXPERIMENTAL. Don’t take anything I say here as the Jitsi Dev Team’s official stance; this is just me sharing my finding from an experiment, using codes that have not been fully vetted yet, nor formally released.

Now I’ve gotten that out of the way, let’s look at the fun stuff. If you’re somewhat familiar with codecs, you know that VP9 offers improvements over VP8. Jitsi has never officially supported VP9 (as far as I know), but in previous versions, it was possible to enable it, with some limits and caveats. In my quest to get the best quality possible, I’ve been bugging the Dev team to point me in directions on how to enable it - understanding the many limitations that exist in its current state (won’t work on Safari on older macOS, ios and even some other mobile devices). Well, it finally paid off!

SUMMARY
I was able to enable VP9 with bridge support using @Jonathan_Lennox’s PR. This involved using code that’s not been officially released or committed. The results were unquestionably fantastic!

BRIEF OF STEPS
Again, this is not officially released yet, so I did so at my own risk. I downgraded to an older version of JM to skip enhancements made to the current versions (which are somewhat incompatible with the PR). Going with instructions from @Jonathan_Lennox and some help from @bbaldino, I built a new JVB as well as the Jitsi-media-transform jar it depends on (from @Jonathan_Lennox’s PR) and dropped those in my installation. I then made the time-tested changes to jicofo for force VP9 and disable VP8.

TEST ENVIRONMENT
Server (MyJitsi)
Bare metal server
CPU - 8 cores 2.33GHz
RAM - 16GB
HDD - 300GB RAID
Available bandwidth (ethernet) - 300Mbps up/down

Clients
Client1 – Mac, OSx 10.13.6, Quad Core, Inter i7 – 2.5GHz, 16GB RAM (Brave)
Client2 – Mac, OSx 10.13.6, Quad Core, Inter i7 – 2.5GHz, 16GB RAM (Chromium)
Client3 – Android Phone (Jitsi Android App)
Client4 – PC Laptop, Windows 10 Pro, Duo Core, Intel i3 - 2.53GHz, 4GB RAM (Jitsi Electron App)
Client5 – Mac, OSx 10.13.6, Quad Core, Inter i7 – 2.5GHz, 16GB RAM (Jitsi Electron App)

Client Camera Resolutions
Client1 - 1280x720 max
Client2 - 1280x720 max
Client3 - 640x480
Client4 - 640x480 max
Client5 - 1280x720 max

RESULTS
First, I confirmed it was actually using the VP9 codec by checking on webrtc internals.

Client Impact
I checked the impact on client machines using the weakest machine I have available (Client4). My findings with 4 clients (Client1, Client2, Client3 and Client4) were as follows:

In Tile View:
CPU - 25.7%
Upload - 456Kbps
Download - 4.1Mbps

In Stage View:
CPU - 19%
Upload - 224Kbps
Download - 1.9Mbps

Server Impact:
Absolutely minimal! Seemed even lighter than with VP8 (and VP8 is already VERY low)
CPU - 4%
RAM - 0.9%

Findings

Screen Shot 2020-12-09 at 2.56.56 PM

With 5 clients connected (in Tile view), I noticed bandwidth usage went even lower, checking on Client4 [I forgot to take the actual bandwidth readings]:
CPU - 28.9%
Upload - 184Kbps
Download - 3.0Mbps

And to me, since no test is complete without getting Jibri’s blessing :grin:, I invited Jibri to join the meeting and recorded for a little over a half-hour. Alas, Jibri was happy too! No undue memory or CPU spikes. :man_dancing:t5:

Small concerning observation: At one point, Client3 lost connection and video froze. :confused: I simply turned off the camera on Client3 and turned it back on and everything went back to normal (in fact, it came back with a better resolution - 480/360). Not sure if this is at all related to my VP9 implementation because I think I remember having the same issue once or twice before this.

CONCLUSION
I am BEYOND excited at my findings! Like I mentioned in my previous post detailing my tests, VP9 on Jitsi is definitely the way to go. The difference is so glaring, once you experience it, you’ll never be satisfied with anything less!

Well, happy Jitsi-ing, my friends! :smiley: Now, I must excuse myself because I need to get some friends on a Jitsi call to do our happy Jitsi dance. :joy:

UPDATE: Edited to add graphs from server monitoring

9 Likes

Hey,@Freddie thanks a lot for sharing your experience with the community!
Would it be possible to gather the exact steps in this thread (like specific commit hashes that you had to revert to, instructions for building the JB and the jitsi-media-transform, etc)?

I tried to follow the Flagship VP9 thread, but there’s outdated information all over the place, where things didn’t eventually work out.

Thanks again for summing up your experience in this game-changing update! It’s really inspiring to see that Jitsi can match the experience (or provide an even better experience) of the closed-source alternatives!

@Freddie I followed below steps for docker. How to verify its VP9(it is showing vp9 when I hover over connection-status but it is because of jicofo)

  1. Created a directory jitsi. Cloned jitsi-media-transform/vp9-support and jitsi-videobridge in jitsi directory.
    commands:

    1. git clone --single-branch --branch vp9-support https://github.com/JonathanLennox/jitsi-media-transform.git
    2. git clone https://github.com/jitsi/jitsi-videobridge.git
  2. Go to jitsi-media-transform directory and execute command mvn install.

  3. Go to jitsi-videobridge directory and execute mvn install -P buildFatJar command.

Step 3 will create jitsi-videobridge-2.1-SNAPSHOT-jar-with-dependencies.jar file in jitsi-videobridge/jvb/target/ directory.

Now copy this jar file to docker-jitsi-meet directory and map this jar file for jvb.

volumes:
    - ${CONFIG}/jvb:/config:Z
    - ./jitsi-videobridge-2.1-SNAPSHOT-jar-with-dependencies.jar:/usr/share/jitsi-videobridge/jitsi-videobridge.jar

I’m also using this PR for Jicofo to enable VP9 and disable VP8 and H264 at jicofo level.

jvb logs

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-set-timezone: executing...
[cont-init.d] 01-set-timezone: exited 0.
[cont-init.d] 10-config: executing...
[cont-init.d] 10-config: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Dec 14, 2020 3:17:02 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Initialized newConfig: merge of /config/jvb.conf: 1,application.conf @ jar:file:/usr/share/jitsi-videobridge/jitsi-videobridge.jar!/application.conf: 1,system properties,reference.conf @ jar:file:/usr/share/jitsi-videobridge/jitsi-videobridge.jar!/reference.conf: 1,reference.conf @ jar:file:/usr/share/jitsi-videobridge/lib/jitsi-media-transform-1.0-203-g67b9a7b.jar!/reference.conf: 1
Dec 14, 2020 3:17:02 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: loading config file at path /config/sip-communicator.properties
Dec 14, 2020 3:17:02 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Initialized legacyConfig: sip communicator props (no description provided)
Dec 14, 2020 3:17:02 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Reloading the Typesafe config source (previously reloaded 0 times).
Dec 14, 2020 3:17:03 PM org.ice4j.ice.harvest.MappingCandidateHarvesters initialize
INFO: Using AwsCandidateHarvester.
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Initializing a new MucClient for [ org.jitsi.xmpp.mucclient.MucClientConfiguration id=shard domain=auth.example.com hostname=xmpp.example.com port=null username=jvb1 mucs=[jvbbrewery@internal-muc.example.com] mucNickname=4052f4d40f29 disableCertificateVerification=true]
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
WARNING: Disabling certificate verification!
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: LastNReducer with reductionScale: 0.75 recoverScale: 1.25 impactTime: PT1M minLastN: 0 maxEnforcedLastN: 40
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: TaskPools detected 32 processors, creating the CPU pool with that many threads
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Dispatching a thread to connect and login.
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
WARNING: Running with open files limit 1048576 (hard 1048576), thread limit null (hard null). These values are too low and they will limit the number of participants that the bridge can serve simultaneously.
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Starting with 60 second interval.
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Started with interval=10000, timeout=PT30S, maxDuration=PT3S, stickyFailures=false.
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Not starting CallstatsService, disabled in configuration.
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Starting public http server
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Base URL: wss://example.com/colibri-ws/jvb1
Dec 14, 2020 3:17:03 PM org.ice4j.ice.harvest.StunMappingCandidateHarvester discover
INFO: Discovered public address 1.2.3.4:33187/udp from STUN server 140.238.252.84:443/udp using local address 172.20.0.5:0/udp
Dec 14, 2020 3:17:03 PM org.eclipse.jetty.util.log.Log initialized
INFO: Logging initialized @1395ms to org.eclipse.jetty.util.log.JavaUtilLog
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Registering servlet at /colibri-ws/*, baseUrl = wss://example.com/colibri-ws/jvb1
Dec 14, 2020 3:17:03 PM org.eclipse.jetty.server.Server doStart
INFO: jetty-9.4.35.v20201120; built: 2020-11-20T21:17:03.964Z; git: bdc54f03a5e0a7e280fab27f55c3c75ee8da89fb; jvm 11.0.9+11-post-Debian-1deb10u1
Dec 14, 2020 3:17:03 PM org.eclipse.jetty.server.handler.ContextHandler doStart
INFO: Started o.e.j.s.ServletContextHandler@5c87bfe2{/,null,AVAILABLE}
Dec 14, 2020 3:17:03 PM org.eclipse.jetty.server.AbstractConnector doStart
INFO: Started ServerConnector@7966baa7{HTTP/1.1, (http/1.1)}{0.0.0.0:9090}
Dec 14, 2020 3:17:03 PM org.eclipse.jetty.server.Server doStart
INFO: Started @1558ms
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Starting private http server
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Connected.
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Logging in.
Dec 14, 2020 3:17:03 PM org.eclipse.jetty.server.Server doStart
INFO: jetty-9.4.35.v20201120; built: 2020-11-20T21:17:03.964Z; git: bdc54f03a5e0a7e280fab27f55c3c75ee8da89fb; jvm 11.0.9+11-post-Debian-1deb10u1
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Authenticated, b=false
Dec 14, 2020 3:17:03 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Joined MUC: jvbbrewery@internal-muc.example.com
Dec 14, 2020 3:17:04 PM org.ice4j.ice.harvest.MappingCandidateHarvesters maybeAdd
INFO: Discarding a mapping harvester: org.ice4j.ice.harvest.AwsCandidateHarvester, face=null, mask=null
Dec 14, 2020 3:17:04 PM org.ice4j.ice.harvest.MappingCandidateHarvesters initialize
INFO: Using org.ice4j.ice.harvest.StunMappingCandidateHarvester, face=/172.20.0.5, mask=/1.2.3.4
Dec 14, 2020 3:17:04 PM org.ice4j.ice.harvest.MappingCandidateHarvesters initialize
INFO: Initialized mapping harvesters (delay=1250ms).  stunDiscoveryFailed=false
Dec 14, 2020 3:17:04 PM org.glassfish.jersey.server.wadl.WadlFeature configure
WARNING: JAXBContext implementation could not be found. WADL feature is disabled.
Dec 14, 2020 3:17:05 PM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
WARNING: A provider org.jitsi.rest.Health registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.jitsi.rest.Health will be ignored.
Dec 14, 2020 3:17:05 PM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
WARNING: A provider org.jitsi.rest.Version registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.jitsi.rest.Version will be ignored.
Dec 14, 2020 3:17:05 PM org.eclipse.jetty.server.handler.ContextHandler doStart
INFO: Started o.e.j.s.ServletContextHandler@113e13f9{/,null,AVAILABLE}
Dec 14, 2020 3:17:05 PM org.eclipse.jetty.server.AbstractConnector doStart
INFO: Started ServerConnector@5f2f577{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
Dec 14, 2020 3:17:05 PM org.eclipse.jetty.server.Server doStart
INFO: Started @3078ms
Dec 14, 2020 3:17:13 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Performed a successful health check in PT0.00147S. Sticky failure: false
Dec 14, 2020 3:17:23 PM org.jitsi.utils.logging2.LoggerImpl log
INFO: Performed a successful health check in PT0.000027S. Sticky failure: false
Dec 14, 2020 3:17:33 PM org.jitsi.utils.logging2.LoggerImpl log

Chrome console logs

Call and Voice quality were superb and CPU consumption was low. I tested it with 4 participants.

Few Issues:

  1. Didn’t work on Ipad( with chrome or safari). when second participant joins I get something went wrong error. Working fine on Android devices. I think it is because of Jicofo PR I’m using. I have other setup which is using stable images( VP9 not supported) and it is working.

  2. Some issues on Mozilla too. some stats are not showing up and video and audio quality aren’t up to chrome level.

1 Like

@metadata you can confirm it’s Vp9 through chrome://webrtc-internals/. You’ll see the codec there listed as VP9. ios (iPad and iPhone ) does not support VP9 yet; even older versions of Safari don’t support VP9. These are constraints from Apple, not Jitsi. I generally don’t use Firefox on my Jitsi installation because it’s just rife with issues. VP9, like I implemented it, is still experimental, so I’m using it in a very controlled environment (Chromium-based clients - including the Electron app - and Android).

1 Like

Maybe I’m missing something, but I don’t understand the point of those figures if you don’t show the corresponding figures with VP8.

For example CPU around 20% and bandwidth between 1680 and 2092, look nice, but is it better or worse than with VP8 ?

Yes, you are missing something (somewhat). Check my previous post (like I referenced in the conclusion) - it will show you the numbers I got doing a similar test with VP8 (and even Zoom). -Platform Comparison - Zoom vs Jitsi (VP8, VP9); A comprehensive Test

Besides that post, I think what’s being highlighted here is the quality. The difference between VP8 and VP9 is simply incredible.

1 Like

Yes its VP9.
Inbound

outbound

Here In step 4. Do we have to explicitly replace jitsi-media-transform like below?

volumes:
   -  ./jitsi-media-transform-1.0-SNAPSHOT.jar:/usr/share/jitsi-videobridge/lib/jitsi-media-transform-1.0-203-g67b9a7b.jar

Here jitsi-media-transform-1.0-203-g67b9a7b.jar is the jar file present in jvb container by default. I’m confuse here whether we have to replace it or not? I tried both by replacing it and not replacing it, Both the time I can see VP9 in webrtc-internals.

I think when we execute mvn install in jitsi-media-transform directory then it caches all the config at /home/alpha/.m2/repository/org/jitsi/jitsi-media-transform/1.0-SNAPSHOT/ and when we execute mvn install -P buildFatJar in jitsi-videobridge directory then it uses this cached jitsi-media-transform.jar file.

You could be right, I’m not certain. Building the fat jar of JVB pulls in all its dependencies and that may very well include those referenced in the media-transform jar (again, I’m not 100% sure of this). However, it’s important to note that the reason why we need this new JVB isn’t just to get VP9 (jicofo configuration alone would have done that), it’s to get VP9 AND SVC through the bridge. The JVB we’re building has a jitsi-media-transform dependency, that’s why we build jitsi-media-transform (from the PR) and then the JVB.

No, you don’t need to change the jar name for media-transform to match the one in your installation. Just remove that one and replace with the one you built. For JVB though, you need the names to match.

1 Like

Hey @Freddie thanks for the reply!

Even though all of the steps were completed successfully, I cannot seem to get the jvb running.

I build the jitsi-media-transform first
cp the produced jar in the /usr/share/jitsi-videobridge/lib directory
then build the jitsi-videobridge jar
and cp jvb/target/jitsi-videobridge-2.1-SNAPSHOT-jar-with-dependencies.jar /usr/share/jitsi-videobridge/jitsi-videobridge.jar (removed the original jitsi-videobridge.jar first)

I get the following errors in the jvb logs

Error: Could not find or load main class org.jitsi.videobridge.MainKt Caused by: java.lang.ClassNotFoundException: org.jitsi.videobridge.MainKt OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

I’m on Linux vb2 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux

Any ideas?

Great post @Freddie

Thanks so much for sharing. I too am very excited to get proper VP9 support (with SVC) working in Jitsi.

Anyone know when we can expect this to be released (in unstable) so we don’t have to hack around to get it working?

One more question. A lot of my friends are heavy users of iOS, so I need to have fallback codecs (VP8 or H264) for when they join.
From what I gathered, right now VP9 is all or nothing deal, but I do hope when this lands in the unstable repo that we can configure a codec priority list with VP8 and H264 as fallbacks. Anyone know if this will be the case?

Anyway, thanks again for sharing your findings Freddie. Much appreciated.

@Freddie thanks for your sharing, what the whole bandwidth consumption on upload and download for this test? I see only consumption per user.

@Peter_Villeneuve it sounds like it’s pretty close to getting there, but I don’t have any definite date. And in fact, the concern you have about the fallback to VP8 is the primary reason why the devs haven’t pushed it out. They are absolutely bent on making sure that fallback works so a whole demographic of users is not cut out from using the upgrade.

1 Like

@Yassine I didn’t actually write down the stats for bandwidth consumption for the server, but I noted it was generally better than on VP8. To be frank, my biggest concern was the impact on the clients since the often-touted downside of VP9 is the heavy CPU demand. Interestingly, I didn’t really see this in my tests.

1 Like

Ran the test again. Captured server bandwidth this time. Graph added.

1 Like

@slack0, I have the same error in the jvb logs:

Error: Could not find or load main class org.jitsi.videobridge.MainKt

Did you manage to find the issue ?

Thanks @Freddie ,I see that only 5 persons, consumes a lot of resources in term of bandwidth.

Download : 600Mbps and Upload: 1.5Gbps.

BR
Yassine

You’re welcome, @Yassine.

How does this compare with VP8, do you know?

I thank is too much that five participants consume in Download : 600Mbps and Upload: 1.5Gbps.
it is too much, imagine if there is 15 persons in the meeting how much bandwidth you need?!

Hello @arzar

I’ll write down a series of steps that got it working for me.
The following instructions were tested on a hetzner cloud vps (CPX11 2 vCPU, 2GB RAM). Keep in mind, I had trouble with building the project in a less powerful vps.

Before you start, make sure you have installed mvn and unzip packages.

Operating System: Debian GNU/Linux 10 (buster)
Kernel: Linux 4.19.0-8-amd64
Architecture: x86-64

Jitsi media transform

  • Clone the jitsi-media-transform fork of @ Jonathan_Lennox (GitHub - JonathanLennox/jitsi-media-transform)
  • Checkout vp9-support branch (latest commit as of this writing is 35439ba76bf400b0235f7df968dc6f2f6e30264e)
  • Run mvn clean install -DskipTests inside the project folder (clean and -DskipTests are optional, feel free to exclude them)
  • After the process is finished, you should see a jitsi-media-transform-1.0-SNAPSHOT.jar inside the target folder of the project. Replace the existing .jar in your server with the one produced here (I suggest to keep the original one as a backup if anything goes wrong).
    Assuming you’re in the jitsi-media-transform directory and a default installation of the jitsi-videobridge project, rm /usr/share/jitsi-videobridge/lib/jitsi-media-transform-whatever.jar && cp ./target/jitsi-media-transform-1.0-SNAPSHOT.jar /usr/share/jitsi-videobridge/lib/ to remove the existing jar and copy/paste the new one in the directory.

Jitsi-videobridge

  • Clone the jitsi-videobridge fork of @ Jonathan_Lennox (GitHub - JonathanLennox/jitsi-videobridge)
  • Checkout vp9-support branch (latest commit as of this writing is e2d34e1974754d044e42f5f57f20974cbadfe595)
  • Run mvn clean package -DskipTests -Dassembly.skipAssembly=false inside the project folder (clean and -DskipTests are optional, feel free to exclude them)
  • After the process is finished you should see some artifacts in the jvb/target directory, inside the project’s folder.

I’m not sure if the next steps are optimal but they worked for me in multiple tries from a clean start.

  • Unzip the jitsi-videobridge-2.1-SNAPSHOT-archive.zip into a folder (e.g. unzip ./jvb/target/jitsi-videobridge-2.1-SNAPSHOT-archive.zip -d ./myFolder
  • Go to the newly created unzipped folder cd ./jvb/target/myFolder You should see a folder named jitsi-videobridge-2.1-SNAPSHOT. Inside that folder you should see a jitsi-videobridge.jar (729KB) a jvb.bat, a jvb.sh and a lib/ folder. You need to replace those files in the /usr/share/jitsi-videobridge directory (again, backup first, just for some piece of mind)

rm -rf /usr/share/jitsi-videobridge/lib /usr/share/jitsi-videobridge/jvb.sh /usr/share/jitsi-videobridge/jitsi-videobridge.jar
cp jitsi-videobridge.jar /usr/share/jitsi-videobridge/
cp -r lib /usr/share/jitsi-videobridge/
cp jvb.sh /usr/share/jitsi-videobridge/

Restart the videobridge service service jitsi-videobridge2 restart
You should not see any errors in logs tail -100 /var/log/jitsi/jvb.log

After a few test calls with some friends, I cannot say that I see any big improvements over the VP8 videobridges. TBH if I didn’t see this indication[1], I wouldn’t even notice the change. I’m not sure if SVC is enabled or not, or if there’s any other way to test that the whole installation was successful. Right now I’ve only attached videobridges built with the above instructions (and stopped old VP8 videobridges), plus I enforced VP9 through jicofo config.

Feel free to report any findings on this, but right now if you also care about Safari support, I think sticking to VP8 is a no-brainer.

[1] Screenshot 2020-12-26 at 13.46.18