Giving feedback on new bandwidth allocation algo

Hi guys,

I installed latest unstable and have been playing around with the new useNewBandwidthAllocationStrategy config setting.

I installed latest unstable on a fresh Debian Buster server, enabled VP9 on the bridge with websockets (both jvb and XMPP) and all seems to work well.

However, when I set useNewBandwidthAllocationStrategy to true I no longer get remote audio/video, just a black screen. Setting useNewBandwidthAllocationStrategy to false fixes that but obviously something isn’t working quite right with the new algo.

Let me know if you need me to grab some logs or whatever else I can do to help.

Thanks again to the team and this wonderful community. I’ve been using jitsi for many years now and it has gotten better and better as time goes by. I remember when the community was tiny and posts would go unanswered for a long time. Since the pandemic started there’s been an explosion of contributions in the community and some very helpful folks that are always willing to spend some of their precious time helping others. Thank you.

@jallamsetty tagging you since I saw in the last community call you’ve been working on this.

Hi @pdarcos, are you running the bridge with @Jonathan_Lennox 's VP9 PR merged ? I don’t see any issues using the new bandwidth allocation strategy and VP9 without his PR.

Hi,

Yes, with Jonathan’s PR merged.

@pdarcos, will you be able to share the JVB logs so that @Jonathan_Lennox can take a look ? Thanks!

The VP9 branch hasn’t been sync’d with master for a while, so it doesn’t contain the code to support the message the new bandwidth allocation algorithm uses.

I’ll try to merge it.

Great. Let me know once that’s done and I can compile your jar again and provide any logs as needed.

I’ve merged with master now.

2 Likes

Unable to create a jar file using latest jitsi-videobridge vp9-support repo. @Jonathan_Lennox

Logs

mvn install -P buildFatJar

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] jvb-parent                                                         [pom]
[INFO] jvb-api-parent                                                     [pom]
[INFO] jvb-api-common                                                     [jar]
[INFO] jvb-api-client                                                     [jar]
[INFO] jitsi-videobridge                                                  [jar]
[INFO]
[INFO] ------------------------< org.jitsi:jvb-parent >------------------------
[INFO] Building jvb-parent 2.1-SNAPSHOT                                   [1/5]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-maven) @ jvb-parent ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ jvb-parent ---
[INFO] Installing /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/pom.xml to /home/metadata/.m2/repository/org/jitsi/jvb-parent/2.1-SNAPSHOT/jvb-parent-2.1-SNAPSHOT.pom
[INFO]
[INFO] ----------------------< org.jitsi:jvb-api-parent >----------------------
[INFO] Building jvb-api-parent 2.1-SNAPSHOT                               [2/5]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-maven) @ jvb-api-parent ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ jvb-api-parent ---
[INFO] Installing /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/pom.xml to /home/metadata/.m2/repository/org/jitsi/jvb-api-parent/2.1-SNAPSHOT/jvb-api-parent-2.1-SNAPSHOT.pom
[INFO]
[INFO] ----------------------< org.jitsi:jvb-api-common >----------------------
[INFO] Building jvb-api-common 2.1-SNAPSHOT                               [3/5]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-maven) @ jvb-api-common ---
[INFO]
[INFO] --- ktlint-maven-plugin:1.7.0:format (format-and-check) @ jvb-api-common ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ jvb-api-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/jvb-api-common/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ jvb-api-common ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- kotlin-maven-plugin:1.3.72:compile (compile) @ jvb-api-common ---
[WARNING] Some JAR files in the classpath have the Kotlin Runtime library bundled into them. This may cause difficult to debug problems if there's a different version of the Kotlin Runtime library in the classpath. Consider removing these libraries from the classpath
[WARNING] /home/metadata/.m2/repository/org/jetbrains/kotlin/kotlin-osgi-bundle/1.3.72/kotlin-osgi-bundle-1.3.72.jar: (-1, -1) Library has Kotlin runtime bundled into it
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ jvb-api-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/jvb-api-common/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ jvb-api-common ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- kotlin-maven-plugin:1.3.72:test-compile (test-compile) @ jvb-api-common ---
[WARNING] Some JAR files in the classpath have the Kotlin Runtime library bundled into them. This may cause difficult to debug problems if there's a different version of the Kotlin Runtime library in the classpath. Consider removing these libraries from the classpath
[WARNING] /home/metadata/.m2/repository/org/jetbrains/kotlin/kotlin-osgi-bundle/1.3.72/kotlin-osgi-bundle-1.3.72.jar: (-1, -1) Library has Kotlin runtime bundled into it
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ jvb-api-common ---
[INFO] Surefire report directory: /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/jvb-api-common/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.jitsi.videobridge.api.types.SupportedApiVersionsTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ jvb-api-common ---
[INFO] Building jar: /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/jvb-api-common/target/jvb-api-common-2.1-SNAPSHOT.jar
[INFO]
[INFO] --- ktlint-maven-plugin:1.7.0:check (format-and-check) @ jvb-api-common ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ jvb-api-common ---
[INFO] Installing /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/jvb-api-common/target/jvb-api-common-2.1-SNAPSHOT.jar to /home/metadata/.m2/repository/org/jitsi/jvb-api-common/2.1-SNAPSHOT/jvb-api-common-2.1-SNAPSHOT.jar
[INFO] Installing /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/jvb-api-common/pom.xml to /home/metadata/.m2/repository/org/jitsi/jvb-api-common/2.1-SNAPSHOT/jvb-api-common-2.1-SNAPSHOT.pom
[INFO]
[INFO] ----------------------< org.jitsi:jvb-api-client >----------------------
[INFO] Building jvb-api-client 2.1-SNAPSHOT                               [4/5]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-maven) @ jvb-api-client ---
[INFO]
[INFO] --- ktlint-maven-plugin:1.7.0:format (format-and-check) @ jvb-api-client ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ jvb-api-client ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/jvb-api-client/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ jvb-api-client ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- kotlin-maven-plugin:1.3.72:compile (compile) @ jvb-api-client ---
[WARNING] Some JAR files in the classpath have the Kotlin Runtime library bundled into them. This may cause difficult to debug problems if there's a different version of the Kotlin Runtime library in the classpath. Consider removing these libraries from the classpath
[WARNING] /home/metadata/.m2/repository/org/jetbrains/kotlin/kotlin-osgi-bundle/1.3.72/kotlin-osgi-bundle-1.3.72.jar: (-1, -1) Library has Kotlin runtime bundled into it
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ jvb-api-client ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/jvb-api-client/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ jvb-api-client ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- kotlin-maven-plugin:1.3.72:test-compile (test-compile) @ jvb-api-client ---
[WARNING] Some JAR files in the classpath have the Kotlin Runtime library bundled into them. This may cause difficult to debug problems if there's a different version of the Kotlin Runtime library in the classpath. Consider removing these libraries from the classpath
[WARNING] /home/metadata/.m2/repository/org/jetbrains/kotlin/kotlin-osgi-bundle/1.3.72/kotlin-osgi-bundle-1.3.72.jar: (-1, -1) Library has Kotlin runtime bundled into it
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ jvb-api-client ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
~~~ Kotest Configuration ~~~
-> Parallelism: 1 thread(s)
-> Default test timeout: 600000ms
-> Default test order: Sequential
-> Default isolation mode: SingleInstance
-> Global soft assertations: False
-> Write spec failure file: False
-> Fail on ignored tests: False
-> Spec execution order: LexicographicSpecExecutionOrder
-> Extensions
  - io.kotest.core.extensions.SystemPropertyTagExtension
  - io.kotest.core.extensions.RuntimeTagExtension
  - io.kotest.core.extensions.IgnoredSpecDiscoveryExtension
  - io.kotest.core.extensions.TagsExcludedDiscoveryExtension
-> Tags: !LongTest

[INFO] Running org.jitsi.videobridge.api.util.WebSocketClientTest
Server running on port 30178
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Server running on port 41514
Server running on port 22087
Server running on port 22174
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.05 s - in org.jitsi.videobridge.api.util.WebSocketClientTest
[INFO] Running org.jitsi.videobridge.api.util.XmppWebSocketClientTest
sending req 1
sending req 2
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.58 s - in org.jitsi.videobridge.api.util.XmppWebSocketClientTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ jvb-api-client ---
[INFO] Building jar: /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/jvb-api-client/target/jvb-api-client-2.1-SNAPSHOT.jar
[INFO]
[INFO] --- ktlint-maven-plugin:1.7.0:check (format-and-check) @ jvb-api-client ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ jvb-api-client ---
[INFO] Installing /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/jvb-api-client/target/jvb-api-client-2.1-SNAPSHOT.jar to /home/metadata/.m2/repository/org/jitsi/jvb-api-client/2.1-SNAPSHOT/jvb-api-client-2.1-SNAPSHOT.jar
[INFO] Installing /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb-api/jvb-api-client/pom.xml to /home/metadata/.m2/repository/org/jitsi/jvb-api-client/2.1-SNAPSHOT/jvb-api-client-2.1-SNAPSHOT.pom
[INFO]
[INFO] --------------------< org.jitsi:jitsi-videobridge >---------------------
[INFO] Building jitsi-videobridge 2.1-SNAPSHOT                            [5/5]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-maven) @ jitsi-videobridge ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ jitsi-videobridge ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO]
[INFO] --- kotlin-maven-plugin:1.3.72:compile (compile) @ jitsi-videobridge ---
[WARNING] Some JAR files in the classpath have the Kotlin Runtime library bundled into them. This may cause difficult to debug problems if there's a different version of the Kotlin Runtime library in the classpath. Consider removing these libraries from the classpath
[WARNING] /home/metadata/.m2/repository/org/jetbrains/kotlin/kotlin-osgi-bundle/1.3.72/kotlin-osgi-bundle-1.3.72.jar: (-1, -1) Library has Kotlin runtime bundled into it
[ERROR] /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb/src/main/kotlin/org/jitsi/videobridge/octo/config/OctoRtpReceiver.kt: (174, 5) 'forceMuteVideo' overrides nothing
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for jvb-parent 2.1-SNAPSHOT:
[INFO]
[INFO] jvb-parent ......................................... SUCCESS [  0.521 s]
[INFO] jvb-api-parent ..................................... SUCCESS [  0.007 s]
[INFO] jvb-api-common ..................................... SUCCESS [  8.663 s]
[INFO] jvb-api-client ..................................... SUCCESS [ 14.351 s]
[INFO] jitsi-videobridge .................................. FAILURE [  8.913 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  32.617 s
[INFO] Finished at: 2021-03-03T12:19:59-06:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.jetbrains.kotlin:kotlin-maven-plugin:1.3.72:compile (compile) on project jitsi-videobridge: Compilation failure
[ERROR] /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb/src/main/kotlin/org/jitsi/videobridge/octo/config/OctoRtpReceiver.kt:[174,5] 'forceMuteVideo' overrides nothing
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :jitsi-videobridge

Ah, sorry, I forgot to push the merged jitsi-media-transform vp9-branch. Check that out and do a new
mvn install there before you build the jvb.

1 Like

new errors

[WARNING] /home/metadata/.m2/repository/org/jetbrains/kotlin/kotlin-osgi-bundle/1.3.72/kotlin-osgi-bundle-1.3.72.jar: (-1, -1) Library has Kotlin runtime bundled into it
[ERROR] /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb/src/test/kotlin/org/jitsi/videobridge/cc/allocation/BitrateControllerTest.kt: (1383, 87) Null can not be a value of a non-null type Array<RtpLayerDesc>
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for jvb-parent 2.1-SNAPSHOT:
[INFO]
[INFO] jvb-parent ......................................... SUCCESS [  0.517 s]
[INFO] jvb-api-parent ..................................... SUCCESS [  0.007 s]
[INFO] jvb-api-common ..................................... SUCCESS [  8.838 s]
[INFO] jvb-api-client ..................................... SUCCESS [ 14.551 s]
[INFO] jitsi-videobridge .................................. FAILURE [ 18.551 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  42.621 s
[INFO] Finished at: 2021-03-03T12:46:06-06:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.jetbrains.kotlin:kotlin-maven-plugin:1.3.72:test-compile (test-compile) on project jitsi-videobridge: Compilation failure
[ERROR] /home/metadata/GoWorkspace/src/github.com/jitsi/jitsi-videobridge/jvb/src/test/kotlin/org/jitsi/videobridge/cc/allocation/BitrateControllerTest.kt:[1383,87] Null can not be a value of a non-null type Array<RtpLayerDesc>
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :jitsi-videobridge

Should be fixed now.

:+1:

[INFO] jvb-parent ......................................... SUCCESS [  0.482 s]
[INFO] jvb-api-parent ..................................... SUCCESS [  0.006 s]
[INFO] jvb-api-common ..................................... SUCCESS [  8.740 s]
[INFO] jvb-api-client ..................................... SUCCESS [ 14.358 s]
[INFO] jitsi-videobridge .................................. SUCCESS [01:33 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:56 min
[INFO] Finished at: 2021-03-03T12:55:43-06:00
[INFO] ------------------------------------------------------------------------

I just compiled the new jars now.

Will test with the new bandwidth algo set to true and will give feedback as soon as I can.

Edit: Just tried it and it seems the new jars broke the installation. Grrrr.
Will take another look when I find some time.

@metadata did your installation break too?

I see this in jvb logs now and as soon as the 2nd party joins I see in the chrome console that jvb is not available

JVB 2021-03-03 21:04:12.032 INFO: [1] JitsiConfig.<clinit>#47: Initialized newConfig: merge of /etc/jitsi/videobridge/jvb.conf: 1,application.conf @ jar:file:/usr/share/jitsi-videobridge/jitsi-videobridge.jar!/application.conf: 1,system properties,reference.conf @ jar:file:/usr/share/jitsi-videobridge/jitsi-videobridge.jar!/reference.conf: 1,reference.conf @ jar:file:/usr/share/jitsi-videobridge/lib/jitsi-media-transform-1.0-SNAPSHOT.jar!/reference.conf: 1,reference.conf @ jar:file:/usr/share/jitsi-videobridge/lib/ice4j-3.0-22-g67ffceb.jar!/reference.conf: 1
JVB 2021-03-03 21:04:12.045 INFO: [1] ReadOnlyConfigurationService.reloadConfiguration#51: loading config file at path /etc/jitsi/videobridge/sip-communicator.properties
JVB 2021-03-03 21:04:12.046 INFO: [1] JitsiConfig.<clinit>#68: Initialized legacyConfig: sip communicator props (no description provided)
JVB 2021-03-03 21:04:12.047 INFO: [1] JitsiConfig$Companion.reloadNewConfig#94: Reloading the Typesafe config source (previously reloaded 0 times).
Exception in thread "main" java.lang.NoSuchMethodError: 'void org.jitsi.xmpp.mucclient.MucClientManager.registerIQ(org.jivesoftware.smack.packet.IQ, boolean)'
	at org.jitsi.videobridge.xmpp.XmppConnection.start(XmppConnection.kt:61)
	at org.jitsi.videobridge.MainKt.main(Main.kt:83)

If you’re using the latest jvb, make sure to use the latest version of all its dependency JVBs. (The zip file that gets built in the standard mvn install will contain them all.).

That error looks like you don’t have the latest version of the jicoco library.

I keep getting “GC overhead limit exceeded” errors. I can certainly use another machine that has more memory, but this is an issue nonetheless. Never had this behavior before.

That was it.
It’s working fine for me now, with and without useNewBandwidthAllocationStrategy set to true.

Thanks Jonathan

@metadata and @pdarcos what kind of resolutions are you reporting on a default call (2 and 3 party)?

@Freddie I still have to test it.

Let me know when you do. The resolution I’m seeing is a bit concerning; want to make sure it’s not something on my end.

Freddie: how much memory do you have?

Does the “GC Exceeded” happen immediately, or after some time?