[jitsi-users] reproducing the Jitsi Videobridge Performance Evaluation


#1

Hi all,

I'd like to evaluate the performance Jitsi Videobridge for a live broadcast video scenario, and I came across this benchmark: https://jitsi.org/Projects/JitsiVideobridgePerformance. I'm trying to run a similar test on my own server, and everything seems to be working, but when I add a fake user with Jitsi-Hammer, no video appears for that user. I see a "Hello World!" chat, and verbose logging indicates that the ICE negotiation is successful. The "-overallstats" option to jitsi-hammer.sh has enough non-zero stats to suggest that video is being sent and received from the fake user's persepective, but I don't see any video in my Chrome client.

I've used https://github.com/jitsi/jitsi-meet/blob/master/doc/manual-install.md to set up the server, which seems to work fine for real Chrome clients -- I can do video chats with 2 or 3 clients and everything works. I'm testing jitsi-hammer by visiting https://myserver.example.com/foo in Chrome and then running "bash jitsi-hammer.sh -XMPPdomain myserver.example.com -XMPPhost myserver.example.com -MUCdomain conference.myserver.example.com -room foo -ivf resources/big-buck-bunny_trailer_track1_eng.ivf -overallstats". In the Chrome client, I can see a "Hello World!" chat from Jitsi_Hammer_0_, but no video thumbnail appears in the "film strip". I see no errors in any of the client or server logs.

Does anyone have any suggestions for how I might debug this?


#2

Hello,

Hi all,

I'd like to evaluate the performance Jitsi Videobridge for a live
broadcast video scenario, and I came across this benchmark:
https://jitsi.org/Projects/JitsiVideobridgePerformance. I'm trying to run
a similar test on my own server, and everything seems to be working, but
when I add a fake user with Jitsi-Hammer, no video appears for that user.
I see a "Hello World!" chat, and verbose logging indicates that the ICE
negotiation is successful. The "-overallstats" option to jitsi-hammer.sh
has enough non-zero stats to suggest that video is being sent and received
from the fake user's persepective, but I don't see any video in my Chrome
client.

I've used
https://github.com/jitsi/jitsi-meet/blob/master/doc/manual-install.md to
set up the server, which seems to work fine for real Chrome clients -- I
can do video chats with 2 or 3 clients and everything works. I'm testing
jitsi-hammer by visiting https://myserver.example.com/foo in Chrome and
then running "bash jitsi-hammer.sh -XMPPdomain myserver.example.com
-XMPPhost myserver.example.com -MUCdomain conference.myserver.example.com
-room foo -ivf resources/big-buck-bunny_trailer_track1_eng.ivf
-overallstats". In the Chrome client, I can see a "Hello World!" chat
from Jitsi_Hammer_0_, but no video thumbnail appears in the "film strip".
I see no errors in any of the client or server logs.

Does anyone have any suggestions for how I might debug this?

You can start by checking chrome://webrtc-internals after a Hammer has joined the room. In the last setRemoteDescription call you should see two SSRCs listed for video (one with a "mixedlabel" and one other).

Next you can try to disable forcing DTLS in the Hammer by commenting out the line in $HAMMER_DIR/.jitsi-hammer/sip-communicators.properties. Note that if that fixes the problem the performance measurements would not be very telling, because there will be no encryption.

Also note that currently the statistics aren't very reliable. We are likely going to work on this soon.

Regards,
Boris

···

On 06/02/15 19:03, Joel Dice wrote:


#3

Thanks for the tips. I do see two SSRCs:

a=ssrc:610871879 cname:mixed
a=ssrc:610871879 msid:mixedmslabel mixedlabelvideo0
a=ssrc:610871879 mslabel:mixedmslabel
a=ssrc:610871879 label:mixedlabelvideo0
a=ssrc:3515327 cname:74a25698-cf15-4751-bd1d-fce4bcb1febf
a=ssrc:3515327 msid:ef1e9c0e-f243-4ccd-b163-d51b1dab8fa0 c52c8b8f-dab5-4b32-8d64-47923342fd10
a=ssrc:3515327 mslabel:ef1e9c0e-f243-4ccd-b163-d51b1dab8fa0
a=ssrc:3515327 label:c52c8b8f-dab5-4b32-8d64-47923342fd10

... but still don't see any video, and I get the same result with the line in sip-communicator.properties commented out.

I should also mention that, of the 8 or so times I've tried this, I saw the first few seconds of the Big Buck Bunny trailer once, then it froze. In all other cases, I never saw any video at all. That leads me to suspect a race condition of some kind, rather than a simple misconfiguration.

Also, I'm using jicofo as the focus, whereas the published performance evaluation refers to the Chrome client being the focus. Could that make a difference?

···

On Fri, 6 Feb 2015, Boris Grozev wrote:

On 06/02/15 19:03, Joel Dice wrote:

I'd like to evaluate the performance Jitsi Videobridge for a live
broadcast video scenario, and I came across this benchmark:
https://jitsi.org/Projects/JitsiVideobridgePerformance. I'm trying to run
a similar test on my own server, and everything seems to be working, but
when I add a fake user with Jitsi-Hammer, no video appears for that user.
I see a "Hello World!" chat, and verbose logging indicates that the ICE
negotiation is successful. The "-overallstats" option to jitsi-hammer.sh
has enough non-zero stats to suggest that video is being sent and received
from the fake user's persepective, but I don't see any video in my Chrome
client.

I've used
https://github.com/jitsi/jitsi-meet/blob/master/doc/manual-install.md to
set up the server, which seems to work fine for real Chrome clients -- I
can do video chats with 2 or 3 clients and everything works. I'm testing
jitsi-hammer by visiting https://myserver.example.com/foo in Chrome and
then running "bash jitsi-hammer.sh -XMPPdomain myserver.example.com
-XMPPhost myserver.example.com -MUCdomain conference.myserver.example.com
-room foo -ivf resources/big-buck-bunny_trailer_track1_eng.ivf
-overallstats". In the Chrome client, I can see a "Hello World!" chat
from Jitsi_Hammer_0_, but no video thumbnail appears in the "film strip".
I see no errors in any of the client or server logs.

Does anyone have any suggestions for how I might debug this?

You can start by checking chrome://webrtc-internals after a Hammer has joined the room. In the last setRemoteDescription call you should see two SSRCs listed for video (one with a "mixedlabel" and one other).

Next you can try to disable forcing DTLS in the Hammer by commenting out the line in $HAMMER_DIR/.jitsi-hammer/sip-communicators.properties. Note that if that fixes the problem the performance measurements would not be very telling, because there will be no encryption.


#4

Hello,

I'd like to evaluate the performance Jitsi Videobridge for a live
broadcast video scenario, and I came across this benchmark:
https://jitsi.org/Projects/JitsiVideobridgePerformance. I'm trying to run
a similar test on my own server, and everything seems to be working, but
when I add a fake user with Jitsi-Hammer, no video appears for that user.
I see a "Hello World!" chat, and verbose logging indicates that the ICE
negotiation is successful. The "-overallstats" option to jitsi-hammer.sh
has enough non-zero stats to suggest that video is being sent and received
from the fake user's persepective, but I don't see any video in my Chrome
client.

I've used
https://github.com/jitsi/jitsi-meet/blob/master/doc/manual-install.md to
set up the server, which seems to work fine for real Chrome clients -- I
can do video chats with 2 or 3 clients and everything works. I'm testing
jitsi-hammer by visiting https://myserver.example.com/foo in Chrome and
then running "bash jitsi-hammer.sh -XMPPdomain myserver.example.com
-XMPPhost myserver.example.com -MUCdomain conference.myserver.example.com
-room foo -ivf resources/big-buck-bunny_trailer_track1_eng.ivf
-overallstats". In the Chrome client, I can see a "Hello World!" chat
from Jitsi_Hammer_0_, but no video thumbnail appears in the "film strip".
I see no errors in any of the client or server logs.

Does anyone have any suggestions for how I might debug this?

You can start by checking chrome://webrtc-internals after a Hammer has
joined the room. In the last setRemoteDescription call you should see
two SSRCs listed for video (one with a "mixedlabel" and one other).

Next you can try to disable forcing DTLS in the Hammer by commenting out
the line in $HAMMER_DIR/.jitsi-hammer/sip-communicators.properties. Note
that if that fixes the problem the performance measurements would not be
very telling, because there will be no encryption.

Thanks for the tips. I do see two SSRCs:

a=ssrc:610871879 cname:mixed
a=ssrc:610871879 msid:mixedmslabel mixedlabelvideo0
a=ssrc:610871879 mslabel:mixedmslabel
a=ssrc:610871879 label:mixedlabelvideo0
a=ssrc:3515327 cname:74a25698-cf15-4751-bd1d-fce4bcb1febf
a=ssrc:3515327 msid:ef1e9c0e-f243-4ccd-b163-d51b1dab8fa0 c52c8b8f-dab5-4b32-8d64-47923342fd10
a=ssrc:3515327 mslabel:ef1e9c0e-f243-4ccd-b163-d51b1dab8fa0
a=ssrc:3515327 label:c52c8b8f-dab5-4b32-8d64-47923342fd10

... but still don't see any video, and I get the same result with the line
in sip-communicator.properties commented out.

I should also mention that, of the 8 or so times I've tried this, I saw
the first few seconds of the Big Buck Bunny trailer once, then it froze.
In all other cases, I never saw any video at all. That leads me to
suspect a race condition of some kind, rather than a simple
misconfiguration.

That's certainly strange. I just tried a few runs on linux-64 and mac and it still works for me (I don't think we've ever tested with other platforms, though). Can you change the org.jitsi.hammer level to INFO in lib/logging.properties and post your log? Hopefully something will come up.

Also, I'm using jicofo as the focus, whereas the published performance
evaluation refers to the Chrome client being the focus. Could that make a
difference?

The github master version of Jitsi Hammer works with Jicofo.

Just to make sure: your Jitsi Meet installation works otherwise (with 2 chrome clients), right? And do you get any javascript errors when the hammer fails to work?

Regards,
Boris

···

On 06/02/15 23:17, Joel Dice wrote:

On Fri, 6 Feb 2015, Boris Grozev wrote:

On 06/02/15 19:03, Joel Dice wrote:


#5

That's certainly strange. I just tried a few runs on linux-64 and mac and it still works for me (I don't think we've ever tested with other platforms, though). Can you change the org.jitsi.hammer level to INFO in lib/logging.properties and post your log? Hopefully something will come up.

I'm running the servers and hammer on x86_64 Debian Linux machines and running the Chrome clients on Windows, Mac, and/or Linux. I've attached the hammer logs.

Also, I'm using jicofo as the focus, whereas the published performance
evaluation refers to the Chrome client being the focus. Could that make a
difference?

The github master version of Jitsi Hammer works with Jicofo.

Just to make sure: your Jitsi Meet installation works otherwise (with 2 chrome clients), right? And do you get any javascript errors when the hammer fails to work?

Well, it was working with 2 Chrome clients on Friday :slight_smile: Now even that isn't working. I get errors in the JavaScript console like this one:

Strophe: error: xmpp is not defined
ReferenceError: xmpp is not defined
     at Object.ContactList.onDisplayNameChange (https://jdpc.ecovate.com/libs/app.bundle.js?v=4:4692:23)
     at onDisplayNameChanged (https://jdpc.ecovate.com/libs/app.bundle.js?v=4:1588:17)
     at EventEmitter.onMucJoined (https://jdpc.ecovate.com/libs/app.bundle.js?v=4:1902:9)
     at EventEmitter.emit (https://jdpc.ecovate.com/libs/app.bundle.js?v=4:81:17)
     at Object.module.exports.Strophe.addConnectionPlugin.onPresence (https://jdpc.ecovate.com/libs/app.bundle.js?v=4:14981:34)
     at Object.Strophe.Handler.run (https://jdpc.ecovate.com/libs/strophe/strophe.min.js?v=1:1:19636)
     at https://jdpc.ecovate.com/libs/strophe/strophe.min.js?v=1:1:27678
     at Object.Strophe.forEachChild (https://jdpc.ecovate.com/libs/strophe/strophe.min.js?v=1:1:10525)
     at Object.Strophe.Connection._dataRecv (https://jdpc.ecovate.com/libs/strophe/strophe.min.js?v=1:1:27466)
     at Object.Strophe.Bosh._onRequestStateChange (https://jdpc.ecovate.com/libs/strophe/strophe.min.js?v=1:1:48024)

Also, when any other user is in the room (hammer or Chrome), I get lots of warnings like this:

No jid for ssrc: 1891479966

Those repeat in the logs for as long as the other user is present.

I'll spend some time this morning trying to figure out what's up with the "xmpp is not defined" messages, since that error seems to prevent anything from working.

hammer.txt (0 Bytes)

···

On Sat, 7 Feb 2015, Boris Grozev wrote:


#6

Nevermind, this was fixed in 57fcee676ae8ccc68cf3a45f22e05d68ee9f5e8f in jitsi-meet.git. And with the latest from the jitsi-meet master branch, the hammer is working as well!

Thanks for your help, Boris.

···

On Mon, 9 Feb 2015, Joel Dice wrote:

Well, it was working with 2 Chrome clients on Friday :slight_smile: Now even that isn't working. I get errors in the JavaScript console like this one:

Strophe: error: xmpp is not defined
ReferenceError: xmpp is not defined
   at Object.ContactList.onDisplayNameChange