Testing Octo Setup

Hello,

I am trying to configure Octo and testing by setting the Bridge Selection Strategy in jicofo’s sip-communicator.properties:

org.jitsi.jicofo.BridgeSelector.BRIDGE_SELECTION_STRATEGY=SplitBridgeSelectionStrategy

I’ve added the following to each of my JVBs sip-communicator.properties:

org.jitsi.videobridge.octo.BIND_ADDRESS=mypublicip
org.jitsi.videobridge.octo.PUBLIC_ADDRESS=mypublicip
org.jitsi.videobridge.octo.BIND_PORT=4096
org.jitsi.videobridge.REGION=useast/useast1/useast2

and in my jitsi-meet config.js I added:

testing: {octo: {probability: 1}},

Looking at my jicofo.log, here are lines related to Bridge Selection / Octo

Jicofo 2020-05-01 19:43:00.975 INFO: [12] org.jitsi.jicofo.bridge.BridgeSelector.log() Using org.jitsi.jicofo.bridge.SplitBridgeSelectionStrategy
Jicofo 2020-05-01 19:43:00.976 INFO: [12] org.jitsi.jicofo.bridge.BridgeSelector.log() Bridge failure reset threshold: 300000
Jicofo 2020-05-01 19:43:00.976 INFO: [12] org.jitsi.jicofo.bridge.BridgeSelector.log() Local region: null
Jicofo 2020-05-01 19:43:00.978 INFO: [12] org.jitsi.jicofo.JitsiMeetServices.log() Using a Bridge MUC detector with MUC: JvbBrewery@internal.auth.meet.mydomain
Jicofo 2020-05-01 19:43:02.096 INFO: [1] org.jitsi.impl.configuration.ConfigurationServiceImpl.log() org.jitsi.jicofo.BRIDGE_MUC=JvbBrewery@internal.auth.meet.mydomain
Jicofo 2020-05-01 19:43:02.097 INFO: [1] org.jitsi.impl.configuration.ConfigurationServiceImpl.log() org.jitsi.jicofo.BridgeSelector.BRIDGE_SELECTION_STRATEGY=SplitBridgeSelectionStrategy
Jicofo 2020-05-01 19:43:04.580 WARNING: [29] org.jitsi.jicofo.bridge.BridgeSelector.log() No pub-sub node mapped for jvbbrewery@internal.auth.meet.mydomain/689c4478-bb04-448c-87cb-547f1c7f939d
Jicofo 2020-05-01 19:43:04.582 INFO: [29] org.jitsi.jicofo.bridge.Bridge.log() Setting max total packet rate of 50800.0
Jicofo 2020-05-01 19:43:04.583 INFO: [29] org.jitsi.jicofo.bridge.Bridge.log() Setting average participant packet rate of 500
Jicofo 2020-05-01 19:43:04.590 INFO: [29] org.jitsi.jicofo.bridge.BridgeSelector.log() Added new videobridge: Bridge[jid=jvbbrewery@internal.auth.meet.mydomain/689c4478-bb04-448c-87cb-547f1c7f939d, relayId=null, region=useast1, stress=0.00]
Jicofo 2020-05-01 19:43:04.597 INFO: [44] org.jitsi.jicofo.bridge.JvbDoctor.log() Scheduled health-check task for: jvbbrewery@internal.auth.meet.mydomain/689c4478-bb04-448c-87cb-547f1c7f939d
Jicofo 2020-05-01 19:43:04.656 WARNING: [29] org.jitsi.jicofo.bridge.BridgeSelector.log() No pub-sub node mapped for jvbbrewery@internal.auth.meet.mydomain/de943362-9278-496f-af79-a85918bb0c1a
Jicofo 2020-05-01 19:43:04.668 INFO: [29] org.jitsi.jicofo.bridge.BridgeSelector.log() Added new videobridge: Bridge[jid=jvbbrewery@internal.auth.meet.mydomain/de943362-9278-496f-af79-a85918bb0c1a, relayId=null, region=useast2, stress=0.00]
Jicofo 2020-05-01 19:43:04.669 INFO: [44] org.jitsi.jicofo.bridge.JvbDoctor.log() Scheduled health-check task for: jvbbrewery@internal.auth.meet.mydomain/de943362-9278-496f-af79-a85918bb0c1a
Jicofo 2020-05-01 19:43:13.612 WARNING: [29] org.jitsi.jicofo.bridge.BridgeSelector.log() No pub-sub node mapped for jvbbrewery@internal.auth.meet.mydomain/d7b3ee74-2171-4a7c-aab4-88d92b5138d5
Jicofo 2020-05-01 19:43:13.613 INFO: [29] org.jitsi.jicofo.bridge.BridgeSelector.log() Added new videobridge: Bridge[jid=jvbbrewery@internal.auth.meet.mydomain/d7b3ee74-2171-4a7c-aab4-88d92b5138d5, relayId=null, region=useast, stress=0.00]
Jicofo 2020-05-01 19:43:13.614 INFO: [44] org.jitsi.jicofo.bridge.JvbDoctor.log() Scheduled health-check task for: jvbbrewery@internal.auth.meet.mydomain/d7b3ee74-2171-4a7c-aab4-88d92b5138d5
Jicofo 2020-05-01 19:45:11.074 INFO: [29] org.jitsi.jicofo.bridge.BridgeSelectionStrategy.log() Selected initial bridge Bridge[jid=jvbbrewery@internal.auth.meet.mydomain/689c4478-bb04-448c-87cb-547f1c7f939d, relayId=null, region=useast1, stress=0.00] with packetRate=0 for participantRegion=null
Jicofo 2020-05-01 19:45:11.094 INFO: [29] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Added participant jid= octooo@conference.meet.mydomain/55b7f324, bridge=jvbbrewery@internal.auth.meet.mydomain/689c4478-bb04-448c-87cb-547f1c7f939d
Jicofo 2020-05-01 19:45:11.094 INFO: [29] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Region info, conference=ffaaa2 octo_enabled= true: [[useast1, null]]
Jicofo 2020-05-01 19:45:11.096 INFO: [29] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Added participant jid= octooo@conference.meet.mydomain/ea224b91, bridge=jvbbrewery@internal.auth.meet.mydomain/689c4478-bb04-448c-87cb-547f1c7f939d
Jicofo 2020-05-01 19:45:11.097 INFO: [29] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Region info, conference=ffaaa2 octo_enabled= true: [[useast1, null, null]]

I can’t see anything wrong from my logs. I see a warning for the PubSub node not being defined for my video bridges, but I thought if I was using MUC that was unnecessary? Note that I also removed XMPP from my JVB APIs since I’m using MUC as well. What am I doing wrong? Or am I misinterpreting the behavior of the SplitBridgeSelectionStrategy?

What is the scenario you’re doing? Starting 2 bridges up with this config and joining some participants? And you’re only seeing them join 1 bridge?

Yes. I have 3 bridges running and connected/detected by jicofo, and even though SplitBridgeSelectionStrategy is set, participants still only join a single bridge.

I seem to have figured out the error. Looking at my JVB logs I discovered octo relay was failing to start. Apparently using my public IP in the bind_address was causing issues, even though port 4096 was open. I changed my bind_address to 127.0.0.1 and octo was tested successfully. Hopefully using the local IP as the bind_address does not cause issues.

1 Like

I have the same behaviour, participants always join on the same jvb… Can you please check the config files, what may be wrong, to have OCTO enabled ? i want to test each participant to join to different jvb with SplitBridgeSelectionStrategy… Installation is on aws ec2 c5.xlarge servers with security group all traffic allowed on each…
When i check with tcpdump -i any src port 10000 always same jvb has traffic, it is not distributed and in stats i always see octo_conferences = 0, but region is correct = “region1”

Jitsi-Meet Server settings

/etc/jitsi/jicofo/sip-communicator.properties

org.jitsi.jicofo.BRIDGE_MUC=JvbBrewery@internal.auth.meet1.mydomain.com
org.jitsi.jicofo.BridgeSelector.BRIDGE_SELECTION_STRATEGY=SplitBridgeSelectionStrategy
#org.jitsi.jicofo.BridgeSelector.BRIDGE_SELECTION_STRATEGY=RegionBasedBridgeSelectionStrategy

/etc/jitsi/videobridge/sip-communicator.properties

org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true
org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=meet-jit-si-turnrelay.jitsi.net:443
org.jitsi.videobridge.ENABLE_STATISTICS=true
org.jitsi.videobridge.STATISTICS_TRANSPORT=muc,colibri
org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=localhost
org.jitsi.videobridge.xmpp.user.shard.DOMAIN=auth.meet1.mydomain.com
org.jitsi.videobridge.xmpp.user.shard.USERNAME=jvb
org.jitsi.videobridge.xmpp.user.shard.PASSWORD=Ia1Nvvrh
org.jitsi.videobridge.xmpp.user.shard.MUC_JIDS=JvbBrewery@internal.auth.meet1.mydomain.com
org.jitsi.videobridge.xmpp.user.shard.MUC_NICKNAME=jvb1
org.jitsi.videobridge.xmpp.user.shard.DISABLE_CERTIFICATE_VERIFICATION=true
org.jitsi.videobridge.octo.BIND_ADDRESS=172.31.36.162
org.jitsi.videobridge.octo.PUBLIC_ADDRESS=publicIP
org.jitsi.videobridge.octo.BIND_PORT=4096
org.jitsi.videobridge.REGION=region1
#org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=172.31.36.162
#org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=publicIP

“/etc/jitsi/meet/meet1.mydomain.com-config.js”

testing: {
    // Enables experimental simulcast support on Firefox.
    enableFirefoxSimulcast: false,

    // P2P test mode disables automatic switching to P2P when there are 2
    // participants in the conference.
    p2pTestMode: false,

    octo: {
    probability: 1
}

    // Enables the test specific features consumed by jitsi-meet-torture
    // testMode: false

    // Disables the auto-play behavior of *all* newly created video element.
    // This is useful when the client runs on a host with limited resources.
    // noAutoPlayVideo: false
},

Second JVB

/etc/jitsi/videobridge/sip-communicator.properties

org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true
org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=meet-jit-si-turnrelay.jitsi.net:443
org.jitsi.videobridge.ENABLE_STATISTICS=true
org.jitsi.videobridge.STATISTICS_TRANSPORT=muc,colibri
org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=meet1.mydomain.com
org.jitsi.videobridge.xmpp.user.shard.DOMAIN=auth.meet1.mydomain.com
org.jitsi.videobridge.xmpp.user.shard.USERNAME=jvb
org.jitsi.videobridge.xmpp.user.shard.PASSWORD=Ia1Nvvrh
org.jitsi.videobridge.xmpp.user.shard.MUC_JIDS=JvbBrewery@internal.auth.meet1.mydomain.com
org.jitsi.videobridge.xmpp.user.shard.MUC_NICKNAME=jvb2
org.jitsi.videobridge.xmpp.user.shard.DISABLE_CERTIFICATE_VERIFICATION=true
org.jitsi.videobridge.octo.BIND_ADDRESS=172.31.1.115
org.jitsi.videobridge.octo.PUBLIC_ADDRESS=publicIP
org.jitsi.videobridge.octo.BIND_PORT=4096
org.jitsi.videobridge.REGION=region1
#org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=172.31.1.115
#org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=publicIP

deploymentInfo: {
    shard: "shard",
    region: "region1",
    userRegion: "region1"
},

In Jicofo logs i see
Jicofo 2020-05-06 11:54:27.772 INFO: [11] org.jitsi.jicofo.bridge.BridgeSelector.log() Using org.jitsi.jicofo.bridge.SingleBridgeSelectionStrategy
Jicofo 2020-05-06 11:54:27.772 INFO: [11] org.jitsi.jicofo.bridge.BridgeSelector.log() Bridge failure reset threshold: 300000
Jicofo 2020-05-06 11:54:27.772 INFO: [11] org.jitsi.jicofo.bridge.BridgeSelector.log() Local region: null
Jicofo 2020-05-06 11:54:28.270 INFO: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() Added videobridge: jvbbrewery@internal.auth.meet1.mydomain.com/jvb1 v: null
Jicofo 2020-05-06 11:54:28.270 WARNING: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() No pub-sub node mapped for jvbbrewery@internal.auth.meet1.mydomain.com/jvb1
Jicofo 2020-05-06 11:54:28.274 INFO: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() Added videobridge: jvbbrewery@internal.auth.meet1.mydomain.com/jvb2 v: null
Jicofo 2020-05-06 11:54:28.274 WARNING: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() No pub-sub node mapped for jvbbrewery@internal.auth.meet1.mydomain.com/jvb2
Jicofo 2020-05-06 11:54:28.276 INFO: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() Added videobridge: jvbbrewery@internal.auth.meet1.mydomain.com/jvb3 v: null
Jicofo 2020-05-06 11:54:28.276 WARNING: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() No pub-sub node mapped for jvbbrewery@internal.auth.meet1.mydomain.com/jvb3
Jicofo 2020-05-06 11:54:28.622 INFO: [1] org.jitsi.impl.configuration.ConfigurationServiceImpl.log() org.jitsi.jicofo.BridgeSelector.BRIDGE_SELECTION_STRATEGY=SplitBridgeSelectionStrategy
Jicofo 2020-05-06 11:54:32.068 INFO: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() Removing JVB: jvbbrewery@internal.auth.meet1.mydomain.com/jvb1
Jicofo 2020-05-06 11:54:38.180 INFO: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() Added videobridge: jvbbrewery@internal.auth.meet1.mydomain.com/jvb1 v: null
Jicofo 2020-05-06 11:54:38.180 WARNING: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() No pub-sub node mapped for jvbbrewery@internal.auth.meet1.mydomain.com/jvb1
Jicofo 2020-05-06 11:56:17.160 INFO: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() Removing JVB: jvbbrewery@internal.auth.meet1.mydomain.com/jvb1
Jicofo 2020-05-06 11:56:20.186 SEVERE: [11] org.jitsi.jicofo.bridge.BridgeSelector.log() Failed to find class for: SplitBridgeSelectionStrategy
at org.jitsi.jicofo.bridge.BridgeSelector.createBridgeSelectionStrategy(BridgeSelector.java:191)
at org.jitsi.jicofo.bridge.BridgeSelector.(BridgeSelector.java:154)
Jicofo 2020-05-06 11:56:20.187 INFO: [11] org.jitsi.jicofo.bridge.BridgeSelector.log() Using org.jitsi.jicofo.bridge.SingleBridgeSelectionStrategy
Jicofo 2020-05-06 11:56:20.187 INFO: [11] org.jitsi.jicofo.bridge.BridgeSelector.log() Bridge failure reset threshold: 300000
Jicofo 2020-05-06 11:56:20.187 INFO: [11] org.jitsi.jicofo.bridge.BridgeSelector.log() Local region: null
Jicofo 2020-05-06 11:56:20.828 INFO: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() Added videobridge: jvbbrewery@internal.auth.meet1.mydomain.com/jvb2 v: null
Jicofo 2020-05-06 11:56:20.828 WARNING: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() No pub-sub node mapped for jvbbrewery@internal.auth.meet1.mydomain.com/jvb2
Jicofo 2020-05-06 11:56:20.831 INFO: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() Added videobridge: jvbbrewery@internal.auth.meet1.mydomain.com/jvb3 v: null
Jicofo 2020-05-06 11:56:20.831 WARNING: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() No pub-sub node mapped for jvbbrewery@internal.auth.meet1.mydomain.com/jvb3
Jicofo 2020-05-06 11:56:21.165 INFO: [1] org.jitsi.impl.configuration.ConfigurationServiceImpl.log() org.jitsi.jicofo.BridgeSelector.BRIDGE_SELECTION_STRATEGY=SplitBridgeSelectionStrategy
Jicofo 2020-05-06 11:56:23.264 INFO: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() Added videobridge: jvbbrewery@internal.auth.meet1.mydomain.com/jvb1 v: null
Jicofo 2020-05-06 11:56:23.264 WARNING: [28] org.jitsi.jicofo.bridge.BridgeSelector.log() No pub-sub node mapped for jvbbrewery@internal.auth.meet1.mydomain.com/jvb1

jicofo.txt (19.0 KB)

Tnx

can you post your prosody logs?
Were you able to register the jvbs in prosody ?
you might also want to edit your txt file. your public ip is showing.

I can, actually MUC works without issues… load can be distributed between jvbs as room based… but OCTO doesn’t seem to be working, participants join room at the same jvb.

I’m also facing the same issue like yours. Can you please have a look. Need to test that whether it's working or not after enabling OCTO . Thank you very much

Any advice?

@omesta I see this in your logs “Jicofo 2020-05-06 11:56:20.186 SEVERE: [11] org.jitsi.jicofo.bridge.BridgeSelector.log() Failed to find class for: SplitBridgeSelectionStrategy”. This makes me think you somehow don’t have this class/feature available. How old is your jicofo installation? Are you sure it is up to date?

Not the latest but march 2020 releases
dpkg -s jicofo | grep Version
Version: 1.0-539-1
dpkg -s jitsi-videobridge2 | grep Version
Version: 2.1-192-g06b3dfe5-1

Hi, your config looks very similar to ours - I noticed that both JVBs have the same region though:
org.jitsi.videobridge.REGION=region1 . If that’s actually the case then maybe giving them unique region names would help.

Also, our meet config.js doesn’t have the

    shard: "shard",
    region: "region1",
    userRegion: "region1"

Though I’m not sure if that’s relevant, or even correct, since Octo isn’t quite working right on my setup.

I set same region for both jvbs because in a weekly call it is said that it can be used for better load balancing in a single region…
also i don’t have any geo regions…

I can remove these shard, region, user region data, i am sometimes adding them sometimes removing, for test purposes.

I see, I hadn’t heard that meeting - I might try it too.
But with the SplitBridge selection strategy setup, it should randomly pick a region without needing the geo regions. That’s how I’ve been testing it at least.
In conferences it shows two servers and users get put in regions 1 or 2 randomly. the problem is that it has low-res video for some reason.

makes sense, i will try that thank you; may be it is splitting between regions not jvbs as you suggested…

and what about statistics, have you enabled colibri/stats ? can you see octo details there?

Colibri/stats is showing Octo details with our setup:

,0,0,0,0,0,0],“videostreams”:8,“jitter_aggregate”:4.839777818999097,“total_ice_succeeded_tcp”:0,“octo_endpoints”:2,“current_timestamp”:“2020-05-12 16:25:13.372”,“total_packets_dropped_octo”:2870,“conferences”:1,“participants”:4,“largest_conference”:4,“total_packets_sent”:640944,“total_data_channel_messages_sent”:0,“total_bytes_received_octo”:922637470,“octo_send_packet_rate”:724,“conferences_by_audio_senders”:[1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],“total_conferences_created”:13,“total_ice_failed”:0,“threads”:55,“videochannels”:2,“total_packets_received_octo”:973546

somehow i don’t see them

I have set region1, region2, region3 to my jvbs but still unable to distribute participants… room is always on a single jvb and all participants go there… no octo related stats in colibri… the configs are quite straight forward and i can’t understand why this is not working at all…

1 Like

Hi @omesta , can you upgrade to actual versions? My octo setup is working, but I’m on latest builds from unstable repo…

Milan

Hi @bbaldino @damencho , I’ve setup octo on our installation with two jvb configured with the same region and splitbridge strategy, but have some questions:

  1. What can cause Octo droped packets? One server had jitsi_total_packets_dropped_octo: 32273
    ans second: jitsi_total_packets_dropped_octo: 58939.

  2. I see some jitsi_total_failed_conferences (15 and 13) on our servers after I’ve enabled octo, before this I had not single one ever, and testing jitsi for 2 months now. Is this to be expected with octo?

  3. I’m using /unstable repo. on my debian install, from what branch on github are those build? Master?

Thank you very much,

kind regards,

Milan