P2p Issues

Dear Jitsi Community

i have a question about peer-to-peer mode which seems to work sometimes and sometimes doesn’t despite using the same UA (e.g. Chrome 86.0.4240.75 on Mac OS) in similar network topologies.

Our setup is based on dedicated (VM) Servers in a data center. All servers run Debian 10 (Buster) with ipv4 and ipv6 fully configured including related DNS entries:

  • jitsi.example.net (main server with jitsi-meet, jitsi-videobridge2, prosody etc.)
  • turn.example.net (dedicated TURN/STUN server)
  • vb2.example.net (additional jitsi-videobridge2 for horizontal scaling)

We limit conferences to two participants to ensure peer-to-peer mode whenever possible and therefore end-to-end encryption for best privacy. Participants have to provide JWT for authentication.

The setup works pretty well:

  • Conferences take place without any problems
  • TURN seems to work as well. Forcing TURN by blocking outgoing 10000/UDP on the client routes the media traffic via turn.example.net. The “connection indicator” in the upper right box shows (turn)
  • Distributing the media data between the two Videobridges does work too

What i do not understand is that peer-to-peer mode fails in some constellations:

console output on the faulty client

Logger.js:154 2020-10-12T11:28:24.920Z [modules/browser/BrowserCapabilities.js] <new a>:  This appears to be chrome, ver: 86.0.4240.75
[...]
Logger.js:154 2020-10-12T11:28:27.692Z [modules/xmpp/JingleSessionPC.js] <A.peerconnection.oniceconnectionstatechange>:  (TIME) ICE checking P2P? true:	 2982.3099999994156
Logger.js:154 2020-10-12T11:28:27.718Z [modules/RTC/BridgeChannel.js] <RTCDataChannel.e.onopen>:  datachannel channel opened
Logger.js:154 2020-10-12T11:28:27.718Z [JitsiConferenceEventManager.js] <a.<anonymous>>:  (TIME) data.channel.opened:	 3009.0650000001915
Logger.js:154 2020-10-12T11:28:27.720Z [modules/xmpp/JingleSessionPC.js] <w.setReceiverVideoConstraint>:  JingleSessionPC[p2p=false,initiator=false,sid=fvpjmvg630hvc] setReceiverVideoConstraint - max frame height: 720
Logger.js:154 2020-10-12T11:28:27.721Z [modules/RTC/BridgeChannel.js] <l.sendReceiverVideoConstraintMessage>:  sending a ReceiverVideoConstraint message with a maxFrameHeight of 720 pixels
Logger.js:154 2020-10-12T11:28:27.721Z [modules/xmpp/JingleSessionPC.js] <w.setReceiverVideoConstraint>:  JingleSessionPC[p2p=true,initiator=true,sid=44876438226c] setReceiverVideoConstraint - max frame height: 720
Logger.js:154 2020-10-12T11:28:27.721Z [modules/xmpp/JingleSessionPC.js] <w.sendContentModify>:  JingleSessionPC[p2p=true,initiator=true,sid=44876438226c] sending content-modify, video senders: both, max frame height: 720
Logger.js:154 2020-10-12T11:28:27.721Z [features/video-quality] <va>:  setReceiverVideoConstraint: 720
Logger.js:154 2020-10-12T11:28:27.722Z [modules/RTC/BridgeChannel.js] <l.sendPinnedEndpointMessage>:  sending pinned changed notification to the bridge for endpoint  null
Logger.js:154 2020-10-12T11:28:27.722Z [modules/RTC/BridgeChannel.js] <l.sendSelectedEndpointsMessage>:  sending selected changed notification to the bridge for endpoints Array(1)
Logger.js:154 2020-10-12T11:28:27.722Z [modules/RTC/BridgeChannel.js] <l.sendReceiverVideoConstraintMessage>:  sending a ReceiverVideoConstraint message with a maxFrameHeight of 720 pixels
Logger.js:154 2020-10-12T11:28:27.825Z [modules/RTC/BridgeChannel.js] <RTCDataChannel.e.onmessage>:  Channel new dominant speaker event:  4e374abe
Logger.js:154 2020-10-12T11:28:27.861Z [modules/xmpp/strophe.jingle.js] <g.onJingle>:  on jingle transport-info from therapie-videositzung-id-y1copuaxinzhai1i@conference.jitsi.example.net/77543589 <iq xmlns=​"jabber:​client" from=​"therapie-videositzung-id-y1copuaxinzhai1i@conference.jitsi.example.net/​77543589" type=​"set" id=​"NGUzNzRhYmUtZThiYy00YjljLTkzMWItYzhiNTg2ODE1YjgxQGppdHNpLnRoZXJhcGllLmRlL3luNzBJY3FIADcxNDY3YWY1LTc1N2EtNDIxMi04Mzc2LTI1OGUzYjI1NzlmODpzZW5kSVEAvdOKVGXiNfBA4hFVykyozw==" to=​"4e374abe-e8bc-4b9c-931b-c8b586815b81@jitsi.example.net/​yn70IcqH">​…​</iq>​
Logger.js:154 2020-10-12T11:28:27.861Z [JitsiConference.js] <re.onTransportInfo>:  P2P addIceCandidates
Logger.js:154 2020-10-12T11:28:27.861Z [modules/xmpp/JingleSessionPC.js] <w.addIceCandidates>:  Queued add (1) ICE candidates task...
Logger.js:154 2020-10-12T11:28:27.861Z [modules/xmpp/strophe.jingle.js] <g.onJingle>:  on jingle transport-info from therapie-videositzung-id-y1copuaxinzhai1i@conference.jitsi.example.net/77543589 <iq xmlns=​"jabber:​client" from=​"therapie-videositzung-id-y1copuaxinzhai1i@conference.jitsi.example.net/​77543589" type=​"set" id=​"NGUzNzRhYmUtZThiYy00YjljLTkzMWItYzhiNTg2ODE1YjgxQGppdHNpLnRoZXJhcGllLmRlL3luNzBJY3FIADg1ZjA5NGMwLTlmNWQtNDcyNi04NTQ1LWNjZTQ4ODU0MTE1ZjpzZW5kSVEAvdOKVGXiNfBA4hFVykyozw==" to=​"4e374abe-e8bc-4b9c-931b-c8b586815b81@jitsi.example.net/​yn70IcqH">​…​</iq>​
Logger.js:154 2020-10-12T11:28:27.862Z [JitsiConference.js] <re.onTransportInfo>:  P2P addIceCandidates
Logger.js:154 2020-10-12T11:28:27.862Z [modules/xmpp/JingleSessionPC.js] <w.addIceCandidates>:  Queued add (1) ICE candidates task...
Logger.js:154 2020-10-12T11:28:27.862Z [modules/xmpp/strophe.jingle.js] <g.onJingle>:  on jingle content-modify from therapie-videositzung-id-y1copuaxinzhai1i@conference.jitsi.example.net/77543589 <iq xmlns=​"jabber:​client" from=​"therapie-videositzung-id-y1copuaxinzhai1i@conference.jitsi.example.net/​77543589" type=​"set" id=​"NGUzNzRhYmUtZThiYy00YjljLTkzMWItYzhiNTg2ODE1YjgxQGppdHNpLnRoZXJhcGllLmRlL3luNzBJY3FIAGMwNWNkMzNmLTMxYzktNGI4Yi1iZTRkLWZkOWEwZjgxMDJiOTpzZW5kSVEAvdOKVGXiNfBA4hFVykyozw==" to=​"4e374abe-e8bc-4b9c-931b-c8b586815b81@jitsi.example.net/​yn70IcqH">​…​</iq>​
Logger.js:154 2020-10-12T11:28:27.862Z [modules/xmpp/JingleSessionPC.js] <w.modifyContents>:  JingleSessionPC[p2p=true,initiator=true,sid=44876438226c] received remote max frame height: 720
Logger.js:154 2020-10-12T11:28:27.862Z [modules/xmpp/JingleSessionPC.js] <w.modifyContents>:  JingleSessionPC[p2p=true,initiator=true,sid=44876438226c] queued "content-modify" task(video senders="both")
Logger.js:154 2020-10-12T11:28:27.863Z [modules/xmpp/JingleSessionPC.js] addIceCandidate ok!
Logger.js:154 2020-10-12T11:28:27.863Z [modules/xmpp/JingleSessionPC.js] addIceCandidate ok!
Logger.js:154 2020-10-12T11:28:28.134Z [modules/xmpp/strophe.jingle.js] <g.onJingle>:  on jingle content-modify from therapie-videositzung-id-y1copuaxinzhai1i@conference.jitsi.example.net/77543589 <iq xmlns=​"jabber:​client" from=​"therapie-videositzung-id-y1copuaxinzhai1i@conference.jitsi.example.net/​77543589" type=​"set" id=​"NGUzNzRhYmUtZThiYy00YjljLTkzMWItYzhiNTg2ODE1YjgxQGppdHNpLnRoZXJhcGllLmRlL3luNzBJY3FIAGM4NTk2MTBlLTRmZGItNGMxZC1hNjJiLTk2OWU1YmMyYTgwMjpzZW5kSVEAvdOKVGXiNfBA4hFVykyozw==" to=​"4e374abe-e8bc-4b9c-931b-c8b586815b81@jitsi.example.net/​yn70IcqH">​…​</iq>​
Logger.js:154 2020-10-12T11:28:28.134Z [modules/xmpp/JingleSessionPC.js] <w.modifyContents>:  JingleSessionPC[p2p=true,initiator=true,sid=44876438226c] received remote max frame height: 720
Logger.js:154 2020-10-12T11:28:28.135Z [modules/xmpp/JingleSessionPC.js] <w.modifyContents>:  JingleSessionPC[p2p=true,initiator=true,sid=44876438226c] queued "content-modify" task(video senders="both")
DevTools failed to load SourceMap: Could not parse content for https://jitsi.example.net/external_api.min.map: Unexpected token < in JSON at position 0
[...]
Logger.js:154 2020-10-12T11:28:42.905Z [modules/xmpp/strophe.jingle.js] <g.onJingle>:  on jingle session-terminate from therapie-videositzung-id-y1copuaxinzhai1i@conference.jitsi.example.net/77543589 <iq xmlns=​"jabber:​client" from=​"therapie-videositzung-id-y1copuaxinzhai1i@conference.jitsi.example.net/​77543589" type=​"set" id=​"NGUzNzRhYmUtZThiYy00YjljLTkzMWItYzhiNTg2ODE1YjgxQGppdHNpLnRoZXJhcGllLmRlL3luNzBJY3FIADUyZjk3YzE2LTM2ZjQtNGUwMi1hYjk3LTAyNTFhYWE2YzcyYTpzZW5kSVEAvdOKVGXiNfBA4hFVykyozw==" to=​"4e374abe-e8bc-4b9c-931b-c8b586815b81@jitsi.example.net/​yn70IcqH">​…​</iq>​
Logger.js:154 2020-10-12T11:28:42.905Z [modules/xmpp/strophe.jingle.js] <g.onJingle>:  terminating... 44876438226c
Logger.js:154 2020-10-12T11:28:42.906Z [modules/xmpp/JingleSessionPC.js] <w.onTerminated>:  Session terminated JingleSessionPC[p2p=true,initiator=true,sid=44876438226c] connectivity-error ICE FAILED
Logger.js:154 2020-10-12T11:28:42.906Z [JitsiConference.js] <re.onCallEnded>:  Call ended: connectivity-error - ICE FAILED P2P ?true

Constallations

  • Network A (Homeoffice, behind NAT Router, ipv4 + ipv6) <–> Network B (Homeoffice, behind NAT Router, ipv4 only) , p2p does not work
  • Network A <–> Network C (Office, behind 2 NAT Routers, ipv4 + ipv6), p2p works
  • Network B <–> Network C. p2p works

This does not make much sense to me so i assume i have misconfigured somewhere in this rather complex setup. Maybe someone has an idea how I can continue at this point?!

Many thanks!

Bests

Harald

Software Versions

  • jitsi-meet 2.0.5076-1
  • jitsi-meet-prosody 1.0.4428-1
  • jitsi-meet-tokens 1.0.4428-1
  • jitsi-meet-web 1.0.4428-1
  • jitsi-meet-web-config 1.0.4428-1
  • jitsi-videobridge2 2.1-351-g0bfaac1c-1
  • jicofo 1.0-636-1
  • prosody 0.11.6-1~bpo10+1
  • coturn 4.5.1.1-1.1+deb10u1

Config Files

/etc/jitsi/meet/jitsi.example.net-config.js

/* eslint-disable no-unused-vars, no-var */
var config = {
    hosts: {
        domain: 'jitsi.example.net',
        muc: 'conference.jitsi.example.net'
    },
    bosh: '//jitsi.example.net/http-bind',
    clientNode: 'http://jitsi.example.net/jitsimeet',
    testing: {
        p2pTestMode: false
    },
    enableNoAudioDetection: true,
    enableNoisyMicDetection: true,
    startAudioOnly: false,
    resolution: 720,
    constraints: {
    "video": {
        "height": {
            "ideal": 720,
            "max": 720,
            "min": 240
        }
    }
},
    enableLayerSuspension: true,
    desktopSharingChromeExtId: null,
    desktopSharingChromeSources: [ 'screen', 'window', 'tab' ],
    desktopSharingChromeMinExtVersion: '0.1',
    channelLastN: -1,
    useStunTurn: true,
    useIPv6: true,
    requireDisplayName: false,
    enableWelcomePage: true,
    defaultLanguage: 'de',
    enableUserRolesBasedOnToken: false,
    disableThirdPartyRequests: false,
    p2p: {
        enabled: true,
        useStunTurn: true,
        stunServers: [
            { urls: 'stun:turn.example.net:443' }
        ],
        preferH264: true
    },
    analytics: {
        matomoEndpoint: 'https://matomo.example.net/',
        matomoSiteID: '42'
    },
    deploymentInfo: {
    },
    makeJsonParserHappy: 'even if last key had a trailing comma'
};
/* eslint-enable no-unused-vars, no-var */

/etc/jitsi/videobridge/config (main server)

JVB_HOSTNAME=jitsi.example.net
JVB_HOST=
JVB_PORT=5347
JVB_SECRET=<my videobridge secrect>
JVB_OPTS="--apis=rest,xmpp"
JAVA_SYS_PROPS="-Dnet.java.sip.communicator.SC_HOME_DIR_LOCATION=/etc/jitsi -Dnet.java.sip.communicator.SC_HOME_DIR_NAME=videobridge -Dnet.java.sip.communicator.SC_LOG_DIR_LOCATION=/var/log/jitsi -Djava.util.logging.config.file=/etc/jitsi/videobridge/logging.properties"

/etc/jitsi/videobridge/sip-communicator.properties

org.ice4j.ice.harvest.DISABLE_AWS_HARVESTER=true
org.ice4j.ice.harvest.STUN_MAPPING_HARVESTER_ADDRESSES=turn.example.net:443
org.jitsi.videobridge.DISABLE_TCP_HARVESTER=true
org.jitsi.videobridge.ENABLE_STATISTICS=true
org.jitsi.videobridge.STATISTICS_TRANSPORT=muc
org.jitsi.videobridge.xmpp.user.shard.HOSTNAME=localhost
org.jitsi.videobridge.xmpp.user.shard.DOMAIN=auth.jitsi.example.net
org.jitsi.videobridge.xmpp.user.shard.USERNAME=jvb
org.jitsi.videobridge.xmpp.user.shard.PASSWORD=<my videobridge password>
org.jitsi.videobridge.xmpp.user.shard.MUC_JIDS=JvbBrewery@internal.auth.jitsi.example.net
org.jitsi.videobridge.xmpp.user.shard.MUC_NICKNAME=a4df7c17-a7c8-5ba9-b6c9-4aba7faec945

/etc/jitsi/jicofo/config

JICOFO_HOST=localhost
JICOFO_HOSTNAME=jitsi.example.net
JICOFO_SECRET=<my jicofo secret>
JICOFO_PORT=5347
JICOFO_AUTH_DOMAIN=auth.jitsi.example.net
JICOFO_AUTH_USER=focus
JICOFO_AUTH_PASSWORD=<my jicofo password>
JICOFO_OPTS=""
JAVA_SYS_PROPS="-Dnet.java.sip.communicator.SC_HOME_DIR_LOCATION=/etc/jitsi -Dnet.java.sip.communicator.SC_HOME_DIR_NAME=jicofo -Dnet.java.sip.communicator.SC_LOG_DIR_LOCATION=/var/log/jitsi -Djava.util.logging.config.file=/etc/jitsi/jicofo/logging.properties"

/etc/jitsi/jicofo/sip-communicator.properties

org.jitsi.jicofo.BRIDGE_MUC=JvbBrewery@internal.auth.jitsi.example.net
org.jitsi.jicofo.DISABLE_AUTO_OWNER=true

/etc/prosody/conf.d/jitsi.example.net.cfg.lua

plugin_paths = { "/usr/share/jitsi-meet/prosody-plugins/" }
muc_mapper_domain_base = "jitsi.example.net";
turncredentials_secret = "<my turn secret>";
turncredentials = {
  { type = "stun", host = "turn.example.net", port = "80" },
  { type = "turn", host = "turn.example.net", port = "80", transport = "udp" },
  { type = "turns", host = "turn.example.net", port = "443", transport = "tcp" }
};
cross_domain_bosh = false;
consider_bosh_secure = true;
VirtualHost "jitsi.example.net"
        authentication = "token"
        app_id = "therapie_web_client"
        app_secret = "<my base64 JWT secret>"
        allow_empty_token = false
        ssl = {
                key = "/etc/prosody/certs/jitsi.example.net.key";
                certificate = "/etc/prosody/certs/jitsi.example.net.crt";
        }
        speakerstats_component = "speakerstats.jitsi.example.net"
        conference_duration_component = "conferenceduration.jitsi.example.net"
        modules_enabled = {
            "bosh";
            "pubsub";
            "ping"; -- Enable mod_ping
            "speakerstats";
            "turncredentials";
            "conference_duration";
        }
        c2s_require_encryption = false
Component "conference.jitsi.example.net" "muc"
    default_storage = "memory"
    muc_max_occupants = 2
    muc_access_whitelist = { 'focus@auth.jitsi.example.net' }
    modules_enabled = {
        "muc_meeting_id";
        "muc_domain_mapper";
        "muc_max_occupants";
        "token_verification";
        "token_affiliation";
    }
    admins = { "focus@auth.jitsi.example.net" }
    muc_room_locking = false
    muc_room_default_public_jids = true
Component "internal.auth.jitsi.example.net" "muc"
    storage = "memory"
    modules_enabled = {
      "ping";
    }
    admins = { "focus@auth.jitsi.example.net", "jvb@auth.jitsi.example.net" }
    muc_room_locking = false
    muc_room_default_public_jids = true
VirtualHost "auth.jitsi.example.net"
    ssl = {
        key = "/etc/prosody/certs/auth.jitsi.example.net.key";
        certificate = "/etc/prosody/certs/auth.jitsi.example.net.crt";
    }
    authentication = "internal_plain"
Component "focus.jitsi.example.net"
    component_secret = "<my jicofo secret>"
Component "speakerstats.jitsi.example.net" "speakerstats_component"
    muc_component = "conference.jitsi.example.net"
Component "conferenceduration.jitsi.example.net" "conference_duration_component"
    muc_component = "conference.jitsi.example.net"
Component "jitsi-videobridge.jitsi.example.net"
    component_secret = "<my videobridge secret>"

/etc/turnserver.conf

listening-port=80
tls-listening-port=443
verbose
fingerprint
use-auth-secret
static-auth-secret=<my turn secret>
realm=turn.example.net
total-quota=0
bps-capacity=0
no-tcp
no-tcp-relay
stale-nonce=600
cert=/etc/coturn/certs/turn.example.net_fullchain.pem
pkey=/etc/coturn/certs/turn.example.net_privkey.pem
cipher-list="ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384"
dh-file=/etc/coturn/certs/dhparam.pem
log-file=/var/log/coturn/coturn.log
no-loopback-peers
no-multicast-peers
denied-peer-ip=0.0.0.0-0.255.255.255
denied-peer-ip=10.0.0.0-10.255.255.255
denied-peer-ip=100.64.0.0-100.127.255.255
denied-peer-ip=127.0.0.0-127.255.255.255
denied-peer-ip=169.254.0.0-169.254.255.255
denied-peer-ip=127.0.0.0-127.255.255.255
denied-peer-ip=172.16.0.0-172.31.255.255
denied-peer-ip=192.0.0.0-192.0.0.255
denied-peer-ip=192.0.2.0-192.0.2.255
denied-peer-ip=192.88.99.0-192.88.99.255
denied-peer-ip=192.168.0.0-192.168.255.255
denied-peer-ip=198.18.0.0-198.19.255.255
denied-peer-ip=198.51.100.0-198.51.100.255
denied-peer-ip=203.0.113.0-203.0.113.255
denied-peer-ip=240.0.0.0-255.255.255.255
keep-address-family
no-cli
no-tlsv1
no-tlsv1_1