JIBRI start and stop after 30s with No Media Session and JVB with timeout for pair

Hi everyone! I’m having some issues when trying to record with Jibri.

The weird fact is that I can successfull run meetings with more than 3 people, but when Jibri is started, he don’t receive any data from JVB and go down with Transitioning from state Running to Error: NoMediaReceived SESSION No media received after about 30s.

I can see a lot of logs from JVB saying INFO: timeout for pair: 192.168.128.5:10000/udp/host -> 192.168.100.1:32352/udp/prflx (stream-c8e8e794.RTP), failing. when jibri start the recording.

My theory is that JVB isn’t sending media data to Jibri for some reason and, without media data, Jibri go down whithin no media timeout.

If someone has any tip I’ll be very gratefull.


Environment:

My environment are bare metal servers, and Jitsi and Jibri are installed using docker in different machines like the scheme bellow:

I have the container ports mapped to host in the JMS host:

ubuntu@live-1-0:~$ docker-compose ps
       Name           Command   State                                                              Ports
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
cefislive_jicofo_1    /init     Up
cefislive_jvb_1       /init     Up      0.0.0.0:10000->10000/udp,:::10000->10000/udp, 0.0.0.0:4443->4443/tcp,:::4443->4443/tcp
cefislive_prosody_1   /init     Up      0.0.0.0:5222->5222/tcp,:::5222->5222/tcp, 0.0.0.0:5280->5280/tcp,:::5280->5280/tcp, 0.0.0.0:5347->5347/tcp,:::5347->5347/tcp
cefislive_web_1       /init     Up      0.0.0.0:8443->443/tcp,:::8443->443/tcp, 0.0.0.0:8000->80/tcp,:::8000->80/tcp

My .env file for JMS host:

# shellcheck disable=SC2034

# Security
#
# Set these to strong passwords to avoid intruders from impersonating a service account
# The service(s) won't start unless these are specified
# Running ./gen-passwords.sh will update .env with strong passwords
# You may skip the Jigasi and Jibri passwords if you are not using those
# DO NOT reuse passwords
#

# XMPP password for Jicofo client connections
JICOFO_AUTH_PASSWORD=<hidden>

# XMPP password for JVB client connections
JVB_AUTH_PASSWORD=<hidden>

# XMPP password for Jigasi MUC client connections
JIGASI_XMPP_PASSWORD=<hidden>

# XMPP recorder password for Jibri client connections
JIBRI_RECORDER_PASSWORD=<hidden>

# XMPP password for Jibri client connections
JIBRI_XMPP_PASSWORD=<hidden>


#
# Basic configuration options
#

# Directory where all configuration will be stored
CONFIG=./jitsi-meet-cfg

# Exposed HTTP port
HTTP_PORT=8000

# Exposed HTTPS port
HTTPS_PORT=8443

# System time zone
TZ=UTC

# Public URL for the web service (required)
PUBLIC_URL=https://cefis.live

# IP address of the Docker host
# See the "Running behind NAT or on a LAN environment" section in the Handbook:
# https://jitsi.github.io/handbook/docs/devops-guide/devops-guide-docker#running-behind-nat-or-on-a-lan-environment
DOCKER_HOST_ADDRESS=189.57.198.250

# Control whether the lobby feature should be enabled or not
#ENABLE_LOBBY=1

# Control whether the A/V moderation should be enabled or not
#ENABLE_AV_MODERATION=1

# Show a prejoin page before entering a conference
ENABLE_PREJOIN_PAGE=1

# Enable the welcome page
#ENABLE_WELCOME_PAGE=1

# Enable the close page
#ENABLE_CLOSE_PAGE=0

# Disable measuring of audio levels
#DISABLE_AUDIO_LEVELS=0

# Enable noisy mic detection
#ENABLE_NOISY_MIC_DETECTION=1

#
# Let's Encrypt configuration
#

# Enable Let's Encrypt certificate generation
#ENABLE_LETSENCRYPT=1

# Domain for which to generate the certificate
#LETSENCRYPT_DOMAIN=meet.example.com

# E-Mail for receiving important account notifications (mandatory)
#LETSENCRYPT_EMAIL=alice@atlanta.net

# Use the staging server (for avoiding rate limits while testing)
#LETSENCRYPT_USE_STAGING=1


#
# Etherpad integration (for document sharing)
#

# Set etherpad-lite URL in docker local network (uncomment to enable)
#ETHERPAD_URL_BASE=http://etherpad.cefis.live:9001

# Set etherpad-lite public URL (uncomment to enable)
#ETHERPAD_PUBLIC_URL=https://etherpad.my.domain

# Name your etherpad instance!
ETHERPAD_TITLE=Video Chat

# The default text of a pad
ETHERPAD_DEFAULT_PAD_TEXT=Welcome to Web Chat!\n\n

# Name of the skin for etherpad
ETHERPAD_SKIN_NAME=colibris

# Skin variants for etherpad
ETHERPAD_SKIN_VARIANTS=super-light-toolbar super-light-editor light-background full-width-editor


#
# Basic Jigasi configuration options (needed for SIP gateway support)
#

# SIP URI for incoming / outgoing calls
#JIGASI_SIP_URI=test@sip2sip.info

# Password for the specified SIP account as a clear text
#JIGASI_SIP_PASSWORD=passw0rd

# SIP server (use the SIP account domain if in doubt)
#JIGASI_SIP_SERVER=sip2sip.info

# SIP server port
#JIGASI_SIP_PORT=5060

# SIP server transport
#JIGASI_SIP_TRANSPORT=UDP

#
# Authentication configuration (see handbook for details)
#

# Enable authentication
ENABLE_AUTH=1

# Enable guest access
ENABLE_GUESTS=1

# Select authentication type: internal, jwt or ldap
AUTH_TYPE=jwt

# JWT authentication
#

# Application identifier
JWT_APP_ID=<hidden>

# Application secret known only to your token generator
JWT_APP_SECRET=<hidden>

# (Optional) Set asap_accepted_issuers as a comma separated list
#JWT_ACCEPTED_ISSUERS=my_web_client,my_app_client

# (Optional) Set asap_accepted_audiences as a comma separated list
#JWT_ACCEPTED_AUDIENCES=my_server1,my_server2


# LDAP authentication (for more information see the Cyrus SASL saslauthd.conf man page)
#

# LDAP url for connection
#LDAP_URL=ldaps://ldap.domain.com/

# LDAP base DN. Can be empty
#LDAP_BASE=DC=example,DC=domain,DC=com

# LDAP user DN. Do not specify this parameter for the anonymous bind
#LDAP_BINDDN=CN=binduser,OU=users,DC=example,DC=domain,DC=com

# LDAP user password. Do not specify this parameter for the anonymous bind
#LDAP_BINDPW=LdapUserPassw0rd

# LDAP filter. Tokens example:
# %1-9 - if the input key is user@mail.domain.com, then %1 is com, %2 is domain and %3 is mail
# %s - %s is replaced by the complete service string
# %r - %r is replaced by the complete realm string
#LDAP_FILTER=(sAMAccountName=%u)

# LDAP authentication method
#LDAP_AUTH_METHOD=bind

# LDAP version
#LDAP_VERSION=3

# LDAP TLS using
#LDAP_USE_TLS=1

# List of SSL/TLS ciphers to allow
#LDAP_TLS_CIPHERS=SECURE256:SECURE128:!AES-128-CBC:!ARCFOUR-128:!CAMELLIA-128-CBC:!3DES-CBC:!CAMELLIA-128-CBC

# Require and verify server certificate
#LDAP_TLS_CHECK_PEER=1

# Path to CA cert file. Used when server certificate verify is enabled
#LDAP_TLS_CACERT_FILE=/etc/ssl/certs/ca-certificates.crt

# Path to CA certs directory. Used when server certificate verify is enabled
#LDAP_TLS_CACERT_DIR=/etc/ssl/certs

# Wether to use starttls, implies LDAPv3 and requires ldap:// instead of ldaps://
# LDAP_START_TLS=1


#
# Advanced configuration options (you generally don't need to change these)
#

# Internal XMPP domain
XMPP_DOMAIN=cefis.live

# Internal XMPP server
XMPP_SERVER=xmpp.cefis.live

# Internal XMPP server URL
XMPP_BOSH_URL_BASE=http://xmpp.cefis.live:5280

# Internal XMPP domain for authenticated services
XMPP_AUTH_DOMAIN=auth.cefis.live

# XMPP domain for the MUC
XMPP_MUC_DOMAIN=muc.cefis.live

# XMPP domain for the internal MUC used for jibri, jigasi and jvb pools
XMPP_INTERNAL_MUC_DOMAIN=internal-muc.cefis.live

# XMPP domain for unauthenticated users
XMPP_GUEST_DOMAIN=guest.cefis.live

# Comma separated list of domains for cross domain policy or "true" to allow all
# The PUBLIC_URL is always allowed
XMPP_CROSS_DOMAIN=true

# Custom Prosody modules for XMPP_DOMAIN (comma separated)
XMPP_MODULES=

# Custom Prosody modules for MUC component (comma separated)
XMPP_MUC_MODULES=

# Custom Prosody modules for internal MUC component (comma separated)
XMPP_INTERNAL_MUC_MODULES=

# MUC for the JVB pool
JVB_BREWERY_MUC=jvbbrewery

# XMPP user for JVB client connections
JVB_AUTH_USER=jvb

# STUN servers used to discover the server's public IP
JVB_STUN_SERVERS=meet-jit-si-turnrelay.jitsi.net:443

# Media port for the Jitsi Videobridge
JVB_PORT=10000

# TCP Fallback for Jitsi Videobridge for when UDP isn't available
JVB_TCP_HARVESTER_DISABLED=true
JVB_TCP_PORT=4443
JVB_TCP_MAPPED_PORT=4443

# A comma separated list of APIs to enable when the JVB is started [default: none]
# See https://github.com/jitsi/jitsi-videobridge/blob/master/doc/rest.md for more information
#JVB_ENABLE_APIS=rest,colibri

# XMPP user for Jicofo client connections.
# NOTE: this option doesn't currently work due to a bug
JICOFO_AUTH_USER=focus

# Base URL of Jicofo's reservation REST API
#JICOFO_RESERVATION_REST_BASE_URL=http://reservation.example.com

# Enable Jicofo's health check REST API (http://<jicofo_base_url>:8888/about/health)
#JICOFO_ENABLE_HEALTH_CHECKS=true

# XMPP user for Jigasi MUC client connections
JIGASI_XMPP_USER=jigasi

# MUC name for the Jigasi pool
JIGASI_BREWERY_MUC=jigasibrewery

# Minimum port for media used by Jigasi
JIGASI_PORT_MIN=20000

# Maximum port for media used by Jigasi
JIGASI_PORT_MAX=20050

# Enable SDES srtp
#JIGASI_ENABLE_SDES_SRTP=1

# Keepalive method
#JIGASI_SIP_KEEP_ALIVE_METHOD=OPTIONS

# Health-check extension
#JIGASI_HEALTH_CHECK_SIP_URI=keepalive

# Health-check interval
#JIGASI_HEALTH_CHECK_INTERVAL=300000
#
# Enable Jigasi transcription
#ENABLE_TRANSCRIPTIONS=1

# Jigasi will record audio when transcriber is on [default: false]
#JIGASI_TRANSCRIBER_RECORD_AUDIO=true

# Jigasi will send transcribed text to the chat when transcriber is on [default: false]
#JIGASI_TRANSCRIBER_SEND_TXT=true

# Jigasi will post an url to the chat with transcription file [default: false]
#JIGASI_TRANSCRIBER_ADVERTISE_URL=true

# Credentials for connect to Cloud Google API from Jigasi
# Please read https://cloud.google.com/text-to-speech/docs/quickstart-protocol
# section "Before you begin" paragraph 1 to 5
# Copy the values from the json to the related env vars
#GC_PROJECT_ID=
#GC_PRIVATE_KEY_ID=
#GC_PRIVATE_KEY=
#GC_CLIENT_EMAIL=
#GC_CLIENT_ID=
#GC_CLIENT_CERT_URL=

# Enable recording
ENABLE_RECORDING=1

# XMPP domain for the jibri recorder
XMPP_RECORDER_DOMAIN=recorder.cefis.live

# XMPP recorder user for Jibri client connections
JIBRI_RECORDER_USER=recorder

# Directory for recordings inside Jibri container
JIBRI_RECORDING_DIR=/config/recordings

# The finalizing script. Will run after recording is complete
#JIBRI_FINALIZE_RECORDING_SCRIPT_PATH=/config/finalize.sh

# XMPP user for Jibri client connections
JIBRI_XMPP_USER=jibri

# MUC name for the Jibri pool
JIBRI_BREWERY_MUC=jibribrewery

# MUC connection timeout
JIBRI_PENDING_TIMEOUT=90

# When jibri gets a request to start a service for a room, the room
# jid will look like: roomName@optional.prefixes.subdomain.xmpp_domain
# We'll build the url for the call by transforming that into:
# https://xmpp_domain/subdomain/roomName
# So if there are any prefixes in the jid (like jitsi meet, which
# has its participants join a muc at conference.xmpp_domain) then
# list that prefix here so it can be stripped out to generate
# the call url correctly
JIBRI_STRIP_DOMAIN_JID=muc

# Directory for logs inside Jibri container
JIBRI_LOGS_DIR=/config/logs

# Configure an external TURN server
# TURN_CREDENTIALS=secret
# TURN_HOST=turnserver.example.com
# TURN_PORT=443
# TURNS_HOST=turnserver.example.com
# TURNS_PORT=443

# Disable HTTPS: handle TLS connections outside of this setup
#DISABLE_HTTPS=1

# Enable FLoC
# Opt-In to Federated Learning of Cohorts tracking
#ENABLE_FLOC=0

# Redirect HTTP traffic to HTTPS
# Necessary for Let's Encrypt, relies on standard HTTPS port (443)
#ENABLE_HTTP_REDIRECT=1

# Send a `strict-transport-security` header to force browsers to use
# a secure and trusted connection. Recommended for production use.
# Defaults to 1 (send the header).
# ENABLE_HSTS=1

# Enable IPv6
# Provides means to disable IPv6 in environments that don't support it (get with the times, people!)
#ENABLE_IPV6=1

# Container restart policy
# Defaults to unless-stopped
RESTART_POLICY=unless-stopped

# Authenticate using external service or just focus external auth window if there is one already.
# TOKEN_AUTH_URL=https://auth.meet.example.com/{room}

# Sentry Error Tracking
# Sentry Data Source Name (Endpoint for Sentry project)
# Example: https://public:private@host:port/1
#JVB_SENTRY_DSN=
#JICOFO_SENTRY_DSN=
#JIGASI_SENTRY_DSN=

# Optional environment info to filter events
#SENTRY_ENVIRONMENT=production

# Optional release info to filter events
#SENTRY_RELEASE=1.0.0

LOG_LEVEL=info

My .env file for Jibri host:

# JIBRI CONFIG

# Public URL for the web service (required)
PUBLIC_URL=https://cefis.live

# Internal XMPP domain for authenticated services
XMPP_AUTH_DOMAIN=auth.cefis.live

# XMPP domain for the internal MUC used for jibri, jigasi and jvb pools
XMPP_INTERNAL_MUC_DOMAIN=internal-muc.cefis.live

# XMPP domain for the jibri recorder
XMPP_RECORDER_DOMAIN=recorder.cefis.live

# Internal XMPP server
XMPP_SERVER=cefis.live

# Internal XMPP domain
XMPP_DOMAIN=cefis.live

# XMPP user for Jibri client connections
JIBRI_XMPP_USER=jibri

# XMPP password for Jibri client connections
JIBRI_XMPP_PASSWORD=<hidden>

# MUC name for the Jibri pool
JIBRI_BREWERY_MUC=jibribrewery

# XMPP recorder user for Jibri client connections
JIBRI_RECORDER_USER=recorder

# XMPP recorder password for Jibri client connections
JIBRI_RECORDER_PASSWORD=<hidden>

# Directory for recordings inside Jibri container
JIBRI_RECORDING_DIR=/config/recordings

# The finalizing script. Will run after recording is complete
JIBRI_FINALIZE_RECORDING_SCRIPT_PATH=/config/finalize.sh

 
# When jibri gets a request to start a service for a room, the room
# jid will look like: roomName@optional.prefixes.subdomain.xmpp_domain
# We'll build the url for the call by transforming that into:
# https://xmpp_domain/subdomain/roomName
# So if there are any prefixes in the jid (like jitsi meet, which
# has its participants join a muc at conference.xmpp_domain) then
# list that prefix here so it can be stripped out to generate
# the call url correctly

JIBRI_STRIP_DOMAIN_JID=muc

# Directory for logs inside Jibri container
JIBRI_LOGS_DIR=/config/logs

DISPLAY=:0=

All this environment is behind a firewall, and the following ports are opened and mapped:

Public IP (10000) -> JMS Host (10000) - UDP
Public IP (5222)  -> JMS Host (5222)  - UDP and TCP
Public IP (5347)  -> JMS Host (5347)  - TCP

Ps.: I have a Nginx as a reverse proxy routing the requests to web container.


Logs:

The following logs was retrived during a test with 3 people in meeting called teste.

Jibri logs: (here we can see that Jibri successfull connect to JMS and we can also see NoMediaReceived error.)
jibri.log (43.0 KB)

JVB logs: (here we can seen a lot off timeout for pair and Suspicious ICE connectivity failure. Checks failed but the remote end was able to reach us..
jvb.log (108.5 KB)

Prosody logs:
prosody.log (16.4 KB)

Jicofo logs:
jicofo.log (40.4 KB)

Jibri Browser logs:
browser.0.txt (1.2 MB)

FFmpeg logs:
ffmpeg.0.txt (15.0 KB)

Web logs:
web.log (66.3 KB)

Welcome to the forum!

Share your jicofo.conf.

Also, what guide did you follow in installing Jibri? Did you customize anything? I’m seeing this error:

_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.

Thanks @Freddie!

My installation is using the Jitsi Meet on Docker. I run only jibri container from docker-compose.yml, there is nothing customized in Jibri, it’s using latest stable oficial docker image.

Jicofo.conf: (I got it from jicofo container in /config/jicofo.conf)

jicofo {

    authentication {
      enabled = true
      // The type of authentication. Supported values are XMPP, JWT or SHIBBOLETH (default).
      type = JWT
      login-url = "cefis.live"
      enable-auto-login=true
    }


    // Configuration related to jitsi-videobridge
    bridge {
      brewery-jid = "jvbbrewery@internal-muc.cefis.live"
    }
    // Configure the codecs and RTP extensions to be used in the offer sent to clients.
    codec {
      video {

      }
    }

    conference {

    }

    jibri {
      brewery-jid = "jibribrewery@internal-muc.cefis.live"
      pending-timeout = "90"

    }

    octo {
      // Whether or not to use Octo. Note that when enabled, its use will be determined by
      // $jicofo.bridge.selection-strategy. There's a corresponding flag in the JVB and these
      // two MUST be in sync (otherwise bridges will crash because they won't know how to
      // deal with octo channels).
      enabled = false
      id = "1"
    }

    sctp {
      enabled = false
    }

    xmpp {
      client {
        enabled = true
        hostname = "xmpp.cefis.live"
        domain = "auth.cefis.live"
        username = "focus"
        password = "25a2b5013a398b97c2c858854e9486d2"
        conference-muc-jid = "muc.cefis.live"
        client-proxy = "focus.cefis.live"
        disable-certificate-verification = true
      }

      trusted-domains = [recorder.cefis.live]
    }
}

Do you by chance have a desktop OS on the jibri machine?

Seems like jibri cannot communicate with JVB. Probably you need HARVESTER lines for JVB to communicate with jibri which is in the local network

Hey everyone, I got it working! I need to keep JVB_TCP_HARVESTER_DISABLED = false and enable a rule in my firewall to forward the traffic:

Public IP (4443)  -> JMS Host (4443)  - TCP

I believe that now Jibri is connect to JVB by TCP, am I right? Is it bad?

But the question in my mind is: why UDP traffic between JVB and Jibri don’t work in a local network, but work for external clients?

No @Freddie, it’s a normal Ubuntu Server 18.04.

I think like you, communication bettween Jibri and JVB wasn’t working in UDP. But why? Do you have any theory?

If HARVESTER lines are missing, JVB learns its external IP from stun and send it to the clients (actually it’s not happening exactly like this but this is not the point for now). The client tries to connect to JVB using this external address but it fails when it’s in the local network