Correct Installation For Jibri 2022

BingeWave · March 22, 2022, 1:37pm

Recently I updated Jibri and it stopped working for my Jitsi setup, so no recording or restreaming. I looked at my error logs to see this:

2022-03-22 12:21:26.135 INFO: [1] JwtInfo$Companion.fromConfig#164: Unable to create JwtInfo: com.typesafe.config.ConfigException$Missing: /etc/jitsi/jibri/jibri.conf: 86: No configuration setting found for key 'signing-key-path'

2022-03-22 12:21:26.261 INFO: [1] MainKt.main#125: Using port 3333 for internal HTTP API

2022-03-22 12:21:26.267 FINE: [17] WebhookClient$updateStatus$1.invokeSuspend#107: Updating 0 subscribers of status

2022-03-22 12:21:26.483 INFO: [1] XmppApi.updatePresence#144: Jibri reports its status is now JibriStatus(busyStatus=IDLE, health=OverallHealth(healthStatus=HEALTHY, details={})), publishing presence to connections

2022-03-22 12:21:26.490 INFO: [1] XmppApi.start#97: Connecting to xmpp environment on meet.bingewave.com with config XmppEnvironmentConfig(name=prod environment, xmppServerHosts=[meet.bingewave.com], xmppDomain=meet.example.com, baseUrl=null, controlLogin=XmppCredentials(domain=auth.meet.bingewave.com, port=null, username=xxxxxx, password=*****), controlMuc=XmppMuc(domain=internal.auth.meet.example.com, roomName=JibriBrewery, nickname=jibri-nickname), sipControlMuc=null, callLogin=XmppCredentials(domain=recorder.meet.bingewave.com, port=null, username=recorder, password=*****), stripFromRoomDomain=conference., usageTimeoutMins=0, trustAllXmppCerts=true, securityMode=null)

2022-03-22 12:21:26.491 INFO: [1] XmppApi.start#109: The trustAllXmppCerts config is enabled for this domain, all XMPP server provided certificates will be accepted

2022-03-22 12:21:26.509 INFO: [28] [hostname=meet.example.com id=meet.example.com] MucClient.initializeConnectAndJoin#272: Initializing a new MucClient for [ org.jitsi.xmpp.mucclient.MucClientConfiguration id=meet.example.com domain=auth.meet.example.com hostname=meet.example.com port=null username=xxxxx mucs=[jibribrewery@internal.auth.meet.example.com] mucNickname=jibri-nickname disableCertificateVerification=true]

2022-03-22 12:21:26.510 INFO: [1] MainKt.main#151: Using port 2222 for HTTP API

2022-03-22 12:21:26.525 WARNING: [28] MucClient.createXMPPTCPConnectionConfiguration#114: Disabling certificate verification!

2022-03-22 12:21:26.546 INFO: [28] [hostname=meet.example.com id=meet.example.com] MucClient.initializeConnectAndJoin#331: Dispatching a thread to connect and login.

2022-03-22 12:21:26.651 INFO: [28] [hostname=meet.example.com id=meet.example.com] MucClient$2.connected#304: Connected.

2022-03-22 12:21:26.651 INFO: [28] [hostname=meet.example.com id=meet.example.com] MucClient.lambda$getConnectAndLoginCallable$8#628: Logging in.

2022-03-22 12:21:26.693 INFO: [28] [hostname=meet.example.com id=meet.example.com] MucClient$2.authenticated#310: Authenticated, b=false

2022-03-22 12:21:26.716 INFO: [28] [hostname=meet.example.com id=meet.example.com] MucClient$MucWrapper.join#748: Joined MUC: jibribrewery@internal.auth.meet.example.com

I think the core issue is this line:


2022-03-22 12:21:26.135 INFO: [1] JwtInfo$Companion.fromConfig#164: Unable to create JwtInfo: com.typesafe.config.ConfigException$Missing: /etc/jitsi/jibri/jibri.conf: 86: No configuration setting found for key 'signing-key-path'

I originally installed Jibri use this tutorial: Jibri setup and configuration - here’s how! FULL GUIDE

And according to another post, this is the latest config, which doesn’t include the jwt-info section: emrah-buster-templates/jibri.conf at master · emrahcom/emrah-buster-templates · GitHub

What is the most update way to install Jibri as of March 2022?

emrah · March 22, 2022, 1:40pm

This log is OK

BingeWave · March 22, 2022, 1:47pm

Then this is log with the issue:

Jicofo 2022-03-22 13:44:26.161 INFO: [528] JibriSession.startInternal#320: Starting session with Jibri jibribrewery@internal.auth.meet.example.com/jibri-nickname

Jicofo 2022-03-22 13:44:26.162 INFO: [528] JibriSession.sendJibriStartIq#474: Starting Jibri jibribrewery@internal.auth.meet.example.com/jibri-nickname for stream ID: rtmp://ingest.livestream.com/live/0c347b52-aedc-4320-80d1-58b64f1d55c3?sign=1648561452-8145f75ea4db in room: 0c347b52-aedc-4320-80d1-58b64f1d55c3@conference.meet.example.com

Jicofo 2022-03-22 13:44:26.412 SEVERE: [528] JibriSession.sendJibriStartIq#533: Unexpected status received in response to the start IQ: <iq xmlns='jabber:client' to='focus@auth.meet.example.com/focus' from='jibribrewery@internal.auth.meet.example.com/jibri-nickname' id='SHY6S-194884' type='result'><jibri xmlns='http://jitsi.org/protocol/jibri' status='off' failure_reason='error' should_retry='true'/></iq>

Jicofo 2022-03-22 13:44:26.412 SEVERE: [528] JibriSession.startInternal#326: Failed to send start Jibri IQ: org.jitsi.jicofo.jibri.JibriSession$StartException$UnexpectedResponse: Unexpected response

org.jitsi.jicofo.jibri.JibriSession$StartException$UnexpectedResponse: Unexpected response

at org.jitsi.jicofo.jibri.JibriSession.sendJibriStartIq(JibriSession.java:537)

at org.jitsi.jicofo.jibri.JibriSession.startInternal(JibriSession.java:322)

at org.jitsi.jicofo.jibri.JibriSession.start(JibriSession.java:286)

at org.jitsi.jicofo.jibri.JibriRecorder.handleStartRequest(JibriRecorder.kt:112)

at org.jitsi.jicofo.jibri.BaseJibri.doHandleIQRequest(BaseJibri.kt:169)

at org.jitsi.jicofo.jibri.BaseJibri.incomingIqQueue$lambda-0(BaseJibri.kt:56)

at org.jitsi.utils.queue.PacketQueue$HandlerAdapter.handleItem(PacketQueue.java:416)

at org.jitsi.utils.queue.AsyncQueueHandler$1.run(AsyncQueueHandler.java:136)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

Jicofo 2022-03-22 13:44:26.412 INFO: [528] JibriDetector.instanceFailed#88: Instance failed: jibribrewery@internal.auth.meet.example.com/jibri-nickname. Will not be selected for the next PT1M

Jicofo 2022-03-22 13:44:26.412 SEVERE: [528] JibriSession.startInternal#308: Unable to find an available Jibri, can't start

Jicofo 2022-03-22 13:44:26.412 INFO: [528] [room=0c347b52-aedc-4320-80d1-58b64f1d55c3@conference.meet.example.com meeting_id=d3817107-3d5b-4f28-92e9-fd68ab59190b] JibriRecorder.handleStartRequest#120: Failed to start a Jibri session, all Jibris were busy

emrah · March 22, 2022, 1:52pm

Did you upgrade JMS too? Seems like their versions don’t match

BingeWave · March 22, 2022, 2:11pm

I ran a dpkg -l | grep jitsi on my jitsi server and got this:

ii  jitsi-meet                             2.0.7001-1                                      all          WebRTC JavaScript video conferences
ii  jitsi-meet-prosody                     1.0.5913-1                                      all          Prosody configuration for Jitsi Meet
ii  jitsi-meet-tokens                      1.0.5913-1                                      all          Prosody token authentication plugin for Jitsi Meet
ii  jitsi-meet-turnserver                  1.0.5913-1                                      all          Configures coturn to be used with Jitsi Meet
ii  jitsi-meet-web                         1.0.5913-1                                      all          WebRTC JavaScript video conferences
ii  jitsi-meet-web-config                  1.0.5913-1                                      all          Configuration for web serving of Jitsi Meet
ii  jitsi-videobridge2                     2.1-634-gff8609ad-1                             all          WebRTC compatible Selective Forwarding Unit (SFU)

And running this on my Jibri server, which only has Jibri, gives me this:

ii  jibri                                  8.0-105-gddca7a2-1                              all          Jibri

emrah · March 22, 2022, 3:47pm

My up-to-date setup has

ii  jitsi-meet            2.0.7001-1   all
ii  jitsi-meet-prosody    1.0.5913-1   all
ii  jitsi-meet-tokens     1.0.5913-1   all
ii  jitsi-meet-turnserver 1.0.5913-1   all
ii  jitsi-meet-web        1.0.5913-1   all
ii  jitsi-meet-web-config 1.0.5913-1   all

ii  jibri          8.0-121-g27323fe-1 all

The following were released after jibri 8.0-105:

8.0-113
8.0-114
8.0-116
8.0-121

dynamite · March 24, 2022, 9:18am

Hi,
My recent setup of JMS and jibri is
JMS:

ii jicofo 1.0-832-1
ii jitsi-meet 2.0.6726-1
ii jitsi-meet-prosody 1.0.5675-1
ii jitsi-meet-turnserver 1.0.5675-1
ii jitsi-meet-web 1.0.5675-1
ii jitsi-meet-web-config 1.0.5675-1
ii jitsi-videobridge2 2.1-595-g3637fda4-1
JIBRI:
jibri 8.0-121-g27323fe-1

I have configured jibri on one machine and JMS on another. Initially, i setup JMS and it has been tested with multiple clients. Now, i want to enable recording feature by following all the guidelines mentioned in the forum. However after start recording, the server always say All recorders are busy

LOG:
2022-03-22 13:07:00.174 INFO: [1] MainKt.handleCommandLineArgs#186: Jibri run with args [–config, /etc/jitsi/jibri/config.json]
2022-03-22 13:07:00.323 INFO: [1] MainKt.setupLegacyConfig#211: Checking legacy config file /etc/jitsi/jibri/config.json
2022-03-22 13:07:00.323 INFO: [1] MainKt.setupLegacyConfig#214: Legacy config file /etc/jitsi/jibri/config.json doesn’t exist
2022-03-22 13:07:00.609 INFO: [1] MainKt.main#55: Jibri starting up with id
2022-03-22 13:07:00.902 INFO: [1] JwtInfo$Companion.fromConfig#154: got jwtConfig: {}

2022-03-22 13:07:00.902 INFO: [1] JwtInfo$Companion.fromConfig#164: Unable to create JwtInfo: com.typesafe.config.ConfigException$Missing: /etc/jitsi/jibri/jibri.conf: 107: No configuration setting found for key ‘signing-key-path’
2022-03-22 13:07:01.084 INFO: [1] MainKt.main#125: Using port 3333 for internal HTTP API
2022-03-22 13:07:01.100 FINE: [17] WebhookClient$updateStatus$1.invokeSuspend#107: Updating 0 subscribers of status

Please help me to fix it.
Thankyou

emrah · March 24, 2022, 10:13am

Is there any error in logs?

Shared part is OK

dynamite · March 24, 2022, 10:20am

Attached is the output of start recording on UI

My jibri machine resources are
8GM RAM
CPU 4

Still recording does not get started. Also, i am not getting any error.

I ran lsof -i | grep 5222 on jibri machine

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 1918 jibri 54u IPv6 32467 0t0 TCP xxx.xxx.xxx.xx:55750->1.2.3.4:xmpp-client (SYN_SENT)
java 1918 jibri 55u IPv6 32468 0t0 TCP xxx.xxx.xxx.xx:37672->1.2.3.4:xmpp-client (ESTABLISHED)

dynamite · March 24, 2022, 10:28am

root@xxx:/etc/jitsi/jibri# service jibri status
● jibri.service - Jibri Process
Loaded: loaded (/etc/systemd/system/jibri.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-03-24 10:12:16 UTC; 12min ago
Main PID: 1918 (java)
Tasks: 47 (limit: 19105)
Memory: 197.2M
CGroup: /system.slice/jibri.service
└─1918 /usr/lib/jvm/adoptopenjdk-8-hotspot-amd64/bin/java -Djava.util.logging.config.file=/etc/jitsi/jibri/logging.properties -Dconfig.file=/etc/jitsi/jibri/jibri.conf -jar /opt/jitsi/jibri/jibri.j>

Mar 24 10:12:16 xxx.xxx.xxx.xx launch.sh[1918]: at java.util.logging.LogManager$RootLogger.accessCheckedHandlers(LogManager.java:1667)
Mar 24 10:12:16 xxx.xxx.xxx.xx launch.sh[1918]: at java.util.logging.Logger.getHandlers(Logger.java:1777)
Mar 24 10:12:16 xxx.xxx.xxx.xx launch.sh[1918]: at java.util.logging.Logger.log(Logger.java:735)
Mar 24 10:12:16 xxx.xxx.xxx.xx launch.sh[1918]: at org.jitsi.utils.logging2.LoggerImpl.log(LoggerImpl.java:129)
Mar 24 10:12:16 xxx.xxx.xxx.xx launch.sh[1918]: at org.jitsi.utils.logging2.LoggerImpl.info(LoggerImpl.java:249)
Mar 24 10:12:16 xxx.xxx.xxx.xx launch.sh[1918]: at org.jitsi.jibri.MainKt.handleCommandLineArgs(Main.kt:186)
Mar 24 10:12:16 xxx.xxx.xxx.xx launch.sh[1918]: at org.jitsi.jibri.MainKt.main(Main.kt:53)
Mar 24 10:12:17 xxx.xxx.xxx.xx launch.sh[1918]: SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.
Mar 24 10:12:17 xxx.xxx.xxx.xx launch.sh[1918]: SLF4J: Defaulting to no-operation (NOP) logger implementation
Mar 24 10:12:17 xxx.xxx.xxx.xx launch.sh[1918]: SLF4J: See SLF4J Error Codes for further details.

This is the output of jibri service.

gpatel-fr · March 24, 2022, 10:30am

that’s nice, you have a Jibri connected. However it’s not the end of it. To understand Jibri, you have to know that the Prosody ‘jibri’ user is connecting when Jibri starts. This is what you see. However, the Prosody user ‘recorder’ is connecting only when you are initiating a recording. If the connection configuration (credentials, domain) are wrong for this user, it will not work and fail in a very similar way as you display. In fact, a nice improvement to Jibri wold be to do a preflight test and check that recorder can connect when starting Jibri, and abort. And abort also when the ‘jibri’ user can’t connect. There is no point to continue in these cases.
BTW the error message about the SLF4J class is not a big issue.

emrah · March 24, 2022, 10:33am

Check your prosody version too. If it’s 0.12.x this may cause some issues

dynamite · March 24, 2022, 10:34am

Okay. Much thanks. @emrah @gpatel-fr

How to check whether my prosody user is working as it is supposed to be ?
I did create jibri user and recorder in prosody as
prosodyctl register recorder recorder.xxx.xxx.xxx.xx password
prosodyctl register jibri auth.xxx.xxx.xxx.xx password

dynamite · March 24, 2022, 10:36am

Its
Prosody 0.11.11

gpatel-fr · March 24, 2022, 10:42am

check the prosody log. When jibri starts, you should see amessage like
Authenticated as jibri@auth…
and when you initiate a recording, a message like
Authenticated as recorder@recorder…

dynamite · March 24, 2022, 10:56am

Here is my prosody log:

Mar 24 15:50:59 c2s55e999111970 info Stream encrypted (TLSv1.2 with ECDHE-RSA-AES128-GCM-SHA256)
Mar 24 15:50:59 c2s55e999111970 info Authenticated as jibri@auth.xxx.xxx.xxx.xx
Mar 24 15:51:00 c2s55e999151a30 info Client connected
Mar 24 15:51:00 c2s55e999151a30 info Stream encrypted (TLSv1.2 with ECDHE-RSA-AES128-GCM-SHA256)
Mar 24 15:51:00 c2s55e999151a30 info Authenticated as jvb@auth.xxx.xxx.xxx.xx
Mar 24 15:51:00 auth.xxx.xxx.xxx.xx:limits_exception info Setting stanza size limits for jvb@auth.xxx.xxx.xxx.xx to 10485760
Mar 24 15:51:03 c2s55e998a6fdc0 info Client connected
Mar 24 15:51:03 c2s55e998a6fdc0 info Stream encrypted (TLSv1.2 with ECDHE-RSA-AES128-GCM-SHA256)
Mar 24 15:51:03 c2s55e998a6fdc0 info Authenticated as focus@auth.xxx.xxx.xxx.xx
Mar 24 15:51:03 auth.xxx.xxx.xxx.xx:limits_exception info Setting stanza size limits for focus@auth.xxx.xxx.xxx.xx to 10485760
Mar 24 15:51:10 mod_bosh info New BOSH session, assigned it sid ‘ae804327-7eea-4470-af04-4afbe2a39461’
Mar 24 15:51:11 boshae804327-7eea-4470-af04-4afbe2a39461 info Authenticated as 12_kmht-lk3ywspm@guest.xxx.xxx.xxx.xx
Mar 24 15:51:12 boshae804327-7eea-4470-af04-4afbe2a39461 info BOSH client disconnected: session close
Mar 24 15:51:57 mod_bosh info New BOSH session, assigned it sid ‘3ec979f8-237e-41dd-8eb6-a0a8c9a841f4’
Mar 24 15:51:57 bosh3ec979f8-237e-41dd-8eb6-a0a8c9a841f4 info Authenticated as hi93qdgdxkzogz0n@guest.xxx.xxx.xxx.xx
Mar 24 15:52:08 mod_bosh info New BOSH session, assigned it sid ‘b80b2436-ece5-4a35-a6c0-ce3a43a90c11’
Mar 24 15:52:09 boshb80b2436-ece5-4a35-a6c0-ce3a43a90c11 info Authenticated as abc@xxx.xxx.xxx.xx
Mar 24 15:52:11 boshb80b2436-ece5-4a35-a6c0-ce3a43a90c11 info BOSH client disconnected: session close

I do not get recoder@… jibri@auth… is present.

dynamite · March 24, 2022, 11:24am

I have a bit of confusion regarding this recording module.

My jibri machine with username jibri and hostname efgrs.xxx.xxx.xx
My jitsi machine with username jitsi and hostname abc123.xxx.xxx.xx

xxx.xxx.xx = MYDOMAINNAME

I have only installed jibri on jibri machine and configured hostname. The entire JMS configurations have been practiced in abc123.xxx.xxx.xx.

My prosody configuration for jibri contains recorder.abc123.xxx.xxx.xx.
Similarly, in JMS-config.js file, hiddenDomain is recorder.abc123.xxx.xxx.xx.
I did prosodyctl register recoder recoder.abc123.xxx.xxx.xx password.

My jicofo contains
jibri: {
brewery-jid: “JibriBrewery@internal.auth.abc123.xxx.xxx.xx”
pending-timeout: 90 seconds
}

On jibri machine in jibri.conf i have also added

call-login {
domain = “recorder.abc123.xxx.xxx.xx”
username = “recorder”
password = “password”
}

However, i dont have any recorder.abc123.xxx.xxx.xx subdomain anywhere. So, is it the format to register recorder.abc.123.xxx.xxx.xx for recording or it should be the hostname of my jibri machine (efgrs.xxx.xxx.xx)

gpatel-fr · March 24, 2022, 11:42am

Jibri is dealing with network address in the xmpp-server-hosts variable. Domains are xmpp domains, that is, setup in Prosody. You should have setup a recorder VirtualHost in your prosody config.

dynamite · March 24, 2022, 12:14pm

– internal muc component, meant to enable pools of jibri and jigasi clients
Component “internal.auth.abc123.xxx.xxx.xx” “muc”
modules_enabled = {
“ping”;
}
storage = “memory”
muc_room_cache_size = 1000

VirtualHost “recorder.abc123.xxx.xxx.xx”
modules_enabled = {
“ping”;
}
authentication = “internal_plain”

This is my prosody config.

gpatel-fr · March 24, 2022, 1:41pm

this is not your prosody config. This is an extract, with some part masked. In your VirtualHost for recorder, there is no mistake. Sorry I can’t help you further with the information you provide. Good luck.