Tip: Pulseaudio support for Jibri

emrah · June 14, 2020, 9:03pm

Pulseaudio support for Jibri

This is a simple guide explaining how to add pulseaudio support to Jibri. While preparing the guide, I greatly benefited from @kpeiruza’s work. It’s tested on Debian Buster and LXC (Linux container).

“Pulseaudio support” means that you no longer need the snd_aloop module.

Packages
Install the following packages on the Jibri host

apt-get install dbus-x11 pulseaudio rtkit

rtkit group
Add the Jibri user to the RealtimeKit group.

adduser jibri rtkit

Pulseaudio autostart
It’s not recommended running the Pulseaudio daemon as a system-wide instance. Run it per-user. I think the best way to do this is to add a ExecStartPost line to the jibri-icewm service. But make this modification with override.conf file to keep the original service file untouched.

mkdir /etc/systemd/system/jibri-icewm.service.d
vim /etc/systemd/system/jibri-icewm.service.d/override.conf

/etc/systemd/system/jibri-icewm.service.d/override.conf

[Service]
ExecStartPost=
ExecStartPost=/usr/bin/start-pulseaudio-x11

Reload the systemd daemons

systemctl daemon-reload

Jibri’s pulseaudio config
To customize the pulseaudio config for the jibri user

mkdir -p /home/jibri/.config/pulse
vim /home/jibri/.config/pulse/client.conf
vim /home/jibri/.config/pulse/daemon.conf
chown jibri:jibri /home/jibri/.config -R

/home/jibri/.config/pulse/client.conf

autospawn = yes

/home/jibri/.config/pulse/daemon.conf

daemonize = yes
high-priority = no
realtime-scheduling = yes
realtime-priority = 5
exit-idle-time = -1
flat-volumes = no

customizing ffmpeg

/usr/local/bin/ffmpeg

#!/bin/bash

ARGS=$@
ARGS=$(echo $ARGS | sed 's/-f alsa/-f pulse/')
ARGS=$(echo $ARGS | sed 's/-i plug:\S*/-i default/')

exec /usr/bin/ffmpeg $ARGS

Execute permission to the fake ffmpeg

chmod 755 /usr/local/bin/ffmpeg

restart the Jibri services

systemctl stop jibri-xorg
systemctl start jibri

Container devices
If Jibri is on the container, it’s needed to allow the container to access the sound devices. This is done by adding the following line to the container config:

lxc.cgroup.devices.allow = c 116:* rwm

kpeiruza · June 14, 2020, 10:15pm

Thank you very much for your guide @emrah !!!

PS: on Docker // Kubernetes, you don’t need mapping /dev/snd. AFAIK, this can work without any sound device

emrah · June 15, 2020, 12:02pm

Testing on LXC without mapping, same result…
I’m editing the post

tu-nv · July 14, 2020, 1:56am

Great tutorial.
The same approach of ffmpeg can be applied to add --no-sanbox mode for chrome. This remove the need of CAP_SYS_ADMIN, so now we can go full swarm in docker!

swathikrishna_guru · August 3, 2020, 7:45am

Do we have a procedure for building it in docker? @kpeiruza

kpeiruza · August 3, 2020, 4:25pm

I’ve a quick & dirt docker under my personal github: https://github.com/kpeiruza/jitsi-images/tree/master/jibri

Regards!

Kenneth

swathikrishna_guru · August 3, 2020, 5:16pm

It seems to be little old, I have managed to build new jar with latest repo and built. All looks ok.

emrah · August 13, 2020, 6:59am

For the current jibri this line should be like

ARGS=$(echo $ARGS | sed 's/-i plug:\S*/-i default/')

tu-nv · August 24, 2020, 9:09am

I face the issue of no audio recorded with new jibri. Already use ARGS=$(echo $ARGS | sed ‘s/-i plug:\S*/-i default/’). Does anyone has any idea?

emrah · August 24, 2020, 9:19am

Do you use alsa or pulse?

tu-nv · August 24, 2020, 9:39am

I use pulse and docker. The only difference is new jibri is used instead of old jibri, so the sed command should replace bsloop instead of cloop. I thought new jibri update cause the error, but if it still works for others, there must be some wrong config in my setup…

emrah · August 24, 2020, 9:46am

Could you add a log command to your fake ffmpeg and paste the $ARGS result?
A line just before the exec line

echo $ARGS >>/tmp/ffmpeg.log

tu-nv · August 24, 2020, 9:59am

The ARGS is:

-y -v info -f x11grab -draw_mouse 0 -r 30 -s 1280x720 -thread_queue_size 4096 -i :0.0+0,0 -f pulse -thread_queue_size 4096 -i default -acodec aac -strict -2 -ar 44100 -b:a 128k -af aresample=async=1 -c:v libx264 -preset veryfast -profile:v main -level 3.1 -pix_fmt yuv420p -r 30 -crf 25 -g 60 -tune zerolatency -f mp4 /config/recordings/lofycizgclxmhlpr/meet1_2020-08-24-07-43-36.mp4

I also remvoe the part b:a 128k -af aresample=async=1 but still no audio

emrah · August 24, 2020, 11:07am

You can add a more log command to be sure pulseaudio is running at the record time

echo $ARGS >>/tmp/ffmpeg.log
ps auxww | grep pulse | grep -v grep >>/tmp/ffmpeg.log

kpeiruza · August 26, 2020, 9:29am

You’ve just built it, right? Looks like you’re using a very recent version of Chromedriver. Not sure if that could be the problem.

Besides that, are you passing /dev/shm into the container?
Was Jibri working before trying to use this Jibri+Pulse Docker?

Regards,

Kenneth

tu-nv · September 10, 2020, 12:45am

Oh sorry for not confirm. its was a chrome configuration mistake. I wrapped chrome like this:

exec /usr/bin/google-chrome-bin --no-sandbox $@

But it does not work correctly. It should be somewhat like this:

ARGS=$@
ARGS=$(echo $ARGS | sed 's/--kiosk/--kiosk --no-sandbox/')
exec /usr/bin/google-chrome-bin $ARGS

kpeiruza · September 10, 2020, 11:05am

Good to know, thanks!

omazin · December 2, 2020, 3:35pm

Hi @kpeiruza, trying to use your jibri + pulseaudio image in my jibri.yml config as follows:

version: '3.5'

services:
    jibri:
        image: kpeiruza/jibri
        restart: ${RESTART_POLICY}
        volumes:
            - ${CONFIG}/jibri:/config:Z
            - /dev/shm:/dev/shm
        cap_add:
            - SYS_ADMIN
            - NET_BIND_SERVICE
        devices:
            - /dev/snd:/dev/snd
        environment:
            - XMPP_AUTH_DOMAIN
            - XMPP_INTERNAL_MUC_DOMAIN
            - XMPP_RECORDER_DOMAIN
            - XMPP_SERVER
            - XMPP_DOMAIN
            - JIBRI_XMPP_USER
            - JIBRI_XMPP_PASSWORD
            - JIBRI_BREWERY_MUC
            - JIBRI_RECORDER_USER
            - JIBRI_RECORDER_PASSWORD
            - JIBRI_RECORDING_DIR
            - JIBRI_FINALIZE_RECORDING_SCRIPT_PATH
            - JIBRI_STRIP_DOMAIN_JID
            - JIBRI_LOGS_DIR
            - DISPLAY=:0
            - TZ
        depends_on:
            - jicofo
        networks:
            meet.jitsi:
networks:
   meet.jitsi:
      external:
         name: custom_jitsi_network

But recording is failing. With jitsi/jibri:latest image it works fine. Could you advice on how to find & fix the problem?

metadata · December 16, 2020, 12:37pm

Is it working with docker with latest jibri config?

EDIT
i tried creating a docker image using @kpeiruza repo . when I started it I’m facing some issue.
jibri logs

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-set-timezone: executing...
[cont-init.d] 01-set-timezone: exited 0.
[cont-init.d] 10-config: executing...
unix:path=/var/run/dbus/system_bus_socket,guid=92f7a23918abb2dd896814275fda01c9
W: [pulseaudio] main.c: Running in system mode, but --disallow-exit not set.
W: [pulseaudio] main.c: Running in system mode, but --disallow-module-loading not set.
N: [pulseaudio] main.c: Running in system mode, forcibly disabling SHM mode.
Home directory not accessible: Permission denied
No PulseAudio daemon running, or not running as session daemon.
Home directory not accessible: Permission denied
No PulseAudio daemon running, or not running as session daemon.
Home directory not accessible: Permission denied
No PulseAudio daemon running, or not running as session daemon.
W: [pulseaudio] core-util.c: Failed to open configuration file '/home/jibri/.config/pulse//daemon.conf': Not a directory
W: [pulseaudio] daemon-conf.c: Failed to open configuration file: Not a directory
Failed to create secure directory (/home/jibri/.config/pulse): File exists
No PulseAudio daemon running, or not running as session daemon.
Failed to create secure directory (/home/jibri/.config/pulse): File exists
No PulseAudio daemon running, or not running as session daemon.
Failed to create secure directory (/home/jibri/.config/pulse): File exists
No PulseAudio daemon running, or not running as session daemon.
[cont-init.d] 10-config: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.

X.Org X Server 1.20.4
X Protocol Version 11, Revision 0
Build Operating System: Linux 4.19.0-12-amd64 x86_64 Debian
Current Operating System: Linux 061c97b60496 4.15.0-111-generic #112-Ubuntu SMP Thu Jul 9 20:32:34 UTC 2020 x86_64
Kernel command line: BOOT_IMAGE=/vmlinuz-4.15.0-111-generic root=UUID=875fbe2a-4103-4485-9dc8-d0be8aad2460 ro
Build Date: 01 December 2020  05:59:57PM
xorg-server 2:1.20.4-1+deb10u2 (https://www.debian.org/support)
Current version of pixman: 0.36.0
        Before reporting problems, check http://wiki.x.org
        to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(++) Log file: "/tmp/xorg.log", Time: Wed Dec 16 18:17:06 2020
(++) Using config file: "/etc/jitsi/jibri/xorg-video-dummy.conf"
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2020-12-16 18:17:07.770 WARNING: [46] org.jitsi.xmpp.mucclient.MucClient.log() Disabling certificate verification!

kpeiruza · December 23, 2020, 11:19am

Hi there,

The repo at github isn’t updated. Feel free to try kpeiruza/jibri:pulse

I’ll upgrade the repo whenever I’ve some spare time … There’s some changes on how to launch jibri in that image, and it has also been patched to allow configuration of XMPP port (which was internally supported by jibri but not on its configuration file).

That image has also a script that pushes videos to Nextcloud, these are the variables:

    - name: XMPP_AUTH_DOMAIN
      value: auth.your.domain.com
    - name: XMPP_INTERNAL_MUC_DOMAIN
      value: internal-muc.your.domain.com
    - name: XMPP_RECORDER_DOMAIN
      value: recorder.your.domain.com
    - name: XMPP_SERVER
      value: your.domain.com
    - name: XMPP_DOMAIN
      value: your.domain.com
    - name: XMPP_PORT
      value: "5222"
    - name: JIBRI_XMPP_USER
      value: jibri
    - name: JIBRI_XMPP_PASSWORD
      value: ANICEPASSWORD
    - name: JIBRI_BREWERY_MUC
      value: jibribrewery
    - name: JIBRI_RECORDER_USER
      value: recorder
    - name: JIBRI_RECORDER_PASSWORD
      value: ANICEPASSWORD
    - name: JIBRI_RECORDING_DIR
      value: /data
    - name: JIBRI_FINALIZE_RECORDING_SCRIPT_PATH
      value: /finalize.sh
    - name: JIBRI_STRIP_DOMAIN_JID
      value: muc
    - name: JIBRI_LOGS_DIR
      value: /logs
    - name: DISPLAY
      value: :0
    - name: TZ
      value: Europe/Madrid
    - name: NC_SERVER
      value: https://MY.NEXTCLOUD.FLOSS.CAT
    - name: NC_SHARE
      value: SHAREID
    - name: NC_PASSWORD
      value: SHAREPASSWORD
    - name: NC_ADMINS_EMAIL
      value: kenneth@floss.cat

Regards!

Kenneth