Error With Jibri And plug:bsnoop in Dockerfile

BingeWave · April 26, 2023, 2:43pm

I think I am hopefully on my last error getting Jibri to work in a Dockertized container.

I am getting the following error:

2023-04-26 14:30:29.444 INFO: [64] [session_id=ldfzryizqaselffm] JibriSubprocess.launch#42: Starting ffmpeg with command ffmpeg -y -v info -f x11grab -draw_mouse 0 -r 60 -s 1920x1080 -thread_queue_size 13500 -i :0.0+0,0 -f alsa -thread_queue_size 13500 -i plug:bsnoop -acodec aac -strict -2 -ar 44100 -b:a 128k -af aresample=async=1 -c:v libx264 -preset medium -maxrate 2976k -bufsize 5952k -pix_fmt yuv420p -r 60 -crf 25 -g 120 -tune zerolatency -f flv rtmp://some-url ([ffmpeg, -y, -v, info, -f, x11grab, -draw_mouse, 0, -r, 60, -s, 1920x1080, -thread_queue_size, 13500, -i, :0.0+0,0, -f, alsa, -thread_queue_size, 13500, -i, plug:bsnoop, -acodec, aac, -strict, -2, -ar, 44100, -b:a, 128k, -af, aresample=async=1, -c:v, libx264, -preset, medium, -maxrate, 2976k, -bufsize, 5952k, -pix_fmt, yuv420p, -r, 60, -crf, 25, -g, 120, -tune, zerolatency, -f, flv, rtmp:/some-url])
2023-04-26 14:30:34.445 FINE: [19] [session_id=ldfzryizqaselffm] ProcessStatePublisher$startProcessAliveChecks$1.invoke#86: Process ffmpeg hasn't written in 2 seconds, publishing periodic update
2023-04-26 14:30:34.446 INFO: [59] [session_id=ldfzryizqaselffm] FfmpegCapturer.onFfmpegProcessUpdate#125: Ffmpeg quit abruptly.  Last output line: plug:bsnoop: Input/output error
2023-04-26 14:30:34.447 INFO: [59] [session_id=ldfzryizqaselffm] FfmpegCapturer.onFfmpegStateMachineStateChange#134: Ffmpeg capturer transitioning from state Starting up to Error: QuitUnexpectedly SESSION plug:bsnoop: Input/output error
2023-04-26 14:30:34.447 INFO: [59] [session_id=ldfzryizqaselffm] StatefulJibriService.onServiceStateChange#39: Streaming service transitioning from state Starting up to Error: QuitUnexpectedly SESSION plug:bsnoop: Input/output error

So the error is the N plug:bsnoop: Input/output error, which I am assuming is the sound card.

On the instance running the docker container, when I run modinfo snd-aloop, I get the following:

filename:       /lib/modules/5.4.0-1103-gcp/kernel/sound/drivers/snd-aloop.ko
license:        GPL
description:    A loopback soundcard
author:         Jaroslav Kysela <perex@perex.cz>
srcversion:     44724425ED81756955CCD37
depends:        snd-pcm,snd
retpoline:      Y
intree:         Y
name:           snd_aloop
vermagic:       5.4.0-1103-gcp SMP mod_unload modversions 
signat:         PKCS#7
signer:         
sig_key:        
sig_hashalgo:   md4
parm:           index:Index value for loopback soundcard. (array of int)
parm:           id:ID string for loopback soundcard. (array of charp)
parm:           enable:Enable this loopback soundcard. (array of bool)
parm:           pcm_substreams:PCM substreams # (1-8) for loopback driver. (array of int)
parm:           pcm_notify:Break capture when PCM format/rate/channels changes. (array of int)

And inside the docker container, when I run modinfo snd-aloop, I also get the following:

filename:       /lib/modules/5.4.0-1103-gcp/kernel/sound/drivers/snd-aloop.ko
license:        GPL
description:    A loopback soundcard
author:         Jaroslav Kysela <perex@perex.cz>
srcversion:     44724425ED81756955CCD37
depends:        snd-pcm,snd
retpoline:      Y
intree:         Y
name:           snd_aloop
vermagic:       5.4.0-1103-gcp SMP mod_unload modversions 
signat:         PKCS#7
signer:         
sig_key:        
sig_hashalgo:   md4
parm:           index:Index value for loopback soundcard. (array of int)
parm:           id:ID string for loopback soundcard. (array of charp)
parm:           enable:Enable this loopback soundcard. (array of bool)
parm:           pcm_substreams:PCM substreams # (1-8) for loopback driver. (array of int)
parm:           pcm_notify:Break capture when PCM format/rate/channels changes. (array of int)

And running lsmod | grep snd_aloop gives me the following:

snd_aloop              24576  0
snd_pcm               106496  1 snd_aloop
snd                    86016  3 snd_timer,snd_aloop,snd_pcm

And the docker container is started like this:

sudo docker run -d -p 80:80 -p 443:443 -p 4443:4443 -p 5222:5222 -p 5347:5347 -p 10000:10000/udp --shm-size=8gb -v /dev/shm:/dev/shm --rm --runtime=nvidia --gpus all -v /dev/snd:/dev/snd --privileged my-custom-jibri-image:latest

My Java Version:

openjdk version "11.0.18" 2023-01-17
OpenJDK Runtime Environment (build 11.0.18+10-post-Ubuntu-0ubuntu118.04.1)
OpenJDK 64-Bit Server VM (build 11.0.18+10-post-Ubuntu-0ubuntu118.04.1, mixed mode, sharing)

My arecord -l

**** List of CAPTURE Hardware Devices ****
card 0: Loopback [Loopback], device 0: Loopback PCM [Loopback PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 0: Loopback [Loopback], device 1: Loopback PCM [Loopback PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7

And when I run cat /home/jibri/.asoundrc

pcm.amix {
  type dmix
  ipc_key 219345
  slave.pcm "hw:Loopback,0,0"
}

pcm.asnoop {
  type dsnoop
  ipc_key 219346
  slave.pcm "hw:Loopback_1,1,0"
}

pcm.aduplex {
  type asym
  playback.pcm "amix"
  capture.pcm "asnoop"
}

pcm.bmix {
  type dmix
  ipc_key 219347
  slave.pcm "hw:Loopback_1,0,0"
}

pcm.bsnoop {
  type dsnoop
  ipc_key 219348
  slave.pcm "hw:Loopback,1,0"
}

pcm.bduplex {
  type asym
  playback.pcm "bmix"
  capture.pcm "bsnoop"
}

pcm.pjsua {
  type plug
  slave.pcm "bduplex"
}

pcm.!default {
  type plug
  slave.pcm "aduplex"
}

So what could be causing the sound card not to be work? (if the error message is indeed a sound card issue).

BingeWave · April 26, 2023, 5:37pm

An update, this is not a fix but in the Jibri config when a change bsnoop to dsnoop, it records but without sound:

/etc/jitsi/jibri/jibri.conf

...
ffmpeg {
    resolution = "1920x1080"
    // The audio source that will be used to capture audio on Linux
    audio-source = "alsa"
    // The audio device that will be used to capture audio on Linux
    //ffmpeg throw input error
    //audio-device = "plug:bsnoop"
    //ffmpeg works but is muted
    audio-device = "plug:dsnoop"
    // -r
    framerate = "60"
    //max
    streamingMaxBitrate = "13000"
    streaming-max-bitrate = "13000"
    //-preset
    videoEncodePreset = "medium"
    video-encode-preset = "medium"
    //queezie
    queueSize="13000"
    queue-size="13000"
  }
...

What are the other options/sound devices that I can use to record the sound?

BingeWave · April 26, 2023, 9:14pm

I figured it out, disregard the dsnoop, use the bsnoop. The final solutio for me was to have HOME=/home/jibri right before the Jibri launch script is executed.

HOME=/home/jibri
/opt/jitsi/jibri/launch.sh

emrah · April 27, 2023, 7:03am

This means that the launch script is run by another user than jibri in Docker setup and this user has no correct .asoundrc file.

BingeWave · April 27, 2023, 4:50pm

Correct, the default user, which in Docker I think, does not have a ~/.asoundrc in their home directory.

Also two other notes I am not sure where to leave to I just give some feedback for them here.

ffmpeg requirement

Right now apt-get install jibri is requiring the ffmpeg be installed via the package manager. The problem is that the default ffmpeg on ubuntu is version, and the highest version it can be updated to via package management is version 4. I had compiled ffmpeg version 5 but the package manager kept over-writing it an I couldn’t use mins apt-get install ffmpeg- to leave ffmpeg out the installation. TDLR, ffmpeg requirement should be optional installation, not required.

Pulse Audio

On the docker installation for Jibri here: GitHub - jitsi/docker-jitsi-meet: Jitsi Meet on Docker

For their jibri installation, they don’t use bsnoop/snd_loop but they use pulse. I think longer term, Jibri should make its way over to that as a whole and standardize that approach to sound because having to stick to Ubuntu 16/18 for bsnoop is outdated.

emrah · April 28, 2023, 6:21am

I checked Docker images for stable-8319. The Java process is run by jibri and this user has .asoundrc in its home directory. I don’t know why this is not the same in your case.

Since jibri cannot be run without ffmpeg, this package should be in the dependency list. Maybe you can mount your compiled ffmpeg into /usr/local/bin. In this case it becomes the default one.