Time to move Jibri from ALSA to Pulseaudio?

Hi there,

I’ve been using a modified Jibri with Pulseaudio for 5 weeks. So far, it has recorded and streamed more than 300 meetings (avg 2h) without issues. A 10th even exceeded 4h.

Why is migrating from ALSA to Pulseaudio a big deal? because most Kubernetes providers don’t allow you to load kernel modules unless you select “standard OS based nodes”. In my particular case, one of our providers doesn’t allow to select the base system, and they’ve a custom kernel without that module.

So, we modified Jibri as stated on another post, and we were able to get several pods running on the same server without needing to reconfigure anything at all. It just works.

I only needed to install 3 additional packages to the Jibri Docker Image and a Script to run Pulseaudio as Jibri user, but the Jibri.jar on it was compiled by myself, so it’s not as neat as I’d wish in order to share it with 3rd parties.

Any plans on making Pulseaudio officialy available for Jibri? if so, I’ll be glad to share the Docker image changes so we can all use official jitsi/jibri images :slight_smile:

Thanks!

4 Likes

Thanks for sharing your findings !

I encourage you to submit a Feature Request on the issue tracker, describing how you think this could be best implemented. Perhaps linking to your post is enough ?

I don’t use Kubernetes but this would certainly help others getting started with Jitsi in such context, specially on medium/large deployments. Citing your example and numbers in Catalonia can help understand the scope.

I believe the " other post " you refer to with details of your changes is this one :

1 Like

i agree @kpeiruza please submit a PR

2 Likes

Hi @masteryoda and @MagicFab, keeping backwards compatibility with jibri-alsa is a bit messy, I need to modify several Kotlin files and I’ve never done kotlin before (DevOps here, not coder).

I see there’s a file with several hardcoded values:
src/main/kotlin/org/jitsi/jibri/capture/ffmpeg/FfmpegCapturer.kt

Probably the 1st step would be to make these the default values, and to be able to change them from the Config file.

Doing so will also open the possibility to capture 1080p instead of 720, or tweak other ffmpeg parameters to increase/decrease video quality.

Could you please share a modified branch that is able to load one of there values from the config file & set a default value if no value has been provided?

Just make that for me and I’ll be able to add all the other parameters + if/else the right audio capture device.

Thanks!

sadly i’m in the same boat as yours. i hope @damencho can guide us here

We were using pulseaudio in the beginning and after few issues where the audio was broken or clicking … we switched to alsa and since then we did not have problems with the audio.

Hi @damencho,

That is why I pointed out our current experience with pulseaudio and the extensive testing we have performed. We’re using 4 core instances to stream meetings from 7 to 40 participants (mostly in tile view and with all open cameras). At the present moment, more than 400 tests and meetings without issues, and we’ll reach 500 before 1st of June…

Maybe it’s time to, at least, allow to configure both systems whilst keeping ALSA as the default.

What I’d need in order to submit a PR is just a sample modification in which one of the parameters present in src/main/kotlin/org/jitsi/jibri/capture/ffmpeg/FfmpegCapturer.kt can be selected from the config file (new to Kotlin and AFAIK, I needed to modify 3-4 files in order to get that done).

As I’m not 100% sure on what might I change, I’d ask “you” to make that tiny change, and I’ll take care of making all the other ffmpeg parameters adjustable from within the config file whilst keeping current values as default.

Could you please do so or ask someone else if that could be done so we can offer a configurable way of using each or other as well as switching quality & resolution parameters on ffmpeg’s capture?

Thank you very much :slight_smile:

1 Like

@bbaldino can you help here?

I’m not familiar with the reasoning for pulse vs alsa, I think this came from @saghul and @Aaron_K_van_Meerten

We had serious problems in the early days of jibri with clicks being introduced to the audio stream by pulseaudio, as well as sometimes pure silence. In addition, the A/V sync was often extremely off (like 3-5 seconds). We never had any of tthese issues when switching to alsa, so we switched and haven’t looked back.

I’m open to the possibility that pulse is now more stable and less likely to introduce such artifacts. The main issue for us was that this was inconsistent behavior. Some days no VMs would have the issue, other weeks every single test we did had the problem, across all VMs in multiple AWS regions. So I think like many other pieces, this could/should be done as an option, rather than a complete replacement. That way others who wish to run on pulse can do so, while continuing to support ALSA as well.

1 Like

Right @Aaron_K_van_Meerten !

Please make just one of the settings of src/main/kotlin/org/jitsi/jibri/capture/ffmpeg/FfmpegCapturer.kt

configurable throug config.js and I’ll do the rest of the work so the current values will be the default but could be overrided in the config :slight_smile:

As stated before, I’m not a coder and I don’t use any IDE, the closest thing I’ve is a vim. If you lead me in that 1st step, I’ll be glad to finish it, but, right now I stopped modifying files blindly (4 so far) because I don’t have as much as I’d wish to do it the trial&error way.

Thank you very much!

That provider kpeiruza talks, I bet is AWS. I’m having same issues. Looking into just building alsa module on non stock ubuntu kernel and it seems to be royal pain at very least. Most things are outdated there, it is out of tree module (not maintained by kernel devs, but created by alsa project, seems all stuff related to that is quite old). No wonder at very least some OS vendors do not include it. IMO jibri should really support alternatives. As for clicking stuff it was possibly load/latency related. For example we had issue with live stream clicking, on more weaker server that has 4 cores. But I’m running docker jitsi meet stack all on same server, and it seems this is just from jvb interrupting cores to forward packets. When we had much bigger event on bigger server with more cores we had no such problem (despite load being similar in both cases), likely just because more cores means jvb is much less likely to execute on core which is handling jibri/sound. So possibly some sound system, kernel, server power tuning could be used to sort problems with running on pulse if any. As option it should be provided, considering that in some cases using alsa is super painful. On AWS I install manually docker version, but before that I have to install stock ubuntu kernel there and make instance boot into it which again is not super fun at all. And seems to be like nonstarter for scaling it with k8s due to them requiring more specific kernel.

@morphies, it wasn’t AWS, so, it’s a quite common issue as you can see… We’ve been using 4 node cores all time, as most conferences host 20-30 participants and Chrome picks 2-2.5 cores alone.

But do you run all stack on one server (including jvb, prosody and web)?

Nope, 1 Prosody + web + jicofo, 10 videobridges and from 1 to 20 jibris in Kubernetes with HPA :slight_smile:

PS: we’ve almost +10 environments with everything in one server.

Main reason is it can run in AWS ECS for example…or other Cloud platforms where user can’t expose /dev/snd

@kpeiruza, we are running the same thing. Our kubernetes cluster is in AWS (EKS). Did you have to install alsa (or pulse) in the host? Or were you able to find another solution?

Thanks for your input!

Hi @acruz,

We’ve a docker image with pulseaudio. No need to setup anything on the nodes.

Regards,

Kenneth

Would you mind sharing your configuration?

Hi @acruz,

Try this: Tip: Pulseaudio support for Jibri

Regards!

1 Like