Performance mode for musicians, comedians, etc

Hi all,

As many countries are currently in lockdown there’s been a dramatic increase in performers making use of video conferencing software to host shows. Events with multiple performers seem to like video conferencing systems for this as it makes it trivial to switch between artists throughout the night; however, the technical needs of performances are very different to those of conversations. As such, I’d like to propose a “performance mode” option which makes a number of automatic changes to the stream settings, such as:

  • Mute all participants besides the currently designated performer
  • Possibly even stop all video from other participants as well to provide maximum bandwidth for the performer
  • Disable all audio pre-processing (e.g. echo cancellation, etc.); although perhaps some optional performance specific effects could be introduced, such as reverb for musicians
  • Increase the audio bitrate (even at the detriment to video quality if necessary, especially for musical performances the audio quality is of paramount importance)
  • Increase the buffer size (latency is no longer a concern, but the smoothness of playback is)

I’d envisage this as being an option that can be switched into and out of during a call, allowing the MC to hold conversations with the artists, audience, etc. between performances as normal, and only switching into performance mode when a performer takes to the stage.

Maybe a good GSoC project for someone?

5 Likes

Hi Mike, I’ll just add my 5cents to this wonderful idea.

  • asymmetrical bandwidth: the performer broadcasts in HD and the general public can attend in SD or low quality, to save bandwidth
  • an integrated music player for the performer. Think for example to a fitness instructor. He plays music then teaches a lesson
  • or (easier) an integrated AUDIO MIXER, to collect audio from the various system sound sources/apps and mix them (in windows there is a 3rd party app called SoundMeeter Potato which does the thing)

“Performance mode” should be a button/option on the performer window, easily accessible.

1 Like

I am a music teacher and I need the best audio quality to teach my classes.

Zoom now has the option to select “original audio”. Something similar might be fine in Jitsi. Discord even allows you to select the audio bitrate, but only up to 96kpps. With a good connection it could be increased up to 192kbps, and maybe reduce the video quality.

Allowing the user to have control over that kind of configuration would be a great bet for professional quality for Jitsi.

1 Like

Here is what you can try: append #config.disableAP=true to your meeting URL. Example: https://meet.jit.si/ThisWillDisableAllAudioProccessing#config.disableAP=true

Note that this will also disable echo cancellation, so you’ll need to use a headset.

2 Likes

Great!! I’ve tried it and it works, although I can’t hear my own sound in the headphones.
However I was now on my old laptop which doesn’t have a good mic and I can’t quite appreciate the quality change.
You know what bitrate I get for doing that?

Local audio is always muted, there is no setting for it.

Opus uses variable bitrate and will adapt based on network conditions. With that setting you are disabling the AEC, NS and AGC filtering.

1 Like

And you know the maximum bitrate Opus allows? My internet is pretty good, I have 600 Mb linear fiber optics.
A bypass, allowing the computer audio or with a minimum compression, for example mp3 at 256kbps and 44hz would be best.

Yesterday I tried out this configuration more thoroughly. It sounds much better, but it’s still not enough. I suspect that Opus is probably limited to 50kbps or similar.

Sorry, I don’t have more ticks up my sleeve at the moment.

1 Like

Ok. Thank you for your help @saghul

There is a discussion on Stackoverflow that suggests that adding maxaveragebitrate to the SDP could improve Opus audioquality: https://stackoverflow.com/a/58898298

As far as I can see, this would require to build Jicofo by yourself and add the option in src/main/java/org/jitsi/jicofo/util/JingleOfferFactory.java

I’d first try to couple this to the stereo option for simplicity, e.g., change line 671ff to

        if (stereo)
        {
            // fmtp: 111 stereo=1
            addParameterExtension(opus, "stereo", "1");
            addParameterExtension(opus, "maxaveragebitrate", "310000");
        }

However, I am not sure if this will actuall force opus to use CELT Fullbandwidth encoding and as I currently have no test environment, I can’t check if that Stackoverflow post is indeed correct.

1 Like

Thank you, @plokta . Maybe it could be a solution, but I think it’s complicated for me.
In addition, I am a Linux user (PcLinuxOS specifically) and use Jitsi in the Chrome browser.

Finally found the time to test the maxaveragebitrate setting mentioned above and can confirm that it does indeed increase the audio quality as can be observed in chrome://webrtc-internals. However, while it does work with Chromium based browsers it has apparently no impact on Firefox and I have no possibility to test with Safari.
It is possible to make this available as an option in config.js but this requires additional changes in lib-jitsi-meet and jitsi-meet as I far as I understand the code. I’ll see if I can put together a PR for this over the weekend.

1 Like

We are very interested in this and happy to hear it is getting some attention. We are also happy to get involved in the development.

It would be very valuable to be able to have config.js settings to boost, and even better, prioritize, audio quality vs video when bandwidth is limited.

Additionally, the ability to toggle “High Definition Sound” on/off (in addition to disabling NS/HPF/AEC, kind of like “Original Sound” on Zoom) in the meeting, would be great.

Our team would be happy to work on it and submit a pull request, or at the very least help with testing.
If anyone has any pointers or already thought of the correct way to do this. Please let us know.

That would be great!!
I’m a musician, if I can help by doing some tests here I am :slight_smile:

I opened a draft PR to start a discussion on how to achieve better audio quality within a Jitsi conference:

3 Likes

Thanks for your work @plokta! FTR, all the necessary PRs for this have landed. Not sure if it got deployed to meet.jit.si yet though.

1 Like

Is there any new news regarding the development of new advances with respect to sound?

Plotka patch works great but it hasn’t made it to a stable release yet. We can help you deploy it on your own servers.

Note that it will not work on the Native App (iOS & Android) has everything is hard coded to Mono / Voice. (It doesn’t work on Safari either)

We are working on a customization of WebRTC framework to support stereo.

Feel free to contact me if anyone is interested in this.

Hi, I am really interested in this topic. We run a swing band with remote musicians using low latency audio via JamKazam and video through Jitsi. We have been doing interactive gigs copying both into Zoom, but with attendant loss of audio quality.
The advantage of Zoom is so many folks know it - and we connect many elderly, isolated folks. From a performers point of view, being interactive rather than just broadcast is good.
The re-connection into the Zoom mode is weighty on CPU.
If Jitsi could work asymmetrically with the band going out in HD, with good sound, and the audience coming back in low definition/simple sound - that would be magic!
Stereo makes a big difference - pity native app doesnt support.
Maybe I need to find how to deploy a server…