URGENT - Low image quality after update

@damencho, if you can jump-in here, I would much appreciate it!

I updated Jitsi this weekend to profit from the latest features, such as the Lobby function.
I repopulated all the files that were crushed by the update and made sure my quality was constrained as follow:

Target = 1080
Ideal = 1080
Max = 1080
Min = 240

P2P is disabled on our instance for many reasons.

The stream won’t reach target, and that is true for all users. It maxes-out at 960x540 for everyone.
It is unbarable to watch on the user’s side too…
The main PC, that we use in a studio to give remote training has a gigabit Internet Link and has always provided top quality for the users.

Is there a new setting, probably not located in the config.js, that could influence video quality?

We are Live, giving training to customers in a $hitty looking video stream… That is unacceptable for us! Please help me ASAP !

By the way, screen sharing looks perfectly good… It is only video quality that is low

Hi @Normand_Nadon, multiple changes went in the last release that would influence the video quality. To be able to identify which one causes this in your deployment, can you please provide us with some more information ?
Will you be able to verify if your main PC is still sending 1080p ? You should be able to see it in the connection stats on the video preview.
Once you are able to confirm that it is not the send side, we can check if the other participants in the call are requesting 1080p. Do you see the following log on the receiving clients ? Can you pls check what resolution is requested ?
[modules/RTC/BridgeChannel.js] <l.sendReceiverVideoConstraintMessage>: sending a ReceiverVideoConstraint message with a maxFrameHeight of

@jallamsetty
I will try your test tonight as there are actually 2 training sessions running and I don’t want to mess on the server while customaers are connected.

I can garantee that the main PC is sending 1080p, and my other computer is capable of 1440p and still gets downscaled to 960x540… It would seem as if jitsi constrains all streams to this lower resolution.
Everyone has set it’s image quality dial to HD.

One thing I realized is that before the update, Wired connection were always tagged as “excellent” and now they are only tagged as “Good”. Also, the feed stats seems to be lacking information about the downstream capability… Can’t tell if it was like that before as I did not have to care about that!
image

To put our setup in perspective, this server has been running since early june and has not been updated since, until this weekend. I also upgraded Prosody to V0.11 to be able to activate the Lobby function.

Can you elaborate on what could cause quality issue?
Also, Can I lock the quality to higher standards and let people lower it if needed?

BTW, I have just checked on meet.jit.si and all my streams are maxed-out to 720p, which I believe to be the maximum that was set on that server…

Yes the recents changes made have restricted the receive video quality to 720p which also affects the send side resolution as well. We will have to make the receive video resolution configurable through config.js options, until then I would suggest you to revert to the older version of jitsi-meet that was working well for you.

Honestly, I had to re-read your answer 10 times to make sure I read correctly…
I am sorry to say that because I know Jitsi is a community based software and people put their hearts and soul in that, but I am pissed-out to a maximum!!!

– BEGIN RANT
Why the hell is this version released on the main branch if it is broken? I am no on the development/unstable branch!
Users should not loose basic features following an update… this is nonsense!
It was one thing that the documentation was severely lacking when I first tried the jitsi stack in march… This I can live with… And BTW, congratulations, it has improved greatly.
It is another thing that upgrading from repos completely breaks your instance, I learned how to go around that.
But now, sending half-baked software in the main repos is unacceptable!
– END RANT

Now, for the solution part:
How am I supposed to roll-back from this update, do you have documentation on that?

Otherwise, How can I force 1080p stream regardless of network capability? All our customers can handle it, I just want to finish this week with a working instance with appreciable video quality and completely flush it on friday night and start-over…

Regards

  • (exhausted) Normand

Ok, I managed to find a working solution… in the end, the option that was not working was “simulcast”

I disabled it in the /etc/jitsi/meet/yourdomain.com-config.js

  • notice the quality constraints we are using too, this is how we manage to have nice video for our particular use case (remote training, using OBS studio and video compositing). one could max it out at 720p without too much of a loss in quality.

The simulcast option was never disabled in the past… I don’t know what it is that made it work poorly after this upgrade, and to be honest, I don’t really care to find out today… I am tired and at least, tomorrow’s sales pitch from my boss will not look like something pulled from a 1999 video camera!

1 Like

I just feel like this simulcast thingy has been more problem than not. Perhaps it’s simulcast itself or just Jitsi’s implementation of it, but it’s almost always resulted in less-than-desirable video quality for me, most of the time. Even with my constraints set to use 720p and with adequate bandwidth, I’ve seen 180p resolution at times. Very disturbing and quite frankly, frustrating.

But this is being provided free of charge, so I don’t want to be that entitled brat. :confused:

I can’t say… I looked at my previous installs and this option was always on and never gave me issues.
Maybe it was just not working at all in the end! And now that it does, it does it poorly… Can’t tell.

Jitsi is a work of art, achieved by dedicated and talented people… I just think it lacks direction.
I don’t think that “new features” are the thing that users need the most at the moment… Good documentation and a solid rework on the packaging and files location (so that updating can work with the apt upgrade command without breaking your instance)…Maybe a front-end configurator would be great too.

That being said, I fixed the issue this way, and our server is solid enough that it does not care about simulcast and can transcode on the fly without issue. Hope my experience can help someone else too.

Simulcast has always been working, we recently made it harder to request higher resolutions like 1080p in order to save cpu/bandwidth on the client machines. So when you are encoding 1080p and the receiver is requesting only 720p, the encoding client will send only the lower 2 stream, i.e., 270p and 540p. When simulcast is disabled, there is only one stream that doesn’t get downscaled until receiver requests 540p.
If your customers have the network capabilities to handle 1080p streams then you can disable Simulcast until we can provide a config.js option to request resolutions higher than 720p.

@jallamsetty maybe it’s Simulcast, or something else, but something in the build-up is not calculating the available bandwidth accurately. Simulcast on its own, from what I’ve read about the concept, should be a plus, not a negative. But from my personal experience setting up Jitsi (and I’ve only been up for a few weeks, so you can take my input with a grain of salt), the assessment of available bandwidth has been widely off.

My Jitsi application is on a dedicated server sitting on Ubuntu 20.04. It’s the only application on that server, nothing else competes with it. I have a 300MBS up/down bandwidth that’s exclusively available to Jitsi when I run it. The other clients connect on bandwidths averaging 100MBS on their end. There is no reason why we should ever see poor resolution, if this is exclusively based on the bandwidth. Something is wrong in the way Jitsi assess what’s available and automatically deprecates the stream quality.

I definitely don’t want to disable Simulcast, so I’m hoping whatever the problem is can be resolved outside of that.

From my understanding, simulcast benefits the server, not the client.
Theoretically, If all clients send max resolution they can achieve to the server (simulcast disabled), the server has to do the transcoding to downgrade quality.
With simulcast, the client sends multiple streams and the server decides which one to serve the other clients based on bandwidth capacity. This puts most of the load on the clients…

Since my server is mostly asleep while hosting jitsi, I don’t mind letting it do the heavy lifting! Most of our customers have well enough bandwidth, it is processing capacity they are lacking (laptop computers)

I agree with you @Freddie, the bandwidth calculation seems to be off by a good margin… It might be the cause of the issue… If we could force it to certain values, that would be great!

@Freddie, it is possible that the bandwidth estimation done by the bridge is slightly off. However, the issue that @Normand_Nadon is facing is because we made it harder to request resolutions higher than 720p. Can you please gather the connection stats when you experience poor video quality ?

Simulcast benefits the client as well, Jitsi bridge doesn’t do transcoding to downgrade quality if simulcast is disabled. If only a single stream is available, the bridge will try it to push it downstream to the clients irrespective of their available downlink resulting in very poor quality video on the clients that don’t have enough downlink. It can overwhelm clients in large calls since the clients will receive all HD streams, most laptop computers are not equipped to decode lots of HD streams and therefore will result in poor video quality.

When simulcast is enabled, the bridge can pick the right stream. The load on the client is slightly higher but we have optimized it to send only the streams that are needed based on how this participant is being viewed by the other participants all the way to not sending anything if no one is viewing this particular user.

Hello @jallamsetty,

I did some tests. My browser is chromium 83.0.4103.116-1~deb10u3 on Debian Buster. I connected to the same room from 3 different tabs on the same browser. Tabs are private (incognito) and no cache issue (cleared before each test)

I disabled simulcast and I fixed the video height in /etc/jitsi/meet/myhost-config.js.

resolution: 480,

constraints: { 
  video: {   
    height: {     
      ideal: 480,
      max: 480, 
      min: 480  
    }      
  }          
},

disableSimulcast: true,

As a result, I was expecting the resolution to stay fixed (480) but that didn’t happen. The resolution varied depending on the network load. I saw that Chromium changed the outgoing stream quality (the resolution and the frame rate) according to the upload condition.

Since there will be more upload traffic when simucast is enabled, the browser may limit the outgoing stream quality more aggressively and can break the simucast working

Hi @emrah, browsers take both cpu and bandwidth into consideration when deciding on the send resolution of the client. If the cpu load on the client machine is high (bound to happen if you run 3 client instances on the same machine), chrome will downsample and change the encode resolution to something lower until it’s able to encode video smoothly . This happens for both simulcast and non-simulcast case.

When the uplink is constricted, what chrome does depends on whether simulcast is turned on/off. If simulcast is on, the higher HD streams will be turned off to match the available uplink. If simulcast is off, the browser will try to send a stream of the said resolution but of lower quality by changing the quantization parameter which will make it pixelated and blurry even though the resolution is high.

You should re-run the test by running only 1 client per machine, you should be able to receive 480p always.

1 Like

480p is not high def! I mean, not in 2020!
1080p has been standard since early 2000’s
Simulcast runs in-between standard resolutions for me, and it gives awful results! (480p would probably scale better in fact!)
The client machines ARE NOT the culprit here… One is an hexacore beast with hyperthreading, 16Gb of RAM and a Quadro Crazy GPU, The other one is an Octacore with 16GB of RAM and an RX480 GPU. Both machines connected to 1GB Internet and I still get low res feeds whatever the layout I use.

@Normand_Nadon, no where I said 480p was high def, I was simply answering about what was probably happening in @emrah’s test.
Your case is a corner case because there is 1080p involved and like I explained before, we made it harder to request higher resolutions (> 720) for simulcast case.
Stop mixing up these 2 different scenarios.

Sorry, I did not realize the answer was not targeted for me…

That being said, where can I change this “threshold” and make it easier to rise to HD ?
I do understand the pixellisation thing now… Thought it was just poor down sampling from the browser!

@Normand_Nadon, no worries! If you want only HD, then you can specify 720 as the ideal/max height in the video constraints in your /etc/jitsi/meet/yourdomain.com-config.js.
All the clients will receive HD even with simulcast enabled.

However, if you want Full HD (1080p), the only way now is to disable simulcast until we are able to add the functionality for requesting 1080p in simulcast case.