High E2E RTT Value and low quality

I see high E2E RTT values for some users and the bridge logs sometimes:

JVB 2020-03-20 10:28:11.047 SEVERE: [471822] org.jitsi.videobridge.EndpointMessageTransport.log() Failed to send a message over a WebRTC data channel (endpoint=879b8235):
JVB 2020-03-20 10:28:11.492 SEVERE: [471850] org.jitsi.videobridge.EndpointMessageTransport.log() Failed to send a message over a WebRTC data channel (endpoint=879b8235):

Some users are using the app, some chrome and some firefox. The group size is 15-30.

Any idea how to improve this?

So 30 people with chrome, you need to have 30*5=150Mbit download to the server and that much upload, this is of course if all are sending hd video.
You can adjust to lower the resolution in the config.js, activate startVideoMuted =5 (only the first 5 will have video). Or startWithVideoMuted: true so everybody will be video muted.

In the max calculation above if you add just one Firefox it will come to 300Mbit upload if all are sending video.
We are working on the Firefox support to bring simulcast and remove this limitation.

Hardware:
• Jicofo, Prosody, etc. (Xeon 12 Cores, 128 GB RAM), 1 Gbps
• Videobridge 1 (Xeon 16 Cores, 64 GB RAM), 1 Gbps
• Videobrdige 2 (Xeon 24 Cores, 196 GB RAM), 10 Gbps

And we also reduced the default quality in jitsi-meet to 320p beforehand.

It is right, that also the user needs 300Mbit bandwidth or only the server? Since some of the person are displayed smaller. I thought the videobridge and client would use a small resolution

Depends whether simulcast is used or not. 300 is for server.

Okay, the mobile application and chrome uses simulcast by default or? Switch all users back to “non”-simulcast if one user uses firefox?

No.

Chrome and mobile use simulcast. The Firefox thing just adds more bandwidth to users and bandwidth to the server (and CPU on both I suppose), it does not change the others behaviour in any way.

Hm okay,
then i wondering why the E2E RTT value is so high… And also the audio and video quality is like “stucking”. Used ubuntu 18 and the same version of jitsi as on jitis.meet offical webseite. Prosody was original at 0.10.X. We upgraded to the prosody 0.11.X and jitsi to the lastest versions since some users appear very long unactive and where not removed from the conference. However we are now stucking in these problems…

The installation can be found here:

I can also provide some logging informations…

How did you install jvb?
Check whether these are applied on your server:

/etc/sysctl.conf

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.ens160.disable_ipv6 = 1


# increase Linux TCP buffer limits
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152

# increase Linux autotuning TCP buffer limits
# min, default, and max number of bytes to use
net.ipv4.tcp_rmem = 4096 87380 2097152
net.ipv4.tcp_wmem = 4096 65536 2097152

I will try the new settings. I already found the information to use these settings. Installation via apt-get. By the way: cpu usage is lower than 20 precent. However the systemload is sometimes higher than 5.

Can you update to the values that jvb is adding on install, those from that file and give it a try.

The lower buffers may explain freezes, there are also os level stats for dropped udp packets you can monitor. Not big enough buffers will cause dropped udp packets.

Also consider switching to jvb2.

Does this help?

 cat /proc/net/snmp6
Ip6InReceives                           3326980
Ip6InHdrErrors                          0
Ip6InTooBigErrors                       0
Ip6InNoRoutes                           0
Ip6InAddrErrors                         0
Ip6InUnknownProtos                      0
Ip6InTruncatedPkts                      0
Ip6InDiscards                           81
Ip6InDelivers                           3326899
Ip6OutForwDatagrams                     0
Ip6OutRequests                          3326961
Ip6OutDiscards                          0
Ip6OutNoRoutes                          424
Ip6ReasmTimeout                         0
Ip6ReasmReqds                           0
Ip6ReasmOKs                             0
Ip6ReasmFails                           0
Ip6FragOKs                              0
Ip6FragFails                            0
Ip6FragCreates                          0
Ip6InMcastPkts                          0
Ip6OutMcastPkts                         62
Ip6InOctets                             965566868
Ip6OutOctets                            965560914
Ip6InMcastOctets                        0
Ip6OutMcastOctets                       3568
Ip6InBcastOctets                        0
Ip6OutBcastOctets                       0
Ip6InNoECTPkts                          3326899
Ip6InECT1Pkts                           0
Ip6InECT0Pkts                           0
Ip6InCEPkts                             0
Icmp6InMsgs                             0
Icmp6InErrors                           0
Icmp6OutMsgs                            62
Icmp6OutErrors                          0
Icmp6InCsumErrors                       0
Icmp6InDestUnreachs                     0
Icmp6InPktTooBigs                       0
Icmp6InTimeExcds                        0
Icmp6InParmProblems                     0
Icmp6InEchos                            0
Icmp6InEchoReplies                      0
Icmp6InGroupMembQueries                 0
Icmp6InGroupMembResponses               0
Icmp6InGroupMembReductions              0
Icmp6InRouterSolicits                   0
Icmp6InRouterAdvertisements             0
Icmp6InNeighborSolicits                 0
Icmp6InNeighborAdvertisements           0
Icmp6InRedirects                        0
Icmp6InMLDv2Reports                     0
Icmp6OutDestUnreachs                    0
Icmp6OutPktTooBigs                      0
Icmp6OutTimeExcds                       0
Icmp6OutParmProblems                    0
Icmp6OutEchos                           0
Icmp6OutEchoReplies                     0
Icmp6OutGroupMembQueries                0
Icmp6OutGroupMembResponses              0
Icmp6OutGroupMembReductions             0
Icmp6OutRouterSolicits                  57
Icmp6OutRouterAdvertisements            0
Icmp6OutNeighborSolicits                1
Icmp6OutNeighborAdvertisements          0
Icmp6OutRedirects                       0
Icmp6OutMLDv2Reports                    4
Icmp6OutType133                         57
Icmp6OutType135                         1
Icmp6OutType143                         4
Udp6InDatagrams                         2
Udp6NoPorts                             0
Udp6InErrors                            0
Udp6OutDatagrams                        2
Udp6RcvbufErrors                        0
Udp6SndbufErrors                        0
Udp6InCsumErrors                        0
Udp6IgnoredMulti                        0
UdpLite6InDatagrams                     0
UdpLite6NoPorts                         0
UdpLite6InErrors                        0
UdpLite6OutDatagrams                    0
UdpLite6RcvbufErrors                    0
UdpLite6SndbufErrors                    0
UdpLite6InCsumErrors                    0

ip4
cat /proc/net/snmp

Ip: Forwarding DefaultTTL InReceives InHdrErrors InAddrErrors ForwDatagrams InUnknownProtos InDiscards InDelivers OutRequests OutDiscards OutNoRoutes ReasmTimeout ReasmReqds ReasmOKs ReasmFails FragOKs FragFails FragCreates
Ip: 1 64 389860478 0 0 57716 0 0 389802549 1078070299 0 0 0 86 43 0 5078 0 10156
Icmp: InMsgs InErrors InCsumErrors InDestUnreachs InTimeExcds InParmProbs InSrcQuenchs InRedirects InEchos InEchoReps InTimestamps InTimestampReps InAddrMasks InAddrMaskReps OutMsgs OutErrors OutDestUnreachs OutTimeExcds OutParmProbs OutSrcQuenchs OutRedirects OutEchos OutEchoReps OutTimestamps OutTimestampReps OutAddrMasks OutAddrMaskReps
Icmp: 516625 2090 1715 513773 128 0 0 0 1008 1 0 0 0 0 5034 0 4026 0 0 0 0 0 1008 0 0 0 0
IcmpMsg: InType0 InType3 InType8 InType11 OutType0 OutType3
IcmpMsg: 1 513773 1008 128 1008 4026
Tcp: RtoAlgorithm RtoMin RtoMax MaxConn ActiveOpens PassiveOpens AttemptFails EstabResets CurrEstab InSegs OutSegs RetransSegs InErrs OutRsts InCsumErrors
Tcp: 1 200 120000 -1 1199311 1253478 25759 6995 60 54827165 79063698 1491717 7290 156285 7288
Udp: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors InCsumErrors IgnoredMulti
Udp: 337200911 121887 11675 1018576413 11659 0 15 444743
UdpLite: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors InCsumErrors IgnoredMulti
UdpLite: 0 0 0 0 0 0 0 0

Further information I also added here Need help for scaling and high E2E values, since I did not find the correct category.