Own VPS install worse performance than meet.jit.si

Hi.

We’ve installed Jitsi with coturn and nginx on a Contabo VPS with 8 cores, 30G RAM and 600MBit/s bandwidth. When doing calls on our instance we often experience disconnections and generally bad performance, even with few users (less than 5). It’s not bad for the single user but for everyone.

I checked server parameters and resources are far from exhaustion (graphs below).

Doing the same calls on meet.jit.si with the same users results in much better experience. Contabo VPSes sucks in term of disk IO, but according to the graph it doesn’t seems the culprit as IOwait is barely visible.

What could cause so much difference in result?

Thanks

What are the disconnects you are talking about? Does the client reload for some reason? What does the js log say in this situation?

If you are talking about quality maybe check chome://webrtc-internals and find the rtt let’s say and see what are the values … its not only the bandwidth, but if the quality of the network is not good and that will affect quality.

If with same participants quality is fine on meet.jit.si this excludes client network problems. So it is a good test to compare.

Hi thanks for the feedback.

About the RTT on Firefox I found it and varies from 2/300 ms peaking to 2000/3000 at times. On Chrome if I found the correct info it’s here:

About the disconnects, what we can see is people occasionally disappearing from the call. Unfortunately they’re not tech users we can ask for debug.

About the comparison to meet.jit.si we made a simple test: a problematic call with three hosts on our server was moved to the global instance and it went much better, no disconnects and good video quality. Before closing we switched back to our instance and the issues showed again.

Is there anything particular required for the on prem installation?

For the JS console, we made a two persons web call and as error I only see:

2021-01-25T14:06:06.580Z [modules/xmpp/xmpp.js] <A.tryParseJSONAndVerify>:  SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at A.tryParseJSONAndVerify (xmpp.js:723)
    at A._onPrivateMessage (xmpp.js:768)
    at I.Handler.run (strophe.umd.js:1875)
    at strophe.umd.js:3157
    at Object.forEachChild (strophe.umd.js:830)
    at I.Connection._dataRecv (strophe.umd.js:3146)
    at O.Bosh._onRequestStateChange (strophe.umd.js:5012)

Thanks

Well RTT of 3 seconds is too much … so disconnections in that environment are to be expected.

The error you saw in the js console is harmless, not sure which jvb json message is producing that, but I have seen reports for those … but yeah you can ignore it.

As said they were peaks… And was with Firefox, which is dead sluggish on my PC (maybe because I have too many open tabs). Chrome’s behaviour looks much better in the graphs, what do you think?

You can compare with meet.jit.si the chrome rtt, but if I read it right 0.6 is 600ms? That also does not look good … you can see there are times with burst of packet loss and jitters and rtt jumps at those moments, these variations are problematic. If the bridge network gives such variations and maybe even worse when there are more participants - high traffic.

We made a second test connected with two landline ADSL and performance has been much much better, even with Firefox.

Of course having a 0.6rtt is bad for performances, but what we would like to figure out is the difference between our installation and jit.si one. Could the VPS provider be somehow the culprit? As said Contabo has bad disk IO performance, but there’s plenty of bandwidth available, tough I haven’t measured connection times or other network specific parameters. Can you suggest anything we can check?

Thanks.

Just an update: we cloned our server to AWS and it’s much more usable. Even when the client connection is poor (we replicated the situation of our first test) video quality always remain HD, video occasionally freeze, but it’s almost always smooth and audio is good.

At this point it seems clear there’s something with Contabo networking which is causing this bad video performance. What parameters can we compare on both servers to understand where’s the culprit? Connection time? Routing?

Thanks

For me the difference is the rtt, the round trip time.
In the first graph you can see it not only jumps but stay above 0.2 for long time, while in the last it stays below 0.1 and jumps to 0.2 occasionally.

Comparing with mtr the two hosts shows Contabo is much slower in the last hops, that’s likely the reason of the laggish experience with it…