SWITCH is the “National Research & Education Network - NREN” for Switzerland. We provide various IT related services to the higher education institutions (universities, universities of applied science, university hospitals etc) sector - including LAN (we run our own 100GBs backbone), Identity Services (based on Shibboleth/ AAI), Cloud IaaS and a lot more. Two weeks ago we saw a huge rise in interest in video conferencing. We use Vidyo and Adobe Connect with a couple of hundred simultaneous licenses each.
We decided to start a Jitsi based service on March 12th and had a running prototype of the service on March 13th. Since then my team has worked hard to scale the system.
Our current setup uses 22 jitsi-meet frontend servers (for various insitutions). The plan was to have them authenticate/authorize via Shibboleth - which worked find in the first prototype (using the August 2019 verision of Jitisi due to a problem in Jicofo with Shibboleth: https://github.com/jitsi/jicofo/issues/404 - I will file a new issue shortly, so that it gets tracked again).
We also soon realized that we had to work on scaling our video bridge infrastructure. In order to have simplified deployment of video bridges, we now work with the nightly version of Jitsi (without Shibboleth Authentication) using a pool of currently 32 videobridges (8 VCPUs, 8 GB RAM each) that connect to all front end instances.
We learnt many things during the last two weeks, and I’m busy sanitizing our deployment scripts (Ansible) to share with others. I hope to make them available this weekend or early next week so others can benefit.
Things we learned and configured (in no particular order):
- Enable LastN (currently 3)
- Start with Audio/Video muted after third particpant
- Enable UDP on videobridge (and in the firewalls )
- Callstasts.io is expensive but very worthwhile to diagnose problems
- Jicofo / Prosody Certificates are a pain, disable the checks
We also built a simple “view” app, that wraps the Jitsi frame (using the external-js API) and removes most of the UI chrome (for example enabling audio/video) except for chat and hangup so that lecturers can use that to lecture and provide a “one way” video channel. This has been a mixed success, but it works.
We ran into the 75 users hard limit, but in practice the conference became unusable after around 25-30 users (mostly due to the browsers having problems handling the UI). We regularly have internal meetings with 15 people and that works very well. I have seen meetings with up to 50 participants, but am not sure, just how well that went.
Our immediate next plans:
- Upgrade to the latest nightly version this weekend
- Enable epoll on Prosody
- Build current versions of https://github.com/jitsi/jitsi-meet-electron and make that available to users
- Build a custome version of Jibri and stream to an internal RTMP server
All in all this project has been challenging (the documentation is all over the place and many things are not documented well, finding stuff on the discussion forums has been difficult, but the level of knowledge from people like @damencho is great) but so far a great success. We have received a lot of good feedback from our (university) community. Me, I am impressed by the design of Jitsi and how well it works.
Thanks for that & onwards!