Loadtesting Jitsi Meet
I needed to do some load testing on my jitsi meet instance to get a feeling for how many participants, audio and videostreams my JVB could handle. You can see my results (on a pretty outdated machine) here.
Today (as requested by a few others on the forum as well) I’m going to describe my Setup and walk through the most important steps.
While this would be possible on Windows as well, I’m going to focus on Linux as that is what I daily-drive and I think it is more convenient as well
A quick warning: Selenium Grids require insane amounts of resources for larger scale tests.
- Setting up a Selenium Grid
- Installing the Maven environment
- Running tests (in this case MalleusJitsificus. I love that name, thank you to whoever named it that way)
There are two tools to do testing on JVBs: Jitsi Hammer and Jitsi Meet Torture. Jitsi Hammer is outdated and doesn’t work anymore, if you want to test up-to-date installations you need to use Jitsi Meet Torture (which also includes a lot of other useful tests as well and could be used in your CI/CD process)
Jitsi Meet Torture requires a Selenium Grid.
A short quote from their Documentation on what Selenium Grid is:
Selenium Grid is a smart proxy server that allows Selenium tests to route commands to remote web browser instances. Its aim is to provide an easy way to run tests in parallel on multiple machines.
With Selenium Grid, one server acts as the hub that routes JSON formatted test commands to one or more registered Grid nodes. Tests contact the hub to obtain access to remote browser instances. The hub has a list of registered servers that it provides access to, and allows control of these instances.
Selenium Grid allows us to run tests in parallel on multiple machines, and to manage different browser versions and browser configurations centrally (instead of in each individual test).
Selenium Grid is not a silver bullet. It solves a subset of common delegation and distribution problems, but will for example not manage your infrastructure, and might not suit your specific needs.
Tl;dr: It launches “real” browser Sessions on Chrome or Firefox and can control them with code you have written. In our case it can open a Webpage of your Jitsi Meet and simulate audio and video. This is as close as you can get to real-life examples without actually having people open browsers and clicking things.
Lets get to it now!
Setting up a Selenium Grid
- A Java installation
- Multiple VMs to run your tests. Cloud instances on GCloud or AWS are maybe not a bad option if you are savy enough to do some automation around them and shut them down when no longer needed.
The grid consists of multiple components:
- The hub
- The nodes
The hub is the brain of your Selenium Grid. It manages all nodes which run your chrome browsers.
When running tests you always connect to your hub which then launches the appropriate amount of chrome browsers on the nodes.
The hub does not require a lot of resources and can be installed on one of your node VMs as well. I’d still recommend running it on its own VM though.
Setting up the Hub
Grab your self the Selenium Server jar: https://www.selenium.dev/downloads/ I will use v.3.141.59 in this example.
Install Java on your VM (I’m using Java 8) and then run
java -jar selenium-server-standalone.jar -role hub you can also use the flag
-host xxx.xxx.xxx.xxx to specify which IP it should bind to.
When you navigate to http://theiporhostname:4444/grid/console You should see an almost empty page with a banner “Grid Console vx.xxx.xx”
Setting up nodes
Nodes are where the magic happens. Jitsi Meet Torture tests mostly request chrome instaces by default, this is why we will only set up chrome instances in this tutorial. You can run firefox as well, the setup is pretty much the same.
In an ideal setup you would launch 1 chrome instance per node (node with 2 CPU, 2GB RAM) but as I didn’t want to spend money on cloud servers I set up 4 VMs with 4 cores, 4gb of ram on my local computer and ran 3 chrome instances on them. I couldn’t run tests on the scale as I wanted to as I soon hit 100% CPU usage on my 8c/16t Ryzen CPU.
This works as well but is kinda janky, if you want to do “real” “professional” tests, I’d suggest going with a lot of small cloud servers with one chrome instance each.
We need the following packages (I’m going off a Debian Buster, you can adapt that to whatever distro you are running):
sudo apt-get update sudo apt-get install -y gnupg curl wget unzip xvfb libxi6 libgconf-2-4 default-jdk sudo curl -sS -o - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - sudo echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list sudo apt-get -y update sudo apt-get -y install google-chrome-stable
Make sure you installed your OS with a Desktop Environment, chrome browser needs some kind of X server. Use GNOME, KDE, whatever floats your boat.
This gives us our “base” install with chrome. To control chrome, we now need to get the ChromeDriver which fits your chrome version. Check the official download page for the matching ChromeDriver and download the ZIP from there, unpack it and make it executable. This is just a single binary blob.
The last part needed is Selenium which connects to your Hub and sends the commands to ChromeDriver. Download the same jar you downloaded for your hub but this time run it like this:
java -Dwebdriver.chrome.driver=path/to/chromedriver -jar selenium-server-standalone.jar -role node -maxSession 1 -hub http://yourip:4444/grid/register -browser browserName=chrome,version=80,platform=Linux,maxInstances=1
You can choose a port with
-port, otherwise it will use 4444 if avaible.
-host again allows you to only bind to a specific IP.
If you want to run tests with video on them, you will need to copy a sample video to each node as well, refer to “Running Tests” for this.
When reloading http://theiporhostname:4444/grid/console you should now see your nodes if they successfully registered.
This might be a lot and kinda overwhelming for new users but don’t worry, we are half way there now.
Installing the Maven environment
Do this on your normal PC or Laptop, you will start and compile your tests here and then send them off to your Selenium Grid. You need a Java JDK installed (I used openjdk-11-jdk-headless)
Jitsi Meet Torture is a collection of different tests written in Java, to compile them you need a correctly configured maven environment.
This is as simple as
sudo apt install maven
mvn -version should show you something like this:
Maven home: /usr/share/maven Java version: 11.0.6, vendor: Debian, runtime: /usr/lib/jvm/java-11-openjdk-amd64 Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "4.19.0-6-amd64", arch: "amd64", family: "unix"
echo $JAVA_HOME returns your path, if not you need to set that Environment variable with
JAVA_HOME=$(readlink -f /usr/bin/javac | sed "s:/bin/javac::")
Clone the jitsi-meet-torture repository:
git clone https://github.com/jitsi/jitsi-meet-torture cd jitsi-meet-torture
To run the MalleusJitsificus test (which can do load testing) there is a nice wrapper script located in
Just put a
--help on there to see all avaible flags, I’m going to give some examples:
./scripts/malleus.sh --conferences=2 --participants=4 --senders=1 --audio-senders=2 --duration=120 --room-name-prefix=hamertesting --hub-url=http://yourSeleniumHub:4444/wd/hub --instance-url=https://theJitsiInstallationYouWantToTest.net
This will connect to the Selenium hub set with
--hub-url and launch chrome browsers pointing to https://theJitsiInstallationYouWantToTest.net/hammertesting0 and https://theJitsiInstallationYouWantToTest.net/hammertesting1
with 4 users in each room, one of them sending video, two of them sending audio, one just watching.
Which video will it play back?
I’m glad you asked because right now it won’t play back any we need to provide the nodes with sample files to play back.
jitsi-meet-torture/src/test/java/org/jitsi/meet/test/MalleusJitsificus.javain your favourite text editor and check the line
private static final String INPUT_VIDEO_FILE
This is by default set to
resources/FourPeople_1280x720_30.y4m which doesn’t exist on the nodes.You can download a video you like from here: https://media.xiph.org/video/derf/y4m/ and place it in /usr/share/jitsi-meet-torture/resources/ and change the file name in the .java accordingly. I suggest you just copy the entire resources folder from the repository on to the nodes, it also includes the audio files.
I hope I didn’t miss any important steps, if yes, I will add them. Should I put this as a PR on Github?