Jitsi-hammer stress test


#21

What is one selenium grid? So you have one machine configured for hub and how many machines for nodes and how many sessions can a node machine receive?


#22

Hi
I have a vm with selenium hub and a Vm that launches 40 clients.
selenium version is 3.14.0
chromium version is 65 (modified to have a fixed video bandwidth)
chromedriver version is 2.38


#23

Now I’m trying with this topology

one server with the videobrdige and jicofo

one server with an instance of selenium and 30 clients.
one client ever 10 seconds

one server with an instance of selenium and 20 clients
one client every 10 seconds.

I launch one server per time, so after the first 30 clients I launch the second pool.
the problem is that after 35 clients chromedriver starts to show me Timed out receiving message from renderer and I don’t see the traffic growing up.

the clients are all in headless mode.
I want to replicate your load test, but I don’t find information about it.
Thank you for your help.

Angelo


#24

So the difference is that we have one machine for hub and for example 50 machines and each of those machine will handle one chrome instance. We never run more than one chrome on a machine.
I think you are experiencing big load on the machine when running multiple chrome instances, you need at least 1.5-2 GB of RAM per instance and some % of CPU doing decoding and encoding … so I think at some point it starts timing out due to load …


#25

Ok,
it is impossible, in my configuration, to create 100 VM for 100 clients.

what I want to achieve is the in/out bandwidth and CPU usage of the VB server.

do you know a solution to emulate a fake client without the rendering part?

thank you very much for your help


#26

Try https://testrtc.com.


#27

no, I need an open source solution. the videobridge server is inside my network, it is not public.
thank you for your help

Angelo


#28

Sorry, so this is what we have. You can try playing with hammer, but this wasn’t used for long time, and it is very hard to maintain it, so probably you are on your own there.
Launching hundreds of vms is very easy with autoscaling in aws, but if you need to do it in your network … Its just you need to distribute the load of chrome instances …


#29

thank you.
I’m very new with selenium (this is my first test with it) and maybe I don’t understand how I have to use it

I’ve launched it with “java -jar selenium.jar”
and after I launched your maven script.
these operations on every container.

You told me that you use only one hub, but I understood that the hub is the application that opens the chromium instances.

but maybe I’m doing wrong.

could you explain me?
Thank you
Angelo


#30

Selenium/webdriver can be ran on the same machine and using remote driver it can connect to a server and the actual webdriver can be there. There is also these settings where selenium server can be two parts hub and node.
You connect to the hub and every node you launch connects to the hub and register itself there. This way you can have one hub and hundreds of nodes from different machine.
If you have powerful machines you can spin up just a few nodes and raise the number of chrome instances slowly so you can find their boundaries, to know what is the max number of chromes you can launch. As we use not so powerful instances in aws we limit every node to one session/chrome.


#31

Hi
I’ve changed my topology
I have 4 servers with 48 cpu each.
I create 20 lxd containers per server.
every container is a selenium node and it manages only one client.

I don’t have errors, but after 50 clients (every client generates 700kbps, the video bandwitdh is fixed, so in the third server after 7Mbps, more or less) I see that the traffic does not increase.
is there a configuration that limits the number of the clients?
thank you very much for your help

Angelo


#32

Nope there is no such by default. If you are using single jvb, is it possible it reaches some limits cpu or bandwidth?


#33

Hi,
thanks for your answer.

the jvb and jicofo are in another server with 72 core. with 50 clients I see that the java prcesses take 5 cpu.

I don’t have bandwidth problems. all the servers are connected via lan.

it is strange, When I started with the last pool I see that there was a peak and after the out bandwidth is always 0.


#34

Have you checked jvb logs?
At some point we were experiencing problems with the limit of opened files and tasks per user on ubuntu bionic, which was capping the limit of participants around 200-250, it can happen you are hitting something similar. For that checkout the second part of advanced configuration: https://github.com/jitsi/jitsi-meet/blob/master/doc/quick-install.md#advanced-configuration
We are working on adding systemd unit files for starting up jvb which will handle that and there will be no need of manual settings. We also added some warning prints in latest jvb on startup when some low values are detected.


#35

Hi, I tried to change the values that are inside “advanced-configuration”, but I don’t see improvements.
I’m sure that I don’t have bandwidth problems.
I see a message “video for fellow jitser has been turned off to save bandwidth”.
could it be that the clients understand that there latency is very high and the last connected switch off the video stream?


#36

Switching off video comes from jvb, had you measured the bandiwdth there? Jvb is doing bandiwdth estimations for the receivers.
What is the distribution of clients across conferences you do?


#37

Hi,
I’ve updated jicofo, jvb and jitsi-meet with the last version and I changed

DefaultLimitNOFILE=99000 DefaultLimitNPROC=99000 DefaultTasksMax=99000

in the JVB container.

with 5 servers (the jvb is on another server), with 20 clients each I don’t have problems. No more.

So thank you very much for your help.

Angelo


#38

Hi,
I’ve modify the torture (malleus branch). I add the command line parameters about the interval between two clients and the name of the video/audio resources.

I want to commit to github in the malleus branch.
hints?
I’m new with public github repos.

Thank you very much for your help
Angelo


#39

So, I would say, create fork, checkout your fork, switch to malleus branch, from it create a new branch lets say malleus_angelo, commit your changes and from that branch create a PR which is agains malleus branch, not against master.


#40

Thanks for the info! I started selenium-hub and registered 2 nodes; then I run the “mvn … test” command as you mentioned, I can see the task has been sent to my 2 nodes, but I cannot make “Dorg.jitsi.malleus.conferences=2” and “Dorg.jitsi.malleus.participants=2” work, because:
I only see 1 room has been created(for example, torture251170, but I am trying to test 2 rooms running in the same time;
also I always saw 3 participants there, it seems that my config(“Dorg.jitsi.malleus.participants=2”) is not read by jitsi-torture. I found the following in ActiveSpeakerTest.java:
WebParticipant participant1 = getParticipant1();
WebParticipant participant2 = getParticipant2();
WebParticipant participant3 = getParticipant3();
how I can make “Dorg.jitsi.malleus.participants=2” work so I can control the participants number in 1 room?

my purpose is use selenium to create several concurrent rooms(for ex., 20 rooms, and for every room I want to join several participants(for ex. 10 participants), could you help me check why I cannot control the room number and participants number in my testing? Thanks!