Failing test when torturing jitsi

We are facing some problems with jitsi currently (people randomly dropping)
Unfortunately it is hard to reproduce - so I wanted to try: GitHub - jitsi/jitsi-meet-torture
I have one failing (muteParticipant1BeforeParticipant2Joins) test after some others pass. But not yet sure what could be an action I could take against this one failing (also not yet convinced it has something to do with our problem/instance - but hesitant to torture a public instance to see if the tests fully pass there : )

ligi@ligi-w520:~/git/jitsi-meet-torture$ mvn test -Djitsi-meet.instance.url="https://meet.ethdevops.io" -Djitsi-meet.tests.toRun="MuteTest,TCPTest"
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/usr/share/maven/lib/guice.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[INFO] Scanning for projects...
[INFO] 
[INFO] --------------------< org.jitsi:jitsi-meet-torture >--------------------
[INFO] Building jitsi-meet-torture 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ jitsi-meet-torture ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/ligi/git/3rd/jitsi-meet-torture/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ jitsi-meet-torture ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ jitsi-meet-torture ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ jitsi-meet-torture ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 127 source files to /home/ligi/git/3rd/jitsi-meet-torture/target/test-classes
[INFO] /home/ligi/git/3rd/jitsi-meet-torture/src/test/java/org/jitsi/meet/test/PSNRTest.java: Some input files use or override a deprecated API.
[INFO] /home/ligi/git/3rd/jitsi-meet-torture/src/test/java/org/jitsi/meet/test/PSNRTest.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/ligi/git/3rd/jitsi-meet-torture/src/test/java/org/jitsi/meet/test/base/AbstractBaseTest.java: Some input files use unchecked or unsafe operations.
[INFO] /home/ligi/git/3rd/jitsi-meet-torture/src/test/java/org/jitsi/meet/test/base/AbstractBaseTest.java: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ jitsi-meet-torture ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
[15] testsToRun=[MuteTest, TCPTest]
[15] testsToExclude=[]
[15] testsToInclude=[]
[15] ---=== Testing MuteTest ===---
[15] No participant type specified for prefix: web.participant1, will use Chrome...
[TestNG-tests-1] INFO io.github.bonigarcia.wdm.WebDriverManager - Using chromedriver 104.0.5112.29 (resolved driver for Chrome 104)
[TestNG-tests-1] INFO io.github.bonigarcia.wdm.WebDriverManager - Exporting webdriver.chrome.driver as /home/ligi/.cache/selenium/chromedriver/linux64/104.0.5112.29/chromedriver
Starting ChromeDriver 104.0.5112.29 (eff877e18f767c77fef0481a1cba402c8cbad404-refs/branch-heads/5112@{#422}) on port 20351
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Aug 03, 2022 1:14:42 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
[16] ChromeDriver created for:905 ms.
[15] Started chrome driver for prefix: web.participant1
[15] web.participant1 is opening URL: https://meet.ethdevops.io/torture163211#config.testing.testMode=true&config.disableNS=true&userInfo.displayName="participant1"&config.disableAEC=true&config.analytics.disabled=true&interfaceConfig.SHOW_CHROME_EXTENSION_BANNER=false&config.disable1On1Mode=true&config.alwaysVisibleToolbar=true&config.p2p.useStunTurn=false&config.prejoinConfig.enabled=false&config.p2p.enabled=false&config.requireDisplayName=false&config.gatherStats=true&config.pcStatsInterval=1500&config.debug=true&config.enableTalkWhileMuted=false&config.callStatsID=false&interfaceConfig.DISABLE_FOCUS_INDICATOR=true
[15] web.participant1 lib-jitsi-meet version: f3a2f61e sessionID: 0fb03ebdc1eda8edf5ef5a402c754752
[15] No participant type specified for prefix: web.participant2, will use Chrome...
[TestNG-tests-1] INFO io.github.bonigarcia.wdm.WebDriverManager - Using chromedriver 104.0.5112.29 (resolved driver for Chrome 104)
[TestNG-tests-1] INFO io.github.bonigarcia.wdm.WebDriverManager - Exporting webdriver.chrome.driver as /home/ligi/.cache/selenium/chromedriver/linux64/104.0.5112.29/chromedriver
Starting ChromeDriver 104.0.5112.29 (eff877e18f767c77fef0481a1cba402c8cbad404-refs/branch-heads/5112@{#422}) on port 29047
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Aug 03, 2022 1:14:48 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
[30] ChromeDriver created for:576 ms.
[15] Started chrome driver for prefix: web.participant2
[15] web.participant2 is opening URL: https://meet.ethdevops.io/torture163211#config.testing.testMode=true&config.disableNS=true&userInfo.displayName="participant2"&config.disableAEC=true&config.analytics.disabled=true&interfaceConfig.SHOW_CHROME_EXTENSION_BANNER=false&config.disable1On1Mode=true&config.alwaysVisibleToolbar=true&config.p2p.useStunTurn=false&config.prejoinConfig.enabled=false&config.p2p.enabled=false&config.requireDisplayName=false&config.gatherStats=true&config.pcStatsInterval=1500&config.debug=true&config.enableTalkWhileMuted=false&config.callStatsID=false&interfaceConfig.DISABLE_FOCUS_INDICATOR=true
[15] web.participant2 lib-jitsi-meet version: f3a2f61e sessionID: 94da112f48e19004ebea5312d3a5c5d2
[15] Start muteParticipant1AndCheck.
[15] End muteParticipant1AndCheck.
[15] Start unmuteParticipant1AndCheck.
[15] End unmuteParticipant1AndCheck.
[15] Start muteParticipant2AndCheck.
[15] End muteParticipant2AndCheck.
[15] Start unmuteParticipant2AndCheck.
[15] End unmuteParticipant2AndCheck.
[15] Start participant1MutesParticipant2AndCheck.
[15] End participant1MutesParticipant2AndCheck.
[15] Start participant2UnmutesAfterParticipant1MutedItAndCheck.
[15] End participant2UnmutesAfterParticipant1MutedItAndCheck.
[15] Start muteParticipant1BeforeParticipant2Joins.
[15] Hung up in web.participant2.
[15] web.participant1 is opening URL: https://meet.ethdevops.io/torture163211#config.testing.testMode=true&config.disableNS=true&userInfo.displayName="participant1"&config.disableAEC=true&config.analytics.disabled=true&interfaceConfig.SHOW_CHROME_EXTENSION_BANNER=false&config.disable1On1Mode=true&config.alwaysVisibleToolbar=true&config.p2p.useStunTurn=false&config.prejoinConfig.enabled=false&config.p2p.enabled=false&config.requireDisplayName=false&config.gatherStats=true&config.pcStatsInterval=1500&config.debug=true&config.enableTalkWhileMuted=false&config.callStatsID=false&interfaceConfig.DISABLE_FOCUS_INDICATOR=true
[15] Not joining web.participant1 in torture163211, already joined.
[15] web.participant2 is opening URL: https://meet.ethdevops.io/torture163211#config.testing.testMode=true&config.disableNS=true&userInfo.displayName="participant2"&config.disableAEC=true&config.analytics.disabled=true&interfaceConfig.SHOW_CHROME_EXTENSION_BANNER=false&config.disable1On1Mode=true&config.alwaysVisibleToolbar=true&config.p2p.useStunTurn=false&config.prejoinConfig.enabled=false&config.p2p.enabled=false&config.requireDisplayName=false&config.gatherStats=true&config.pcStatsInterval=1500&config.debug=true&config.enableTalkWhileMuted=false&config.callStatsID=false&interfaceConfig.DISABLE_FOCUS_INDICATOR=true
[15] web.participant2 lib-jitsi-meet version: f3a2f61e sessionID: 94da112f48e19004ebea5312d3a5c5d2
[15] End muteParticipant1BeforeParticipant2Joins.
[15] Closing web.participant1
[15] Closing web.participant2
[15] ---=== Testing TCPTest ===---
[15] No participant type specified for prefix: web.participant1, will use Chrome...
[TestNG-tests-1] INFO io.github.bonigarcia.wdm.WebDriverManager - Using chromedriver 104.0.5112.29 (resolved driver for Chrome 104)
[TestNG-tests-1] INFO io.github.bonigarcia.wdm.WebDriverManager - Exporting webdriver.chrome.driver as /home/ligi/.cache/selenium/chromedriver/linux64/104.0.5112.29/chromedriver
Starting ChromeDriver 104.0.5112.29 (eff877e18f767c77fef0481a1cba402c8cbad404-refs/branch-heads/5112@{#422}) on port 30619
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Aug 03, 2022 1:15:19 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
[38] ChromeDriver created for:487 ms.
[15] Started chrome driver for prefix: web.participant1
[15] web.participant1 is opening URL: https://meet.ethdevops.io/torture871738#config.testing.testMode=true&config.disableNS=true&userInfo.displayName="participant1"&config.disableAEC=true&config.analytics.disabled=true&interfaceConfig.SHOW_CHROME_EXTENSION_BANNER=false&config.disable1On1Mode=true&config.alwaysVisibleToolbar=true&config.p2p.useStunTurn=false&config.prejoinConfig.enabled=false&config.p2p.enabled=false&config.requireDisplayName=false&config.gatherStats=true&config.pcStatsInterval=1500&config.debug=true&config.enableTalkWhileMuted=false&config.callStatsID=false&interfaceConfig.DISABLE_FOCUS_INDICATOR=true
[15] web.participant1 lib-jitsi-meet version: f3a2f61e sessionID: 63ce5e1e4b581ca94c32997093380d0f
[15] No participant type specified for prefix: web.participant2, will use Chrome...
[TestNG-tests-1] INFO io.github.bonigarcia.wdm.WebDriverManager - Using chromedriver 104.0.5112.29 (resolved driver for Chrome 104)
[TestNG-tests-1] INFO io.github.bonigarcia.wdm.WebDriverManager - Exporting webdriver.chrome.driver as /home/ligi/.cache/selenium/chromedriver/linux64/104.0.5112.29/chromedriver
Starting ChromeDriver 104.0.5112.29 (eff877e18f767c77fef0481a1cba402c8cbad404-refs/branch-heads/5112@{#422}) on port 32577
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Aug 03, 2022 1:15:24 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
[43] ChromeDriver created for:605 ms.
[15] Started chrome driver for prefix: web.participant2
[15] web.participant2 is opening URL: https://meet.ethdevops.io/torture871738#config.testing.testMode=true&config.disableNS=true&userInfo.displayName="participant2"&config.disableAEC=true&config.analytics.disabled=true&interfaceConfig.SHOW_CHROME_EXTENSION_BANNER=false&config.disable1On1Mode=true&config.alwaysVisibleToolbar=true&config.p2p.useStunTurn=false&config.prejoinConfig.enabled=false&config.p2p.enabled=false&config.requireDisplayName=false&config.gatherStats=true&config.pcStatsInterval=1500&config.debug=true&config.enableTalkWhileMuted=false&config.callStatsID=false&interfaceConfig.DISABLE_FOCUS_INDICATOR=true
[15] web.participant2 lib-jitsi-meet version: f3a2f61e sessionID: cf5ecf0290d9b973e3cfaf1c6fb48a71
[15] Start tcpTest.
[15] Hung up in web.participant2.
[15] web.participant1 is opening URL: https://meet.ethdevops.io/torture871738#config.testing.testMode=true&config.disableNS=true&userInfo.displayName="participant1"&config.disableAEC=true&config.analytics.disabled=true&interfaceConfig.SHOW_CHROME_EXTENSION_BANNER=false&config.disable1On1Mode=true&config.alwaysVisibleToolbar=true&config.p2p.useStunTurn=false&config.prejoinConfig.enabled=false&config.p2p.enabled=false&config.requireDisplayName=false&config.gatherStats=true&config.pcStatsInterval=1500&config.debug=true&config.enableTalkWhileMuted=false&config.callStatsID=false&interfaceConfig.DISABLE_FOCUS_INDICATOR=true
[15] Not joining web.participant1 in torture871738, already joined.
[15] web.participant2 is opening URL: https://meet.ethdevops.io/torture871738#config.testing.testMode=true&config.disableNS=true&userInfo.displayName="participant2"&config.disableAEC=true&config.analytics.disabled=true&interfaceConfig.SHOW_CHROME_EXTENSION_BANNER=false&config.webrtcIceUdpDisable=true&config.disable1On1Mode=true&config.alwaysVisibleToolbar=true&config.p2p.useStunTurn=false&config.prejoinConfig.enabled=false&config.p2p.enabled=false&config.requireDisplayName=false&config.gatherStats=true&config.pcStatsInterval=1500&config.debug=true&config.enableTalkWhileMuted=false&config.callStatsID=false&interfaceConfig.DISABLE_FOCUS_INDICATOR=true
[15] web.participant2 lib-jitsi-meet version: f3a2f61e sessionID: cf5ecf0290d9b973e3cfaf1c6fb48a71
[15] TestFailure:
org.openqa.selenium.TimeoutException: Expected condition failed: waiting for WebParticipant[web.participant1]@1206387002#waitForSendReceiveData (tried for 20 second(s) with 500 milliseconds interval)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: 'ligi-w520', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-122-generic', java.version: '11.0.15'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 104.0.5112.79, chrome: {chromedriverVersion: 104.0.5112.29 (eff877e18f76..., userDataDir: /tmp/.com.google.Chrome.DHSLap}, goog:chromeOptions: {debuggerAddress: localhost:44715}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: LINUX, platformName: LINUX, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Session ID: 63ce5e1e4b581ca94c32997093380d0f
	at org.openqa.selenium.support.ui.WebDriverWait.timeoutException(WebDriverWait.java:95)
	at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:272)
	at org.jitsi.meet.test.util.TestUtils.waitForCondition(TestUtils.java:545)
	at org.jitsi.meet.test.util.TestUtils.waitForCondition(TestUtils.java:559)
	at org.jitsi.meet.test.base.Participant.waitForCondition(Participant.java:392)
	at org.jitsi.meet.test.base.Participant.waitForSendReceiveData(Participant.java:440)
	at org.jitsi.meet.test.base.Participant.waitForSendReceiveData(Participant.java:429)
	at org.jitsi.meet.test.web.WebTestBase.ensureTwoParticipantsInternal(WebTestBase.java:159)
	at org.jitsi.meet.test.web.WebTestBase.ensureTwoParticipants(WebTestBase.java:126)
	at org.jitsi.meet.test.web.WebTestBase.ensureTwoParticipants(WebTestBase.java:104)
	at org.jitsi.meet.test.TCPTest.tcpTest(TCPTest.java:82)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:571)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:707)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:979)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
	at org.testng.TestRunner.privateRun(TestRunner.java:648)
	at org.testng.TestRunner.run(TestRunner.java:505)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
	at org.testng.SuiteRunner.access$000(SuiteRunner.java:40)
	at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:489)
	at org.testng.internal.thread.ThreadUtil$1.call(ThreadUtil.java:52)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Aug 03, 2022 1:15:58 PM org.jitsi.meet.test.base.FailureListener saveMeetDebugLog
INFO: get debug log for:web.participant1
Aug 03, 2022 1:15:58 PM org.jitsi.meet.test.base.FailureListener saveMeetDebugLog
INFO: get debug log for:web.participant2
[15] Saving browser logs for:web.participant1
[15] Saving browser logs for:web.participant2
[15] End tcpTest.
[15] Closing web.participant1
[15] Closing web.participant2
[ERROR] Tests run: 444, Failures: 1, Errors: 0, Skipped: 436, Time elapsed: 80.78 s <<< FAILURE! - in TestSuite
[ERROR] tcpTest(org.jitsi.meet.test.TCPTest)  Time elapsed: 25.754 s  <<< FAILURE!
org.openqa.selenium.TimeoutException: 
Expected condition failed: waiting for WebParticipant[web.participant1]@1206387002#waitForSendReceiveData (tried for 20 second(s) with 500 milliseconds interval)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: 'ligi-w520', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-122-generic', java.version: '11.0.15'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 104.0.5112.79, chrome: {chromedriverVersion: 104.0.5112.29 (eff877e18f76..., userDataDir: /tmp/.com.google.Chrome.DHSLap}, goog:chromeOptions: {debuggerAddress: localhost:44715}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: LINUX, platformName: LINUX, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Session ID: 63ce5e1e4b581ca94c32997093380d0f
	at org.jitsi.meet.test.TCPTest.tcpTest(TCPTest.java:82)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   TCPTest.tcpTest:82->WebTestBase.ensureTwoParticipants:104->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:159 » Timeout
[INFO] 
[ERROR] Tests run: 440, Failures: 1, Errors: 0, Skipped: 432
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:27 min
[INFO] Finished at: 2022-08-03T13:16:01+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project jitsi-meet-torture: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/ligi/git/3rd/jitsi-meet-torture/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

What is the machine where you start chrome? Do you have enough RAM to run the chrome instances?

Thanks for the swift reply!
I have 12G total and 7.7G free - that should be enough right?
I have another machine with 32G - but hesitant to install chrome on it and unfortunately it seems to not work with chromium.

This maybe not be enough to spawn 3 browsers… I guess this may be what is happening…

Thanks! I can try on another machine that has more RAM later