Page crash in Jibri Docker

Hi everyone.
I setup a complete jitsi meet platform on a debian 10 vm:

ii  jitsi-meet                           1.0.4101-1                   all          WebRTC JavaScript video conferences
ii  jitsi-meet-prosody                   1.0.3729-1                   all          Prosody configuration for Jitsi Meet
ii  jitsi-meet-web                       1.0.3729-1                   all          WebRTC JavaScript video conferences
ii  jitsi-meet-web-config                1.0.3729-1                   all          Configuration for web serving of Jitsi Meet
ii  jitsi-videobridge                    1126-1                       amd64        WebRTC compatible Selective Forwarding Unit (SFU)
ii  prosody                              0.11.2-1                     amd64        Lightweight Jabber/XMPP server

I tweaked prosody configuration to force the videoconference’s moderators to authenticate (as prosody users).
All is working smoothly. I really thank all the contributors for this amazing software.

On a different VM (Ubuntu 16.04), i setup a docker environnment (19.03.8) and installed one Jibri docker (2020-05-02T13:41:00.513369517Z).
Jibri correctly registers on prosody and start recording the videocall session while asked to do so… but i think i am getting a crash:

Summary
2020-05-11 16:08:08.369 FINE: [38] org.jitsi.xmpp.mucclient.MucClient.log() Received an IQ with type set: IQ Stanza (jibri http://jitsi.org/protocol/jibri) [to=jibri@auth.coyotemeet.acme.it/5XqgQAtH,from=jibribrewery@internal.auth.coyotemeet.acme.it/focus,id=amlicmlAYXV0aC5sb3F1YWNlbWVldC5uZW1zLml0LzVYcWdRQXRIAGZMWnBHLTU3OACuV0P7bK/Iqf5TvQABl1Xr,type=set,]
2020-05-11 16:08:08.376 INFO: [38] org.jitsi.jibri.api.xmpp.XmppApi.handleJibriIq() Received JibriIq <iq to='jibri@auth.coyotemeet.acme.it/5XqgQAtH' from='jibribrewery@internal.auth.coyotemeet.acme.it/focus' id='amlicmlAYXV0aC5sb3F1YWNlbWVldC5uZW1zLml0LzVYcWdRQXRIAGZMWnBHLTU3OACuV0P7bK/Iqf5TvQABl1Xr' type='set'><jibri xmlns='http://jitsi.org/protocol/jibri' action='start' recording_mode='file' room='test@conference.coyotemeet.acme.it' session_id='jraoszbqbrvpylbb' app_data='{"file_recording_metadata":{"share":true}}'/></iq> from environment [MucClient id=coyotemeet.acme.it hostname=coyotemeet.acme.it]
2020-05-11 16:08:08.383 INFO: [38] org.jitsi.jibri.api.xmpp.XmppApi.handleStartJibriIq() Received start request
2020-05-11 16:08:08.398 INFO: [38] org.jitsi.jibri.api.xmpp.XmppApi.handleStartJibriIq() Sending 'pending' response to start IQ
2020-05-11 16:08:08.406 INFO: [39] org.jitsi.jibri.api.xmpp.XmppApi.run() Starting service
2020-05-11 16:08:08.550 INFO: [39] org.jitsi.jibri.api.xmpp.XmppApi.handleStartService() Parsed call url info: CallUrlInfo(baseUrl=https://coyotemeet.acme.it, callName=test, urlParams=[])
2020-05-11 16:08:08.553 INFO: [39] org.jitsi.jibri.JibriManager.startFileRecording() Starting a file recording with params: FileRecordingRequestParams(callParams=CallParams(callUrlInfo=CallUrlInfo(baseUrl=https://coyotemeet.acme.it, callName=test, urlParams=[])), sessionId=jraoszbqbrvpylbb, callLoginParams=XmppCredentials(domain=recorder.coyotemeet.acme.it, username=recorder, password=2LcAq5cxBUZ8Anx6)) finalize script path: /config/finalize.sh and recordings directory: /config/recordings
2020-05-11 16:08:10.897 INFO: [39] org.openqa.selenium.remote.ProtocolHandshake.createSession() Detected dialect: OSS
2020-05-11 16:08:10.958 INFO: [39] org.jitsi.jibri.selenium.JibriSelenium.<init>() Starting empty call check with a timeout of PT30S
2020-05-11 16:08:11.000 FINE: [39] org.jitsi.jibri.capture.ffmpeg.FfmpegCapturer.<init>() Detected os as OS: LINUX
2020-05-11 16:08:11.022 INFO: [39] org.jitsi.jibri.service.impl.FileRecordingJibriService.<init>() Writing recording to /config/recordings/jraoszbqbrvpylbb
2020-05-11 16:08:11.027 FINE: [39] org.jitsi.jibri.statsd.JibriStatsDClient.incrementCounter() Incrementing statsd counter: start:recording
2020-05-11 16:08:11.038 INFO: [39] org.jitsi.jibri.status.JibriStatusManager.log() Busy status has changed: IDLE -> BUSY
2020-05-11 16:08:11.039 INFO: [39] org.jitsi.jibri.api.xmpp.XmppApi.updatePresence() Jibri reports its status is now JibriStatus(busyStatus=BUSY, health=OverallHealth(healthStatus=HEALTHY, details={})), publishing presence to connections
2020-05-11 16:08:11.040 INFO: [39] org.jitsi.xmpp.mucclient.MucClientManager.log() Setting a presence extension: org.jitsi.xmpp.extensions.jibri.JibriStatusPacketExt@45c4ce0d
2020-05-11 16:08:11.041 FINE: [39] org.jitsi.xmpp.mucclient.MucClientManager.log() Replacing presence extension: org.jitsi.xmpp.extensions.jibri.JibriStatusPacketExt@771158fb
2020-05-11 16:08:11.048 FINE: [30] org.jitsi.xmpp.extensions.DefaultPacketExtensionProvider.parse() Could not add a provider for element busy-status from namespace http://jitsi.org/protocol/jibri
2020-05-11 16:08:11.053 FINE: [30] org.jitsi.xmpp.extensions.DefaultPacketExtensionProvider.parse() Could not add a provider for element health-status from namespace http://jitsi.org/protocol/health
2020-05-11 16:08:13.069 FINE: [49] org.jitsi.jibri.selenium.pageobjects.CallPage.visit() Visiting url https://coyotemeet.acme.it/test#config.iAmRecorder=true&config.externalConnectUrl=null&config.startWithAudioMuted=true&config.startWithVideoMuted=true&interfaceConfig.APP_NAME="Jibri"&config.analytics.disabled=true&config.p2p.enabled=false
2020-05-11 16:08:17.433 INFO: [49] org.jitsi.jibri.selenium.pageobjects.CallPage.visit() Waited 3018 milliseconds for call page to load
2020-05-11 16:08:17.504 INFO: [49] org.jitsi.jibri.selenium.JibriSelenium.onSeleniumStateChange() Transitioning from state Starting up to Running
2020-05-11 16:08:17.511 INFO: [49] org.jitsi.jibri.service.impl.FileRecordingJibriService.invoke() Selenium joined the call, starting the capturer
2020-05-11 16:08:17.586 INFO: [49] org.jitsi.jibri.util.JibriSubprocess.ffmpeg.launch() Starting ffmpeg with command ffmpeg -y -v info -f x11grab -draw_mouse 0 -r 30 -s 1280x720 -thread_queue_size 4096 -i :0.0+0,0 -f alsa -thread_queue_size 4096 -i plug:cloop -acodec aac -strict -2 -ar 44100 -c:v libx264 -preset veryfast -profile:v main -level 3.1 -pix_fmt yuv420p -r 30 -crf 25 -g 60 -tune zerolatency -f mp4 /config/recordings/jraoszbqbrvpylbb/test_2020-05-11-16-08-11.mp4 ([ffmpeg, -y, -v, info, -f, x11grab, -draw_mouse, 0, -r, 30, -s, 1280x720, -thread_queue_size, 4096, -i, :0.0+0,0, -f, alsa, -thread_queue_size, 4096, -i, plug:cloop, -acodec, aac, -strict, -2, -ar, 44100, -c:v, libx264, -preset, veryfast, -profile:v, main, -level, 3.1, -pix_fmt, yuv420p, -r, 30, -crf, 25, -g, 60, -tune, zerolatency, -f, mp4, /config/recordings/jraoszbqbrvpylbb/test_2020-05-11-16-08-11.mp4])
2020-05-11 16:08:19.336 INFO: [53] org.jitsi.jibri.capture.ffmpeg.FfmpegCapturer.onFfmpegStateMachineStateChange() Ffmpeg capturer transitioning from state Starting up to Running
2020-05-11 16:08:19.338 INFO: [53] org.jitsi.jibri.service.impl.FileRecordingJibriService.onServiceStateChange() File recording service transitioning from state Starting up to Running
2020-05-11 16:08:19.351 INFO: [53] org.jitsi.jibri.api.xmpp.XmppApi.invoke() Current service started up successfully, sending on iq <iq to='jibribrewery@internal.auth.coyotemeet.acme.it/focus' id='agEUR-35' type='set'><jibri xmlns='http://jitsi.org/protocol/jibri' status='on'/></iq>
2020-05-11 16:08:38.954 SEVERE: [50] org.jitsi.jibri.selenium.JibriSelenium.invoke() Error while running call status checks: org.openqa.selenium.WebDriverException: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
  (Session info: chrome=78.0.3904.97)
  (Driver info: chromedriver=78.0.3904.105 (60e2d8774a8151efa6a00b1f358371b1e0e07ee2-refs/branch-heads/3904@{#877}),platform=Linux 4.4.0-178-generic x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
System info: host: 'e249e1cd76da', ip: '172.22.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-178-generic', java.version: '1.8.0_252'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: chrome, chrome: {chromedriverVersion: 78.0.3904.105 (60e2d8774a81..., userDataDir: /tmp/.com.google.Chrome.7q8dN6}, cssSelectorsEnabled: true, databaseEnabled: false, goog:chromeOptions: {debuggerAddress: localhost:34060}, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: LINUX, platformName: LINUX, proxy: Proxy(), rotatable: false, setWindowRect: true, strictFileInteractability: false, takesHeapSnapshot: true, takesScreenshot: true, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unexpectedAlertBehaviour: ignore, unhandledPromptBehavior: ignore, version: 78.0.3904.97, webStorageEnabled: true}
Session ID: cd3328c7ed3afc9e78787a8153585dbe with stack:
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214)
org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166)
org.openqa.selenium.remote.http.JsonHttpResponseCodec.reconstructValue(JsonHttpResponseCodec.java:40)
org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:80)
org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:44)
org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:543)
org.openqa.selenium.remote.RemoteWebDriver.executeScript(RemoteWebDriver.java:480)
org.jitsi.jibri.selenium.pageobjects.CallPage.getNumParticipants(CallPage.kt:71)
org.jitsi.jibri.selenium.status_checks.EmptyCallStatusCheck.isCallEmpty(EmptyCallStatusCheck.kt:44)
org.jitsi.jibri.selenium.status_checks.EmptyCallStatusCheck.run(EmptyCallStatusCheck.kt:29)
org.jitsi.jibri.selenium.JibriSelenium$startRecurringCallStatusChecks$1$event$1.invoke(JibriSelenium.kt:201)
org.jitsi.jibri.selenium.JibriSelenium$startRecurringCallStatusChecks$1$event$1.invoke(JibriSelenium.kt:112)
kotlin.sequences.TransformingSequence$iterator$1.next(Sequences.kt:172)
org.jitsi.jibri.selenium.JibriSelenium$startRecurringCallStatusChecks$1.invoke(JibriSelenium.kt:301)
org.jitsi.jibri.selenium.JibriSelenium$startRecurringCallStatusChecks$1.invoke(JibriSelenium.kt:112)
org.jitsi.jibri.util.extensions.SchedulerExecutorServiceExtsKt$sam$java_lang_Runnable$0.run(SchedulerExecutorServiceExts.kt)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

2020-05-11 16:08:38.969 INFO: [49] org.jitsi.jibri.selenium.JibriSelenium.onSeleniumStateChange() Transitioning from state Running to Error: SESSION Chrome hung
2020-05-11 16:08:38.972 INFO: [49] org.jitsi.jibri.service.impl.FileRecordingJibriService.onServiceStateChange() File recording service transitioning from state Running to Error: SESSION Chrome hung
2020-05-11 16:08:38.978 INFO: [49] org.jitsi.jibri.api.xmpp.XmppApi.invoke() Current service had an error, sending error iq <iq to='jibribrewery@internal.auth.coyotemeet.acme.it/focus' id='agEUR-37' type='set'><jibri xmlns='http://jitsi.org/protocol/jibri' status='off' failure_reason='error'/></iq>
2020-05-11 16:08:38.987 FINE: [49] org.jitsi.jibri.statsd.JibriStatsDClient.incrementCounter() Incrementing statsd counter: stop:recording
2020-05-11 16:08:38.993 INFO: [49] org.jitsi.jibri.JibriManager.stopService() Stopping the current service
2020-05-11 16:08:38.998 INFO: [49] org.jitsi.jibri.service.impl.FileRecordingJibriService.stop() Stopping capturer
2020-05-11 16:08:38.999 INFO: [49] org.jitsi.jibri.util.JibriSubprocess.ffmpeg.stop() Stopping ffmpeg process
2020-05-11 16:08:39.345 INFO: [49] org.jitsi.jibri.util.JibriSubprocess.ffmpeg.stop() ffmpeg exited with value 255
2020-05-11 16:08:39.347 INFO: [53] org.jitsi.jibri.capture.ffmpeg.FfmpegCapturer.onFfmpegStateMachineStateChange() Ffmpeg capturer transitioning from state Running to Finished
2020-05-11 16:08:39.348 INFO: [49] org.jitsi.jibri.service.impl.FileRecordingJibriService.stop() Quitting selenium
2020-05-11 16:08:39.360 INFO: [49] org.jitsi.jibri.service.impl.FileRecordingJibriService.stop() Participants in this recording: []

Is it a Chrome issue or what?
By the way, i got a small (3s) mp4 file with some video/audio… meaning that the recording started and lasted a few seconds at least.

Thanks a lot for your help!
S

Hello everyone.
Just an update.
I cloned the https://github.com/jitsi/docker-jitsi-meet project, edited the jibri Dockerfile, enabling latest chrome version, and built everything. Then I started the jibri image.

Same error:
2020-05-15 02:31:59.359 INFO: [54] org.jitsi.jibri.api.xmpp.XmppApi.invoke() Current service started up successfully, sending on iq
2020-05-15 02:32:12.922 INFO: [51] org.jitsi.jibri.selenium.JibriSelenium.run() Jibri client receive bitrates: {audio={download=0, upload=0}, download=2699, upload=0, video={download=2699, upload=0}}, all clients muted? false
2020-05-15 02:32:23.116 FINE: [51] org.jitsi.jibri.util.ProcessStatePublisher.ffmpeg.invoke() Process ffmpeg hasn’t written in 2 seconds, publishing periodic update
2020-05-15 02:32:23.159 FINE: [51] org.jitsi.jibri.util.ProcessStatePublisher.ffmpeg.invoke() Process ffmpeg hasn’t written in 2 seconds, publishing periodic update
2020-05-15 02:32:23.172 FINE: [51] org.jitsi.jibri.util.ProcessStatePublisher.ffmpeg.invoke() Process ffmpeg hasn’t written in 2 seconds, publishing periodic update
2020-05-15 02:32:23.177 FINE: [51] org.jitsi.jibri.util.ProcessStatePublisher.ffmpeg.invoke() Process ffmpeg hasn’t written in 2 seconds, publishing periodic update
2020-05-15 02:32:28.764 FINE: [51] org.jitsi.jibri.util.ProcessStatePublisher.ffmpeg.invoke() Process ffmpeg hasn’t written in 2 seconds, publishing periodic update
2020-05-15 02:32:45.459 SEVERE: [51] org.jitsi.jibri.selenium.JibriSelenium.invoke() Error while running call status checks: org.openqa.selenium.WebDriverException: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: chrome=81.0.4044.138)
(Driver info: chromedriver=81.0.4044.138 (8c6c7ba89cc9453625af54f11fd83179e23450fa-refs/branch-heads/4044@{#999}),platform=Linux 4.4.0-178-generic x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
Build info: version: ‘unknown’, revision: ‘unknown’, time: ‘unknown’
System info: host: ‘3c34979b067c’, ip: ‘172.23.0.2’, os.name: ‘Linux’, os.arch: ‘amd64’, os.version: ‘4.4.0-178-generic’, java.version: ‘1.8.0_252’
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: chrome, chrome: {chromedriverVersion: 81.0.4044.138 (8c6c7ba89cc9…, userDataDir: /tmp/.com.google.Chrome.oM1F9s}, cssSelectorsEnabled: true, databaseEnabled: false, goog:chromeOptions: {debuggerAddress: localhost:43036}, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: LINUX, platformName: LINUX, proxy: Proxy(), rotatable: false, setWindowRect: true, strictFileInteractability: false, takesHeapSnapshot: true, takesScreenshot: true, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unexpectedAlertBehaviour: ignore, unhandledPromptBehavior: ignore, version: 81.0.4044.138, webStorageEnabled: true, webauthn:virtualAuthenticators: true}
Session ID: 3f4b9e546dbf450d6d17b33e7d967147 with stack:
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
[…]

Could be related to my server (jitsi-meet) version?
Or could be something related in my docker setup?

Thanks a lot.
S.

Ok. Solved.
Chrome was killed due to lack of available RAM.

In /var/log/kern.log:
May 18 13:56:42 dockerhost01 kernel: [1195769.169665] Out of memory: Kill process 65030 (chrome) score 344 or sacrifice child
May 18 13:56:42 dockerhost01 kernel: [1195769.169724] Killed process 65030 (chrome) total-vm:4903684kB, anon-rss:73288kB, file-rss:6048kB

I will try again on a host with bigger RAM.

Thanks anyway.

Sandro,

Did you solve it after increase the RAM on host? How much are you using?

I’m afraid this is not enough. https://github.com/jitsi/jibri/issues/267

Hey guys. I had the similar issue and I increasing RAM fixed it. Thank y’all

Hi guys.
I was trying this on a small test vm with 1GB RAM.
I actually just read that it should be planned 4GB RAM for each running jibri docker.
So… i’ll try it on a bigger VM asap