Conversion Failed

SergioComeron · March 11, 2020, 5:17pm

Hi!
We are implementing jibri to stream the jitsi sessions to YouTube and it works on a machine with two cores and 4 gigabytes of ram. This configuration was short and after a time the streaming was falling. We resize a machine with 4 cores but now when launching the streaming it does not work and in the log the only thing that appears is that ffmpeg has had a conversion failed.
Anyone have any idea what might be happening?
I leave the log here:

2020-03-11 14:44:07.494 INFO: [67] org.openqa.selenium.remote.ProtocolHandshake.createSession() Detected dialect: OSS
2020-03-11 14:44:07.503 INFO: [67] org.jitsi.jibri.selenium.JibriSelenium.<init>() Starting empty call check with a timeout of PT30S
2020-03-11 14:44:07.503 FINE: [67] org.jitsi.jibri.statsd.JibriStatsDClient.incrementCounter() Incrementing statsd counter: start:live_stream
2020-03-11 14:44:07.503 INFO: [67] org.jitsi.jibri.status.JibriStatusManager.log() Busy status has changed: IDLE -> BUSY
2020-03-11 14:44:07.504 INFO: [67] org.jitsi.jibri.api.xmpp.XmppApi.updatePresence() Jibri reports its status is now JibriStatus(busyStatus=BUSY, health=OverallHealth(healthStatus=HEALTHY, details={})), publishing presence to connections
2020-03-11 14:44:07.505 INFO: [67] org.jitsi.xmpp.mucclient.MucClientManager.log() Setting a presence extension: org.jitsi.xmpp.extensions.jibri.JibriStatusPacketExt@7560e7a6
2020-03-11 14:44:07.505 FINE: [67] org.jitsi.xmpp.mucclient.MucClientManager.log() Replacing presence extension: org.jitsi.xmpp.extensions.jibri.JibriStatusPacketExt@2be63c1b
2020-03-11 14:44:07.516 FINE: [31] org.jitsi.xmpp.extensions.DefaultPacketExtensionProvider.parse() Could not add a provider for element busy-status from namespace http://jitsi.org/protocol/jibri
2020-03-11 14:44:07.516 FINE: [31] org.jitsi.xmpp.extensions.DefaultPacketExtensionProvider.parse() Could not add a provider for element health-status from namespace http://jitsi.org/protocol/health
2020-03-11 14:44:08.438 FINE: [75] org.jitsi.jibri.selenium.pageobjects.CallPage.visit() Visiting url https://meet.jitsi/prueba5#config.iAmRecorder=true&config.externalConnectUrl=null&config.startWithAudioMuted=true&config.startWithVideoMuted=true&interfaceConfig.APP_NAME="Jibri"&config.analytics.disabled=true&config.p2p.enabled=false
2020-03-11 14:44:10.698 INFO: [75] org.jitsi.jibri.selenium.pageobjects.CallPage.visit() Waited 1724 milliseconds for call page to load
2020-03-11 14:44:10.725 INFO: [75] org.jitsi.jibri.selenium.JibriSelenium.onSeleniumStateChange() Transitioning from state Starting up to Running
2020-03-11 14:44:10.726 INFO: [75] org.jitsi.jibri.service.impl.StreamingJibriService.invoke() Selenium joined the call, starting capturer
2020-03-11 14:44:10.760 INFO: [75] org.jitsi.jibri.util.JibriSubprocess.ffmpeg.launch() Starting ffmpeg with command ffmpeg -y -v info -f x11grab -draw_mouse 0 -r 30 -s 1280x720 -thread_queue_size 4096 -i :0.0+0,0 -f alsa -thread_queue_size 4096 -i plug:cloop -acodec aac -strict -2 -ar 44100 -c:v libx264 -preset veryfast -maxrate 2976k -bufsize 5952k -pix_fmt yuv420p -r 30 -crf 25 -g 60 -tune zerolatency -f flv rtmp://a.rtmp.youtube.com/live2/m34d-egre-2966-7vbq ([ffmpeg, -y, -v, info, -f, x11grab, -draw_mouse, 0, -r, 30, -s, 1280x720, -thread_queue_size, 4096, -i, :0.0+0,0, -f, alsa, -thread_queue_size, 4096, -i, plug:cloop, -acodec, aac, -strict, -2, -ar, 44100, -c:v, libx264, -preset, veryfast, -maxrate, 2976k, -bufsize, 5952k, -pix_fmt, yuv420p, -r, 30, -crf, 25, -g, 60, -tune, zerolatency, -f, flv, rtmp://a.rtmp.youtube.com/live2/m34d-egre-2966-7vbq])
2020-03-11 14:44:12.382 INFO: [79] org.jitsi.jibri.capture.ffmpeg.FfmpegCapturer.onFfmpegStateMachineStateChange() Ffmpeg capturer transitioning from state Starting up to Running
2020-03-11 14:44:12.383 INFO: [79] org.jitsi.jibri.service.impl.StreamingJibriService.onServiceStateChange() Streaming service transitioning from state Starting up to Running
2020-03-11 14:44:12.383 INFO: [79] org.jitsi.jibri.api.xmpp.XmppApi.invoke() Current service started up successfully, sending on iq <iq [to='jibribrewery@internal-muc.meet.jitsi](mailto:to='jibribrewery@internal-muc.meet.jitsi)/focus' id='Ro2Ca-54' type='set'><jibri xmlns='[http://jitsi.org/protocol/jibri' status='on](http://jitsi.org/protocol/jibri'%20status='on)'/></iq>
2020-03-11 14:44:25.768 INFO: [55] org.jitsi.jibri.selenium.JibriSelenium.run() Jibri client receive bitrates: {audio={download=0, upload=0}, download=0, upload=0, video={download=0, upload=0}}, all clients muted? false
2020-03-11 14:44:31.773 FINE: [55] org.jitsi.jibri.util.ProcessStatePublisher.ffmpeg.invoke() Process ffmpeg hasn't written in 2 seconds, publishing periodic update
2020-03-11 14:44:31.774 INFO: [79] org.jitsi.jibri.capture.ffmpeg.FfmpegCapturer.onFfmpegProcessUpdate() Ffmpeg quit abruptly.  Last output line: Conversion failed!
2020-03-11 14:44:31.774 INFO: [79] org.jitsi.jibri.capture.ffmpeg.FfmpegCapturer.onFfmpegStateMachineStateChange() Ffmpeg capturer transitioning from state Running to Error: SESSION Ffmpeg failed to start
2020-03-11 14:44:31.774 INFO: [79] org.jitsi.jibri.service.impl.StreamingJibriService.onServiceStateChange() Streaming service transitioning from state Running to Error: SESSION Ffmpeg failed to start
2020-03-11 14:44:31.775 INFO: [79] org.jitsi.jibri.api.xmpp.XmppApi.invoke() Current service had an error, sending error iq <iq [to='jibribrewery@internal-muc.meet.jitsi](mailto:to='jibribrewery@internal-muc.meet.jitsi)/focus' id='Ro2Ca-56' type='set'><jibri xmlns='[http://jitsi.org/protocol/jibri' status='off' failure_reason='error](http://jitsi.org/protocol/jibri'%20status='off'%20failure_reason='error)'/></iq>
2020-03-11 14:44:31.775 FINE: [79] org.jitsi.jibri.statsd.JibriStatsDClient.incrementCounter() Incrementing statsd counter: stop:live_stream
2020-03-11 14:44:31.775 INFO: [79] org.jitsi.jibri.JibriManager.stopService() Stopping the current service
2020-03-11 14:44:31.776 INFO: [79] org.jitsi.jibri.service.impl.StreamingJibriService.stop() Stopping capturer
2020-03-11 14:44:31.776 INFO: [79] org.jitsi.jibri.util.JibriSubprocess.ffmpeg.stop() Stopping ffmpeg process
2020-03-11 14:44:31.778 INFO: [79] org.jitsi.jibri.util.JibriSubprocess.ffmpeg.stop() ffmpeg exited with value 1
2020-03-11 14:44:31.778 INFO: [79] org.jitsi.jibri.service.impl.StreamingJibriService.stop() Stopped capturer
2020-03-11 14:44:31.779 INFO: [79] org.jitsi.jibri.service.impl.StreamingJibriService.stop() Quitting selenium
2020-03-11 14:44:31.820 INFO: [79] org.jitsi.jibri.selenium.JibriSelenium.leaveCallAndQuitBrowser() Got 13 log entries for type browser
2020-03-11 14:44:31.868 INFO: [79] org.jitsi.jibri.selenium.JibriSelenium.leaveCallAndQuitBrowser() Got 689 log entries for type driver
2020-03-11 14:44:31.991 INFO: [79] org.jitsi.jibri.selenium.JibriSelenium.leaveCallAndQuitBrowser() Got 0 log entries for type client
2020-03-11 14:44:31.992 INFO: [79] org.jitsi.jibri.selenium.JibriSelenium.leaveCallAndQuitBrowser() Leaving web call
2020-03-11 14:44:32.237 INFO: [79] org.jitsi.jibri.selenium.JibriSelenium.leaveCallAndQuitBrowser() Quitting chrome driver
2020-03-11 14:44:32.310 INFO: [79] org.jitsi.jibri.selenium.JibriSelenium.leaveCallAndQuitBrowser() Chrome driver quit
2020-03-11 14:44:32.310 INFO: [79] org.jitsi.jibri.service.impl.StreamingJibriService.stop() Quit selenium
2020-03-11 14:44:32.310 INFO: [79] org.jitsi.jibri.status.JibriStatusManager.log() Busy status has changed: BUSY -> IDLE
2020-03-11 14:44:32.311 INFO: [79] org.jitsi.jibri.api.xmpp.XmppApi.updatePresence() Jibri reports its status is now JibriStatus(busyStatus=IDLE, health=OverallHealth(healthStatus=HEALTHY, details={})), publishing presence to connections
2020-03-11 14:44:32.311 INFO: [79] org.jitsi.xmpp.mucclient.MucClientManager.log() Setting a presence extension: org.jitsi.xmpp.extensions.jibri.JibriStatusPacketExt@7544df0
2020-03-11 14:44:32.311 FINE: [79] org.jitsi.xmpp.mucclient.MucClientManager.log() Replacing presence extension: org.jitsi.xmpp.extensions.jibri.JibriStatusPacketExt@7560e7a6
2020-03-11 14:44:32.315 FINE: [31] org.jitsi.xmpp.extensions.DefaultPacketExtensionProvider.parse() Could not add a provider for element busy-status from namespace http://jitsi.org/protocol/jibri
2020-03-11 14:44:32.316 FINE: [31] org.jitsi.xmpp.extensions.DefaultPacketExtensionProvider.parse() Could not add a provider for element health-status from namespace http://jitsi.org/protocol/health

bbaldino · March 11, 2020, 5:22pm

We’ve started seeing this recently as well. If you check the ffmpeg log, you’ll likely see that before the Conversion failed! line there was a line about Connection reset by peer. We’re not sure what the root cause of this is (something on our side, aws, or YouTube’s) and haven’t had a chance to dig into it yet.

bbaldino · March 11, 2020, 5:48pm

Where were you running this, btw? Was this your own deployment? Is the Jibri running on an amazon machine? Have you been running Jibri there for a while and only seen this error recently?

SergioComeron · March 11, 2020, 6:07pm

Hi!
We run this on Google Cloud and deployment width the docker of jitsi.
Before the error we can run correctly on a machine with two cores. After resizing the machine to 4 cores it never worked again.

bbaldino · March 11, 2020, 6:40pm

So it’s happening every time? Does it happen right away when the stream starts or after some time? If after some time, is the amount of time it takes to fail consistent?

SergioComeron · March 12, 2020, 4:20pm

Hi!
yes, it’s happening every time and after some time. I would say that time is not consistent