Jibri Docker on AWS - getting "The following addresses failed" error from 2nd instance

I have Jisti+Jicofo+Prosody on one EC2 instance and Jibri Docker on two other EC2 instances. When I start two meetings and start recording in one meeting, recording works fine. But when I start recording in 2nd meeting while the 1st one is progress, I get an error from 2nd Jibri instance -

All 3 EC2 instances have their Elastic IP and share a common set of records on Route53. What could be the reason for this error in 2nd instance ? Appreciate your help.

Jibri 2022-11-11 04:17:43.039 WARNING: [29] [hostname=meet.MYDOMAIN.com id=meet.MYDOMAIN.com] MucClient.lambda$getConnectAndLoginCallable$7#630: Error connecting:
org.jivesoftware.smack.SmackException$EndpointConnectionException: The following addresses failed: ‘RFC 6120 A/AAAA Endpoint + [meet.MYDOMAIN.com:5222] (meet.MYDOMAIN.com/ELASTIC IP OF JITSI/JICOFO:5222)’ failed because: java.net.SocketTimeoutException: connect timed out
at org.jivesoftware.smack.SmackException$EndpointConnectionException.from(SmackException.java:334)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:664)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:849)
at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:526)

Are they both able to reach the configured XMPP server? I’d suggest you simplify and try to connect using something like telnet or netcat.

I found the issue and fixed it. You will not believe how silly the cause was - the inbound rule for Jitsi instance was restricting access to only one elastic IP !! That was why it was working and every other instance was failing !!! One of the silliest reasons possible. Thank you very much for patiently checking the issue and pointing out the logic.

Glad you got it fixed! A general note to whoever will be reading this in future - firewall restriction in the cloud are very easy to overlook and are often the cause of problems, very often.

On most platforms by default you always have all incoming traffic blocked and it’s a good thing, but you have to pay special attention to designing the security rules correctly. Remember you have several layers of restrictions - vpcs, security groups, host firewalls, docker networking - if in trouble, double check one by one.

As @saghul said, telnet and nc are your best friends if something goes wrong. You can also check the access from within the docker containers, if needed.

2 Likes

A question on Jibri instance status - how do I monitor status of a Jibri instance to know if it is available or if it is busy recording at that moment ? Any endpoint call that I can make to get status of a Jibri instance using Nickname ? I am trying to find out a good approach for an auto-scaling policy, to add/reduce instances based on count of free Jibri instances.

You may ask jicofo

curl -s http://127.0.0.1:8888/stats | jq ".jibri_detector.count"
curl -s http://127.0.0.1:8888/stats | jq ".jibri_detector.available"
1 Like

Great thanks, works perfectly

Additionally to count/availability from Jicofo, you can also query each individual Jibri, if you need to:

curl -s http://IP_ADDRESS:3333/jibri/api/v1.0/health

By default it’s 3333 tcp from outside and 2222 tcp from localhost - both configurable in jibri.conf.

Very good, thank you