Recording not working in docker based jitsi deployment

Hi

So, we have web, jibri, prosody, jicofo, jvb running on seperate kube pods. When I start a meeting and check the recordings folder, it shows empty. Also receiving the following errors:

Jibri:
[31] [hostname=jitsi-prosody.jitsi.svc.cluster.local id=jitsi-prosody.jitsi.svc.cluster.local] MucClient.lambda$getConnectAndLoginCallable$9#651: Failed to login. Disconnecting to trigger a re-connect.
org.jivesoftware.smack.sasl.SASLErrorException: SASLError using SCRAM-SHA-1: not-authorized
at org.jivesoftware.smack.SASLAuthentication.authenticationFailed(SASLAuthentication.java:286)
at org.jivesoftware.smack.AbstractXMPPConnection.lambda$new$2(AbstractXMPPConnection.java:407)
at org.jivesoftware.smack.NonzaCallback$ClassAndConsumer.accept(NonzaCallback.java:177)
at org.jivesoftware.smack.NonzaCallback$ClassAndConsumer.access$200(NonzaCallback.java:166)
at org.jivesoftware.smack.NonzaCallback.onNonzaReceived(NonzaCallback.java:46)
at org.jivesoftware.smack.AbstractXMPPConnection.parseAndProcessNonza(AbstractXMPPConnection.java:1444)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.access$1700(XMPPTCPConnection.java:131)
at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader.parsePackets(XMPPTCPConnection.java:1010)
at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader.access$700(XMPPTCPConnection.java:916)
at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader$1.run(XMPPTCPConnection.java:939)
at java.base/java.lang.Thread.run(Thread.java:829)

JVB:
hostname=jitsi-prosody.jitsi.svc.cluster.local id=shard0] MucClient.lambda$getConnectAndLoginCallable$9#640: Error connecting:

org.jivesoftware.smack.SmackException$EndpointConnectionException: The following addresses failed: ‘RFC 6120 A/AAAA Endpoint + [jitsi-prosody.jitsi.svc.cluster.local:5222] (jitsi-prosody.jitsi.svc.cluster.local/10.100.68.125:5222)’ failed because: java.net.ConnectException: Connection refused (Connection refused)

at org.jivesoftware.smack.SmackException$EndpointConnectionException.from(SmackException.java:334)

at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectUsingConfiguration(XMPPTCPConnection.java:664)

at org.jivesoftware.smack.tcp.XMPPTCPConnection.connectInternal(XMPPTCPConnection.java:849)

at org.jivesoftware.smack.AbstractXMPPConnection.connect(AbstractXMPPConnection.java:526)

at org.jitsi.xmpp.mucclient.MucClient.lambda$getConnectAndLoginCallable$9(MucClient.java:635)

at org.jitsi.retry.RetryStrategy$TaskRunner.run(RetryStrategy.java:167)

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

at java.base/java.lang.Thread.run(Thread.java:829)

Jicofo:
icofo 2023-02-07 03:33:15.118 SEVERE: [40] [xmpp_connection=client] XmppProvider.fireComponentsChanged$lambda-5$lambda-4#198: An error occurred while executing componentsChanged() on org.jitsi.jicofo.jibri.JibriDetector@2fcfeca7

java.lang.AbstractMethodError: Receiver class org.jitsi.jicofo.jibri.JibriDetector does not define or inherit an implementation of the resolved method ‘abstract void componentsChanged(java.util.Set)’ of interface org.jitsi.jicofo.xmpp.XmppProvider$Listener.

at org.jitsi.jicofo.xmpp.XmppProvider.fireComponentsChanged$lambda-5$lambda-4(XmppProvider.kt:196)

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

at java.base/java.lang.Thread.run(Thread.java:829)

What seems to be the issue here?
The normal meeting flow is working fine

Looks like the JVB and Jibri are not able to connect to Prosody. I suspect some cross Pod network shenanigans, but my k8s knowledge is limited.