Jicofo is not able to create conference

I’ve installed Jitsi from source and followed a combination of instructions from the manual installation steps, readme’s, and by looking at the scripts used by Docker and the Debian package.

This is what I end up with:

Prosody:

VirtualHost "meet.ent.domain.tld"
    authentication = "anonymous"
    ssl = {
        key = "/var/db/prosody/meet.ent.domain.tld.key";
        certificate = "/var/db/prosody/meet.ent.domain.tld.crt";
    }
    modules_enabled = {
        "bosh";
        "pubsub";
    }
    c2s_require_encryption = false
	
VirtualHost "auth.meet.ent.domain.tld"
    ssl = {
        key = "/var/db/prosody/meet.ent.domain.tld.key";
        certificate = "/var/db/prosody/meet.ent.domain.tld.crt";
    }
    authentication = "internal_plain"
	
	admins = { "focus@auth.meet.ent.domain.tld" }
	
Component "conference.meet.ent.domain.tld" "muc"
Component "internal.auth.meet.ent.domain.tld" "muc"
	storage = "memory"
	modules_enabled = { "ping"; }
	admins = { "focus@auth.meet.ent.domain.tld", "jvb@auth.meet.ent.domain.tld" }
Component "jitsi-videobridge.meet.ent.domain.tld"
    component_secret = "password123"
Component "focus.meet.ent.domain.tld"
    component_secret = "password123"

config.js

var config = {

    hosts: {

        domain: 'meet.ent.domain.tld',
		bridge: 'jitsi-videobridge.meet.ent.domain.tld',
        focus: 'focus.meet.ent.domain.tld',
        muc: 'conference.meet.ent.domain.tld'
    },

    bosh: '//meet.ent.domain.tld/http-bind',
    clientNode: 'http://jitsi.org/jitsimeet',

sip-communicator.properties (for jitsi-videobridge)

org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS=172.16.7.248
org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS=X.X.X.X

# Enable broadcasting stats/presence in a MUC
org.jitsi.videobridge.ENABLE_STATISTICS=true
org.jitsi.videobridge.STATISTICS_TRANSPORT=muc

# Connect to the first XMPP server
org.jitsi.videobridge.xmpp.user.xmppserver1.HOSTNAME=localhost
org.jitsi.videobridge.xmpp.user.xmppserver1.DOMAIN=auth.meet.ent.domain.tld
org.jitsi.videobridge.xmpp.user.xmppserver1.USERNAME=jvb
org.jitsi.videobridge.xmpp.user.xmppserver1.PASSWORD=password123
org.jitsi.videobridge.xmpp.user.xmppserver1.MUC_JIDS=JvbBrewery@internal.auth.meet.ent.domain.tld
org.jitsi.videobridge.xmpp.user.xmppserver1.MUC_NICKNAME=<removed>
org.jitsi.videobridge.xmpp.user.xmppserver1.DISABLE_CERTIFICATE_VERIFICATION=true

sip-communicator.properties (for jicofo)

org.jitsi.jicofo.BRIDGE_MUC=JvbBrewery@internal.auth.meet.ent.domain.tld
org.jitsi.jicofo.ALWAYS_TRUST_MODE_ENABLED=true

Startup command for jicofo

JVB_XMPP_HOST=localhost
JVB_XMPP_DOMAIN=meet.ent.domain.tld
JVB_XMPP_PORT=5347
JVB_XMPP_SECRET=password123p
JVB_XMPP_USER_DOMAIN=auth.meet.ent.domain.tld
JVB_XMPP_USER_NAME=focus
JVB_XMPP_USER_SECRET=password123
MAX_MEMORY=3072m

java -Xmx${MAX_MEMORY} \
                -XX:+HeapDumpOnOutOfMemoryError \
                -XX:HeapDumpPath=/tmp \
                -Djava.util.logging.config.file=/etc/jitsi/jicofo/logging.properties \
                -Dnet.java.sip.communicator.SC_HOME_DIR_LOCATION=/etc/jitsi \
                -Dnet.java.sip.communicator.SC_HOME_DIR_NAME=jicofo \
                -Dnet.java.sip.communicator.SC_LOG_DIR_LOCATION=/var/log/ \
		-Djavax.net.ssl.trustStore=/etc/jitsi/jicofo/truststore.jks \
                -cp /path/to/jcofo.jar \
                org.jitsi.jicofo.Main \
                --host=${JVB_XMPP_HOST} \
                --domain=${JVB_XMPP_DOMAIN} \
                --port=${JVB_XMPP_PORT} \
                --secret=${JVB_XMPP_SECRET} \
		--user_domain=${JVB_XMPP_USER_DOMAIN} \
		--user_name=${JVB_XMPP_USER_NAME} \
		--user_password=${JVB_XMPP_USER_SECRET}

Startup command for jitsi-videobridge

JVB_XMPP_HOST=localhost
JVB_XMPP_DOMAIN=meet.ent.domain.tld
JVB_XMPP_PORT=5347
JVB_XMPP_SECRET=password123
VIDEOBRIDGE_MAX_MEMORY=3072m

java -Xmx${VIDEOBRIDGE_MAX_MEMORY} \
                -XX:+UseConcMarkSweepGC \
                -XX:+HeapDumpOnOutOfMemoryError \
                -XX:HeapDumpPath=/tmp \
                -Djava.util.logging.config.file=/etc/jitsi/videobridge/logging.properties \
                -Dnet.java.sip.communicator.SC_HOME_DIR_LOCATION=/etc/jitsi \
                -Dnet.java.sip.communicator.SC_HOME_DIR_NAME=videobridge \
                -Dnet.java.sip.communicator.SC_LOG_DIR_LOCATION=/var/log/ \
                -cp /path/to/jitsi-videobridge.jar} \
                org.jitsi.videobridge.MainKt \
                --host=${JVB_XMPP_HOST} \
                --domain=${JVB_XMPP_DOMAIN} \
                --port=${JVB_XMPP_PORT} \
                --secret=${JVB_XMPP_SECRET} \
                --apis=xmpp,rest

Both jicofo and jitsi-videobridge startup correctly (more or less)

icofo 2020-12-18 18:17:35.364 INFO: [11] org.eclipse.jetty.util.log.initialized() Logging initialized @1157ms to org.eclipse.jetty.util.log.Slf4jLog
Jicofo 2020-12-18 18:17:35.489 INFO: [11] org.eclipse.jetty.server.Server.doStart() jetty-9.4.z-SNAPSHOT; built: 2019-02-15T16:53:49.381Z; git: eb70b240169fcf1abbd86af36482d1c49826fa0b; jvm 1.8.0_265-b01
Jicofo 2020-12-18 18:17:38.083 INFO: [11] org.eclipse.jetty.server.handler.ContextHandler.doStart() Started o.e.j.s.ServletContextHandler@338f309e{/,null,AVAILABLE}
Jicofo 2020-12-18 18:17:38.098 INFO: [11] org.eclipse.jetty.server.AbstractConnector.doStart() Started ServerConnector@2fb43f06{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
Jicofo 2020-12-18 18:17:38.098 INFO: [11] org.eclipse.jetty.server.Server.doStart() Started @3894ms
Jicofo 2020-12-18 18:17:38.105 INFO: [11] org.jitsi.jicofo.health.Health.log() Health checks are disabled.
Jicofo 2020-12-18 18:17:38.106 INFO: [1] org.jitsi.impl.configuration.ConfigurationServiceImpl.log() org.jitsi.jicofo.BRIDGE_MUC=JvbBrewery@internal.auth.meet.ent.domain.tld
Jicofo 2020-12-18 18:17:38.106 INFO: [1] org.jitsi.impl.configuration.ConfigurationServiceImpl.log() org.jitsi.jicofo.ALWAYS_TRUST_MODE_ENABLED=true
Jicofo 2020-12-18 18:17:38.111 INFO: [1] org.jitsi.xmpp.component.ComponentBase.log() Component org.jitsi.jicofo. config: 
Jicofo 2020-12-18 18:17:38.112 INFO: [1] org.jitsi.xmpp.component.ComponentBase.log()   ping interval: 10000 ms
Jicofo 2020-12-18 18:17:38.112 INFO: [1] org.jitsi.xmpp.component.ComponentBase.log()   ping timeout: 5000 ms
Jicofo 2020-12-18 18:17:38.112 INFO: [1] org.jitsi.xmpp.component.ComponentBase.log()   ping threshold: 3
2020-12-18 18:17:29.549 INFO: [1] JitsiConfig.<clinit>#46: Initialized newConfig: merge of system properties,system properties,reference.conf @ jar:file:/usr/share/java/classes/jitsi-videobridge.jar!/reference.conf: 1
2020-12-18 18:17:29.556 INFO: [1] ReadOnlyConfigurationService.reloadConfiguration#51: loading config file at path /etc/jitsi/videobridge/sip-communicator.properties
2020-12-18 18:17:29.558 INFO: [1] JitsiConfig.<clinit>#67: Initialized legacyConfig: sip communicator props (no description provided)
2020-12-18 18:17:29.560 INFO: [1] JitsiConfig$Companion.reloadNewConfig#83: Reloading the Typesafe config source (previously reloaded 0 times).
2020-12-18 18:17:29.904 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.initialize: Using AwsCandidateHarvester.
2020-12-18 18:17:30.097 WARNING: [12] MucClient.createXMPPTCPConnectionConfiguration#116: Disabling certificate verification!
2020-12-18 18:17:30.113 INFO: [1] LastNReducer.<init>#60: LastNReducer with reductionScale: 0.75 recoverScale: 1.25 impactTime: PT1M minLastN: 0 maxEnforcedLastN: 40
2020-12-18 18:17:30.116 INFO: [1] TaskPools.<clinit>#81: TaskPools detected 4 processors, creating the CPU pool with that many threads
2020-12-18 18:17:30.135 WARNING: [1] UlimitCheck.printUlimits#111: Running with open files limit null (hard null), thread limit null (hard null). These values are too low and they will limit the number of participants that the bridge can serve simultaneously.
2020-12-18 18:17:30.136 INFO: [1] VideobridgeExpireThread.start#88: Starting with 60 second interval.
2020-12-18 18:17:30.137 INFO: [1] HealthChecker.start#118: Started with interval=10000, timeout=PT30S, maxDuration=PT3S, stickyFailures=false.
2020-12-18 18:17:30.174 INFO: [1] MainKt.main#101: Not starting CallstatsService, disabled in configuration.
2020-12-18 18:17:30.177 INFO: [1] MainKt.main#119: Not starting public http server
2020-12-18 18:17:30.179 INFO: [1] MainKt.main#128: Starting private http server
2020-12-18 18:17:30.260 INFO: [1] org.eclipse.jetty.util.log.Log.initialized: Logging initialized @1196ms to org.eclipse.jetty.util.log.JavaUtilLog
2020-12-18 18:17:30.344 INFO: [1] org.eclipse.jetty.server.Server.doStart: jetty-9.4.z-SNAPSHOT; built: 2019-02-15T16:53:49.381Z; git: eb70b240169fcf1abbd86af36482d1c49826fa0b; jvm 1.8.0_265-b01
2020-12-18 18:17:30.391 INFO: [12] [hostname=localhost id=xmppserver1] MucClient$1.connected#271: Connected.
2020-12-18 18:17:30.391 INFO: [12] [hostname=localhost id=xmppserver1] MucClient.lambda$getConnectAndLoginCallable$7#680: Logging in.
2020-12-18 18:17:30.441 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.maybeAdd: Discarding a mapping harvester: org.ice4j.ice.harvest.AwsCandidateHarvester, face=null, mask=null
2020-12-18 18:17:30.441 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.initialize: Using org.ice4j.ice.harvest.MappingCandidateHarvester, face=/172.16.7.248, mask=/63.247.147.165
2020-12-18 18:17:30.441 INFO: [11] org.ice4j.ice.harvest.MappingCandidateHarvesters.initialize: Initialized mapping harvesters (delay=860ms).  stunDiscoveryFailed=false
2020-12-18 18:17:30.482 INFO: [12] [hostname=localhost id=xmppserver1] MucClient$MucWrapper.join#788: Joined MUC: jvbbrewery@internal.auth.meet.ent.domain.tld
2020-12-18 18:17:31.751 WARNING: [1] org.glassfish.jersey.internal.inject.Providers.checkProviderRuntime: A provider org.jitsi.rest.Version registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.jitsi.rest.Version will be ignored. 
2020-12-18 18:17:31.751 WARNING: [1] org.glassfish.jersey.internal.inject.Providers.checkProviderRuntime: A provider org.jitsi.rest.Health registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.jitsi.rest.Health will be ignored. 
2020-12-18 18:17:31.909 INFO: [1] org.eclipse.jetty.server.handler.ContextHandler.doStart: Started o.e.j.s.ServletContextHandler@66238be2{/,null,AVAILABLE}
2020-12-18 18:17:31.923 INFO: [1] org.eclipse.jetty.server.AbstractConnector.doStart: Started ServerConnector@51684e4a{HTTP/1.1,[http/1.1]}{127.0.0.1:8080}
2020-12-18 18:17:31.924 INFO: [1] org.eclipse.jetty.server.Server.doStart: Started @2861ms
2020-12-18 18:17:40.142 INFO: [16] HealthChecker.run#170: Performed a successful health check in PT0.001S. Sticky failure: false
2020-12-18 18:17:50.143 INFO: [16] HealthChecker.run#170: Performed a successful health check in PT0S. Sticky failure: false
2020-12-18 18:18:00.143 INFO: [16] HealthChecker.run#170: Performed a successful health check in PT0S. Sticky failure: false

Now, I try to host a meeting and join a room, jicofo fails with:

Jicofo 2020-12-18 17:33:03.294 INFO: [52] org.jitsi.jicofo.xmpp.FocusComponent.log() Focus request for room: test@conference.meet.ent.domain.tld
Jicofo 2020-12-18 17:33:03.294 INFO: [52] org.jitsi.jicofo.FocusManager.log() Created new focus for test@conference.meet.ent.domain.tld@auth.meet.ent.domain.tld. Conference count 1,options: channelLastN=-1 enableLipSync=false openSctp=true disableRtx=false 
Jicofo 2020-12-18 17:33:03.295 SEVERE: [52] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() disposeConference error
Jicofo 2020-12-18 17:33:03.295 SEVERE: [52] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Chat room already left!
Jicofo 2020-12-18 17:33:03.295 INFO: [52] org.jitsi.jicofo.FocusManager.log() Disposed conference for room: test@conference.meet.ent.domain.tld conference count: 0
Jicofo 2020-12-18 17:33:03.296 INFO: [52] org.jitsi.jicofo.FocusManager.log() Exception while trying to start the conference
Jicofo 2020-12-18 17:33:03.296 WARNING: [52] org.jitsi.jicofo.xmpp.FocusComponent.processIQ() (serving component 'Jitsi Meet Focus') Unexpected exception while processing IQ stanza: <iq from="p-n_ffidub3oa8dh@meet.ent.domain.tld/11Pwq7qL" to="focus.meet.ent.domain.tld" type="set" id="1f3e2b1a-3f5b-4ba3-ae77-21ff2fac4321:sendIQ"><conference xmlns="http://jitsi.org/protocol/focus" room="test@conference.meet.ent.domain.tld" machine-uid="c39c947ff52fc74f71af81b61553c832"><property name="channelLastN" value="-1"/><property name="disableRtx" value="false"/><property name="enableLipSync" value="false"/><property name="openSctp" value="true"/></conference></iq>

Prosody Log doesn’t show anything wrong:

Dec 18 18:17:05 jcp8019e3880	info	component disconnected: focus.meet.ent.domain.tld (closed)
Dec 18 18:17:10 c2s80193d5c0	info	Client disconnected: closed
Dec 18 18:17:30 c2s8019a54c0	info	Client connected
Dec 18 18:17:30 c2s8019a54c0	info	Stream encrypted (TLSv1.2 with ECDHE-RSA-AES256-GCM-SHA384)
Dec 18 18:17:30 c2s8019a54c0	info	Authenticated as jvb@auth.meet.ent.domain.tld
Dec 18 18:17:38 jcp801a17e40	info	Incoming Jabber component connection
Dec 18 18:17:38 focus.meet.ent.domain.tld:component	info	External component successfully authenticated
Dec 18 18:22:21 mod_bosh	info	New BOSH session, assigned it sid '<redacted>'
Dec 18 18:22:22 bosh<redacted>	info	Authenticated as xpemxnsjdmu4niid@meet.ent.domain.tld
Dec 18 18:22:29 bosh<redacted>	info	BOSH client disconnected: session close

My suspicion is that something might be mis-configured between Jicofo and Prosody, but I don’t know exactly what.

I turned on health checks for jicofo by adding this to the sip-communicator.properties file:

org.jitsi.jicofo.health.ENABLE_HEALTH_CHECKS=true

This is what is showing in the log:

icofo 2020-12-18 19:41:16.383 SEVERE: [23] org.jitsi.health.HealthChecker.log() Health check failed in PT0S:
java.lang.RuntimeException: No JitsiMeetServices available

I downgraded to these version, leaving all configuration in place (from above) and it started working.

jicofo-1.0.555_2
jitsi-meet-1.0.4048_2
jitsi-videobridge-2.1.183_3

I think there might be a problem with the 2.0.5142 releases.