Failed to bind even a single host candidate for component:Component id=1 parent stream=stream

We are developing from lib-jitsi-meet and created api to create/join room in our installed jitsi server(local machine). we called this api from normal html and from another website (all are running local network in different machines) that that need this api to create/join room.

all is working fine but there sometimes without any reason jicofo starts removing video bridge and we could not understand what the problem is. Though restarting jicofo fix that. here is videobridge log(removed at 10.43):

JVB 2020-01-11 10:43:21.674 INFO: [20] org.jitsi.videobridge.health.Health.log() Performed a successful health check in 7ms. Sticky failure: false
JVB 2020-01-11 10:43:26.274 FINE: [187] org.jitsi.videobridge.xmpp.ComponentImpl.processIQ() (serving component 'JitsiVideobridge') Processing IQ (packetId aC5fJ-4988): <iq type="get" to="jitsi-videobridge.192.168.0.150" from="focus@auth.192.168.0.150/focus10369002038" id="aC5fJ-4988"><healthcheck xmlns="http://jitsi.org/protocol/healthcheck"/></iq>
JVB 2020-01-11 10:43:26.274 FINE: [187] org.jitsi.videobridge.xmpp.ComponentImpl.processIQRequest() (serving component 'JitsiVideobridge') Processing IQ request (packetId aC5fJ-4988).
JVB 2020-01-11 10:43:26.275 FINE: [187] org.jitsi.videobridge.xmpp.ComponentImpl.processIQ() (serving component 'JitsiVideobridge') Responding to IQ (packetId aC5fJ-4988) with: <iq to="focus@auth.192.168.0.150/focus10369002038" from="jitsi-videobridge.192.168.0.150" id="aC5fJ-4988" type="result"/>
JVB 2020-01-11 10:43:26.386 FINE: [210] org.jitsi.videobridge.xmpp.ComponentImpl.processIQ() (serving component 'JitsiVideobridge') Processing IQ (packetId PVQpp-2280): <iq id="PVQpp-2280" type="result" to="jitsi-videobridge.192.168.0.150" from="192.168.0.150"/>
JVB 2020-01-11 10:43:27.107 WARNING: [17416] org.jitsi.impl.neomedia.rtp.TransportCCEngine.log() Not sending transport-cc feedback, delta or packetcount too big.
JVB 2020-01-11 10:43:30.461 WARNING: [18017] org.jitsi.videobridge.AbstractEndpointMessageTransport.log() Unable to find endpoint e1630088 to send EndpointMessage
JVB 2020-01-11 10:43:30.789 INFO: [19] org.jitsi.videobridge.VideobridgeExpireThread.log() Running expire()
JVB 2020-01-11 10:43:31.674 INFO: [20] org.jitsi.videobridge.Videobridge.log() CAT=stat create_conf,conf_id=2fc706cff1951927 conf_name=null,logging=false,conf_count=2,ch_count=6,v_streams=4
JVB 2020-01-11 10:43:31.675 SEVERE: [20] org.jitsi.videobridge.health.Health.log() Health check failed in 1ms:
java.io.IOException: Failed to bind even a single host candidate for component:Component id=1 parent stream=stream
no local candidates.
no remote candidates. preferredPort=14018 minPort=14018 maxPort=14118 foundAtLeastOneUsableInterface=false foundAtLeastOneUsableAddress=false
	at org.ice4j.ice.harvest.HostCandidateHarvester.harvest(HostCandidateHarvester.java:436)
	at org.ice4j.ice.Agent.gatherCandidates(Agent.java:675)
	at org.ice4j.ice.Agent.createComponent(Agent.java:601)
	at org.jitsi.videobridge.IceUdpTransportManager.createIceAgent(IceUdpTransportManager.java:1082)
	at org.jitsi.videobridge.IceUdpTransportManager.<init>(IceUdpTransportManager.java:566)
	at org.jitsi.videobridge.IceUdpTransportManager.<init>(IceUdpTransportManager.java:527)
	at org.jitsi.videobridge.Channel.createTransportManager(Channel.java:323)
	at org.jitsi.videobridge.Channel.initialize(Channel.java:674)
	at org.jitsi.videobridge.RtpChannel.initialize(RtpChannel.java:895)
	at org.jitsi.videobridge.Content.createRtpChannel(Content.java:340)
	at org.jitsi.videobridge.Content.createRtpChannel(Content.java:264)
	at org.jitsi.videobridge.health.Health.check(Health.java:152)
	at org.jitsi.videobridge.health.Health.doCheck(Health.java:244)
	at org.jitsi.videobridge.health.Health.doRun(Health.java:495)
	at org.jitsi.utils.concurrent.PeriodicRunnableWithObject.run(PeriodicRunnableWithObject.java:87)
	at org.jitsi.utils.concurrent.RecurringRunnableExecutor.run(RecurringRunnableExecutor.java:216)
	at org.jitsi.utils.concurrent.RecurringRunnableExecutor.runInThread(RecurringRunnableExecutor.java:292)
	at org.jitsi.utils.concurrent.RecurringRunnableExecutor.access$000(RecurringRunnableExecutor.java:36)
	at org.jitsi.utils.concurrent.RecurringRunnableExecutor$1.run(RecurringRunnableExecutor.java:328)
JVB 2020-01-11 10:43:36.257 FINE: [212] org.jitsi.videobridge.xmpp.ComponentImpl.processIQ() (serving component 'JitsiVideobridge') Processing IQ (packetId aC5fJ-5002): <iq type="get" to="jitsi-videobridge.192.168.0.150" from="focus@auth.192.168.0.150/focus10369002038" id="aC5fJ-5002"><query xmlns="http://jabber.org/protocol/disco#info"/></iq>
JVB 2020-01-11 10:43:36.257 FINE: [212] org.jitsi.videobridge.xmpp.ComponentImpl.processIQRequest() (serving component 'JitsiVideobridge') Processing IQ request (packetId aC5fJ-5002).
JVB 2020-01-11 10:43:36.257 FINE: [212] org.jitsi.videobridge.xmpp.ComponentImpl.processIQ() (serving component 'JitsiVideobridge') Responding to IQ (packetId aC5fJ-5002) with: <iq type="result" id="aC5fJ-5002" from="jitsi-videobridge.192.168.0.150" to="focus@auth.192.168.0.150/focus10369002038"><query xmlns="http://jabber.org/protocol/disco#info"><identity category="component" type="conference" name="JitsiVideobridge"/><feature var="http://jabber.org/protocol/disco#info"/><feature var="urn:xmpp:ping"/><feature var="jabber:iq:last"/><feature var="urn:xmpp:time"/><feature var="http://jitsi.org/protocol/colibri"/><feature var="http://jitsi.org/protocol/healthcheck"/><feature var="urn:xmpp:jingle:apps:dtls:0"/><feature var="urn:xmpp:jingle:transports:ice-udp:1"/><feature var="urn:xmpp:jingle:transports:raw-udp:1"/><feature var="jabber:iq:version"/></query></iq>
JVB 2020-01-11 10:43:36.274 FINE: [235] org.jitsi.videobridge.xmpp.ComponentImpl.processIQ() (serving component 'JitsiVideobridge') Processing IQ (packetId aC5fJ-5004): <iq type="get" to="jitsi-videobridge.192.168.0.150" from="focus@auth.192.168.0.150/focus10369002038" id="aC5fJ-5004"><healthcheck xmlns="http://jitsi.org/protocol/healthcheck"/></iq>
JVB 2020-01-11 10:43:36.274 FINE: [235] org.jitsi.videobridge.xmpp.ComponentImpl.processIQRequest() (serving component 'JitsiVideobridge') Processing IQ request (packetId aC5fJ-5004).
java.lang.Exception: java.io.IOException: Failed to bind even a single host candidate for component:Component id=1 parent stream=stream
no local candidates.
no remote candidates. preferredPort=14018 minPort=14018 maxPort=14118 foundAtLeastOneUsableInterface=false foundAtLeastOneUsableAddress=false

jitsi-videobridge.log (249.6 KB)

here is my jicofo log(10.43):

Jicofo 2020-01-11 10:43:36.277 WARNING: [90] org.jitsi.jicofo.JvbDoctor.log() Health check failed on: jitsi-videobridge.192.168.0.150 error: <error type='cancel'><internal-server-error xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/><text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas' xml:lang='en'>java.io.IOException: Failed to bind even a single host candidate for component:Component id=1 parent stream=stream
no local candidates.
no remote candidates. preferredPort=14018 minPort=14018 maxPort=14118 foundAtLeastOneUsableInterface=false foundAtLeastOneUsableAddress=false</text></error>
Jicofo 2020-01-11 10:43:36.277 INFO: [58] org.jitsi.jicofo.BridgeSelector.log() Removing JVB: jitsi-videobridge.192.168.0.150
Jicofo 2020-01-11 10:43:36.277 INFO: [58] org.jitsi.jicofo.JvbDoctor.log() Stopping health-check task for: jitsi-videobridge.192.168.0.150
Jicofo 2020-01-11 10:43:36.277 SEVERE: [58] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() One of our bridges failed: jitsi-videobridge.192.168.0.150
Jicofo 2020-01-11 10:43:36.278 INFO: [58] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Region info, conference=ffdf77 octo_enabled= false: [[null, null]]
Jicofo 2020-01-11 10:43:36.278 INFO: [58] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Expiring channels for: emerging-bd@conference.192.168.0.150/42b9fcab on: Bridge[jid=jitsi-videobridge.192.168.0.150, relayId=null, region=null]
Jicofo 2020-01-11 10:43:36.278 INFO: [58] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Region info, conference=ffdf77 octo_enabled= false: [[null]]
Jicofo 2020-01-11 10:43:36.278 INFO: [58] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Expiring channels for: emerging-bd@conference.192.168.0.150/a293701a on: Bridge[jid=jitsi-videobridge.192.168.0.150, relayId=null, region=null]
Jicofo 2020-01-11 10:43:36.278 SEVERE: [58] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Can not invite participant -- no bridge available.
Jicofo 2020-01-11 10:43:36.278 SEVERE: [58] org.jitsi.jicofo.JitsiMeetConferenceImpl.log() Can not invite participant -- no bridge available.

jicofo.log (56.6 KB)

actually what is the cause that creating this problem? how should I solve this as it happens anytime but othertime it is ok…
thanx in advance :heart:
@damencho

Cannot bind on any port between 14018 and 14118, this is on health-checking which marks jvb as unhealthy and had been removed from jicofo’s pool of jvb instances.

1 Like

why “cant bind”? is some service is running in that port without me knowing or else? how can I check? and how can I ensure it wont happen again?
Thanx for the reply :heart:

Can’t bind as those are already used. netstat -anp can show you the ports by process so you can check.
To prevent it you must identify why it is happening.

ok… what if I just make the range bigger so that it will try to bind any port available within big range… ?
I will check the ports … but as I said… it happens suddenly and not always… is there any way I can manage that no service will be given this port without jitsi video bridge? as the service seems to occupy that port suddenly and after restarting jicofo it is solved :confused: how this problem connects to jicofo…!
Thanx :heart:

When you restart jicofo it checks jvb whether it is healthy, it seems healthy and starts using it.
You can either stop health-checks from jicofo, risking trying to use unhealthy jvb. Or try to identify the problem, it may be bug in the code. Are you using latest jvb (jvb2)?
Normally when we detect unhealthy jvb we drain it and in the meantime, we spin up a new one.

mm can u plz say how to check jvb version? prosody version was in log and we installed the server through quickstart more than a month ago so I am not sure which version I am using …
why restarting jicofo find jvb healthy where without restarting it finds same thing unhealthy?

So if you don’t know, then you are using jvb1.
Well, jvb reports itself unhealthy, it is removed from the pool of jvbs in jicofo and then nobody checks it, when you restart jicofo you provoke a new check.

I also faced it few minutes earlier… and I needed to restart both jicofo and video-bridge.
but if this continues it will be really a big problem,so what do u suggest? using jvb2 or turn off health chcking(though I dont know why it is reporting itself unhealthy)
Thanx :heart:

It is known while it reports unhealthy, because of not able to bind to those ports. But it is not known why those are taken and it may indicate a bug that later can affect use of the jvb.
The proper way to go is first check whether you experience it with jvb2, as we will not be fixing anything in jvb1 as we are moving soon unstable to use jvb2 and later will follow and stable.
If this is still happening, the proper way of investigate this is to add a cron job that will check the health and once it is unhealthy do collect-dump-logs.sh and send us the result so we can take a look.