Jicofo stale conferences metric

I have a strange issue with Jicofo - it reports an increasing number of conferences and participants, while the on the JVB there are none. They seem to pile up when there is usage, then they are not always being released and this number increases. When it becomes very high we saw Jicofo losing connection to JVB (health checks started to fail and then it got disconnected), but I’m not sure there is a direct relation and we don’t risk replicating this every time, so when I see these ghost Jicofo conferences and participants rise too much, I just restart Jicofo when there is no usage and the stats are back to zero.

The versions are:
Jicofo - 1.0-636, latest stable
JVB - 2.1-351, all the Jitsi code is from latest stable
JRE - openjdk-11-jre-headless 11.0.8+10-0ubuntu1~18.04.1 (latest security update from Ubuntu 18.04 Bionic)

Here is an example:

curl -s http://localhost:8080/colibri/stats | jq '.conferences'
curl -s http://localhost:8888/stats | jq '.conferences'

This is with no conferences at all at the moment. After Jicofo restart it goes to 0. I have also graphs of this metric, that show how this jicofo stat metric rises alongside with the conferences on JVB, but it doesn’t always go down when the conferences end and in time it piles up.

Do you have any ideas?

1 Like

Can you do /usr/share/jicofo/collect-dump-logs.sh on such jicofo that does not have conferences from jvb point of view but stats show it in jicofo, like the example above, and send us the file in a private message?

The issue is fixed by upgrading Prosody from 0.10 to 0.11
Now there are no stale connections. Thanks, @damencho!

1 Like