Debugging prosody messages: "Client tried to use sid 'XXX' which we don't know about"

We have a working Jitsi install consisting of 1 Jitsi Meet frontend machine (prosody, jicofo, nginx) and 6-8 Videobridge machines. When the system isn’t under much load everything works great, but once we reach ~1200 or so simultaneous participants in 100+ rooms we begin getting reports that users are unable to start conferences correctly (I realize that’s a very vague problem description, we are working to reproduce the issue better).

One thing that jumps out at me: Around the time of the reported issues there are often several messages in prosody.log that say Client tried to use sid 'XXX' which we don't know about. When I search for the relevant sid I always find a corresponding New BOSH session, assigned it sid XXX from 5-10 minutes earlier.

I believe that every time a conference is started we see New BOSH session, assigned it sid XXX in prosody.log. I believe this is Jicofo connecting to the room, but if anyone can point to more detailed documentation I would really appreciate it.

Maybe something is timing out and somehow prosody is throwing away the sid that it assigned minutes before? Has anyone seen something like this?

In case it is relevant, versions:

  • Prosody: 0.11.2
  • Jicofo: 1.0.644
  • Videobrige: 2.1.376-g9f12bfe2

Many thanks for reading!

Jicofo connects using the tcp connection on port 5222. Clients using bosh the first time they connect a new sid is created for them, the sid is removed after 60 seconds of inactivity, there are pings from the client every 10-15 secs so this mean the client had been disconnected for more than 60 seconds and need to reload…