Errors slowing prosody

Hello,

I have update all components to the latest version but encountered some errors in the process. I have made a rollback to 2.0.4857 (September 1st).
During the update, I have enabled lobby and patched the mod_websocket.lua with mod_websocket_smacks.patch and I suspect one of these to cause an error.
We are using an external turnserver and websockets since 5 months but we had no errors in prosody.err
We have Prosody 0.11.5 with mod_smacks and mod_pinger enabled. We use secure domains.
The configuration is using storage = “memory” since 5 months.

Here are the error logs:

error Traceback[c2s]: /usr/lib/prosody/util/serialization.lua:38: Can’t serialize function
stack traceback:
[C]: in function ‘error’
/usr/lib/prosody/util/serialization.lua:38: in function </usr/lib/prosody/util/serialization.lua:37>
(…tail calls…)
/usr/lib/prosody/util/serialization.lua:196: in function ‘serialize_table’
/usr/lib/prosody/util/serialization.lua:194: in function ‘serialize_table’
/usr/lib/prosody/util/serialization.lua:194: in function ‘serialize_table’
/usr/lib/prosody/util/serialization.lua:220: in function </usr/lib/prosody/util/serialization.lua:218>
(…tail calls…)
/usr/lib/prosody/modules/mod_storage_memory.lua:39: in function </usr/lib/prosody/modules/mod_storage_memory.lua:37>
(…tail calls…)
/usr/lib/prosody/modules/muc/mod_muc.lua:158: in function ‘on_evict’
/usr/lib/prosody/util/cache.lua:57: in function ‘set’
/usr/lib/prosody/modules/muc/mod_muc.lua:176: in function ‘track_room’
/usr/lib/prosody/modules/muc/mod_muc.lua:204: in function </usr/lib/prosody/modules/muc/mod_muc.lua:191>
(…tail calls…)
/usr/lib/prosody/modules/muc/mod_muc.lua:433: in function ‘?’
/usr/lib/prosody/util/events.lua:79: in function </usr/lib/prosody/util/events.lua:75>
(…tail calls…)
/usr/lib/prosody/core/stanza_router.lua:180: in function ‘core_post_stanza’
/usr/lib/prosody/core/stanza_router.lua:127: in function ‘core_process_stanza’
/usr/lib/prosody/modules/mod_c2s.lua:275: in function ‘func’
/usr/lib/prosody/util/async.lua:127: in function </usr/lib/prosody/util/async.lua:125>

We have the same error with bosh (users from mobile app I guess):

error Traceback[bosh]: /usr/lib/prosody/util/serialization.lua:38: Can’t serialize function
stack traceback:
SAME AS ABOVE EXCEPT FOR THE LAST LINES
(…tail calls…)
/usr/lib/prosody/core/stanza_router.lua:180: in function ‘core_post_stanza’
/usr/lib/prosody/core/stanza_router.lua:127: in function ‘core_process_stanza’
/usr/lib/prosody/modules/mod_c2s.lua:275: in function ‘func’
/usr/lib/prosody/util/async.lua:127: in function </usr/lib/prosody/util/async.lua:125>

Do you have some clue on what I can do to solve my issue ? It seems like prosody is not closing some sockets. When the errors appear, I even seems slower to authenticate and we are often disconnected from conferences.

I can provide more information if needed.

Any luck figuring this out? We’re seeing very similar Prosody error in our logs, causing people to not connect to the same conference.

You can test latest from unstable repo, I think that error should be fixed there. If you test it, report whether you still see it, thanks.

I’ll take a look at the unstable repo, but I’m pretty nervous about trying that in production. We’re still on an older package for 18.04, and we’re only seeing the problem intermittently and while under load, I haven’t been able to reproduce on demand.

For what it’s worth, we’re using Prosody 0.11.7, but we’re not using any of the modules listed by the OP.

Can you give me a little more detail on what the fix was or where you think the problem might have been? Maybe I can manually patch?

It is in the lobby module https://github.com/jitsi/jitsi-meet/pull/8014
You can make the changes in the lua by hand and reload module using admin_telnet, live without restarting prosody …

Thanks. We’re actually not currently using the lobby feature, so I decided to try just disabling that prosody module entirely in the prosody virtual host. So far, no errors, though I’m not sure if there are any risks with disabling that module (we also have the default speakerstats module disabled, which was throwing warnings in the prosody logs).

If we still run into problems, I will try re-enabling and patching manually, or try the unstable branch.

Thanks for the report! If you see it please report. Nope, there is no problem of disabling the lobby, the web client detects that it is not loaded and disables lobby automatically.
We will soon push it to stable …

Hi, we were experiencing the same error with the versions on Ubuntu 20:
ii jitsi-meet 2.0.5142-1 all WebRTC JavaScript video conferences
ii jitsi-meet-prosody 1.0.4466-1 all Prosody configuration for Jitsi Meet
ii jitsi-meet-web 1.0.4466-1 all WebRTC JavaScript video conferences
ii jitsi-meet-web-config 1.0.4466-1 all Configuration for web serving of Jitsi Meet
ii jitsi-videobridge2 2.1-376-g9f12bfe2-1 all WebRTC compatible Selective Forwarding Unit (SFU)
ii prosody 0.11.7-1~focal4 amd64 Lightweight Jabber/XMPP server

Prosody is installed from this repo http://packages.prosody.im/debian to be able to use token authentication.

We are using epoll and use_libevent = true

I solved the problem for now be deactivating all lobby related entries in /etc/prosody/conf.avail/HOSTNAME.cfg.lua

Should the error be fixed in these versions?

Nope, this is the current stable jitsi-meet 2.0.5142-1. The new RC that I mentioned is 2.0.5390-1.