[Solved] How to upgrade Prosody for Jitsi

I’m not referring to your old deployment. You mentioned starting with a fresh instance and still experiencing problems. If you get the latest prosody first and then do a fresh installation like I mentioned, you won’t need to tinker with websockets as they now come by default. Get things working first before making modifications.

All of the logs and config pointed to in this thread are the new fresh instance.

It doesn’t work for more than a handful of users with the initial install if the new prosody is included.

Hmm, is this a factor (I thought I saw thread that said ignore it, but maybe not?)
tail -f /var/log/prosody/*err
May 25 17:32:25 certmanager error SSL/TLS: Failed to load ‘/etc/prosody/certs/localhost.key’: Previous error (see logs), or other system error. (for localhost)
May 25 17:32:25 localhost:tls error Error creating contexts for s2sout: error loading private key (system lib)
May 25 17:32:25 certmanager error SSL/TLS: Failed to load ‘/etc/prosody/certs/localhost.key’: Previous error (see logs), or other system error. (for localhost)
May 25 17:32:25 localhost:tls error Error creating contexts for s2sin: error loading private key (system lib)
May 25 17:32:26 portmanager error Error binding encrypted port for https: No certificate present in SSL/TLS configuration for https port 5281
May 25 17:32:26 portmanager error Error binding encrypted port for https: No certificate present in SSL/TLS configuration for https port 5281
May 25 17:35:14 portmanager error Error binding encrypted port for https: No certificate present in SSL/TLS configuration for https port 5281
May 25 17:35:14 portmanager error Error binding encrypted port for https: No certificate present in SSL/TLS configuration for https port 5281
May 25 17:46:17 portmanager error Error binding encrypted port for https: No certificate present in SSL/TLS configuration for https port 5281
May 25 17:46:17 portmanager error Error binding encrypted port for https: No certificate present in SSL/TLS configuration for https port 5281

and /var/log/prosody/prosody.log:

May 25 17:35:16 c2s5625c9d1d760 info Client connected
May 25 17:35:16 c2s5625c9cf69c0 info Client connected
May 25 17:35:16 c2s5625c9d1d760 info Stream encrypted (TLSv1.3 with TLS_AES_256_GCM_SHA384)
May 25 17:35:16 c2s5625c9cf69c0 info Stream encrypted (TLSv1.3 with TLS_AES_256_GCM_SHA384)
May 25 17:35:16 c2s5625c9cf69c0 info Authenticated as jvb@auth.lmtgt1.dev2dev.net
May 25 17:35:16 c2s5625c9d1d760 info Authenticated as focus@auth.lmtgt1.dev2dev.net
May 25 17:35:16 c2s5625c9d11850 info Client connected
May 25 17:35:16 c2s5625c9d11850 info Authenticated as ksapzuicasoqmadi@lmtgt1.dev2dev.net
May 25 17:35:56 c2s5625c9d0ad30 info Client connected
May 25 17:35:56 c2s5625c9d0ad30 info Authenticated as nx-mmm3z9dwhpi0t@lmtgt1.dev2dev.net
May 25 17:36:45 c2s5625c9eb5f90 info Client connected
May 25 17:36:49 c2s5625c9a9e270 info Client connected
May 25 17:36:52 c2s5625c9e99850 info Client connected
May 25 17:36:54 c2s5625c9d11850 info Client disconnected: connection closed
May 25 17:36:55 c2s5625c9db6120 info Client connected
May 25 17:36:55 c2s5625c9db6120 info Authenticated as crzvqphjkaveeklr@lmtgt1.dev2dev.net
May 25 17:37:45 c2s5625c9eb5f90 info Client disconnected: closed
May 25 17:37:49 c2s5625c9a9e270 info Client disconnected: closed
May 25 17:37:52 c2s5625c9e99850 info Client disconnected: closed
May 25 17:38:07 c2s5625c9d0ad30 info Client disconnected: connection closed
May 25 17:38:08 c2s5625c9f24950 info Client connected
May 25 17:38:09 c2s5625c9f24950 info Authenticated as 6zb4ezrhizlsxlhg@lmtgt1.dev2dev.net
May 25 17:38:51 c2s5625c9db6120 info Client disconnected: connection closed
May 25 17:38:52 c2s5625c9ea74e0 info Client connected
May 25 17:38:53 c2s5625c9ea74e0 info Authenticated as xjtn54mvikb1hhy8@lmtgt1.dev2dev.net
May 25 17:40:06 c2s5625c9f24950 info Client disconnected: connection closed
May 25 17:40:07 c2s5625c9e52540 info Client connected
May 25 17:40:07 c2s5625c9e52540 info Authenticated as 9qqqrg9cefxikapq@lmtgt1.dev2dev.net
May 25 17:40:50 c2s5625c9ea74e0 info Client disconnected: connection closed
May 25 17:40:51 c2s5625c9e88e90 info Client connected
May 25 17:40:51 c2s5625c9e88e90 info Authenticated as -dzcffym5mldgdc6@lmtgt1.dev2dev.net
May 25 17:41:50 c2s5625c9e52540 info Client disconnected: connection closed
May 25 17:41:51 c2s5625c9f81d50 info Client connected
May 25 17:41:51 c2s5625c9f81d50 info Authenticated as dazmbaykmpmrwmml@lmtgt1.dev2dev.net
May 25 17:42:22 c2s5625c9e88e90 info Client disconnected: connection closed
May 25 17:42:23 c2s5625c9c07210 info Client connected
May 25 17:42:23 c2s5625c9c07210 info Authenticated as vvu_blsxeqjfy-jq@lmtgt1.dev2dev.net
May 25 17:43:49 speakerstats.lmtgt1.dev2dev.net:speakerstats_component warn A module has been configured that triggers external events.
May 25 17:43:49 speakerstats.lmtgt1.dev2dev.net:speakerstats_component warn Implement this lib to trigger external events.
May 25 17:43:53 c2s5625c9c07210 info Client disconnected: connection closed
May 25 17:43:54 c2s5625c9fd8010 info Client connected
May 25 17:43:54 c2s5625c9fd8010 info Authenticated as _tsohcb3kvdan_th@lmtgt1.dev2dev.net
May 25 17:44:01 c2s5625c9f81d50 info Client disconnected: connection closed
May 25 17:44:02 c2s5625c9f34710 info Client connected
May 25 17:44:03 c2s5625c9f34710 info Authenticated as wz2hysoucy43lhtj@lmtgt1.dev2dev.net
May 25 17:45:18 speakerstats.lmtgt1.dev2dev.net:speakerstats_component warn A module has been configured that triggers external events.
May 25 17:45:18 speakerstats.lmtgt1.dev2dev.net:speakerstats_component warn Implement this lib to trigger external events.
May 25 17:45:29 c2s5625c9fd8010 info Client disconnected: connection closed
May 25 17:45:30 c2s5625c9f34710 info Client disconnected: connection closed
May 25 17:45:30 c2s5625c9c149c0 info Client connected
May 25 17:45:30 c2s5625c9c149c0 info Authenticated as izb71w7-p0rwqdqp@lmtgt1.dev2dev.net
May 25 17:45:31 c2s5625c9f17be0 info Client connected
May 25 17:45:31 c2s5625c9f17be0 info Authenticated as ac1ojf63ijclxdyt@lmtgt1.dev2dev.net
May 25 17:45:47 mod_posix warn Received SIGTERM
May 25 17:45:47 startup info Shutting down: Received SIGTERM
May 25 17:45:47 c2s5625c9f17be0 info Client disconnected: connection closed
May 25 17:45:47 c2s5625c9cf69c0 info Client disconnected: connection closed
May 25 17:45:47 c2s5625c9d1d760 info Client disconnected: connection closed
May 25 17:45:47 speakerstats.lmtgt1.dev2dev.net:speakerstats_component warn A module has been configured that triggers external events.
May 25 17:45:47 speakerstats.lmtgt1.dev2dev.net:speakerstats_component warn Implement this lib to trigger external events.
May 25 17:45:47 c2s5625c9c149c0 info Client disconnected: connection closed
May 25 17:45:47 general info Shutting down…
May 25 17:45:47 general info Shutdown status: Cleaning up
May 25 17:45:47 general info Shutdown complete
May 25 17:46:16 startup info Hello and welcome to Prosody version 0.11.9
May 25 17:46:16 startup info Prosody is using the epoll backend for connection handling
May 25 17:46:17 portmanager info Activated service ‘s2s’ on [*]:5269, [::]:5269
May 25 17:46:17 portmanager info Activated service ‘http’ on [*]:5280, [::]:5280
May 25 17:46:17 portmanager error Error binding encrypted port for https: No certificate present in SSL/TLS configuration for https port 5281
May 25 17:46:17 portmanager error Error binding encrypted port for https: No certificate present in SSL/TLS configuration for https port 5281
May 25 17:46:17 portmanager info Activated service ‘https’ on no ports
May 25 17:46:17 portmanager info Activated service ‘c2s’ on [*]:5222, [::]:5222
May 25 17:46:17 portmanager info Activated service ‘legacy_ssl’ on no ports
May 25 17:46:17 lmtgt1.dev2dev.net:tls info Certificates loaded
May 25 17:46:17 auth.lmtgt1.dev2dev.net:tls info Certificates loaded
May 25 17:46:17 general info Starting conference duration timer for conference.lmtgt1.dev2dev.net
May 25 17:46:17 conferenceduration.lmtgt1.dev2dev.net:conference_duration_component info No muc component found, will listen for it: conference.lmtgt1.dev2dev.net
May 25 17:46:17 conferenceduration.lmtgt1.dev2dev.net:tls info Certificates loaded
May 25 17:46:17 conferenceduration.lmtgt1.dev2dev.net:conference_duration_component info Hook to muc events on conference.lmtgt1.dev2dev.net
May 25 17:46:17 conference.lmtgt1.dev2dev.net:tls info Certificates loaded
May 25 17:46:17 conference.lmtgt1.dev2dev.net:muc_domain_mapper info Loading mod_muc_domain_mapper for host lmtgt1.dev2dev.net!
May 25 17:46:17 conference.lmtgt1.dev2dev.net:muc_domain_mapper info Loading mod_muc_domain_mapper for host conference.lmtgt1.dev2dev.net!
May 25 17:46:17 conference.lmtgt1.dev2dev.net:muc_domain_mapper info Loading mod_muc_domain_mapper for host auth.lmtgt1.dev2dev.net!
May 25 17:46:17 conference.lmtgt1.dev2dev.net:muc_domain_mapper info Loading mod_muc_domain_mapper for host conferenceduration.lmtgt1.dev2dev.net!
May 25 17:46:17 conference.lmtgt1.dev2dev.net:muc_domain_mapper info Loading mod_muc_domain_mapper for host focus.lmtgt1.dev2dev.net!
May 25 17:46:17 focus.lmtgt1.dev2dev.net:tls info Certificates loaded
May 25 17:46:17 conference.lmtgt1.dev2dev.net:muc_domain_mapper info Loading mod_muc_domain_mapper for host speakerstats.lmtgt1.dev2dev.net!
May 25 17:46:17 general info Starting speakerstats for conference.lmtgt1.dev2dev.net
May 25 17:46:17 speakerstats.lmtgt1.dev2dev.net:speakerstats_component info Hook to muc events on conference.lmtgt1.dev2dev.net
May 25 17:46:17 speakerstats.lmtgt1.dev2dev.net:tls info Certificates loaded
May 25 17:46:17 conference.lmtgt1.dev2dev.net:muc_domain_mapper info Loading mod_muc_domain_mapper for host lobby.lmtgt1.dev2dev.net!
May 25 17:46:17 lmtgt1.dev2dev.net:muc_lobby_rooms info Lobby component loaded lobby.lmtgt1.dev2dev.net
May 25 17:46:17 lobby.lmtgt1.dev2dev.net:tls info Certificates loaded
May 25 17:46:17 conference.lmtgt1.dev2dev.net:muc_domain_mapper info Loading mod_muc_domain_mapper for host internal.auth.lmtgt1.dev2dev.net!
May 25 17:46:17 internal.auth.lmtgt1.dev2dev.net:tls info Certificates loaded
May 25 17:46:18 c2s559ea2cabe90 info Client connected
May 25 17:46:18 c2s559ea2c3b300 info Client connected
May 25 17:46:18 c2s559ea2c3b300 info Stream encrypted (TLSv1.3 with TLS_AES_256_GCM_SHA384)
May 25 17:46:18 c2s559ea2cabe90 info Stream encrypted (TLSv1.3 with TLS_AES_256_GCM_SHA384)
May 25 17:46:18 c2s559ea2cabe90 info Authenticated as focus@auth.lmtgt1.dev2dev.net
May 25 17:46:18 c2s559ea2c3b300 info Authenticated as jvb@auth.lmtgt1.dev2dev.net
May 25 17:46:20 c2s559ea2ccb300 info Client connected
May 25 17:46:20 c2s559ea2ccb300 info Authenticated as okmethvz7g4rzoxo@lmtgt1.dev2dev.net
May 25 17:46:23 c2s559ea27ffd60 info Client connected
May 25 17:46:23 c2s559ea27ffd60 info Authenticated as jz3emkmim32owmui@lmtgt1.dev2dev.net
May 25 17:47:48 speakerstats.lmtgt1.dev2dev.net:speakerstats_component warn A module has been configured that triggers external events.
May 25 17:47:48 speakerstats.lmtgt1.dev2dev.net:speakerstats_component warn Implement this lib to trigger external events.
May 25 17:47:53 c2s559ea2ccb300 info Client disconnected: connection closed
May 25 17:47:54 c2s559ea2d021e0 info Client connected
May 25 17:47:54 c2s559ea2d021e0 info Authenticated as 1kmlnsd6radlveij@lmtgt1.dev2dev.net
May 25 17:48:01 c2s559ea27ffd60 info Client disconnected: connection closed
May 25 17:48:02 c2s559ea2c16b00 info Client connected
May 25 17:48:02 c2s559ea2c16b00 info Authenticated as mx4outnbbiiinhfs@lmtgt1.dev2dev.net
May 25 17:49:25 c2s559ea2d021e0 info Client disconnected: connection closed
May 25 17:49:26 c2s559ea2c04f20 info Client connected
May 25 17:49:26 c2s559ea2c04f20 info Authenticated as vooxya2bsazivocw@lmtgt1.dev2dev.net
May 25 17:50:04 c2s559ea2c16b00 info Client disconnected: connection closed
May 25 17:50:05 c2s559ea2c4ffe0 info Client connected
May 25 17:50:05 c2s559ea2c4ffe0 info Authenticated as upik2fwz2jfr58aj@lmtgt1.dev2dev.net
May 25 17:51:10 speakerstats.lmtgt1.dev2dev.net:speakerstats_component warn A module has been configured that triggers external events.
May 25 17:51:10 speakerstats.lmtgt1.dev2dev.net:speakerstats_component warn Implement this lib to trigger external events.
May 25 17:51:13 c2s559ea2c04f20 info Client disconnected: connection closed
May 25 17:51:14 c2s559ea2aa8e60 info Client connected
May 25 17:51:14 c2s559ea2aa8e60 info Authenticated as tolbx_4uudvh60mu@lmtgt1.dev2dev.net
May 25 17:51:37 c2s559ea2c4ffe0 info Client disconnected: connection closed
May 25 17:51:38 c2s559ea2b1cd30 info Client connected
May 25 17:51:38 c2s559ea2b1cd30 info Authenticated as l1z3s-nt7iudbd_c@lmtgt1.dev2dev.net

I feel like you’re not really paying attention to what I’m saying. This, for instance, is not from a default installation. None of this shows up by default in a fresh installation.

Again:

Sorry, I am not trying to frustrate you, I appreciate you taking the time to help.

As I stated at the beginning of this thread, that is what I started with.

I created a fresh instance.
I prep the system, and add the newer prosody repo.
I then run the new, fresh apt install jitsi-meet.
I then test it for use. It works, but only for a handful of users, then starts disconnecting after juts a few (4-5 users).
I then try adding the further tweaks listed in those howto links, and it only becomes worse.

I will go ahead and once again create another fresh instance without any of those instruction’s tweaks, and post those config files and see if that helps further. Thank you for your patience in working through this.

It’s ok, no worries. It’s just easier to debug when it’s a default installation. If you’re only getting 4-5 users on a default installation, then something is going on. We need to sort that out first, thereafter, you can add the customizations. So yeah, do the default installation and let’s see how it goes.

Just curious though, what do you mean by ‘prep the system’?

1 Like

May you need to bump the limits as I mentioned in another thread … Its the latest prosody has some limits breaking jicofo connection

1 Like
1 Like

When I state “prep the system” I mean:

create aws instance, load-testing subnet, assign security groups, volumes, etc.
assign elastic ip
Ssh in.

Set hostname.fqdn
disable dynamic cloud hostname changes by changing preserve_hostname in /etc/cloud/cloud.cfg to true. ​

reboot
apt update and apt upgrade all default packages.
reboot for kernel to apply changes, etc.

install java 11 headless: apt install openjdk-11-jre-headless

follow self-hosting guide quick start instructions here: Self-Hosting Guide - Debian/Ubuntu server · Jitsi Meet Handbook

apt install gnupg2
Add the newer prosody repo
as per here: [How to] How do I update Prosody?
just add the repo without running the prosody install yet:

echo deb http://packages.prosody.im/debian $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list

wget https://prosody.im/files/prosody-debian-packages.key -O- | sudo apt-key add -

apt-get update

Then back at jitsi setup run:
apt install nginx-full
apt install apt-transport-https
apt-add-repository universe (usually already enabled, but double check)
apt update to verify looks good (no typos, etc.)

Add jitsi repo
curl https://download.jitsi.org/jitsi-key.gpg.key | sudo sh -c ‘gpg --dearmor > /usr/share/keyrings/jitsi-keyring.gpg’

echo ‘deb [signed-by=/usr/share/keyrings/jitsi-keyring.gpg] https://download.jitsi.org stable/’ | sudo tee /etc/apt/sources.list.d/jitsi-stable.list > /dev/null

apt update to verify looks good

edit /etc/hosts appropriately.
3.18.144.164 lmtgt1.dev2dev.net

Then run the rest of the setup/install instruction including apt install jitsi-meet.

apt install jitsi-meet

enter the fqdn when prompted

during question about cert, choose the self-signed for now, and run letsencrypt script later.

I don’t add access controls, since this is for simple baseline quick and dirty load tests and turned off in between. (I have enabled protected/login for other tests, and when the basics are working at scale, will bring those back in again. Right now I am trying to keep it at the basic for baseline.

After install in place with temp cert, run the letsencrypt script.

/usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh

Then environmental check:

systemctl show --property DefaultLimitNPROC
DefaultLimitNPROC=254048

systemctl show --property DefaultLimitNOFILE
DefaultLimitNOFILE=524288

systemctl show --property DefaultTasksMax
DefaultTasksMax=76214

All above the recommended 65000 in the quickstart docs.
Should any of these be larger for goal of 4-5k participants?

Connect and verify jitsi working with 2 laptops running audio and video.
Checks out fine.

Then run baseline load test:
Only 1 other user joins, then I both laptops get kicked out and neither can rejoin any rooms while the load test is running (Just keep getting kicked).

Logs look like this when this is happening:

tail -f /var/log/jitsi/*log
==> /var/log/jitsi/jicofo.log <==
Jicofo 2021-05-25 19:11:02.729 WARNING: [36] BridgeSelectionStrategy.select#111: Failed to select initial bridge for participantRegion=null
Jicofo 2021-05-25 19:11:02.730 SEVERE: [36] [room=loadtest41@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.selectBridge#673: Can not invite participant, no bridge available: 22cf068e
Jicofo 2021-05-25 19:11:02.730 SEVERE: [36] [room=loadtest41@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.inviteParticipant#743: Failed to select a bridge for Participant[loadtest41@conference.lmtgt1.dev2dev.net/22cf068e]@356332273
Jicofo 2021-05-25 19:11:02.730 WARNING: [36] BridgeSelectionStrategy.select#111: Failed to select initial bridge for participantRegion=null
Jicofo 2021-05-25 19:11:02.730 SEVERE: [36] [room=loadtest41@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.selectBridge#673: Can not invite participant, no bridge available: 23ad87de
Jicofo 2021-05-25 19:11:02.730 SEVERE: [36] [room=loadtest41@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.inviteParticipant#743: Failed to select a bridge for Participant[loadtest41@conference.lmtgt1.dev2dev.net/23ad87de]@459466372
Jicofo 2021-05-25 19:11:02.730 INFO: [36] [room=loadtest41@conference.lmtgt1.dev2dev.net] ChatRoomRoleAndPresence.memberPresenceChanged#130: Chat room event PresenceUpdated member=ChatMember[loadtest41@conference.lmtgt1.dev2dev.net/23ad87de, jid: null]@1002037791
Jicofo 2021-05-25 19:11:02.730 INFO: [36] [room=loadtest31@conference.lmtgt1.dev2dev.net] ChatRoomRoleAndPresence.memberPresenceChanged#130: Chat room event Joined member=ChatMember[loadtest31@conference.lmtgt1.dev2dev.net/5983b167, jid: null]@1384016953
Jicofo 2021-05-25 19:11:02.730 WARNING: [36] [room=loadtest31@conference.lmtgt1.dev2dev.net] ChatRoomRoleAndPresence.electNewOwner#177: Focus role unknown
Jicofo 2021-05-25 19:11:02.730 INFO: [36] [room=loadtest31@conference.lmtgt1.dev2dev.net] ChatRoomRoleAndPresence.electNewOwner#181: Obtained focus role: OWNER

==> /var/log/jitsi/jvb.log <==
JVB 2021-05-25 19:11:02.665 WARNING: [207] [confId=3fc89e2aeb396954 epId=be1ca62c gid=12500 stats_id=Gwen-N7H conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 67e9decf
JVB 2021-05-25 19:11:02.669 WARNING: [207] [confId=3fc89e2aeb396954 epId=b2c66cad gid=12500 stats_id=Esta-MeH conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 67e9decf
JVB 2021-05-25 19:11:02.669 WARNING: [192] [confId=3fc89e2aeb396954 epId=a8f51fd2 gid=12500 stats_id=Blanca-D4A conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 67e9decf
JVB 2021-05-25 19:11:02.674 WARNING: [192] [confId=c517332eb7628b5d epId=7136198d gid=5659 stats_id=Edwardo-dNx conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 85d191ec
JVB 2021-05-25 19:11:02.741 WARNING: [192] [confId=9e7f39691eb453b8 epId=fd9ea629 gid=42769 stats_id=Carolyn-kdT conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: de4cfe8b
JVB 2021-05-25 19:11:02.746 WARNING: [192] [confId=9e7f39691eb453b8 epId=ae43624e gid=42769 stats_id=Mack-p0n conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: de4cfe8b
JVB 2021-05-25 19:11:02.772 INFO: [192] [confId=9e7f39691eb453b8 epId=de4cfe8b local_ufrag=btvj61f6if003j gid=42769 conf_name=loadtest58@conference.lmtgt1.dev2dev.net ufrag=btvj61f6if003j] Agent.gatherCandidates#622: Gathering candidates for component stream-de4cfe8b.RTP.
JVB 2021-05-25 19:11:02.772 INFO: [207] [confId=5b2a7a2d501a974 epId=07e78538 local_ufrag=4ldq61f6if003j gid=59966 conf_name=loadtest48@conference.lmtgt1.dev2dev.net ufrag=4ldq61f6if003j] Agent.gatherCandidates#622: Gathering candidates for component stream-07e78538.RTP.
JVB 2021-05-25 19:11:02.774 INFO: [207] [confId=5b2a7a2d501a974 epId=07e78538 gid=59966 conf_name=loadtest48@conference.lmtgt1.dev2dev.net] Endpoint.setTransportInfo#662: Ignoring empty DtlsFingerprint extension:
JVB 2021-05-25 19:11:02.775 INFO: [192] [confId=9e7f39691eb453b8 epId=de4cfe8b gid=42769 conf_name=loadtest58@conference.lmtgt1.dev2dev.net] Endpoint.setTransportInfo#662: Ignoring empty DtlsFingerprint extension:
JVB 2021-05-25 19:11:03.530 WARNING: [207] [confId=5b2a7a2d501a974 epId=77b91288 gid=59966 stats_id=Allan-ahM conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 84cd4bdc
JVB 2021-05-25 19:11:03.530 WARNING: [192] [confId=5b2a7a2d501a974 epId=03baf5ad gid=59966 stats_id=Marquise-1yk conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 84cd4bdc
JVB 2021-05-25 19:11:03.541 WARNING: [192] [confId=5b2a7a2d501a974 epId=377eedfe gid=59966 stats_id=Lawson-J7Z conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 84cd4bdc
JVB 2021-05-25 19:11:03.570 WARNING: [207] [confId=9e7f39691eb453b8 epId=ae43624e gid=42769 stats_id=Mack-p0n conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 4d9928df
JVB 2021-05-25 19:11:03.620 WARNING: [207] [confId=9e7f39691eb453b8 epId=fd9ea629 gid=42769 stats_id=Carolyn-kdT conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 4d9928df
JVB 2021-05-25 19:11:03.621 WARNING: [207] [confId=9e7f39691eb453b8 epId=777d95de gid=42769 stats_id=Earline-f3L conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 4d9928df
JVB 2021-05-25 19:11:03.622 WARNING: [207] [confId=9e7f39691eb453b8 epId=333f1281 gid=42769 stats_id=Giovanny-Wy8 conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 4d9928df
JVB 2021-05-25 19:11:03.622 WARNING: [207] [confId=5b2a7a2d501a974 epId=03baf5ad gid=59966 stats_id=Marquise-1yk conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f9227ad6
JVB 2021-05-25 19:11:03.667 WARNING: [207] [confId=e9754803553e1e5a epId=342b51b5 gid=52251 stats_id=Lauren-uwO conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 832621ff
JVB 2021-05-25 19:11:03.671 WARNING: [207] [confId=f4db77e4558a6b37 epId=b5e4924d gid=22495 stats_id=Alyson-hIO conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 79138703
JVB 2021-05-25 19:11:03.673 WARNING: [207] [confId=f4db77e4558a6b37 epId=9a15d2cb gid=22495 stats_id=Delbert-IBz conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 79138703

==> /var/log/jitsi/jicofo.log <==
Jicofo 2021-05-25 19:11:03.699 INFO: [222] [room=loadtest30@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.stop#428: Stopped.
Jicofo 2021-05-25 19:11:03.699 INFO: [182] [room=loadtest32@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.stop#428: Stopped.
Jicofo 2021-05-25 19:11:03.699 WARNING: [222] FocusManager.conferenceRequest#244: Exception while trying to start the conference
org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 15000ms (~15s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=413yQ-2444)), : fromFilter (OrFilter: (FromMatchesFilter (full): loadtest30@conference.lmtgt1.dev2dev.net)).
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:111)
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:98)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:260)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:214)
at org.jivesoftware.smackx.muc.MultiUserChat.getConfigurationForm(MultiUserChat.java:781)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.joinAs(ChatRoomImpl.java:237)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.join(ChatRoomImpl.java:215)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.joinTheRoom(JitsiMeetConferenceImpl.java:466)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.start(JitsiMeetConferenceImpl.java:316)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:239)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:193)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:172)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.handleConferenceIq(ConferenceIqHandler.kt:65)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.access$handleConferenceIq(ConferenceIqHandler.kt:36)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler$handleIQRequest$2.run(ConferenceIqHandler.kt:153)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Jicofo 2021-05-25 19:11:03.700 WARNING: [182] FocusManager.conferenceRequest#244: Exception while trying to start the conference
org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 15000ms (~15s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=413yQ-2445)), : fromFilter (OrFilter: (FromMatchesFilter (full): loadtest32@conference.lmtgt1.dev2dev.net)).
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:111)
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:98)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:260)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:214)
at org.jivesoftware.smackx.muc.MultiUserChat.getConfigurationForm(MultiUserChat.java:781)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.joinAs(ChatRoomImpl.java:237)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.join(ChatRoomImpl.java:215)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.joinTheRoom(JitsiMeetConferenceImpl.java:466)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.start(JitsiMeetConferenceImpl.java:316)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:239)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:193)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:172)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.handleConferenceIq(ConferenceIqHandler.kt:65)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.access$handleConferenceIq(ConferenceIqHandler.kt:36)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler$handleIQRequest$2.run(ConferenceIqHandler.kt:153)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)

==> /var/log/jitsi/jvb.log <==
JVB 2021-05-25 19:11:03.701 WARNING: [207] [confId=a20c8568d5fab362 epId=25d3be7a gid=8153 stats_id=Cathy-6UV conf_name=loadtest44@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 34a74b14

==> /var/log/jitsi/jicofo.log <==
Jicofo 2021-05-25 19:11:03.706 INFO: [200] [room=loadtest53@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.terminateParticipant#1130: Terminating 7f1f7e5e, reason: general-error, send session-terminate: true
Jicofo 2021-05-25 19:11:03.706 INFO: [200] [room=loadtest53@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.terminateParticipant#1155: Removed participant 7f1f7e5e removed=true

==> /var/log/jitsi/jvb.log <==
JVB 2021-05-25 19:11:03.716 WARNING: [207] [confId=f4db77e4558a6b37 epId=7abf45ec gid=22495 stats_id=Kristian-yIk conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 79138703
JVB 2021-05-25 19:11:03.716 WARNING: [207] [confId=f4db77e4558a6b37 epId=031e74d0 gid=22495 stats_id=Olen-quX conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 79138703
JVB 2021-05-25 19:11:03.733 WARNING: [207] [confId=c517332eb7628b5d epId=5fbdde59 gid=5659 stats_id=Clovis-wxH conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: c366b595
JVB 2021-05-25 19:11:03.734 WARNING: [207] [confId=5b2a7a2d501a974 epId=77b91288 gid=59966 stats_id=Allan-ahM conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f9227ad6
JVB 2021-05-25 19:11:03.736 WARNING: [207] [confId=c517332eb7628b5d epId=757f7b17 gid=5659 stats_id=Elenor-iZJ conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: c366b595
JVB 2021-05-25 19:11:03.755 WARNING: [207] [confId=c517332eb7628b5d epId=7136198d gid=5659 stats_id=Edwardo-dNx conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: c366b595
JVB 2021-05-25 19:11:03.772 WARNING: [207] [confId=ed23db561d6c91cc epId=54760cbe gid=32346 stats_id=Willy-zkk conf_name=loadtest0@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 64cf95a6
JVB 2021-05-25 19:11:03.775 WARNING: [207] [confId=ed23db561d6c91cc epId=7ca427c2 gid=32346 stats_id=Clifford-AuR conf_name=loadtest0@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 64cf95a6
JVB 2021-05-25 19:11:03.777 INFO: [192] [confId=3fc89e2aeb396954 epId=67e9decf local_ufrag=60jht1f6if0130 gid=12500 conf_name=loadtest49@conference.lmtgt1.dev2dev.net ufrag=60jht1f6if0130] Agent.gatherCandidates#622: Gathering candidates for component stream-67e9decf.RTP.
JVB 2021-05-25 19:11:03.777 INFO: [207] [confId=c517332eb7628b5d epId=85d191ec local_ufrag=1pe1d1f6if0130 gid=5659 conf_name=loadtest55@conference.lmtgt1.dev2dev.net ufrag=1pe1d1f6if0130] Agent.gatherCandidates#622: Gathering candidates for component stream-85d191ec.RTP.
JVB 2021-05-25 19:11:03.780 INFO: [207] [confId=c517332eb7628b5d epId=85d191ec gid=5659 conf_name=loadtest55@conference.lmtgt1.dev2dev.net] Endpoint.setTransportInfo#662: Ignoring empty DtlsFingerprint extension:
JVB 2021-05-25 19:11:03.780 INFO: [192] [confId=3fc89e2aeb396954 epId=67e9decf gid=12500 conf_name=loadtest49@conference.lmtgt1.dev2dev.net] Endpoint.setTransportInfo#662: Ignoring empty DtlsFingerprint extension:
JVB 2021-05-25 19:11:03.784 WARNING: [192] [confId=5b2a7a2d501a974 epId=377eedfe gid=59966 stats_id=Lawson-J7Z conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f9227ad6
JVB 2021-05-25 19:11:03.945 WARNING: [192] [confId=f4db77e4558a6b37 epId=9a15d2cb gid=22495 stats_id=Delbert-IBz conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f3e4656a
JVB 2021-05-25 19:11:04.007 WARNING: [192] [confId=f4db77e4558a6b37 epId=b5e4924d gid=22495 stats_id=Alyson-hIO conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f3e4656a
JVB 2021-05-25 19:11:04.021 WARNING: [192] [confId=f4db77e4558a6b37 epId=031e74d0 gid=22495 stats_id=Olen-quX conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f3e4656a
JVB 2021-05-25 19:11:04.104 WARNING: [207] [confId=f4db77e4558a6b37 epId=7abf45ec gid=22495 stats_id=Kristian-yIk conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f3e4656a
JVB 2021-05-25 19:11:04.569 WARNING: [192] [confId=e9754803553e1e5a epId=342b51b5 gid=52251 stats_id=Lauren-uwO conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 6bff0854
JVB 2021-05-25 19:11:04.572 WARNING: [192] [confId=3fc89e2aeb396954 epId=a8f51fd2 gid=12500 stats_id=Blanca-D4A conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ac658f44
JVB 2021-05-25 19:11:04.572 WARNING: [192] [confId=3fc89e2aeb396954 epId=be1ca62c gid=12500 stats_id=Gwen-N7H conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ac658f44
JVB 2021-05-25 19:11:04.613 WARNING: [192] [confId=3fc89e2aeb396954 epId=b2c66cad gid=12500 stats_id=Esta-MeH conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ac658f44
JVB 2021-05-25 19:11:04.618 WARNING: [192] [confId=d80b9097ae979ad6 epId=db314a8b gid=22435 stats_id=Benedict-gXA conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 3b7b17f9
JVB 2021-05-25 19:11:04.634 WARNING: [192] [confId=d80b9097ae979ad6 epId=8d0e7ca1 gid=22435 stats_id=Terrill-jQN conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 3b7b17f9
JVB 2021-05-25 19:11:04.635 WARNING: [192] [confId=d80b9097ae979ad6 epId=0a70351b gid=22435 stats_id=Esperanza-6p7 conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 3b7b17f9
JVB 2021-05-25 19:11:04.646 WARNING: [192] [confId=d80b9097ae979ad6 epId=9366ed40 gid=22435 stats_id=Brooks-o9f conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 3b7b17f9
JVB 2021-05-25 19:11:04.676 WARNING: [192] [confId=c517332eb7628b5d epId=757f7b17 gid=5659 stats_id=Elenor-iZJ conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 849d2c00
JVB 2021-05-25 19:11:04.676 WARNING: [207] [confId=c517332eb7628b5d epId=5fbdde59 gid=5659 stats_id=Clovis-wxH conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 849d2c00
JVB 2021-05-25 19:11:04.679 WARNING: [207] [confId=c517332eb7628b5d epId=7136198d gid=5659 stats_id=Edwardo-dNx conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 849d2c00
JVB 2021-05-25 19:11:04.787 INFO: [207] [confId=5b2a7a2d501a974 epId=84cd4bdc local_ufrag=d88ru1f6if022j gid=59966 conf_name=loadtest48@conference.lmtgt1.dev2dev.net ufrag=d88ru1f6if022j] Agent.gatherCandidates#622: Gathering candidates for component stream-84cd4bdc.RTP.
JVB 2021-05-25 19:11:04.790 INFO: [207] [confId=5b2a7a2d501a974 epId=84cd4bdc gid=59966 conf_name=loadtest48@conference.lmtgt1.dev2dev.net] Endpoint.setTransportInfo#662: Ignoring empty DtlsFingerprint extension:
JVB 2021-05-25 19:11:05.547 WARNING: [207] [confId=9e7f39691eb453b8 epId=fd9ea629 gid=42769 stats_id=Carolyn-kdT conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 45258650
JVB 2021-05-25 19:11:05.548 WARNING: [207] [confId=9e7f39691eb453b8 epId=ae43624e gid=42769 stats_id=Mack-p0n conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 45258650
JVB 2021-05-25 19:11:05.602 WARNING: [207] [confId=9e7f39691eb453b8 epId=333f1281 gid=42769 stats_id=Giovanny-Wy8 conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 45258650
JVB 2021-05-25 19:11:05.604 WARNING: [207] [confId=9e7f39691eb453b8 epId=777d95de gid=42769 stats_id=Earline-f3L conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 45258650
JVB 2021-05-25 19:11:05.632 WARNING: [207] [confId=27d1ce7dcdbadc8a epId=22cf068e gid=8746 stats_id=Aric-qmS conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 23ad87de
JVB 2021-05-25 19:11:05.640 WARNING: [207] [confId=27d1ce7dcdbadc8a epId=86a82331 gid=8746 stats_id=Garland-kOE conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 23ad87de
JVB 2021-05-25 19:11:05.652 WARNING: [207] [confId=a20c8568d5fab362 epId=25d3be7a gid=8153 stats_id=Cathy-6UV conf_name=loadtest44@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 1524f81c
JVB 2021-05-25 19:11:05.659 WARNING: [207] [confId=27d1ce7dcdbadc8a epId=88885d24 gid=8746 stats_id=Daron-t3g conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 23ad87de
JVB 2021-05-25 19:11:05.662 WARNING: [207] [confId=f4db77e4558a6b37 epId=031e74d0 gid=22495 stats_id=Olen-quX conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: cffb4f6e
JVB 2021-05-25 19:11:05.670 WARNING: [207] [confId=f4db77e4558a6b37 epId=9a15d2cb gid=22495 stats_id=Delbert-IBz conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: cffb4f6e
JVB 2021-05-25 19:11:05.681 WARNING: [207] [confId=f4db77e4558a6b37 epId=b5e4924d gid=22495 stats_id=Alyson-hIO conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: cffb4f6e
JVB 2021-05-25 19:11:05.699 WARNING: [207] [confId=f4db77e4558a6b37 epId=7abf45ec gid=22495 stats_id=Kristian-yIk conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: cffb4f6e

==> /var/log/jitsi/jicofo.log <==
Jicofo 2021-05-25 19:11:05.708 INFO: [156] [room=loadtest33@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.stop#428: Stopped.
Jicofo 2021-05-25 19:11:05.708 WARNING: [156] FocusManager.conferenceRequest#244: Exception while trying to start the conference
org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 15000ms (~15s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=413yQ-2471)), : fromFilter (OrFilter: (FromMatchesFilter (full): loadtest33@conference.lmtgt1.dev2dev.net)).
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:111)
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:98)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:260)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:214)
at org.jivesoftware.smackx.muc.MultiUserChat.getConfigurationForm(MultiUserChat.java:781)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.joinAs(ChatRoomImpl.java:237)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.join(ChatRoomImpl.java:215)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.joinTheRoom(JitsiMeetConferenceImpl.java:466)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.start(JitsiMeetConferenceImpl.java:316)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:239)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:193)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:172)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.handleConferenceIq(ConferenceIqHandler.kt:65)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.access$handleConferenceIq(ConferenceIqHandler.kt:36)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler$handleIQRequest$2.run(ConferenceIqHandler.kt:153)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)

==> /var/log/jitsi/jvb.log <==
JVB 2021-05-25 19:11:05.721 WARNING: [207] [confId=9e7f39691eb453b8 epId=333f1281 gid=42769 stats_id=Giovanny-Wy8 conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 9f04bccc
JVB 2021-05-25 19:11:05.729 WARNING: [207] [confId=a20c8568d5fab362 epId=25d3be7a gid=8153 stats_id=Cathy-6UV conf_name=loadtest44@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 1627943d
JVB 2021-05-25 19:11:05.758 WARNING: [207] [confId=9e7f39691eb453b8 epId=fd9ea629 gid=42769 stats_id=Carolyn-kdT conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 9f04bccc
JVB 2021-05-25 19:11:05.814 INFO: [207] [confId=5b2a7a2d501a974 epId=f9227ad6 local_ufrag=3gald1f6if032m gid=59966 conf_name=loadtest48@conference.lmtgt1.dev2dev.net ufrag=3gald1f6if032m] Agent.gatherCandidates#622: Gathering candidates for component stream-f9227ad6.RTP.
JVB 2021-05-25 19:11:05.817 INFO: [207] [confId=5b2a7a2d501a974 epId=f9227ad6 gid=59966 conf_name=loadtest48@conference.lmtgt1.dev2dev.net] Endpoint.setTransportInfo#662: Ignoring empty DtlsFingerprint extension:
JVB 2021-05-25 19:11:05.913 WARNING: [207] [confId=9e7f39691eb453b8 epId=777d95de gid=42769 stats_id=Earline-f3L conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 9f04bccc
JVB 2021-05-25 19:11:05.933 WARNING: [207] [confId=9e7f39691eb453b8 epId=ae43624e gid=42769 stats_id=Mack-p0n conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 9f04bccc

==> /var/log/jitsi/jicofo.log <==
Jicofo 2021-05-25 19:11:06.062 INFO: [26] [room=loadtest42@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.stop#428: Stopped.
Jicofo 2021-05-25 19:11:06.062 INFO: [26] [room=loadtest31@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.stop#428: Stopped.
Jicofo 2021-05-25 19:11:06.062 INFO: [26] [room=loadtest23@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.stop#428: Stopped.
Jicofo 2021-05-25 19:11:06.062 INFO: [26] [room=loadtest16@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.stop#428: Stopped.
Jicofo 2021-05-25 19:11:06.716 WARNING: [175] FocusManager.conferenceRequest#244: Exception while trying to start the conference
org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 15000ms (~15s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=413yQ-2488)), : fromFilter (OrFilter: (FromMatchesFilter (full): loadtest42@conference.lmtgt1.dev2dev.net)).
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:111)
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:98)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:260)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:214)
at org.jivesoftware.smackx.muc.MultiUserChat.getConfigurationForm(MultiUserChat.java:781)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.joinAs(ChatRoomImpl.java:237)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.join(ChatRoomImpl.java:215)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.joinTheRoom(JitsiMeetConferenceImpl.java:466)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.start(JitsiMeetConferenceImpl.java:316)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:239)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:193)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:172)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.handleConferenceIq(ConferenceIqHandler.kt:65)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.access$handleConferenceIq(ConferenceIqHandler.kt:36)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler$handleIQRequest$2.run(ConferenceIqHandler.kt:153)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Jicofo 2021-05-25 19:11:06.716 WARNING: [179] FocusManager.conferenceRequest#244: Exception while trying to start the conference
org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 15000ms (~15s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=413yQ-2495)), : fromFilter (OrFilter: (FromMatchesFilter (full): loadtest23@conference.lmtgt1.dev2dev.net)).
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:111)
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:98)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:260)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:214)
at org.jivesoftware.smackx.muc.MultiUserChat.getConfigurationForm(MultiUserChat.java:781)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.joinAs(ChatRoomImpl.java:237)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.join(ChatRoomImpl.java:215)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.joinTheRoom(JitsiMeetConferenceImpl.java:466)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.start(JitsiMeetConferenceImpl.java:316)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:239)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:193)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:172)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.handleConferenceIq(ConferenceIqHandler.kt:65)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.access$handleConferenceIq(ConferenceIqHandler.kt:36)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler$handleIQRequest$2.run(ConferenceIqHandler.kt:153)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Jicofo 2021-05-25 19:11:06.716 WARNING: [216] FocusManager.conferenceRequest#244: Exception while trying to start the conference
org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 15000ms (~15s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=413yQ-2498)), : fromFilter (OrFilter: (FromMatchesFilter (full): loadtest16@conference.lmtgt1.dev2dev.net)).
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:111)
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:98)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:260)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:214)
at org.jivesoftware.smackx.muc.MultiUserChat.getConfigurationForm(MultiUserChat.java:781)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.joinAs(ChatRoomImpl.java:237)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.join(ChatRoomImpl.java:215)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.joinTheRoom(JitsiMeetConferenceImpl.java:466)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.start(JitsiMeetConferenceImpl.java:316)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:239)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:193)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:172)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.handleConferenceIq(ConferenceIqHandler.kt:65)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.access$handleConferenceIq(ConferenceIqHandler.kt:36)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler$handleIQRequest$2.run(ConferenceIqHandler.kt:153)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Jicofo 2021-05-25 19:11:06.716 WARNING: [194] FocusManager.conferenceRequest#244: Exception while trying to start the conference
org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 15000ms (~15s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=413yQ-2503)), : fromFilter (OrFilter: (FromMatchesFilter (full): loadtest36@conference.lmtgt1.dev2dev.net)).
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:111)
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:98)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:260)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:214)
at org.jivesoftware.smackx.muc.MultiUserChat.sendConfigurationForm(MultiUserChat.java:801)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.joinAs(ChatRoomImpl.java:257)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.join(ChatRoomImpl.java:215)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.joinTheRoom(JitsiMeetConferenceImpl.java:466)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.start(JitsiMeetConferenceImpl.java:316)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:239)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:193)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:172)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.handleConferenceIq(ConferenceIqHandler.kt:65)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.access$handleConferenceIq(ConferenceIqHandler.kt:36)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler$handleIQRequest$2.run(ConferenceIqHandler.kt:153)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Jicofo 2021-05-25 19:11:06.717 WARNING: [210] FocusManager.conferenceRequest#244: Exception while trying to start the conference
org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 15000ms (~15s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=413yQ-2492)), : fromFilter (OrFilter: (FromMatchesFilter (full): loadtest31@conference.lmtgt1.dev2dev.net)).
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:111)
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:98)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:260)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:214)
at org.jivesoftware.smackx.muc.MultiUserChat.getConfigurationForm(MultiUserChat.java:781)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.joinAs(ChatRoomImpl.java:237)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.join(ChatRoomImpl.java:215)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.joinTheRoom(JitsiMeetConferenceImpl.java:466)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.start(JitsiMeetConferenceImpl.java:316)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:239)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:193)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:172)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.handleConferenceIq(ConferenceIqHandler.kt:65)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.access$handleConferenceIq(ConferenceIqHandler.kt:36)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler$handleIQRequest$2.run(ConferenceIqHandler.kt:153)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Jicofo 2021-05-25 19:11:06.717 INFO: [193] [room=loadtest37@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.stop#428: Stopped.

==> /var/log/jitsi/jvb.log <==
JVB 2021-05-25 19:11:06.717 WARNING: [207] [confId=27d1ce7dcdbadc8a epId=22cf068e gid=8746 stats_id=Aric-qmS conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: d5b0fd5c

==> /var/log/jitsi/jicofo.log <==
Jicofo 2021-05-25 19:11:06.717 WARNING: [193] FocusManager.conferenceRequest#244: Exception while trying to start the conference
org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 15000ms (~15s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=413yQ-2505)), : fromFilter (OrFilter: (FromMatchesFilter (full): loadtest37@conference.lmtgt1.dev2dev.net)).
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:111)
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:98)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:260)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:214)
at org.jivesoftware.smackx.muc.MultiUserChat.sendConfigurationForm(MultiUserChat.java:801)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.joinAs(ChatRoomImpl.java:257)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.join(ChatRoomImpl.java:215)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.joinTheRoom(JitsiMeetConferenceImpl.java:466)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.start(JitsiMeetConferenceImpl.java:316)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:239)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:193)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:172)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.handleConferenceIq(ConferenceIqHandler.kt:65)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.access$handleConferenceIq(ConferenceIqHandler.kt:36)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler$handleIQRequest$2.run(ConferenceIqHandler.kt:153)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Jicofo 2021-05-25 19:11:06.717 WARNING: [166] FocusManager.conferenceRequest#244: Exception while trying to start the conference
org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 15000ms (~15s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=413yQ-2501)), : fromFilter (OrFilter: (FromMatchesFilter (full): loadtest35@conference.lmtgt1.dev2dev.net)).
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:111)
at org.jivesoftware.smack.SmackException$NoResponseException.newWith(SmackException.java:98)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:260)
at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:214)
at org.jivesoftware.smackx.muc.MultiUserChat.sendConfigurationForm(MultiUserChat.java:801)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.joinAs(ChatRoomImpl.java:257)
at org.jitsi.impl.protocol.xmpp.ChatRoomImpl.join(ChatRoomImpl.java:215)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.joinTheRoom(JitsiMeetConferenceImpl.java:466)
at org.jitsi.jicofo.JitsiMeetConferenceImpl.start(JitsiMeetConferenceImpl.java:316)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:239)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:193)
at org.jitsi.jicofo.FocusManager.conferenceRequest(FocusManager.java:172)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.handleConferenceIq(ConferenceIqHandler.kt:65)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler.access$handleConferenceIq(ConferenceIqHandler.kt:36)
at org.jitsi.jicofo.xmpp.ConferenceIqHandler$handleIQRequest$2.run(ConferenceIqHandler.kt:153)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Jicofo 2021-05-25 19:11:06.718 WARNING: [67] JvbDoctor$HealthCheckTask.doHealthCheck#233: jvbbrewery@internal.auth.lmtgt1.dev2dev.net/f40e5aa1-1094-4e41-b913-b473039e6de7 health-check timed out, but will give it another try after: 5000

==> /var/log/jitsi/jvb.log <==
JVB 2021-05-25 19:11:06.726 WARNING: [207] [confId=27d1ce7dcdbadc8a epId=88885d24 gid=8746 stats_id=Daron-t3g conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: d5b0fd5c
JVB 2021-05-25 19:11:06.749 WARNING: [207] [confId=5b2a7a2d501a974 epId=377eedfe gid=59966 stats_id=Lawson-J7Z conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 89152c84
JVB 2021-05-25 19:11:06.753 WARNING: [207] [confId=5b2a7a2d501a974 epId=03baf5ad gid=59966 stats_id=Marquise-1yk conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 89152c84
JVB 2021-05-25 19:11:06.753 WARNING: [192] [confId=5b2a7a2d501a974 epId=77b91288 gid=59966 stats_id=Allan-ahM conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 89152c84

==> /var/log/jitsi/jicofo.log <==
Jicofo 2021-05-25 19:11:06.759 INFO: [170] [room=loadtest57@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.terminateParticipant#1130: Terminating da4308d4, reason: general-error, send session-terminate: true
Jicofo 2021-05-25 19:11:06.759 INFO: [170] [room=loadtest57@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.terminateParticipant#1155: Removed participant da4308d4 removed=true

==> /var/log/jitsi/jvb.log <==
JVB 2021-05-25 19:11:06.768 WARNING: [192] [confId=27d1ce7dcdbadc8a epId=86a82331 gid=8746 stats_id=Garland-kOE conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: d5b0fd5c
JVB 2021-05-25 19:11:06.916 WARNING: [192] [confId=27d1ce7dcdbadc8a epId=22cf068e gid=8746 stats_id=Aric-qmS conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 741005a7
JVB 2021-05-25 19:11:06.973 WARNING: [192] [confId=27d1ce7dcdbadc8a epId=86a82331 gid=8746 stats_id=Garland-kOE conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 741005a7
JVB 2021-05-25 19:11:07.105 WARNING: [207] [confId=27d1ce7dcdbadc8a epId=88885d24 gid=8746 stats_id=Daron-t3g conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 741005a7
JVB 2021-05-25 19:11:07.111 WARNING: [207] [confId=d80b9097ae979ad6 epId=7f1f7e5e gid=22435 stats_id=Verdie-XpJ conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 3b7b17f9
JVB 2021-05-25 19:11:07.112 WARNING: [207] [confId=d80b9097ae979ad6 epId=7f1f7e5e gid=22435 stats_id=Verdie-XpJ conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: b02b6a6a
JVB 2021-05-25 19:11:07.112 WARNING: [207] [confId=d80b9097ae979ad6 epId=7f1f7e5e gid=22435 stats_id=Verdie-XpJ conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ce15a83c
JVB 2021-05-25 19:11:07.112 WARNING: [207] [confId=d80b9097ae979ad6 epId=7f1f7e5e gid=22435 stats_id=Verdie-XpJ conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f36a93b4
JVB 2021-05-25 19:11:07.112 WARNING: [207] [confId=d80b9097ae979ad6 epId=7f1f7e5e gid=22435 stats_id=Verdie-XpJ conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 7b6786db
JVB 2021-05-25 19:11:07.581 WARNING: [192] [confId=e9754803553e1e5a epId=342b51b5 gid=52251 stats_id=Lauren-uwO conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 009f7802
JVB 2021-05-25 19:11:07.589 WARNING: [192] [confId=5b2a7a2d501a974 epId=377eedfe gid=59966 stats_id=Lawson-J7Z conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: b7167f1c
JVB 2021-05-25 19:11:07.590 WARNING: [192] [confId=5b2a7a2d501a974 epId=03baf5ad gid=59966 stats_id=Marquise-1yk conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: b7167f1c
JVB 2021-05-25 19:11:07.592 WARNING: [192] [confId=5b2a7a2d501a974 epId=77b91288 gid=59966 stats_id=Allan-ahM conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: b7167f1c
JVB 2021-05-25 19:11:07.595 WARNING: [192] [confId=c517332eb7628b5d epId=757f7b17 gid=5659 stats_id=Elenor-iZJ conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 52a50828
JVB 2021-05-25 19:11:07.596 WARNING: [192] [confId=c517332eb7628b5d epId=5fbdde59 gid=5659 stats_id=Clovis-wxH conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 52a50828
JVB 2021-05-25 19:11:07.597 WARNING: [192] [confId=c517332eb7628b5d epId=7136198d gid=5659 stats_id=Edwardo-dNx conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 52a50828
JVB 2021-05-25 19:11:07.624 WARNING: [192] [confId=f4db77e4558a6b37 epId=b5e4924d gid=22495 stats_id=Alyson-hIO conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ede9274e
JVB 2021-05-25 19:11:07.624 WARNING: [192] [confId=f4db77e4558a6b37 epId=9a15d2cb gid=22495 stats_id=Delbert-IBz conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ede9274e
JVB 2021-05-25 19:11:07.626 WARNING: [192] [confId=f4db77e4558a6b37 epId=031e74d0 gid=22495 stats_id=Olen-quX conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ede9274e
JVB 2021-05-25 19:11:07.626 WARNING: [192] [confId=f4db77e4558a6b37 epId=7abf45ec gid=22495 stats_id=Kristian-yIk conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ede9274e
JVB 2021-05-25 19:11:07.722 WARNING: [192] [confId=5b2a7a2d501a974 epId=03baf5ad gid=59966 stats_id=Marquise-1yk conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 2263e607
JVB 2021-05-25 19:11:07.726 WARNING: [192] [confId=d80b9097ae979ad6 epId=7f1f7e5e gid=22435 stats_id=Verdie-XpJ conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: b02b6a6a
JVB 2021-05-25 19:11:07.727 WARNING: [192] [confId=d80b9097ae979ad6 epId=0a70351b gid=22435 stats_id=Esperanza-6p7 conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: b02b6a6a
JVB 2021-05-25 19:11:07.728 WARNING: [192] [confId=d80b9097ae979ad6 epId=8d0e7ca1 gid=22435 stats_id=Terrill-jQN conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: b02b6a6a

==> /var/log/jitsi/jicofo.log <==
Jicofo 2021-05-25 19:11:07.731 INFO: [167] [room=loadtest57@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.terminateParticipant#1130: Terminating 2df7d64c, reason: general-error, send session-terminate: true
Jicofo 2021-05-25 19:11:07.731 INFO: [190] [room=loadtest44@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.terminateParticipant#1130: Terminating 4acd896f, reason: general-error, send session-terminate: true
Jicofo 2021-05-25 19:11:07.731 INFO: [167] [room=loadtest57@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.terminateParticipant#1155: Removed participant 2df7d64c removed=true
Jicofo 2021-05-25 19:11:07.731 INFO: [190] [room=loadtest44@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.terminateParticipant#1155: Removed participant 4acd896f removed=true

==> /var/log/jitsi/jvb.log <==
JVB 2021-05-25 19:11:07.746 WARNING: [192] [confId=c517332eb7628b5d epId=5fbdde59 gid=5659 stats_id=Clovis-wxH conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 86a9dc8e
JVB 2021-05-25 19:11:07.752 WARNING: [192] [confId=d80b9097ae979ad6 epId=db314a8b gid=22435 stats_id=Benedict-gXA conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: b02b6a6a
JVB 2021-05-25 19:11:07.759 WARNING: [192] [confId=f4db77e4558a6b37 epId=031e74d0 gid=22495 stats_id=Olen-quX conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: c6949ce2
JVB 2021-05-25 19:11:07.763 WARNING: [192] [confId=a20c8568d5fab362 epId=25d3be7a gid=8153 stats_id=Cathy-6UV conf_name=loadtest44@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f8e17b04
JVB 2021-05-25 19:11:07.764 WARNING: [192] [confId=5b2a7a2d501a974 epId=377eedfe gid=59966 stats_id=Lawson-J7Z conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 2263e607
JVB 2021-05-25 19:11:07.787 WARNING: [192] [confId=f4db77e4558a6b37 epId=b5e4924d gid=22495 stats_id=Alyson-hIO conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: c6949ce2
JVB 2021-05-25 19:11:07.800 WARNING: [192] [confId=5b2a7a2d501a974 epId=77b91288 gid=59966 stats_id=Allan-ahM conf_name=loadtest48@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 2263e607
JVB 2021-05-25 19:11:07.809 WARNING: [192] [confId=d80b9097ae979ad6 epId=9366ed40 gid=22435 stats_id=Brooks-o9f conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: b02b6a6a
JVB 2021-05-25 19:11:07.828 WARNING: [192] [confId=c517332eb7628b5d epId=757f7b17 gid=5659 stats_id=Elenor-iZJ conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 86a9dc8e
JVB 2021-05-25 19:11:07.848 WARNING: [192] [confId=f4db77e4558a6b37 epId=9a15d2cb gid=22495 stats_id=Delbert-IBz conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: c6949ce2
JVB 2021-05-25 19:11:07.854 WARNING: [192] [confId=c517332eb7628b5d epId=7136198d gid=5659 stats_id=Edwardo-dNx conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 86a9dc8e
JVB 2021-05-25 19:11:07.870 WARNING: [192] [confId=c517332eb7628b5d epId=7136198d gid=5659 stats_id=Edwardo-dNx conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 5fd4f647
JVB 2021-05-25 19:11:07.904 WARNING: [192] [confId=f4db77e4558a6b37 epId=7abf45ec gid=22495 stats_id=Kristian-yIk conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: c6949ce2
JVB 2021-05-25 19:11:07.921 WARNING: [192] [confId=c517332eb7628b5d epId=757f7b17 gid=5659 stats_id=Elenor-iZJ conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 5fd4f647
JVB 2021-05-25 19:11:07.927 WARNING: [192] [confId=c517332eb7628b5d epId=5fbdde59 gid=5659 stats_id=Clovis-wxH conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 5fd4f647
JVB 2021-05-25 19:11:07.947 WARNING: [192] [confId=c517332eb7628b5d epId=5fbdde59 gid=5659 stats_id=Clovis-wxH conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 0295f796
JVB 2021-05-25 19:11:07.958 WARNING: [192] [confId=c517332eb7628b5d epId=7136198d gid=5659 stats_id=Edwardo-dNx conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 0295f796
JVB 2021-05-25 19:11:08.022 WARNING: [192] [confId=c517332eb7628b5d epId=757f7b17 gid=5659 stats_id=Elenor-iZJ conf_name=loadtest55@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 0295f796
JVB 2021-05-25 19:11:08.556 WARNING: [192] [confId=3fc89e2aeb396954 epId=b2c66cad gid=12500 stats_id=Esta-MeH conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 372bfaec
JVB 2021-05-25 19:11:08.559 WARNING: [192] [confId=3fc89e2aeb396954 epId=a8f51fd2 gid=12500 stats_id=Blanca-D4A conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 372bfaec
JVB 2021-05-25 19:11:08.562 WARNING: [192] [confId=d80b9097ae979ad6 epId=7f1f7e5e gid=22435 stats_id=Verdie-XpJ conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ce15a83c
JVB 2021-05-25 19:11:08.563 WARNING: [192] [confId=d80b9097ae979ad6 epId=db314a8b gid=22435 stats_id=Benedict-gXA conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ce15a83c
JVB 2021-05-25 19:11:08.563 WARNING: [192] [confId=d80b9097ae979ad6 epId=9366ed40 gid=22435 stats_id=Brooks-o9f conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ce15a83c
JVB 2021-05-25 19:11:08.564 WARNING: [192] [confId=d80b9097ae979ad6 epId=0a70351b gid=22435 stats_id=Esperanza-6p7 conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ce15a83c
JVB 2021-05-25 19:11:08.565 WARNING: [192] [confId=d80b9097ae979ad6 epId=8d0e7ca1 gid=22435 stats_id=Terrill-jQN conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: ce15a83c
JVB 2021-05-25 19:11:08.568 WARNING: [192] [confId=3fc89e2aeb396954 epId=be1ca62c gid=12500 stats_id=Gwen-N7H conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 372bfaec
JVB 2021-05-25 19:11:08.589 WARNING: [192] [confId=e9754803553e1e5a epId=342b51b5 gid=52251 stats_id=Lauren-uwO conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: db7d84ac
JVB 2021-05-25 19:11:08.591 WARNING: [192] [confId=9e7f39691eb453b8 epId=fd9ea629 gid=42769 stats_id=Carolyn-kdT conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: d669713c
JVB 2021-05-25 19:11:08.592 WARNING: [192] [confId=9e7f39691eb453b8 epId=777d95de gid=42769 stats_id=Earline-f3L conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: d669713c
JVB 2021-05-25 19:11:08.592 WARNING: [192] [confId=9e7f39691eb453b8 epId=333f1281 gid=42769 stats_id=Giovanny-Wy8 conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: d669713c
JVB 2021-05-25 19:11:08.593 WARNING: [192] [confId=9e7f39691eb453b8 epId=ae43624e gid=42769 stats_id=Mack-p0n conf_name=loadtest58@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: d669713c
JVB 2021-05-25 19:11:08.597 WARNING: [192] [confId=a20c8568d5fab362 epId=25d3be7a gid=8153 stats_id=Cathy-6UV conf_name=loadtest44@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: d63e56ce
JVB 2021-05-25 19:11:08.692 WARNING: [192] [confId=27d1ce7dcdbadc8a epId=22cf068e gid=8746 stats_id=Aric-qmS conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 7025ba3e
JVB 2021-05-25 19:11:08.696 WARNING: [192] [confId=27d1ce7dcdbadc8a epId=86a82331 gid=8746 stats_id=Garland-kOE conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 7025ba3e
JVB 2021-05-25 19:11:08.697 WARNING: [192] [confId=27d1ce7dcdbadc8a epId=88885d24 gid=8746 stats_id=Daron-t3g conf_name=loadtest41@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 7025ba3e
JVB 2021-05-25 19:11:08.699 WARNING: [192] [confId=d80b9097ae979ad6 epId=7f1f7e5e gid=22435 stats_id=Verdie-XpJ conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f36a93b4
JVB 2021-05-25 19:11:08.721 WARNING: [192] [confId=d80b9097ae979ad6 epId=7f1f7e5e gid=22435 stats_id=Verdie-XpJ conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 7b6786db

==> /var/log/jitsi/jicofo.log <==
Jicofo 2021-05-25 19:11:08.730 INFO: [196] [room=loadtest44@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.terminateParticipant#1130: Terminating a82ce431, reason: general-error, send session-terminate: true
Jicofo 2021-05-25 19:11:08.730 INFO: [196] [room=loadtest44@conference.lmtgt1.dev2dev.net] JitsiMeetConferenceImpl.terminateParticipant#1155: Removed participant a82ce431 removed=true

==> /var/log/jitsi/jvb.log <==
JVB 2021-05-25 19:11:08.759 WARNING: [192] [confId=d80b9097ae979ad6 epId=db314a8b gid=22435 stats_id=Benedict-gXA conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f36a93b4
JVB 2021-05-25 19:11:08.768 WARNING: [192] [confId=d80b9097ae979ad6 epId=0a70351b gid=22435 stats_id=Esperanza-6p7 conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f36a93b4
JVB 2021-05-25 19:11:08.777 WARNING: [192] [confId=d80b9097ae979ad6 epId=db314a8b gid=22435 stats_id=Benedict-gXA conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 7b6786db
JVB 2021-05-25 19:11:08.781 WARNING: [192] [confId=a20c8568d5fab362 epId=25d3be7a gid=8153 stats_id=Cathy-6UV conf_name=loadtest44@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 3918e29a
JVB 2021-05-25 19:11:08.802 WARNING: [192] [confId=d80b9097ae979ad6 epId=8d0e7ca1 gid=22435 stats_id=Terrill-jQN conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f36a93b4
JVB 2021-05-25 19:11:08.824 WARNING: [192] [confId=d80b9097ae979ad6 epId=9366ed40 gid=22435 stats_id=Brooks-o9f conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: f36a93b4
JVB 2021-05-25 19:11:08.858 WARNING: [192] [confId=d80b9097ae979ad6 epId=0a70351b gid=22435 stats_id=Esperanza-6p7 conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 7b6786db
JVB 2021-05-25 19:11:08.897 WARNING: [192] [confId=d80b9097ae979ad6 epId=8d0e7ca1 gid=22435 stats_id=Terrill-jQN conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 7b6786db
JVB 2021-05-25 19:11:08.917 WARNING: [192] [confId=d80b9097ae979ad6 epId=9366ed40 gid=22435 stats_id=Brooks-o9f conf_name=loadtest53@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 7b6786db
JVB 2021-05-25 19:11:08.962 INFO: [31] HealthChecker.run#171: Performed a successful health check in PT0.000005S. Sticky failure: false
JVB 2021-05-25 19:11:09.257 WARNING: [207] [confId=e9754803553e1e5a epId=da4308d4 gid=52251 stats_id=Orville-2io conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 832621ff
JVB 2021-05-25 19:11:09.257 WARNING: [207] [confId=e9754803553e1e5a epId=da4308d4 gid=52251 stats_id=Orville-2io conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 6bff0854
JVB 2021-05-25 19:11:09.257 WARNING: [207] [confId=e9754803553e1e5a epId=da4308d4 gid=52251 stats_id=Orville-2io conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 009f7802
JVB 2021-05-25 19:11:09.257 WARNING: [207] [confId=e9754803553e1e5a epId=da4308d4 gid=52251 stats_id=Orville-2io conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: db7d84ac
JVB 2021-05-25 19:11:09.257 WARNING: [207] [confId=e9754803553e1e5a epId=da4308d4 gid=52251 stats_id=Orville-2io conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 6edabe96
JVB 2021-05-25 19:11:09.257 WARNING: [207] [confId=e9754803553e1e5a epId=da4308d4 gid=52251 stats_id=Orville-2io conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: a4e4f4c0
JVB 2021-05-25 19:11:09.597 WARNING: [192] [confId=e9754803553e1e5a epId=342b51b5 gid=52251 stats_id=Lauren-uwO conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 6edabe96
JVB 2021-05-25 19:11:09.628 WARNING: [192] [confId=3fc89e2aeb396954 epId=be1ca62c gid=12500 stats_id=Gwen-N7H conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 26059a06
JVB 2021-05-25 19:11:09.631 WARNING: [192] [confId=f4db77e4558a6b37 epId=9a15d2cb gid=22495 stats_id=Delbert-IBz conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 3d765b4b
JVB 2021-05-25 19:11:09.632 WARNING: [192] [confId=f4db77e4558a6b37 epId=031e74d0 gid=22495 stats_id=Olen-quX conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 3d765b4b
JVB 2021-05-25 19:11:09.634 WARNING: [192] [confId=f4db77e4558a6b37 epId=b5e4924d gid=22495 stats_id=Alyson-hIO conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 3d765b4b
JVB 2021-05-25 19:11:09.639 WARNING: [192] [confId=3fc89e2aeb396954 epId=b2c66cad gid=12500 stats_id=Esta-MeH conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 26059a06
JVB 2021-05-25 19:11:09.640 WARNING: [192] [confId=3fc89e2aeb396954 epId=a8f51fd2 gid=12500 stats_id=Blanca-D4A conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 26059a06
JVB 2021-05-25 19:11:09.643 WARNING: [192] [confId=f4db77e4558a6b37 epId=7abf45ec gid=22495 stats_id=Kristian-yIk conf_name=loadtest52@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 3d765b4b
JVB 2021-05-25 19:11:09.652 WARNING: [192] [confId=e9754803553e1e5a epId=da4308d4 gid=52251 stats_id=Orville-2io conf_name=loadtest57@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 6edabe96
JVB 2021-05-25 19:11:09.663 WARNING: [192] [confId=3fc89e2aeb396954 epId=b2c66cad gid=12500 stats_id=Esta-MeH conf_name=loadtest49@conference.lmtgt1.dev2dev.net] EndpointMessageTransport.endpointMessage#534: Unable to find endpoint to send EndpointMessage to: 062a9af3
^C
root@lmtgt1:/home/ubuntu#

My assumption from those logs was that because I hadn’t made any of the websockets or smack changes listed in the upgrade prosody instructions that it is not fully configured yet, and that was why I moved on to those steps, since this doesn’t work as-is for more than a few participants.

I am holding off making any of those changes for now in the hopes that something else stands out.

Are those logs helpful? Want me to post the current config files, or do the logs jump out anything obvious?

Thanks!

This is a bad sign: Can not invite participant, no bridge available

That would help me. For this dev test, I assume this server is acting as both the JMS and the video bridge? (sorry if you already mentioned this). I don’t think upgrading Prosody is your issue. Looks like a config issue to me.

Edit: By the way, the start up portion of the logs is usually the most telling of config issues.

Yes, this is all-in-one server basic baseline, all defaults.

Here are the config files from this fresh setup without any of the recommended changes in the setup instructions:

cat /etc/nginx/sites-available/lmtgt1.dev2dev.net.conf 
server_names_hash_bucket_size 64;

types {
# nginx's default mime.types doesn't include a mapping for wasm
    application/wasm     wasm;
}
server {
    listen 80;
    listen [::]:80;
    server_name lmtgt1.dev2dev.net;

    location ^~ /.well-known/acme-challenge/ {
        default_type "text/plain";
        root         /usr/share/jitsi-meet;
    }
    location = /.well-known/acme-challenge/ {
        return 404;
    }
    location / {
        return 301 https://$host$request_uri;
    }
}
server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name lmtgt1.dev2dev.net;

    # Mozilla Guideline v5.4, nginx 1.17.7, OpenSSL 1.1.1d, intermediate configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
    ssl_prefer_server_ciphers off;

    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:10m;  # about 40000 sessions
    ssl_session_tickets off;

    add_header Strict-Transport-Security "max-age=63072000" always;

    ssl_certificate /etc/letsencrypt/live/lmtgt1.dev2dev.net/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/lmtgt1.dev2dev.net/privkey.pem;

    root /usr/share/jitsi-meet;

    # ssi on with javascript for multidomain variables in config.js
    ssi on;
    ssi_types application/x-javascript application/javascript;

    index index.html index.htm;
    error_page 404 /static/404.html;

    gzip on;
    gzip_types text/plain text/css application/javascript application/json image/x-icon application/octet-stream application/wasm;
    gzip_vary on;
    gzip_proxied no-cache no-store private expired auth;
    gzip_min_length 512;

    location = /config.js {
        alias /etc/jitsi/meet/lmtgt1.dev2dev.net-config.js;
    }

    location = /external_api.js {
        alias /usr/share/jitsi-meet/libs/external_api.min.js;
    }

    # ensure all static content can always be found first
    location ~ ^/(libs|css|static|images|fonts|lang|sounds|connection_optimization|.well-known)/(.*)$
    {
        add_header 'Access-Control-Allow-Origin' '*';
        alias /usr/share/jitsi-meet/$1/$2;

        # cache all versioned files
        if ($arg_v) {
            expires 1y;
        }
    }

    # BOSH
    location = /http-bind {
        proxy_pass       http://localhost:5280/http-bind;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $http_host;
    }

    # xmpp websockets
    location = /xmpp-websocket {
        proxy_pass http://127.0.0.1:5280/xmpp-websocket?prefix=$prefix&$args;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $http_host;
        tcp_nodelay on;
    }

    # colibri (JVB) websockets for jvb1
    location ~ ^/colibri-ws/default-id/(.*) {
        proxy_pass http://127.0.0.1:9090/colibri-ws/default-id/$1$is_args$args;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        tcp_nodelay on;
    }

    # load test minimal client, uncomment when used
    #location ~ ^/_load-test/([^/?&:'"]+)$ {
    #    rewrite ^/_load-test/(.*)$ /load-test/index.html break;
    #}
    #location ~ ^/_load-test/libs/(.*)$ {
    #    add_header 'Access-Control-Allow-Origin' '*';
    #    alias /usr/share/jitsi-meet/load-test/libs/$1;
    #}

    location ~ ^/([^/?&:'"]+)$ {
        try_files $uri @root_path;
    }

    location @root_path {
        rewrite ^/(.*)$ / break;
    }

    location ~ ^/([^/?&:'"]+)/config.js$
    {
        set $subdomain "$1.";
        set $subdir "$1/";

        alias /etc/jitsi/meet/lmtgt1.dev2dev.net-config.js;
    }

    # Anything that didn't match above, and isn't a real file, assume it's a room name and redirect to /
    location ~ ^/([^/?&:'"]+)/(.*)$ {
        set $subdomain "$1.";
        set $subdir "$1/";
        rewrite ^/([^/?&:'"]+)/(.*)$ /$2;
    }

    # BOSH for subdomains
    location ~ ^/([^/?&:'"]+)/http-bind {
        set $subdomain "$1.";
        set $subdir "$1/";
        set $prefix "$1";

        rewrite ^/(.*)$ /http-bind;
    }

    # websockets for subdomains
    location ~ ^/([^/?&:'"]+)/xmpp-websocket {
        set $subdomain "$1.";
        set $subdir "$1/";
        set $prefix "$1";

        rewrite ^/(.*)$ /xmpp-websocket;
    }
}
cat /etc/prosody/prosody.cfg.lua 
-- Prosody XMPP Server Configuration
--
-- Information on configuring Prosody can be found on our
-- website at https://prosody.im/doc/configure
--
-- Tip: You can check that the syntax of this file is correct
-- when you have finished by running this command:
--     prosodyctl check config
-- If there are any errors, it will let you know what and where
-- they are, otherwise it will keep quiet.
--
-- Good luck, and happy Jabbering!


---------- Server-wide settings ----------
-- Settings in this section apply to the whole server and are the default settings
-- for any virtual hosts

-- This is a (by default, empty) list of accounts that are admins
-- for the server. Note that you must create the accounts separately
-- (see https://prosody.im/doc/creating_accounts for info)
-- Example: admins = { "user1@example.com", "user2@example.net" }
admins = { }

-- Enable use of libevent for better performance under high load
-- For more information see: https://prosody.im/doc/libevent
--use_libevent = true

-- Prosody will always look in its source directory for modules, but
-- this option allows you to specify additional locations where Prosody
-- will look for modules first. For community modules, see https://modules.prosody.im/
--plugin_paths = {}

-- This is the list of modules Prosody will load on startup.
-- It looks for mod_modulename.lua in the plugins folder, so make sure that exists too.
-- Documentation for bundled modules can be found at: https://prosody.im/doc/modules
modules_enabled = {

	-- Generally required
		"roster"; -- Allow users to have a roster. Recommended ;)
		"saslauth"; -- Authentication for clients and servers. Recommended if you want to log in.
		"tls"; -- Add support for secure TLS on c2s/s2s connections
		"dialback"; -- s2s dialback support
		"disco"; -- Service discovery

	-- Not essential, but recommended
		"carbons"; -- Keep multiple clients in sync
		"pep"; -- Enables users to publish their avatar, mood, activity, playing music and more
		"private"; -- Private XML storage (for room bookmarks, etc.)
		"blocklist"; -- Allow users to block communications with other users
		"vcard4"; -- User profiles (stored in PEP)
		"vcard_legacy"; -- Conversion between legacy vCard and PEP Avatar, vcard
		"limits"; -- Enable bandwidth limiting for XMPP connections

	-- Nice to have
		"version"; -- Replies to server version requests
		"uptime"; -- Report how long server has been running
		"time"; -- Let others know the time here on this server
		"ping"; -- Replies to XMPP pings with pongs
		"register"; -- Allow users to register on this server using a client and change passwords
		--"mam"; -- Store messages in an archive and allow users to access it
		--"csi_simple"; -- Simple Mobile optimizations

	-- Admin interfaces
		"admin_adhoc"; -- Allows administration via an XMPP client that supports ad-hoc commands
		--"admin_telnet"; -- Opens telnet console interface on localhost port 5582

	-- HTTP modules
		--"bosh"; -- Enable BOSH clients, aka "Jabber over HTTP"
		--"websocket"; -- XMPP over WebSockets
		--"http_files"; -- Serve static files from a directory over HTTP

	-- Other specific functionality
		--"groups"; -- Shared roster support
		--"server_contact_info"; -- Publish contact information for this service
		--"announce"; -- Send announcement to all online users
		--"welcome"; -- Welcome users who register accounts
		--"watchregistrations"; -- Alert admins of registrations
		--"motd"; -- Send a message to users when they log in
		--"legacyauth"; -- Legacy authentication. Only used by some old clients and bots.
		--"proxy65"; -- Enables a file transfer proxy service which clients behind NAT can use
}

-- These modules are auto-loaded, but should you want
-- to disable them then uncomment them here:
modules_disabled = {
	-- "offline"; -- Store offline messages
	-- "c2s"; -- Handle client connections
	-- "s2s"; -- Handle server-to-server connections
	-- "posix"; -- POSIX functionality, sends server to background, enables syslog, etc.
}

-- Disable account creation by default, for security
-- For more information see https://prosody.im/doc/creating_accounts
allow_registration = false

-- Force clients to use encrypted connections? This option will
-- prevent clients from authenticating unless they are using encryption.

c2s_require_encryption = true

-- Force servers to use encrypted connections? This option will
-- prevent servers from authenticating unless they are using encryption.

s2s_require_encryption = true

-- Force certificate authentication for server-to-server connections?

s2s_secure_auth = false

-- Some servers have invalid or self-signed certificates. You can list
-- remote domains here that will not be required to authenticate using
-- certificates. They will be authenticated using DNS instead, even
-- when s2s_secure_auth is enabled.

--s2s_insecure_domains = { "insecure.example" }

-- Even if you disable s2s_secure_auth, you can still require valid
-- certificates for some domains by specifying a list here.

--s2s_secure_domains = { "jabber.org" }

-- Enable rate limits for incoming client and server connections

limits = {
  c2s = {
    rate = "10kb/s";
  };
  s2sin = {
    rate = "30kb/s";
  };
}

-- Required for init scripts and prosodyctl
pidfile = "/var/run/prosody/prosody.pid"

-- Select the authentication backend to use. The 'internal' providers
-- use Prosody's configured data storage to store the authentication data.

authentication = "internal_hashed"

-- Select the storage backend to use. By default Prosody uses flat files
-- in its configured data directory, but it also supports more backends
-- through modules. An "sql" backend is included by default, but requires
-- additional dependencies. See https://prosody.im/doc/storage for more info.

--storage = "sql" -- Default is "internal"

-- For the "sql" backend, you can uncomment *one* of the below to configure:
--sql = { driver = "SQLite3", database = "prosody.sqlite" } -- Default. 'database' is the filename.
--sql = { driver = "MySQL", database = "prosody", username = "prosody", password = "secret", host = "localhost" }
--sql = { driver = "PostgreSQL", database = "prosody", username = "prosody", password = "secret", host = "localhost" }


-- Archiving configuration
-- If mod_mam is enabled, Prosody will store a copy of every message. This
-- is used to synchronize conversations between multiple clients, even if
-- they are offline. This setting controls how long Prosody will keep
-- messages in the archive before removing them.

archive_expires_after = "1w" -- Remove archived messages after 1 week

-- You can also configure messages to be stored in-memory only. For more
-- archiving options, see https://prosody.im/doc/modules/mod_mam

-- Logging configuration
-- For advanced logging see https://prosody.im/doc/logging
log = {
	info = "/var/log/prosody/prosody.log"; -- Change 'info' to 'debug' for verbose logging
	error = "/var/log/prosody/prosody.err";
	-- "*syslog"; -- Uncomment this for logging to syslog
	-- "*console"; -- Log to the console, useful for debugging with daemonize=false
}

-- Uncomment to enable statistics
-- For more info see https://prosody.im/doc/statistics
-- statistics = "internal"

-- Certificates
-- Every virtual host and component needs a certificate so that clients and
-- servers can securely verify its identity. Prosody will automatically load
-- certificates/keys from the directory specified here.
-- For more information, including how to use 'prosodyctl' to auto-import certificates
-- (from e.g. Let's Encrypt) see https://prosody.im/doc/certificates

-- Location of directory to find certificates in (relative to main config file):
certificates = "certs"

-- HTTPS currently only supports a single certificate, specify it here:
--https_certificate = "/etc/prosody/certs/localhost.crt"

----------- Virtual hosts -----------
-- You need to add a VirtualHost entry for each domain you wish Prosody to serve.
-- Settings under each VirtualHost entry apply *only* to that host.

VirtualHost "localhost"

--VirtualHost "example.com"
--	certificate = "/path/to/example.crt"

------ Components ------
-- You can specify components to add hosts that provide special services,
-- like multi-user conferences, and transports.
-- For more information on components, see https://prosody.im/doc/components

---Set up a MUC (multi-user chat) room server on conference.example.com:
--Component "conference.example.com" "muc"
--- Store MUC messages in an archive and allow users to access it
--modules_enabled = { "muc_mam" }

---Set up an external component (default component port is 5347)
--
-- External components allow adding various services, such as gateways/
-- transports to other networks like ICQ, MSN and Yahoo. For more info
-- see: https://prosody.im/doc/components#adding_an_external_component
--
--Component "gateway.example.com"
--	component_secret = "password"

Include "conf.d/*.cfg.lua"
cat /etc/jitsi/meet/lmtgt1.dev2dev.net
cat: /etc/jitsi/meet/lmtgt1.dev2dev.net: No such file or directory
root@lmtgt1:/home/ubuntu# cat /etc/jitsi/meet/lmtgt1.dev2dev.net-config.js 
/* eslint-disable no-unused-vars, no-var */

var config = {
    // Connection
    //

    hosts: {
        // XMPP domain.
        domain: 'lmtgt1.dev2dev.net',

        // When using authentication, domain for guest users.
        // anonymousdomain: 'guest.example.com',

        // Domain for authenticated users. Defaults to <domain>.
        // authdomain: 'lmtgt1.dev2dev.net',

        // Focus component domain. Defaults to focus.<domain>.
        // focus: 'focus.lmtgt1.dev2dev.net',

        // XMPP MUC domain. FIXME: use XEP-0030 to discover it.
        muc: 'conference.<!--# echo var="subdomain" default="" -->lmtgt1.dev2dev.net'
    },

    // BOSH URL. FIXME: use XEP-0156 to discover it.
    bosh: '//lmtgt1.dev2dev.net/http-bind',

    // Websocket URL
    // websocket: 'wss://lmtgt1.dev2dev.net/xmpp-websocket',

    // The name of client node advertised in XEP-0115 'c' stanza
    clientNode: 'http://jitsi.org/jitsimeet',

    // The real JID of focus participant - can be overridden here
    // Do not change username - FIXME: Make focus username configurable
    // https://github.com/jitsi/jitsi-meet/issues/7376
    // focusUserJid: 'focus@auth.lmtgt1.dev2dev.net',


    // Testing / experimental features.
    //

    testing: {
        // Disables the End to End Encryption feature. Useful for debugging
        // issues related to insertable streams.
        // disableE2EE: false,

        // P2P test mode disables automatic switching to P2P when there are 2
        // participants in the conference.
        p2pTestMode: false

        // Enables the test specific features consumed by jitsi-meet-torture
        // testMode: false

        // Disables the auto-play behavior of *all* newly created video element.
        // This is useful when the client runs on a host with limited resources.
        // noAutoPlayVideo: false

        // Enable / disable 500 Kbps bitrate cap on desktop tracks. When enabled,
        // simulcast is turned off for the desktop share. If presenter is turned
        // on while screensharing is in progress, the max bitrate is automatically
        // adjusted to 2.5 Mbps. This takes a value between 0 and 1 which determines
        // the probability for this to be enabled. This setting has been deprecated.
        // desktopSharingFrameRate.max now determines whether simulcast will be enabled
        // or disabled for the screenshare.
        // capScreenshareBitrate: 1 // 0 to disable - deprecated.

        // Enable callstats only for a percentage of users.
        // This takes a value between 0 and 100 which determines the probability for
        // the callstats to be enabled.
        // callStatsThreshold: 5 // enable callstats for 5% of the users.
    },

    // Disables ICE/UDP by filtering out local and remote UDP candidates in
    // signalling.
    // webrtcIceUdpDisable: false,

    // Disables ICE/TCP by filtering out local and remote TCP candidates in
    // signalling.
    // webrtcIceTcpDisable: false,


    // Media
    //

    // Audio

    // Disable measuring of audio levels.
    // disableAudioLevels: false,
    // audioLevelsInterval: 200,

    // Enabling this will run the lib-jitsi-meet no audio detection module which
    // will notify the user if the current selected microphone has no audio
    // input and will suggest another valid device if one is present.
    enableNoAudioDetection: true,

    // Enabling this will show a "Save Logs" link in the GSM popover that can be
    // used to collect debug information (XMPP IQs, SDP offer/answer cycles)
    // about the call.
    // enableSaveLogs: false,

    // Enabling this will run the lib-jitsi-meet noise detection module which will
    // notify the user if there is noise, other than voice, coming from the current
    // selected microphone. The purpose it to let the user know that the input could
    // be potentially unpleasant for other meeting participants.
    enableNoisyMicDetection: true,

    // Start the conference in audio only mode (no video is being received nor
    // sent).
    // startAudioOnly: false,

    // Every participant after the Nth will start audio muted.
    // startAudioMuted: 10,

    // Start calls with audio muted. Unlike the option above, this one is only
    // applied locally. FIXME: having these 2 options is confusing.
    // startWithAudioMuted: false,

    // Enabling it (with #params) will disable local audio output of remote
    // participants and to enable it back a reload is needed.
    // startSilent: false

    // Enables support for opus-red (redundancy for Opus).
    // enableOpusRed: false,

    // Specify audio quality stereo and opusMaxAverageBitrate values in order to enable HD audio.
    // Beware, by doing so, you are disabling echo cancellation, noise suppression and AGC.
    // audioQuality: {
    //     stereo: false,
    //     opusMaxAverageBitrate: null // Value to fit the 6000 to 510000 range.
    // },

    // Video

    // Sets the preferred resolution (height) for local video. Defaults to 720.
    // resolution: 720,

    // How many participants while in the tile view mode, before the receiving video quality is reduced from HD to SD.
    // Use -1 to disable.
    // maxFullResolutionParticipants: 2,

    // w3c spec-compliant video constraints to use for video capture. Currently
    // used by browsers that return true from lib-jitsi-meet's
    // util#browser#usesNewGumFlow. The constraints are independent from
    // this config's resolution value. Defaults to requesting an ideal
    // resolution of 720p.
    // constraints: {
    //     video: {
    //         height: {
    //             ideal: 720,
    //             max: 720,
    //             min: 240
    //         }
    //     }
    // },

    // Enable / disable simulcast support.
    // disableSimulcast: false,

    // Enable / disable layer suspension.  If enabled, endpoints whose HD
    // layers are not in use will be suspended (no longer sent) until they
    // are requested again.
    // enableLayerSuspension: false,

    // Every participant after the Nth will start video muted.
    // startVideoMuted: 10,

    // Start calls with video muted. Unlike the option above, this one is only
    // applied locally. FIXME: having these 2 options is confusing.
    // startWithVideoMuted: false,

    // If set to true, prefer to use the H.264 video codec (if supported).
    // Note that it's not recommended to do this because simulcast is not
    // supported when  using H.264. For 1-to-1 calls this setting is enabled by
    // default and can be toggled in the p2p section.
    // This option has been deprecated, use preferredCodec under videoQuality section instead.
    // preferH264: true,

    // If set to true, disable H.264 video codec by stripping it out of the
    // SDP.
    // disableH264: false,

    // Desktop sharing

    // Optional desktop sharing frame rate options. Default value: min:5, max:5.
    // desktopSharingFrameRate: {
    //     min: 5,
    //     max: 5
    // },

    // Try to start calls with screen-sharing instead of camera video.
    // startScreenSharing: false,

    // Recording

    // Whether to enable file recording or not.
    // fileRecordingsEnabled: false,
    // Enable the dropbox integration.
    // dropbox: {
    //     appKey: '<APP_KEY>' // Specify your app key here.
    //     // A URL to redirect the user to, after authenticating
    //     // by default uses:
    //     // 'https://lmtgt1.dev2dev.net/static/oauth.html'
    //     redirectURI:
    //          'https://lmtgt1.dev2dev.net/subfolder/static/oauth.html'
    // },
    // When integrations like dropbox are enabled only that will be shown,
    // by enabling fileRecordingsServiceEnabled, we show both the integrations
    // and the generic recording service (its configuration and storage type
    // depends on jibri configuration)
    // fileRecordingsServiceEnabled: false,
    // Whether to show the possibility to share file recording with other people
    // (e.g. meeting participants), based on the actual implementation
    // on the backend.
    // fileRecordingsServiceSharingEnabled: false,

    // Whether to enable live streaming or not.
    // liveStreamingEnabled: false,

    // Transcription (in interface_config,
    // subtitles and buttons can be configured)
    // transcribingEnabled: false,

    // Enables automatic turning on captions when recording is started
    // autoCaptionOnRecord: false,

    // Misc

    // Default value for the channel "last N" attribute. -1 for unlimited.
    channelLastN: -1,

    // Provides a way for the lastN value to be controlled through the UI.
    // When startLastN is present, conference starts with a last-n value of startLastN and channelLastN
    // value will be used when the quality level is selected using "Manage Video Quality" slider.
    // startLastN: 1,

    // Provides a way to use different "last N" values based on the number of participants in the conference.
    // The keys in an Object represent number of participants and the values are "last N" to be used when number of
    // participants gets to or above the number.
    //
    // For the given example mapping, "last N" will be set to 20 as long as there are at least 5, but less than
    // 29 participants in the call and it will be lowered to 15 when the 30th participant joins. The 'channelLastN'
    // will be used as default until the first threshold is reached.
    //
    // lastNLimits: {
    //     5: 20,
    //     30: 15,
    //     50: 10,
    //     70: 5,
    //     90: 2
    // },

    // Provides a way to translate the legacy bridge signaling messages, 'LastNChangedEvent',
    // 'SelectedEndpointsChangedEvent' and 'ReceiverVideoConstraint' into the new 'ReceiverVideoConstraints' message
    // that invokes the new bandwidth allocation algorithm in the bridge which is described here
    // - https://github.com/jitsi/jitsi-videobridge/blob/master/doc/allocation.md.
    // useNewBandwidthAllocationStrategy: false,

    // Specify the settings for video quality optimizations on the client.
    // videoQuality: {
    //    // Provides a way to prevent a video codec from being negotiated on the JVB connection. The codec specified
    //    // here will be removed from the list of codecs present in the SDP answer generated by the client. If the
    //    // same codec is specified for both the disabled and preferred option, the disable settings will prevail.
    //    // Note that 'VP8' cannot be disabled since it's a mandatory codec, the setting will be ignored in this case.
    //    disabledCodec: 'H264',
    //
    //    // Provides a way to set a preferred video codec for the JVB connection. If 'H264' is specified here,
    //    // simulcast will be automatically disabled since JVB doesn't support H264 simulcast yet. This will only
    //    // rearrange the the preference order of the codecs in the SDP answer generated by the browser only if the
    //    // preferred codec specified here is present. Please ensure that the JVB offers the specified codec for this
    //    // to take effect.
    //    preferredCodec: 'VP8',
    //
    //    // Provides a way to enforce the preferred codec for the conference even when the conference has endpoints
    //    // that do not support the preferred codec. For example, older versions of Safari do not support VP9 yet.
    //    // This will result in Safari not being able to decode video from endpoints sending VP9 video.
    //    // When set to false, the conference falls back to VP8 whenever there is an endpoint that doesn't support the
    //    // preferred codec and goes back to the preferred codec when that endpoint leaves.
    //    // enforcePreferredCodec: false,
    //
    //    // Provides a way to configure the maximum bitrates that will be enforced on the simulcast streams for
    //    // video tracks. The keys in the object represent the type of the stream (LD, SD or HD) and the values
    //    // are the max.bitrates to be set on that particular type of stream. The actual send may vary based on
    //    // the available bandwidth calculated by the browser, but it will be capped by the values specified here.
    //    // This is currently not implemented on app based clients on mobile.
    //    maxBitratesVideo: {
    //          H264: {
    //              low: 200000,
    //              standard: 500000,
    //              high: 1500000
    //          },
    //          VP8 : {
    //              low: 200000,
    //              standard: 500000,
    //              high: 1500000
    //          },
    //          VP9: {
    //              low: 100000,
    //              standard: 300000,
    //              high:  1200000
    //          }
    //    },
    //
    //    // The options can be used to override default thresholds of video thumbnail heights corresponding to
    //    // the video quality levels used in the application. At the time of this writing the allowed levels are:
    //    //     'low' - for the low quality level (180p at the time of this writing)
    //    //     'standard' - for the medium quality level (360p)
    //    //     'high' - for the high quality level (720p)
    //    // The keys should be positive numbers which represent the minimal thumbnail height for the quality level.
    //    //
    //    // With the default config value below the application will use 'low' quality until the thumbnails are
    //    // at least 360 pixels tall. If the thumbnail height reaches 720 pixels then the application will switch to
    //    // the high quality.
    //    minHeightForQualityLvl: {
    //        360: 'standard',
    //        720: 'high'
    //    },
    //
    //    // Provides a way to resize the desktop track to 720p (if it is greater than 720p) before creating a canvas
    //    // for the presenter mode (camera picture-in-picture mode with screenshare).
    //    resizeDesktopForPresenter: false
    // },

    // // Options for the recording limit notification.
    // recordingLimit: {
    //
    //    // The recording limit in minutes. Note: This number appears in the notification text
    //    // but doesn't enforce the actual recording time limit. This should be configured in
    //    // jibri!
    //    limit: 60,
    //
    //    // The name of the app with unlimited recordings.
    //    appName: 'Unlimited recordings APP',
    //
    //    // The URL of the app with unlimited recordings.
    //    appURL: 'https://unlimited.recordings.app.com/'
    // },

    // Disables or enables RTX (RFC 4588) (defaults to false).
    // disableRtx: false,

    // Disables or enables TCC support in this client (default: enabled).
    // enableTcc: true,

    // Disables or enables REMB support in this client (default: enabled).
    // enableRemb: true,

    // Enables ICE restart logic in LJM and displays the page reload overlay on
    // ICE failure. Current disabled by default because it's causing issues with
    // signaling when Octo is enabled. Also when we do an "ICE restart"(which is
    // not a real ICE restart), the client maintains the TCC sequence number
    // counter, but the bridge resets it. The bridge sends media packets with
    // TCC sequence numbers starting from 0.
    // enableIceRestart: false,

    // Enables forced reload of the client when the call is migrated as a result of
    // the bridge going down.
    // enableForcedReload: true,

    // Use TURN/UDP servers for the jitsi-videobridge connection (by default
    // we filter out TURN/UDP because it is usually not needed since the
    // bridge itself is reachable via UDP)
    // useTurnUdp: false

    // UI
    //

    // Disables responsive tiles.
    // disableResponsiveTiles: false,

    // Hides lobby button
    // hideLobbyButton: false,

    // Require users to always specify a display name.
    // requireDisplayName: true,

    // Whether to use a welcome page or not. In case it's false a random room
    // will be joined when no room is specified.
    enableWelcomePage: true,

    // Disable app shortcuts that are registered upon joining a conference
    // disableShortcuts: false,

    // Disable initial browser getUserMedia requests.
    // This is useful for scenarios where users might want to start a conference for screensharing only
    // disableInitialGUM: false,

    // Enabling the close page will ignore the welcome page redirection when
    // a call is hangup.
    // enableClosePage: false,

    // Disable hiding of remote thumbnails when in a 1-on-1 conference call.
    // disable1On1Mode: false,

    // Default language for the user interface.
    // defaultLanguage: 'en',

    // Disables profile and the edit of all fields from the profile settings (display name and email)
    // disableProfile: false,

    // Whether or not some features are checked based on token.
    // enableFeaturesBasedOnToken: false,

    // When enabled the password used for locking a room is restricted to up to the number of digits specified
    // roomPasswordNumberOfDigits: 10,
    // default: roomPasswordNumberOfDigits: false,

    // Message to show the users. Example: 'The service will be down for
    // maintenance at 01:00 AM GMT,
    // noticeMessage: '',

    // Enables calendar integration, depends on googleApiApplicationClientID
    // and microsoftApiApplicationClientID
    // enableCalendarIntegration: false,

    // When 'true', it shows an intermediate page before joining, where the user can configure their devices.
    // prejoinPageEnabled: false,

    // If etherpad integration is enabled, setting this to true will
    // automatically open the etherpad when a participant joins.  This
    // does not affect the mobile app since opening an etherpad
    // obscures the conference controls -- it's better to let users
    // choose to open the pad on their own in that case.
    // openSharedDocumentOnJoin: false,

    // If true, shows the unsafe room name warning label when a room name is
    // deemed unsafe (due to the simplicity in the name) and a password is not
    // set or the lobby is not enabled.
    // enableInsecureRoomNameWarning: false,

    // Whether to automatically copy invitation URL after creating a room.
    // Document should be focused for this option to work
    // enableAutomaticUrlCopy: false,

    // Base URL for a Gravatar-compatible service. Defaults to libravatar.
    // gravatarBaseURL: 'https://seccdn.libravatar.org/avatar/',

    // Moved from interfaceConfig(TOOLBAR_BUTTONS).
    // The name of the toolbar buttons to display in the toolbar, including the
    // "More actions" menu. If present, the button will display. Exceptions are
    // "livestreaming" and "recording" which also require being a moderator and
    // some other values in config.js to be enabled. Also, the "profile" button will
    // not display for users with a JWT.
    // Notes:
    // - it's impossible to choose which buttons go in the "More actions" menu
    // - it's impossible to control the placement of buttons
    // - 'desktop' controls the "Share your screen" button
    // - if `toolbarButtons` is undefined, we fallback to enabling all buttons on the UI
    // toolbarButtons: [
    //    'microphone', 'camera', 'closedcaptions', 'desktop', 'embedmeeting', 'fullscreen',
    //    'fodeviceselection', 'hangup', 'profile', 'chat', 'recording',
    //    'livestreaming', 'etherpad', 'sharedvideo', 'shareaudio', 'settings', 'raisehand',
    //    'videoquality', 'filmstrip', 'invite', 'feedback', 'stats', 'shortcuts',
    //    'tileview', 'select-background', 'download', 'help', 'mute-everyone', 'mute-video-everyone', 'security'
    // ],

    // Stats
    //

    // Whether to enable stats collection or not in the TraceablePeerConnection.
    // This can be useful for debugging purposes (post-processing/analysis of
    // the webrtc stats) as it is done in the jitsi-meet-torture bandwidth
    // estimation tests.
    // gatherStats: false,

    // The interval at which PeerConnection.getStats() is called. Defaults to 10000
    // pcStatsInterval: 10000,

    // To enable sending statistics to callstats.io you must provide the
    // Application ID and Secret.
    // callStatsID: '',
    // callStatsSecret: '',

    // Enables sending participants' display names to callstats
    // enableDisplayNameInStats: false,

    // Enables sending participants' emails (if available) to callstats and other analytics
    // enableEmailInStats: false,

    // Controls the percentage of automatic feedback shown to participants when callstats is enabled.
    // The default value is 100%. If set to 0, no automatic feedback will be requested
    // feedbackPercentage: 100,

    // Privacy
    //

    // If third party requests are disabled, no other server will be contacted.
    // This means avatars will be locally generated and callstats integration
    // will not function.
    // disableThirdPartyRequests: false,


    // Peer-To-Peer mode: used (if enabled) when there are just 2 participants.
    //

    p2p: {
        // Enables peer to peer mode. When enabled the system will try to
        // establish a direct connection when there are exactly 2 participants
        // in the room. If that succeeds the conference will stop sending data
        // through the JVB and use the peer to peer connection instead. When a
        // 3rd participant joins the conference will be moved back to the JVB
        // connection.
        enabled: true,

        // Sets the ICE transport policy for the p2p connection. At the time
        // of this writing the list of possible values are 'all' and 'relay',
        // but that is subject to change in the future. The enum is defined in
        // the WebRTC standard:
        // https://www.w3.org/TR/webrtc/#rtcicetransportpolicy-enum.
        // If not set, the effective value is 'all'.
        // iceTransportPolicy: 'all',

        // If set to true, it will prefer to use H.264 for P2P calls (if H.264
        // is supported). This setting is deprecated, use preferredCodec instead.
        // preferH264: true,

        // Provides a way to set the video codec preference on the p2p connection. Acceptable
        // codec values are 'VP8', 'VP9' and 'H264'.
        // preferredCodec: 'H264',

        // If set to true, disable H.264 video codec by stripping it out of the
        // SDP. This setting is deprecated, use disabledCodec instead.
        // disableH264: false,

        // Provides a way to prevent a video codec from being negotiated on the p2p connection.
        // disabledCodec: '',

        // How long we're going to wait, before going back to P2P after the 3rd
        // participant has left the conference (to filter out page reload).
        // backToP2PDelay: 5,

        // The STUN servers that will be used in the peer to peer connections
        stunServers: [

            // { urls: 'stun:lmtgt1.dev2dev.net:3478' },
            { urls: 'stun:meet-jit-si-turnrelay.jitsi.net:443' }
        ]
    },

    analytics: {
        // The Google Analytics Tracking ID:
        // googleAnalyticsTrackingId: 'your-tracking-id-UA-123456-1'

        // Matomo configuration:
        // matomoEndpoint: 'https://your-matomo-endpoint/',
        // matomoSiteID: '42',

        // The Amplitude APP Key:
        // amplitudeAPPKey: '<APP_KEY>'

        // Configuration for the rtcstats server:
        // By enabling rtcstats server every time a conference is joined the rtcstats
        // module connects to the provided rtcstatsEndpoint and sends statistics regarding
        // PeerConnection states along with getStats metrics polled at the specified
        // interval.
        // rtcstatsEnabled: true,

        // In order to enable rtcstats one needs to provide a endpoint url.
        // rtcstatsEndpoint: wss://rtcstats-server-pilot.jitsi.net/,

        // The interval at which rtcstats will poll getStats, defaults to 1000ms.
        // If the value is set to 0 getStats won't be polled and the rtcstats client
        // will only send data related to RTCPeerConnection events.
        // rtcstatsPolIInterval: 1000,

        // Array of script URLs to load as lib-jitsi-meet "analytics handlers".
        // scriptURLs: [
        //      "libs/analytics-ga.min.js", // google-analytics
        //      "https://example.com/my-custom-analytics.js"
        // ],
    },

    // Logs that should go be passed through the 'log' event if a handler is defined for it
    // apiLogLevels: ['warn', 'log', 'error', 'info', 'debug'],

    // Information about the jitsi-meet instance we are connecting to, including
    // the user region as seen by the server.
    deploymentInfo: {
        // shard: "shard1",
        // region: "europe",
        // userRegion: "asia"
    },

    // Decides whether the start/stop recording audio notifications should play on record.
    // disableRecordAudioNotification: false,

    // Disables the sounds that play when other participants join or leave the
    // conference (if set to true, these sounds will not be played).
    // disableJoinLeaveSounds: false,

    // Information for the chrome extension banner
    // chromeExtensionBanner: {
    //     // The chrome extension to be installed address
    //     url: 'https://chrome.google.com/webstore/detail/jitsi-meetings/kglhbbefdnlheedjiejgomgmfplipfeb',

    //     // Extensions info which allows checking if they are installed or not
    //     chromeExtensionsInfo: [
    //         {
    //             id: 'kglhbbefdnlheedjiejgomgmfplipfeb',
    //             path: 'jitsi-logo-48x48.png'
    //         }
    //     ]
    // },

    // Local Recording
    //

    // localRecording: {
    // Enables local recording.
    // Additionally, 'localrecording' (all lowercase) needs to be added to
    // TOOLBAR_BUTTONS in interface_config.js for the Local Recording
    // button to show up on the toolbar.
    //
    //     enabled: true,
    //

    // The recording format, can be one of 'ogg', 'flac' or 'wav'.
    //     format: 'flac'
    //

    // },

    // Options related to end-to-end (participant to participant) ping.
    // e2eping: {
    //   // The interval in milliseconds at which pings will be sent.
    //   // Defaults to 10000, set to <= 0 to disable.
    //   pingInterval: 10000,
    //
    //   // The interval in milliseconds at which analytics events
    //   // with the measured RTT will be sent. Defaults to 60000, set
    //   // to <= 0 to disable.
    //   analyticsInterval: 60000,
    //   },

    // If set, will attempt to use the provided video input device label when
    // triggering a screenshare, instead of proceeding through the normal flow
    // for obtaining a desktop stream.
    // NOTE: This option is experimental and is currently intended for internal
    // use only.
    // _desktopSharingSourceDevice: 'sample-id-or-label',

    // If true, any checks to handoff to another application will be prevented
    // and instead the app will continue to display in the current browser.
    // disableDeepLinking: false,

    // A property to disable the right click context menu for localVideo
    // the menu has option to flip the locally seen video for local presentations
    // disableLocalVideoFlip: false,

    // A property used to unset the default flip state of the local video.
    // When it is set to 'true', the local(self) video will not be mirrored anymore.
    // doNotFlipLocalVideo: false,

    // Mainly privacy related settings

    // Disables all invite functions from the app (share, invite, dial out...etc)
    // disableInviteFunctions: true,

    // Disables storing the room name to the recents list
    // doNotStoreRoom: true,

    // Deployment specific URLs.
    // deploymentUrls: {
    //    // If specified a 'Help' button will be displayed in the overflow menu with a link to the specified URL for
    //    // user documentation.
    //    userDocumentationURL: 'https://docs.example.com/video-meetings.html',
    //    // If specified a 'Download our apps' button will be displayed in the overflow menu with a link
    //    // to the specified URL for an app download page.
    //    downloadAppsUrl: 'https://docs.example.com/our-apps.html'
    // },

    // Options related to the remote participant menu.
    // remoteVideoMenu: {
    //     // If set to true the 'Kick out' button will be disabled.
    //     disableKick: true,
    //     // If set to true the 'Grant moderator' button will be disabled.
    //     disableGrantModerator: true
    // },

    // If set to true all muting operations of remote participants will be disabled.
    // disableRemoteMute: true,

    // Enables support for lip-sync for this client (if the browser supports it).
    // enableLipSync: false

    /**
     External API url used to receive branding specific information.
     If there is no url set or there are missing fields, the defaults are applied.
     None of the fields are mandatory and the response must have the shape:
     {
         // The hex value for the colour used as background
         backgroundColor: '#fff',
         // The url for the image used as background
         backgroundImageUrl: 'https://example.com/background-img.png',
         // The anchor url used when clicking the logo image
         logoClickUrl: 'https://example-company.org',
         // The url used for the image used as logo
         logoImageUrl: 'https://example.com/logo-img.png'
     }
    */
    // dynamicBrandingUrl: '',

    // Sets the background transparency level. '0' is fully transparent, '1' is opaque.
    // backgroundAlpha: 1,

    // The URL of the moderated rooms microservice, if available. If it
    // is present, a link to the service will be rendered on the welcome page,
    // otherwise the app doesn't render it.
    // moderatedRoomServiceUrl: 'https://moderated.lmtgt1.dev2dev.net',

    // If true, tile view will not be enabled automatically when the participants count threshold is reached.
    // disableTileView: true,

    // Hides the conference subject
    // hideConferenceSubject: true,

    // Hides the conference timer.
    // hideConferenceTimer: true,

    // Hides the participants stats
    // hideParticipantsStats: true,

    // Sets the conference subject
    // subject: 'Conference Subject',

    // This property is related to the use case when jitsi-meet is used via the IFrame API. When the property is true
    // jitsi-meet will use the local storage of the host page instead of its own. This option is useful if the browser
    // is not persisting the local storage inside the iframe.
    // useHostPageLocalStorage: true,

    // List of undocumented settings used in jitsi-meet
    /**
     _immediateReloadThreshold
     debug
     debugAudioLevels
     deploymentInfo
     dialInConfCodeUrl
     dialInNumbersUrl
     dialOutAuthUrl
     dialOutCodesUrl
     disableRemoteControl
     displayJids
     etherpad_base
     externalConnectUrl
     firefox_fake_device
     googleApiApplicationClientID
     iAmRecorder
     iAmSipGateway
     microsoftApiApplicationClientID
     peopleSearchQueryTypes
     peopleSearchUrl
     requireDisplayName
     tokenAuthUrl
     */

    /**
     * This property can be used to alter the generated meeting invite links (in combination with a branding domain
     * which is retrieved internally by jitsi meet) (e.g. https://meet.jit.si/someMeeting
     * can become https://brandedDomain/roomAlias)
     */
    // brandingRoomAlias: null,

    // List of undocumented settings used in lib-jitsi-meet
    /**
     _peerConnStatusOutOfLastNTimeout
     _peerConnStatusRtcMuteTimeout
     abTesting
     avgRtpStatsN
     callStatsConfIDNamespace
     callStatsCustomScriptUrl
     desktopSharingSources
     disableAEC
     disableAGC
     disableAP
     disableHPF
     disableNS
     enableTalkWhileMuted
     forceJVB121Ratio
     forceTurnRelay
     hiddenDomain
     ignoreStartMuted
     websocketKeepAlive
     websocketKeepAliveUrl
     */

    /**
        Use this array to configure which notifications will be shown to the user
        The items correspond to the title or description key of that notification
        Some of these notifications also depend on some other internal logic to be displayed or not,
        so adding them here will not ensure they will always be displayed

        A falsy value for this prop will result in having all notifications enabled (e.g null, undefined, false)
    */
    // notifications: [
    //     'connection.CONNFAIL', // shown when the connection fails,
    //     'dialog.cameraNotSendingData', // shown when there's no feed from user's camera
    //     'dialog.kickTitle', // shown when user has been kicked
    //     'dialog.liveStreaming', // livestreaming notifications (pending, on, off, limits)
    //     'dialog.lockTitle', // shown when setting conference password fails
    //     'dialog.maxUsersLimitReached', // shown when maximmum users limit has been reached
    //     'dialog.micNotSendingData', // shown when user's mic is not sending any audio
    //     'dialog.passwordNotSupportedTitle', // shown when setting conference password fails due to password format
    //     'dialog.recording', // recording notifications (pending, on, off, limits)
    //     'dialog.remoteControlTitle', // remote control notifications (allowed, denied, start, stop, error)
    //     'dialog.reservationError',
    //     'dialog.serviceUnavailable', // shown when server is not reachable
    //     'dialog.sessTerminated', // shown when there is a failed conference session
    //     'dialog.sessionRestarted', // show when a client reload is initiated because of bridge migration
    //     'dialog.tokenAuthFailed', // show when an invalid jwt is used
    //     'dialog.transcribing', // transcribing notifications (pending, off)
    //     'dialOut.statusMessage', // shown when dial out status is updated.
    //     'liveStreaming.busy', // shown when livestreaming service is busy
    //     'liveStreaming.failedToStart', // shown when livestreaming fails to start
    //     'liveStreaming.unavailableTitle', // shown when livestreaming service is not reachable
    //     'lobby.joinRejectedMessage', // shown when while in a lobby, user's request to join is rejected
    //     'lobby.notificationTitle', // shown when lobby is toggled and when join requests are allowed / denied
    //     'localRecording.localRecording', // shown when a local recording is started
    //     'notify.disconnected', // shown when a participant has left
    //     'notify.grantedTo', // shown when moderator rights were granted to a participant
    //     'notify.invitedOneMember', // shown when 1 participant has been invited
    //     'notify.invitedThreePlusMembers', // shown when 3+ participants have been invited
    //     'notify.invitedTwoMembers', // shown when 2 participants have been invited
    //     'notify.kickParticipant', // shown when a participant is kicked
    //     'notify.mutedRemotelyTitle', // shown when user is muted by a remote party
    //     'notify.mutedTitle', // shown when user has been muted upon joining,
    //     'notify.newDeviceAudioTitle', // prompts the user to use a newly detected audio device
    //     'notify.newDeviceCameraTitle', // prompts the user to use a newly detected camera
    //     'notify.passwordRemovedRemotely', // shown when a password has been removed remotely
    //     'notify.passwordSetRemotely', // shown when a password has been set remotely
    //     'notify.raisedHand', // shown when a partcipant used raise hand,
    //     'notify.startSilentTitle', // shown when user joined with no audio
    //     'prejoin.errorDialOut',
    //     'prejoin.errorDialOutDisconnected',
    //     'prejoin.errorDialOutFailed',
    //     'prejoin.errorDialOutStatus',
    //     'prejoin.errorStatusCode',
    //     'prejoin.errorValidation',
    //     'recording.busy', // shown when recording service is busy
    //     'recording.failedToStart', // shown when recording fails to start
    //     'recording.unavailableTitle', // shown when recording service is not reachable
    //     'toolbar.noAudioSignalTitle', // shown when a broken mic is detected
    //     'toolbar.noisyAudioInputTitle', // shown when noise is detected for the current microphone
    //     'toolbar.talkWhileMutedPopup', // shown when user tries to speak while muted
    //     'transcribing.failedToStart' // shown when transcribing fails to start
    // ]

    // Allow all above example options to include a trailing comma and
    // prevent fear when commenting out the last value.
    makeJsonParserHappy: 'even if last key had a trailing comma'

    // no configuration value should follow this line.
};

/* eslint-enable no-unused-vars, no-var */

cat /etc/prosody/conf.avail/lmtgt1.dev2dev.net.cfg.lua 
plugin_paths = { "/usr/share/jitsi-meet/prosody-plugins/" }

-- domain mapper options, must at least have domain base set to use the mapper
muc_mapper_domain_base = "lmtgt1.dev2dev.net";

external_service_secret = "io5eXkbITYPCuRHe";
external_services = {
     { type = "stun", host = "lmtgt1.dev2dev.net", port = 3478 },
     { type = "turn", host = "lmtgt1.dev2dev.net", port = 3478, transport = "udp", secret = true, ttl = 86400, algorithm = "turn" },
     { type = "turns", host = "lmtgt1.dev2dev.net", port = 5349, transport = "tcp", secret = true, ttl = 86400, algorithm = "turn" }
};

cross_domain_bosh = false;
consider_bosh_secure = true;
-- https_ports = { }; -- Remove this line to prevent listening on port 5284

-- https://ssl-config.mozilla.org/#server=haproxy&version=2.1&config=intermediate&openssl=1.1.0g&guideline=5.4
ssl = {
    protocol = "tlsv1_2+";
    ciphers = "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384"
}

VirtualHost "lmtgt1.dev2dev.net"
    -- enabled = false -- Remove this line to enable this host
    authentication = "anonymous"
    -- Properties below are modified by jitsi-meet-tokens package config
    -- and authentication above is switched to "token"
    --app_id="example_app_id"
    --app_secret="example_app_secret"
    -- Assign this host a certificate for TLS, otherwise it would use the one
    -- set in the global section (if any).
    -- Note that old-style SSL on port 5223 only supports one certificate, and will always
    -- use the global one.
    ssl = {
        key = "/etc/prosody/certs/lmtgt1.dev2dev.net.key";
        certificate = "/etc/prosody/certs/lmtgt1.dev2dev.net.crt";
    }
    speakerstats_component = "speakerstats.lmtgt1.dev2dev.net"
    conference_duration_component = "conferenceduration.lmtgt1.dev2dev.net"
    -- we need bosh
    modules_enabled = {
        "bosh";
        "pubsub";
        "ping"; -- Enable mod_ping
        "speakerstats";
        "external_services";
        "conference_duration";
        "muc_lobby_rooms";
    }
    c2s_require_encryption = false
    lobby_muc = "lobby.lmtgt1.dev2dev.net"
    main_muc = "conference.lmtgt1.dev2dev.net"
    -- muc_lobby_whitelist = { "recorder.lmtgt1.dev2dev.net" } -- Here we can whitelist jibri to enter lobby enabled rooms

Component "conference.lmtgt1.dev2dev.net" "muc"
    storage = "memory"
    modules_enabled = {
        "muc_meeting_id";
        "muc_domain_mapper";
        --"token_verification";
    }
    admins = { "focus@auth.lmtgt1.dev2dev.net" }
    muc_room_locking = false
    muc_room_default_public_jids = true

-- internal muc component
Component "internal.auth.lmtgt1.dev2dev.net" "muc"
    storage = "memory"
    modules_enabled = {
        "ping";
    }
    admins = { "focus@auth.lmtgt1.dev2dev.net", "jvb@auth.lmtgt1.dev2dev.net" }
    muc_room_locking = false
    muc_room_default_public_jids = true

VirtualHost "auth.lmtgt1.dev2dev.net"
    ssl = {
        key = "/etc/prosody/certs/auth.lmtgt1.dev2dev.net.key";
        certificate = "/etc/prosody/certs/auth.lmtgt1.dev2dev.net.crt";
    }
    authentication = "internal_hashed"

-- Proxy to jicofo's user JID, so that it doesn't have to register as a component.
Component "focus.lmtgt1.dev2dev.net" "client_proxy"
    target_address = "focus@auth.lmtgt1.dev2dev.net"

Component "speakerstats.lmtgt1.dev2dev.net" "speakerstats_component"
    muc_component = "conference.lmtgt1.dev2dev.net"

Component "conferenceduration.lmtgt1.dev2dev.net" "conference_duration_component"
    muc_component = "conference.lmtgt1.dev2dev.net"

Component "lobby.lmtgt1.dev2dev.net" "muc"
    storage = "memory"
    restrict_room_creation = true
    muc_room_locking = false
    muc_room_default_public_jids = true


And here are the logs as I restart the services with this command:
service nginx restart && service prosody restart && service jicofo restart

The logs were a bit too long combined with the configs, so I posted the logfile portion here:
https://www2.techtalkhawke.com/news/standalone-all-in-one-jitsi-server-max-capacity-600-users-no-matter-how-much-hardware-at-least-in-aws/newsitem_view#1621971135683245

Hope that helps. Thank you kindly for your assistance.

This is exactly the problems with the limits. Try adding the limits cconfiguration from my post and restart prosody, jicofo and jvb and see how it goes.

1 Like

There we go! Now back to successful baseline of smooth running at 600 users.=!
Thank you so much!

When you said limits before, I was conflating the system limits such as sysctl and others like these:
DefaultLimitNOFILE
DefaultLimitNPROC
DefaultTasksMax
ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 254048
max locked memory (kbytes, -l) 65536
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 254048
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

and such.

That code snippet you provided was exactly the info I need to find what you meant. My apologies for the confusion on my part.

Thank you very much for helping get oriented and back on track! As always the Jitsi community and folks are just tremendous and wonderfully helpful.

Now to see how it goes ramping up to 1k+ loadtest again (the other thread).

Mark this as fixed for this particular Prosody upgrade issue. Thanks again!

@damencho why is this happening with a vanilla installation though? I don’t see mod_limits enabled by default, so why is this an issue in a fresh installation per rpg’s assertion?

EDIT - Actually, wait - it does look like it’s enabled by default after all. My bad.

1 Like

If you do new prosody install, I think it is enabled by default.
I’m testing it now.
I’m currently testing this: feat: Adds new module to unlimit jicofo and jvb connections. by damencho · Pull Request #9267 · jitsi/jitsi-meet · GitHub upgrading stable to this.
To add an exception for jicofo if somebody stumbles on it…

2 Likes

Hi Damencho, i have this exact problem with the installation of prosody 0.11.9. How can remove limits?

Thanks Damencho for your quick response!! I put this where? in conf.avail/mydomain.lua? or in jicofo.config? sorry for my question.

Add it in /etc/prosody/prosody.lua.cfg