Using Firefox I’ve noticed similar effects occurring intermittently: video freezing, or no video at all at the start of a conference. These issues are not evident with webkit-based browsers (that is, those using the chrome engine, Brave, Vivaldi, Opera, etc.).
Furthermore, in a conference with 2 participants FF is apparently not using p2p mode, rather connected to remote port 10000 used by jvb. OTOH the webkit browsers evidently are using p2p mode. (Info provided by clicking the green connection symbol for the host participant, that is, oneself.)
This difference is reflected in CPU utilization on the server. For 2-party meetings under FF jvb uses ~22% CPU vs. Vivaldi at <3%:
Currently I’m running FF Nightly—performance can change one day to the next. However these issues have been quite consistent over the last couple of months. Until Mozilla fixes FF I’m inclined to use a different browser for Jitsi meetings.