Muted track and "black video" bug

Developers
#1

Hey,

I’ve finally managed to replicate a bug we’re randomly facing in production (we have an up-to-date self-hosted Jitsi Meet server and use lib-jitsi-meet.js built from tag jitsi-meet_4639 to build a custom UI):

  • create a new room and join with one browser
  • join the room with another browser
  • kill the second browser (don’t close it, but kill the process, i.e. “not-graceful” shutdown, happens in real life apparently, maybe when running out of battery, or simply when the browser crashes for some reason I guess)
  • the first browser now sees a “black video” in place of the killed browser for a while, then receives JitsiMeetJS.events.conference.USER_LEFT (which allows us to clean up the UI at this point)
  • any participant joining before the JitsiMeetJS.events.conference.USER_LEFT is sent do see a “black video” too

UX isn’t great when this happens, so I’m trying to catch those “black videos” before the JitsiMeetJS.events.conference.USER_LEFT is sent so that I can “preclean up” the UI.

I logged the JitsiTrack objects I receive, when the track is “working” (i.e. no issue, video is “moving”, all good):

disposed: false
hasBeenMuted: false
isP2P: false
muted: false
type: "video"
videoType: "camera"
stream: MediaStream
  active: true
  ...
track: MediaStreamTrack
  enabled: true
  kind: "video"
  muted: false
  ...
...

When the track brings a “black video”:

disposed: false
hasBeenMuted: false
isP2P: false
muted: false             // here, muted:false, while
type: "video"
videoType: "camera"
stream: MediaStream
  active: false          // here, active:false
  ...
track: MediaStreamTrack
  enabled: true
  kind: "video"
  muted: true            // and here, muted:true
  ...
...

Any reason why the JitsiTrack’s muted can be false while the actual enclosed MediaStreamTrack’s muted is true? Do they represent a different “muted” concept?

MDN on MediaStreamTrack#muted (https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrack/muted):

The muted read-only property of the MediaStreamTrack interface returns a Boolean value indicating whether or not the track is currently unable to provide media output. […] To implement a way for users to mute and unmute a track, use the enabled property.

Should I start looking for this property to catch those “black videos”, or are the Jitsi JS objects abstracting those for me already? Do you recommend using the native MediaStreamTrack#onunmute event handler? I couldn’t find my way out through the docs to be honest (https://github.com/jitsi/lib-jitsi-meet/blob/master/doc/API.md).

Thanks for Jitsi, brilliant opensource tech, keep it up!

Chris

#2

Hmm, actually, Chrome updates the console.logged object states, so those logs I’ve posted aren’t in the state they are when I first receive then…

When the track brings a “black video”:

  • I first get jitsiTrack.track.muted === false
  • then the native jitsiTrack.track.onunmute event handle is called (short after, say less than 2 seconds)
  • and finally, jitsiTrack.track.muted === true (which is what I see in the console as Chrome updated the debugged track’s state)

I’m currently hacking around with audio#onloadeddata and video#onloadeddata instead, as they seem to not be called when the video is “black”, which could help me not display those “black videos”…

Stay tuned, and I’m still interested about an “official way” of avoiding those short-living “black videos” :wink:

#3

hi @chrischris, any luck on this issue?

#4

I’m having these exact issues anytime a new user JOINS the room and publishes their video!

1 Like
#5

We’re facing similar issues in our custom front-end,
although not on the same conditions, it looks like it’s pretty random.

Sometimes it works other times it doesn’t.

It appears like this is happening almost always when testing in the same machine with different browsers/tabs opened, while it happens less frequently when testing across different devices.

I’ve tried to analyze the situation more deeply and found the same stuff inside the JittsiRemoteTrack object as OP.

I’m also getting this console log any time this bug happens:
first I get -> Halt: There are no SSRC groups in the remote description.
then -> Imploding SIM group: 1292046596 2935148463 1016015531
the numbers always vary and I was able to link only the first number to one of the “muted” tracks’ ssrc.

This bug never happened before this phase of development, showing tracks always went smooth.
Before this phase the localTracks of each user were added to the conference as soon as they connected, now we have an intermediate phase were the localTracks are created and showed in a “pre-join” like window, when the user advances the localTracks are then added to the conference.

Do you have any idea on how we can solve this, and any related info without a solution is well accepted.
@chrischris were you able to solve it?

#6

Hey,

I didn’t really manage to solve it yet either unfortunately :confused:

I did build an abstraction on top of lib-jitsi-meeti though, that seemed to have improved the steadiness of the conferences a bit (mostly by relying on the DOM native events as I said above). I’ve just created a gist so that you could have a look at it if you wish: https://gist.github.com/sp00m/928a92e9283e8c3a4ac5daf4481f3a06

By no means in a finished nor production-ready state, plus lacks documentation badly, so use at your own risk :wink: And feel free to comment on it to feed back any improvement you made.