Identifying New Track As Desktop

We have a presentation mode where we show a user’s cam stream and screen share at the same time, like a picture in picture setup. Previously we used a 2nd connection so the user could have 2 simultaneous video streams. Now that Jitsi supports 2 video streams, we are changing it to use multiple video streams on the same connection.

The problem is, identifying which video track is the cam & which is the desktop. There is no guaranteed connection order, so it’s not always the case that v0 is the cam & v1 is the desktop.

I found a property in the track called videoType, that can be read with track.getVideoType() that has the info I need i.e. videoType = “desktop” for a screen share, but there is a problem: when onTrackAdded fires for a remote track, that property is incorrectly set to camera for both the camera and the screen share. After the onTrackAdded is complete, it is correctly set to videoType: “desktop”. I guess it’s only set after the onTrackAdded event

You can confirm this by reading all tracks for a remote participant during onTrackAdded, and then reading them again after the event.

Is this by design? Is it a mistake? Is there another way I can check a remote track’s source in onTrackAdded?

2 Likes

We have exactly the same error. Did you manage to fix it?

I came up with a workaround. I send a message over the conference with the source name and participant ID when starting the screen share. Basically a notification that a screen share started. Then the other participants save a look-up table of participant ID and screen source name to use in onTrackAdded.

It’s not the nicest solution but it works

So the same way you send the track_removed like event right?

Yes

I don’t know if it’s any help, but my workaround was to ignore desktop tracks from the TRACK_ADDED listener and instead handle them from the TRACK_VIDEOTYPE_CHANGED listener.