[jitsi-dev] [lib-jitsi-meet] weird behavior when joining with no media and then adding a video track later


#1

I am using build 1758 of lib-jitsi-meet and have run into a problem I'm not sure how to solve.

I have User 1 join with audio and video and then have User 2 join with no audio or video and after joining then I turn on video for User 2 (by calling createLocalTracks and then passing the track to replaceTrack on JitsiConference with the old track set to null). User 2 successfully adds the track and it shows up for the loopback video but User 1 doesn't see User 2's video. After some investigation, I can see in webrtc internals of User 1 that the video ssrc for User 2 is receiving data but it is not in the list of remote tracks. That ssrc's cname is prefixed with recvonly- which I imagine is the problem. Looking at User 2's webrtc internals I can see that the direction for the video mline was switched from recvonly to sendrecv when the video track was added. But no signaling happened to notify User 1 of that change.

I know there was some refactoring recently that allows for video muting and stream switching to happen without a jingle negotiation. Are the corresponding changes on the videobridge to make that work (the videobridge I am using is a couple months out of date)? Or does that no signaling stream switching not work with going from no streams (which lib-jitsi-meet generates a recvonly ssrc for) to adding a video stream?

Devin Wilson
Software Engineer 2

ReadyTalk, a PGi Company | 1900 16th Street, Suite 600 | Denver, CO 80202

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.


#2

Are the corresponding changes on the videobridge to make that work (the

videobridge I am using is a couple months out of date)?
Nope, nothing on the bridge side needed here

Or does that no signaling stream switching not work with going from no

streams (which lib-jitsi-meet generates a recvonly ssrc for) to adding a
video stream?
The flow we do on meet.jit.si for this is a bit funky...even if there are
no streams we generate ssrcs that we will use in the future up front and
signal separately that they are muted, so it's quite possible you've found
a bug here that we just haven't hit (because we create/"add" the dummy
streams up front, even if the client isn't sending anything at that time).

···

On Fri, Feb 24, 2017 at 5:01 PM, George Politis <gp@jitsi.org> wrote:

Hi Devin,

I believe the track at the receiver should have been created as sendrecv
(with msids etc.) right from the beginning, but I’m sure we have some
"black magic” that’s broken in your use case. If you try the same scenario
on beta.meet.jit.si, what do you get?

Best,
George

On Feb 24, 2017, at 6:46 PM, Devin Wilson <devin.wilson@foxden.io> wrote:

I am using build 1758 of lib-jitsi-meet and have run into a problem I’m
not sure how to solve.

I have User 1 join with audio and video and then have User 2 join with no
audio or video and after joining then I turn on video for User 2 (by
calling createLocalTracks and then passing the track to replaceTrack on
JitsiConference with the old track set to null). User 2 successfully adds
the track and it shows up for the loopback video but User 1 doesn’t see
User 2’s video. After some investigation, I can see in webrtc internals of
User 1 that the video ssrc for User 2 is receiving data but it is not in
the list of remote tracks. That ssrc’s cname is prefixed with recvonly-
which I imagine is the problem. Looking at User 2’s webrtc internals I can
see that the direction for the video mline was switched from recvonly to
sendrecv when the video track was added. But no signaling happened to
notify User 1 of that change.

I know there was some refactoring recently that allows for video muting
and stream switching to happen without a jingle negotiation. Are the
corresponding changes on the videobridge to make that work (the videobridge
I am using is a couple months out of date)? Or does that no signaling
stream switching not work with going from no streams (which lib-jitsi-meet
generates a recvonly ssrc for) to adding a video stream?

Devin Wilson
Software Engineer 2

ReadyTalk, a PGi Company | 1900 16th Street, Suite 600 | Denver, CO 80202

*Disclaimer*

The information contained in this communication from the sender is
confidential. It is intended solely for use by the recipient and others
authorized to receive it. If you are not the recipient, you are hereby
notified that any disclosure, copying, distribution or taking action in
relation of the contents of this information is strictly prohibited and may
be unlawful.

This email has been scanned for viruses and malware, and may have been
automatically archived.
_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#3

Hi Devin,

I believe the track at the receiver should have been created as sendrecv (with msids etc.) right from the beginning, but I’m sure we have some "black magic” that’s broken in your use case. If you try the same scenario on beta.meet.jit.si, what do you get?

Best,
George

···

On Feb 24, 2017, at 6:46 PM, Devin Wilson <devin.wilson@foxden.io> wrote:

I am using build 1758 of lib-jitsi-meet and have run into a problem I’m not sure how to solve.

I have User 1 join with audio and video and then have User 2 join with no audio or video and after joining then I turn on video for User 2 (by calling createLocalTracks and then passing the track to replaceTrack on JitsiConference with the old track set to null). User 2 successfully adds the track and it shows up for the loopback video but User 1 doesn’t see User 2’s video. After some investigation, I can see in webrtc internals of User 1 that the video ssrc for User 2 is receiving data but it is not in the list of remote tracks. That ssrc’s cname is prefixed with recvonly- which I imagine is the problem. Looking at User 2’s webrtc internals I can see that the direction for the video mline was switched from recvonly to sendrecv when the video track was added. But no signaling happened to notify User 1 of that change.

I know there was some refactoring recently that allows for video muting and stream switching to happen without a jingle negotiation. Are the corresponding changes on the videobridge to make that work (the videobridge I am using is a couple months out of date)? Or does that no signaling stream switching not work with going from no streams (which lib-jitsi-meet generates a recvonly ssrc for) to adding a video stream?

Devin Wilson
Software Engineer 2

ReadyTalk, a PGi Company | 1900 16th Street, Suite 600 | Denver, CO 80202

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived.

_______________________________________________
dev mailing list
dev@jitsi.org <mailto:dev@jitsi.org>
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#4

I guess the problem is that we do not clear primary video SSRC when a
change from "no device" to "video device" occurs. Then the signalling
will look wrong, because the logic there cares only about SSRC values
and not the attributes ("msid", "cname" etc.). The recvonly SSRC will
not be removed, but only the RTX (if exists) will be added. The
recvonly SSRC does not have "msid", so remote participant does not
recognise that as a stream.

···

On Fri, Feb 24, 2017 at 7:50 PM, Brian Baldino <brian@jitsi.org> wrote:

Are the corresponding changes on the videobridge to make that work (the
videobridge I am using is a couple months out of date)?

Nope, nothing on the bridge side needed here

Or does that no signaling stream switching not work with going from no
streams (which lib-jitsi-meet generates a recvonly ssrc for) to adding a
video stream?

The flow we do on meet.jit.si for this is a bit funky...even if there are no
streams we generate ssrcs that we will use in the future up front and signal
separately that they are muted, so it's quite possible you've found a bug
here that we just haven't hit (because we create/"add" the dummy streams up
front, even if the client isn't sending anything at that time).


#5

Thanks for the responses.

Pawel: yeah, the signaling doesn't happen at all when I add the video stream. Confusingly, adding an audio stream does work correctly.

Brian/George: I tried this scenario on beta.meet.jit.si and it does work. Interestingly, the recvonly ssrc is not generated when I join without video. It would appear that the generated recvonly ssrc is what causes the problem.

After pulling in the latest lib-jitsi-meet into my fork I don't experience the issue anymore. It appears that the recent refactoring around the jingle session causes the recvonly ssrc to not be generated when there is no local video track when the jingle session is set up.

Devin

···

-----Original Message-----
From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Pawel Domas
Sent: Monday, February 27, 2017 6:42 AM
To: Jitsi Developers
Subject: Re: [jitsi-dev] [lib-jitsi-meet] weird behavior when joining with no media and then adding a video track later

On Fri, Feb 24, 2017 at 7:50 PM, Brian Baldino <brian@jitsi.org> wrote:

Are the corresponding changes on the videobridge to make that work (the
videobridge I am using is a couple months out of date)?

Nope, nothing on the bridge side needed here

Or does that no signaling stream switching not work with going from no
streams (which lib-jitsi-meet generates a recvonly ssrc for) to adding a
video stream?

The flow we do on meet.jit.si for this is a bit funky...even if there are no
streams we generate ssrcs that we will use in the future up front and signal
separately that they are muted, so it's quite possible you've found a bug
here that we just haven't hit (because we create/"add" the dummy streams up
front, even if the client isn't sending anything at that time).

I guess the problem is that we do not clear primary video SSRC when a
change from "no device" to "video device" occurs. Then the signalling
will look wrong, because the logic there cares only about SSRC values
and not the attributes ("msid", "cname" etc.). The recvonly SSRC will
not be removed, but only the RTX (if exists) will be added. The
recvonly SSRC does not have "msid", so remote participant does not
recognise that as a stream.

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.


#6

Hmm that might be bad... we want to generate the recvonly SSRC
otherwise the user with no video device will experience often video
freezes. Maybe something was broken during refactoring

···

On Wed, Mar 1, 2017 at 10:13 AM, Devin Wilson <devin.wilson@foxden.io> wrote:

After pulling in the latest lib-jitsi-meet into my fork I don't experience
the issue anymore. It appears that the recent refactoring around the jingle
session causes the recvonly ssrc to not be generated when there is no local
video track when the jingle session is set up.

Devin


#7

Yeah, I figured there was a reason it was there though there is a bug when the recvonly ssrc is generated and then you later add a real video track. In the js console I do see a warning that the recvonly ssrc was not generated because there was no session. It might be worth adding a test around both cases.

···

-----Original Message-----
From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Pawel Domas
Sent: Wednesday, March 01, 2017 9:23 AM
To: Jitsi Developers
Subject: Re: [jitsi-dev] [lib-jitsi-meet] weird behavior when joining with no media and then adding a video track later

On Wed, Mar 1, 2017 at 10:13 AM, Devin Wilson <devin.wilson@foxden.io> wrote:

After pulling in the latest lib-jitsi-meet into my fork I don't experience
the issue anymore. It appears that the recent refactoring around the jingle
session causes the recvonly ssrc to not be generated when there is no local
video track when the jingle session is set up.

Devin

Hmm that might be bad... we want to generate the recvonly SSRC
otherwise the user with no video device will experience often video
freezes. Maybe something was broken during refactoring

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.


#8

Sorry to open this discussion again, but how can I create a recvonly track?

···

On Wed, Mar 1, 2017 at 2:08 PM, Devin Wilson <devin.wilson@foxden.io> wrote:

Yeah, I figured there was a reason it was there though there is a bug when
the recvonly ssrc is generated and then you later add a real video track.
In the js console I do see a warning that the recvonly ssrc was not
generated because there was no session. It might be worth adding a test
around both cases.

-----Original Message-----
From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Pawel Domas
Sent: Wednesday, March 01, 2017 9:23 AM
To: Jitsi Developers
Subject: Re: [jitsi-dev] [lib-jitsi-meet] weird behavior when joining with
no media and then adding a video track later

On Wed, Mar 1, 2017 at 10:13 AM, Devin Wilson <devin.wilson@foxden.io> > wrote:
> After pulling in the latest lib-jitsi-meet into my fork I don't
experience
> the issue anymore. It appears that the recent refactoring around the
jingle
> session causes the recvonly ssrc to not be generated when there is no
local
> video track when the jingle session is set up.
>
> Devin

Hmm that might be bad... we want to generate the recvonly SSRC
otherwise the user with no video device will experience often video
freezes. Maybe something was broken during refactoring

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

*Disclaimer*

The information contained in this communication from the sender is
confidential. It is intended solely for use by the recipient and others
authorized to receive it. If you are not the recipient, you are hereby
notified that any disclosure, copying, distribution or taking action in
relation of the contents of this information is strictly prohibited and may
be unlawful.

This email has been scanned for viruses and malware, and may have been
automatically archived.

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev