[jitsi-dev] Video mute


#1

(I'll point out up front that we're not using xmpp/jicofo, this involves
our own signaling and the REST api to the bridge)

Hi all,
We were recently doing some testing with video mute where we would destroy
the local camera stream and update the sdp to 'recvonly' for the video
mline. This had the unfortunate side-effect of freezing the *remote* incoming
video and we had no idea why. After digging deeper, we found that when
this new description was sent up to our signaling, we updated the bridge
appropriately (or, we *thought* appropriately). This included changing the
video channel for this client to "recvonly". This direction change caused
the bridge to stop forwarding video to the muted client. I changed this to
'invert' at the bridge level (client recvonly->bridge sendonly, etc.) and
that worked, and after thinking about it, that does make sense, although
the channel allocation on the bridge makes it feel more like a 'proxy' for
the client rather than an endpoint that happens to forward to the client,
but just semantics I suppose.

When I say it "worked" that was mainly the case, but it freezes quickly due
to NACK/PLI seem to cease working after doing this. I suspect that this
may have something to do with the RTCP sender SSRC--I've seen chrome having
issues with this before (issues with RTCP feedback from recvonly streams).
I'll take a look at chrome logs on the receiving side, but has anyone seen
this issue before?

-brian


#2

Hi Brian,

This sounds familiar. As you correctly point out, even though the video channel at the client machine is recvonly, it still needs to send RTCP packets (NACKs/PLIs/etc) using an RTCP sender SSRC. The bridge needs to know that RTCP sender SSRC so that it can decide whether it’s an audio or a video RTCP packet, when mux/bundle is used. If you don’t signal this SSRC to the bridge, the packet will be discarded. You can try setting an SSRC in the local SDP of the recvonly video channel and then announce it to the bridge and see if this helps.

To fix this issue Mozilla has implemented this [1] and there’s this issue [2] on the Chrome side. We’ve recently dealt with a related issue here [3] and filed a bug report agains WebRTC here [4]. The information around this issue is literally scattered across mailing lists, bug trackers and source code comments which makes it nearly impossible to find out what’s going on without first wasting a few hours trying to figure out what’s going on.

Hope this helps.

Best,
George

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1160280
[2]: https://code.google.com/p/webrtc/issues/detail?id=4740
[3]: https://github.com/jitsi/jitsi-meet/issues/320
[4]: https://code.google.com/p/webrtc/issues/detail?id=4883

···

It turns out that Chrome is sending PLIs that use old video stream
SSRC instead of the new one. I'm going to check what happens on the
bridge with that packet.

On Aug 24, 2015, at 4:54 PM, Brian Baldino <brian@highfive.com> wrote:

(I'll point out up front that we're not using xmpp/jicofo, this involves our own signaling and the REST api to the bridge)

Hi all,
We were recently doing some testing with video mute where we would destroy the local camera stream and update the sdp to 'recvonly' for the video mline. This had the unfortunate side-effect of freezing the remote incoming video and we had no idea why. After digging deeper, we found that when this new description was sent up to our signaling, we updated the bridge appropriately (or, we thought appropriately). This included changing the video channel for this client to "recvonly". This direction change caused the bridge to stop forwarding video to the muted client. I changed this to 'invert' at the bridge level (client recvonly->bridge sendonly, etc.) and that worked, and after thinking about it, that does make sense, although the channel allocation on the bridge makes it feel more like a 'proxy' for the client rather than an endpoint that happens to forward to the client, but just semantics I suppose.

When I say it "worked" that was mainly the case, but it freezes quickly due to NACK/PLI seem to cease working after doing this. I suspect that this may have something to do with the RTCP sender SSRC--I've seen chrome having issues with this before (issues with RTCP feedback from recvonly streams). I'll take a look at chrome logs on the receiving side, but has anyone seen this issue before?

-brian
_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#3

Thanks for this, George! We are leveraging the jitsi-meet js code, so we
are generating an ssrc for the recvonly stream. We also send an update to
the bridge that includes this new SSRC, but after taking a look, it looks
like the REST interface of the bridge doesn't even look at the 'ssrcs'
field--is this because it instead just waits to receive packets on an
ssrc? I noticed this comment in the ColibriConferenceIq.java file:
/**
         * Adds a specific (RTP) SSRC to the list of SSRCs seen/received on
this
         * <tt>Channel</tt>. Invoked by the Jitsi Videobridge server, not
its
         * clients.

···

*
         * @param ssrc the (RTP) SSRC to be added to the list of SSRCs
         * seen/received on this <tt>Channel</tt>
         * @return <tt>true</tt> if the list of SSRCs seen/received on this
         * <tt>Channel</tt> has been modified as part of the method call;
         * otherwise, <tt>false</tt>
         */
        public synchronized boolean addSSRC(int ssrc)

If I manually add the receive-only ssrc, I can get it to show up on the
channel's receive ssrcs, but I seem to see this frequently after every one:
2015-07-25 19:06:45.617 SEVERE: [95]
org.jitsi.impl.neomedia.transform.rtcp.StatisticsEngine.error() Failed to
analyze an incoming RTCP packet for the purposes of statistics.
java.io.IOException: Invalid RTCP Version
        at net.sf.fmj.media.rtp.RTCPHeader.<init>(RTCPHeader.java:145)
        at
net.sf.fmj.media.rtp.RTCPReport.readSourceDescription(RTCPReport.java:293)
        at net.sf.fmj.media.rtp.RTCPReport.<init>(RTCPReport.java:123)
        at
net.sf.fmj.media.rtp.RTCPSenderReport.<init>(RTCPSenderReport.java:70)
        at
org.jitsi.impl.neomedia.transform.rtcp.StatisticsEngine.parseRTCPReport(StatisticsEngine.java:985)
        at
org.jitsi.impl.neomedia.transform.rtcp.StatisticsEngine.updateReceivedMediaStreamStats(StatisticsEngine.java:1147)
        at
org.jitsi.impl.neomedia.transform.rtcp.StatisticsEngine.reverseTransform(StatisticsEngine.java:1017)
        at
org.jitsi.impl.neomedia.transform.SinglePacketTransformer.reverseTransform(SinglePacketTransformer.java:129)
        at
org.jitsi.impl.neomedia.transform.TransformEngineChain$PacketTransformerChain.reverseTransform(TransformEngineChain.java:268)
        at
org.jitsi.impl.neomedia.transform.TransformInputStream.createRawPacket(TransformInputStream.java:75)
        at
org.jitsi.impl.neomedia.RTPConnectorInputStream.runInReceiveThread(RTPConnectorInputStream.java:830)
        at
org.jitsi.impl.neomedia.RTPConnectorInputStream.access$000(RTPConnectorInputStream.java:33)
        at
org.jitsi.impl.neomedia.RTPConnectorInputStream$3.run(RTPConnectorInputStream.java:630)

I know the statistics isn't a crucial failure, but I'm guessing something
else is wrong. I'll keep digging deeper into that. I'm curious how jitsi
gets the recvonly ssrc to be processed via xmpp though?

-brian

On Mon, Aug 24, 2015 at 10:24 PM, George Politis <gp@jitsi.org> wrote:

Hi Brian,

This sounds familiar. As you correctly point out, even though the video
channel at the client machine is recvonly, it still needs to send RTCP
packets (NACKs/PLIs/etc) using an RTCP sender SSRC. The bridge needs to
know that RTCP sender SSRC so that it can decide whether it’s an audio or a
video RTCP packet, when mux/bundle is used. If you don’t signal this SSRC
to the bridge, the packet will be discarded. You can try setting an SSRC in
the local SDP of the recvonly video channel and then announce it to the
bridge and see if this helps.

To fix this issue Mozilla has implemented this [1] and there’s this issue
[2] on the Chrome side. We’ve recently dealt with a related issue here [3]
and filed a bug report agains WebRTC here [4]. The information around this
issue is literally scattered across mailing lists, bug trackers and source
code comments which makes it nearly impossible to find out what’s going on
without first wasting a few hours trying to figure out what’s going on.

Hope this helps.

Best,
George

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1160280
[2]: https://code.google.com/p/webrtc/issues/detail?id=4740
[3]: https://github.com/jitsi/jitsi-meet/issues/320
[4]: https://code.google.com/p/webrtc/issues/detail?id=4883

> It turns out that Chrome is sending PLIs that use old video stream
> SSRC instead of the new one. I'm going to check what happens on the
> bridge with that packet.

> On Aug 24, 2015, at 4:54 PM, Brian Baldino <brian@highfive.com> wrote:
>
> (I'll point out up front that we're not using xmpp/jicofo, this involves
our own signaling and the REST api to the bridge)
>
> Hi all,
> We were recently doing some testing with video mute where we would
destroy the local camera stream and update the sdp to 'recvonly' for the
video mline. This had the unfortunate side-effect of freezing the remote
incoming video and we had no idea why. After digging deeper, we found that
when this new description was sent up to our signaling, we updated the
bridge appropriately (or, we thought appropriately). This included
changing the video channel for this client to "recvonly". This direction
change caused the bridge to stop forwarding video to the muted client. I
changed this to 'invert' at the bridge level (client recvonly->bridge
sendonly, etc.) and that worked, and after thinking about it, that does
make sense, although the channel allocation on the bridge makes it feel
more like a 'proxy' for the client rather than an endpoint that happens to
forward to the client, but just semantics I suppose.
>
> When I say it "worked" that was mainly the case, but it freezes quickly
due to NACK/PLI seem to cease working after doing this. I suspect that
this may have something to do with the RTCP sender SSRC--I've seen chrome
having issues with this before (issues with RTCP feedback from recvonly
streams). I'll take a look at chrome logs on the receiving side, but has
anyone seen this issue before?
>
> -brian
> _______________________________________________
> dev mailing list
> dev@jitsi.org
> Unsubscribe instructions and other list options:
> http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#4

So it seems as though I'm seeing exactly what you guys have seen here:
https://code.google.com/p/webrtc/issues/detail?id=4883

Although video mute seems to work consistently on meet.jit.si...is there a
workaround?

···

On Tue, Aug 25, 2015 at 12:08 PM, Brian Baldino <brian@highfive.com> wrote:

Thanks for this, George! We are leveraging the jitsi-meet js code, so we
are generating an ssrc for the recvonly stream. We also send an update to
the bridge that includes this new SSRC, but after taking a look, it looks
like the REST interface of the bridge doesn't even look at the 'ssrcs'
field--is this because it instead just waits to receive packets on an
ssrc? I noticed this comment in the ColibriConferenceIq.java file:
/**
         * Adds a specific (RTP) SSRC to the list of SSRCs seen/received
on this
         * <tt>Channel</tt>. Invoked by the Jitsi Videobridge server, not
its
         * clients.
         *
         * @param ssrc the (RTP) SSRC to be added to the list of SSRCs
         * seen/received on this <tt>Channel</tt>
         * @return <tt>true</tt> if the list of SSRCs seen/received on this
         * <tt>Channel</tt> has been modified as part of the method call;
         * otherwise, <tt>false</tt>
         */
        public synchronized boolean addSSRC(int ssrc)

If I manually add the receive-only ssrc, I can get it to show up on the
channel's receive ssrcs, but I seem to see this frequently after every one:
2015-07-25 19:06:45.617 SEVERE: [95]
org.jitsi.impl.neomedia.transform.rtcp.StatisticsEngine.error() Failed to
analyze an incoming RTCP packet for the purposes of statistics.
java.io.IOException: Invalid RTCP Version
        at net.sf.fmj.media.rtp.RTCPHeader.<init>(RTCPHeader.java:145)
        at
net.sf.fmj.media.rtp.RTCPReport.readSourceDescription(RTCPReport.java:293)
        at net.sf.fmj.media.rtp.RTCPReport.<init>(RTCPReport.java:123)
        at
net.sf.fmj.media.rtp.RTCPSenderReport.<init>(RTCPSenderReport.java:70)
        at
org.jitsi.impl.neomedia.transform.rtcp.StatisticsEngine.parseRTCPReport(StatisticsEngine.java:985)
        at
org.jitsi.impl.neomedia.transform.rtcp.StatisticsEngine.updateReceivedMediaStreamStats(StatisticsEngine.java:1147)
        at
org.jitsi.impl.neomedia.transform.rtcp.StatisticsEngine.reverseTransform(StatisticsEngine.java:1017)
        at
org.jitsi.impl.neomedia.transform.SinglePacketTransformer.reverseTransform(SinglePacketTransformer.java:129)
        at
org.jitsi.impl.neomedia.transform.TransformEngineChain$PacketTransformerChain.reverseTransform(TransformEngineChain.java:268)
        at
org.jitsi.impl.neomedia.transform.TransformInputStream.createRawPacket(TransformInputStream.java:75)
        at
org.jitsi.impl.neomedia.RTPConnectorInputStream.runInReceiveThread(RTPConnectorInputStream.java:830)
        at
org.jitsi.impl.neomedia.RTPConnectorInputStream.access$000(RTPConnectorInputStream.java:33)
        at
org.jitsi.impl.neomedia.RTPConnectorInputStream$3.run(RTPConnectorInputStream.java:630)

I know the statistics isn't a crucial failure, but I'm guessing something
else is wrong. I'll keep digging deeper into that. I'm curious how jitsi
gets the recvonly ssrc to be processed via xmpp though?

-brian

On Mon, Aug 24, 2015 at 10:24 PM, George Politis <gp@jitsi.org> wrote:

Hi Brian,

This sounds familiar. As you correctly point out, even though the video
channel at the client machine is recvonly, it still needs to send RTCP
packets (NACKs/PLIs/etc) using an RTCP sender SSRC. The bridge needs to
know that RTCP sender SSRC so that it can decide whether it’s an audio or a
video RTCP packet, when mux/bundle is used. If you don’t signal this SSRC
to the bridge, the packet will be discarded. You can try setting an SSRC in
the local SDP of the recvonly video channel and then announce it to the
bridge and see if this helps.

To fix this issue Mozilla has implemented this [1] and there’s this issue
[2] on the Chrome side. We’ve recently dealt with a related issue here [3]
and filed a bug report agains WebRTC here [4]. The information around this
issue is literally scattered across mailing lists, bug trackers and source
code comments which makes it nearly impossible to find out what’s going on
without first wasting a few hours trying to figure out what’s going on.

Hope this helps.

Best,
George

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1160280
[2]: https://code.google.com/p/webrtc/issues/detail?id=4740
[3]: https://github.com/jitsi/jitsi-meet/issues/320
[4]: https://code.google.com/p/webrtc/issues/detail?id=4883

> It turns out that Chrome is sending PLIs that use old video stream
> SSRC instead of the new one. I'm going to check what happens on the
> bridge with that packet.

> On Aug 24, 2015, at 4:54 PM, Brian Baldino <brian@highfive.com> wrote:
>
> (I'll point out up front that we're not using xmpp/jicofo, this
involves our own signaling and the REST api to the bridge)
>
> Hi all,
> We were recently doing some testing with video mute where we would
destroy the local camera stream and update the sdp to 'recvonly' for the
video mline. This had the unfortunate side-effect of freezing the remote
incoming video and we had no idea why. After digging deeper, we found that
when this new description was sent up to our signaling, we updated the
bridge appropriately (or, we thought appropriately). This included
changing the video channel for this client to "recvonly". This direction
change caused the bridge to stop forwarding video to the muted client. I
changed this to 'invert' at the bridge level (client recvonly->bridge
sendonly, etc.) and that worked, and after thinking about it, that does
make sense, although the channel allocation on the bridge makes it feel
more like a 'proxy' for the client rather than an endpoint that happens to
forward to the client, but just semantics I suppose.
>
> When I say it "worked" that was mainly the case, but it freezes quickly
due to NACK/PLI seem to cease working after doing this. I suspect that
this may have something to do with the RTCP sender SSRC--I've seen chrome
having issues with this before (issues with RTCP feedback from recvonly
streams). I'll take a look at chrome logs on the receiving side, but has
anyone seen this issue before?
>
> -brian
> _______________________________________________
> dev mailing list
> dev@jitsi.org
> Unsubscribe instructions and other list options:
> http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev