[jitsi-dev] Feature of relaytype-mixer


#1

I have been having a go with mixer because it appears to me that many RTP interfaces cannot cope with multiple ssrcs well. The sound quality goes awful and it is pretty clear that this arises from the RTP interface not coping for some reason or other.

In RtpChannel I discovered that when you get a DTMF packet it stops and starts the RTPSourceStream I have bodged this in my copy of the code as you can see in the following extract. However, this may be something that is worth doing properly in the master branch hence the email. Am I right about RTP often not coping with more than one audio stream (this obviously depends upon the implementation as the protocol obviously does). Is this an OK way of dealing with this issue?

                 if \(RTPLevelRelayType\.MIXER\.equals\(getRTPLevelRelayType\(\)\)\)
                 \{
                     Map<Byte, MediaFormat> payloadTypes
                         = stream\.getDynamicRTPPayloadTypes\(\);

                     if \(payloadTypes \!= null\)
                     \{
                         int pt = data\[off \+ 1\] & 0x7f;

                         if \(pt\!=101\) \{ // JAMH 21/10/17 don't do this for telephone\-events

                         MediaFormat format = payloadTypes\.get\(\(byte\) pt\);

                         if \(\(format \!= null\)
                                 && \!format\.equals\(stream\.getFormat\(\)\)\)
                         \{
                             stream\.setFormat\(format\);
                             synchronized \(streamSyncRoot\)
                             \{   // otherwise races with stream\.start\(\)

stream.setDirection(MediaDirection.SENDRECV);
}
notify = true;
}
}
}


#2

John,

Can you explain a bit more on what problem you are trying to solve?

Thanks,

/Kaiduan

···

On Sat, Oct 21, 2017 at 12:47 PM, John Hemming <john@hemming.email> wrote:

I have been having a go with mixer because it appears to me that many RTP
interfaces cannot cope with multiple ssrcs well. The sound quality goes
awful and it is pretty clear that this arises from the RTP interface not
coping for some reason or other.

In RtpChannel I discovered that when you get a DTMF packet it stops and
starts the RTPSourceStream I have bodged this in my copy of the code as
you can see in the following extract. However, this may be something that
is worth doing properly in the master branch hence the email. Am I right
about RTP often not coping with more than one audio stream (this obviously
depends upon the implementation as the protocol obviously does). Is this
an OK way of dealing with this issue?

                    if (RTPLevelRelayType.MIXER.equal
s(getRTPLevelRelayType()))
                    {
                        Map<Byte, MediaFormat> payloadTypes
                            = stream.getDynamicRTPPayloadTypes();

                        if (payloadTypes != null)
                        {
                            int pt = data[off + 1] & 0x7f;

                            if (pt!=101) { // JAMH 21/10/17 don't do this
for telephone-events

                            MediaFormat format = payloadTypes.get((byte)
pt);

                            if ((format != null)
                                    && !format.equals(stream.getFormat()))
                            {
                                stream.setFormat(format);
                                synchronized (streamSyncRoot)
                                { // otherwise races with stream.start()
stream.setDirection(MediaDirection.SENDRECV);
                                }
                                notify = true;
                            }
                            }
                        }
                    }

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

--
Founder of Goodstartsoft
https://www.goodstartsoft.com


#3

If you want to disable DTMF you should just remove it from signaling. If that's not an option, then filter on the format, not on the payload type number (101 is for dynamic assignment, it could be used for anything).

Boris

···

On 21/10/2017 11:47, John Hemming wrote:

I have been having a go with mixer because it appears to me that many
RTP interfaces cannot cope with multiple ssrcs well. The sound
quality goes awful and it is pretty clear that this arises from the RTP
interface not coping for some reason or other.

In RtpChannel I discovered that when you get a DTMF packet it stops and
starts the RTPSourceStream I have bodged this in my copy of the code as
you can see in the following extract. However, this may be something
that is worth doing properly in the master branch hence the email. Am I
right about RTP often not coping with more than one audio stream (this
obviously depends upon the implementation as the protocol obviously
does). Is this an OK way of dealing with this issue?

   if
(RTPLevelRelayType.MIXER.equals(getRTPLevelRelayType()))
   {
   Map<Byte, MediaFormat> payloadTypes
   = stream.getDynamicRTPPayloadTypes();

   if (payloadTypes != null)
   {
   int pt = data[off + 1] & 0x7f;

   if (pt!=101) { // JAMH 21/10/17 don't do
this for telephone-events


#4

Hi John,

Adding back the dev mailing list.

A good suggestion.

Also on Mixer there is an intermittent problem with "Failed to unprotect SRTP packet"

Which has similarities to:

https://github.com/jitsi/jitsi-videobridge/issues/230

Setting up a webrtc connection with translation on video and mixing on audio I get the above or sometimes it doesn't work for a bit then starts working. Sometimes it works straight away.

There is no issue on the video connection (which is translating), nor is there an issue with rtp connections (obviously that problem couldn't happen). I am getting the message from Sawbuck.

Looking at the packets they do seem to start with a quite high packet number and if the error message is something that happens when there is a jump (particularly down) in packet numbers it could be related to that.

Does anyone have any hints on this?

Yes, a jump in sequence numbers (by more than 2^15) would explain all of this. I think we start sending with one SSRC and then switch to another. If both of these are chosen randomly that also explains the intermittency -- the jump is big enough about half the time.

But I don't know why we do the switch or how to fix it.

Regards,
Boris

···

On 23/10/2017 13:16, John Hemming wrote:


#5

Sorry. Copying in the dev list I have done a bit of research on this and it is clear that when a new stream is added to the mixer something does something to the packet number. I don't think it is a new SSRCInfo or a resetting of that, so it must be a resetting of the internal packet number. I have been tracking that back, but am going to look at this again tomorrow.

However, the jump in packet sequence numbers occurs around the time of the new stream event (either just before or just afterwards).

···

On 23/10/2017 19:23, Boris Grozev wrote:

Hi John,

Adding back the dev mailing list.

On 23/10/2017 13:16, John Hemming wrote:

A good suggestion.

Also on Mixer there is an intermittent problem with "Failed to unprotect SRTP packet"

Which has similarities to:

https://github.com/jitsi/jitsi-videobridge/issues/230

Setting up a webrtc connection with translation on video and mixing on audio I get the above or sometimes it doesn't work for a bit then starts working. Sometimes it works straight away.

There is no issue on the video connection (which is translating), nor is there an issue with rtp connections (obviously that problem couldn't happen). I am getting the message from Sawbuck.

Looking at the packets they do seem to start with a quite high packet number and if the error message is something that happens when there is a jump (particularly down) in packet numbers it could be related to that.

Does anyone have any hints on this?

Yes, a jump in sequence numbers (by more than 2^15) would explain all of this. I think we start sending with one SSRC and then switch to another. If both of these are chosen randomly that also explains the intermittency -- the jump is big enough about half the time.

But I don't know why we do the switch or how to fix it.

Regards,
Boris

--

john.hemming.name <http://john.hemming.name>


#6

Actually you are not wrong. I have been messing around with the ssrcs and had used the same default values for something that normally would have different values. That essentially resolves the issue.

Thank you for your help.

···

On 23/10/2017 22:27, Boris Grozev wrote:

On 23/10/2017 15:02, John Hemming wrote:

Sorry. Copying in the dev list I have done a bit of research on this and it is clear that when a new stream is added to the mixer something does something to the packet number. I don't think it is a new SSRCInfo or a resetting of that, so it must be a resetting of the internal packet number. I have been tracking that back, but am going to look at this again tomorrow.

Yes, my comment about switching SSRCs is wrong (I don't know what I was thinking). I meant that something triggers a new starting sequence number to be generated.

Boris

--

john.hemming.name <http://john.hemming.name>