[jitsi-dev] [jitsi/libjitsi] Fixes and improvements for RTX. (#129)


#1

Recognizes packets retransmitted with RTX and doesn't request them again.
You can view, comment on, or merge this pull request online at:

  https://github.com/jitsi/libjitsi/pull/129

-- Commit Summary --

  * Supports the RTX format in RetranmissionRequester.
  * Uses an int instead of a short to store RTP sequence numbers.
  * Creates a stream's RetransmissionRequester instance early.

-- File Changes --

    M src/org/jitsi/impl/neomedia/MediaStreamImpl.java (21)
    M src/org/jitsi/impl/neomedia/RawPacket.java (8)
    M src/org/jitsi/impl/neomedia/VideoMediaStreamImpl.java (31)
    R src/org/jitsi/impl/neomedia/transform/RetransmissionRequesterImpl.java (114)
    M src/org/jitsi/impl/neomedia/transform/rewriting/ExtendedSequenceNumberInterval.java (9)
    M src/org/jitsi/impl/neomedia/transform/rewriting/SsrcGroupRewriter.java (4)
    M src/org/jitsi/impl/neomedia/transform/rewriting/SsrcRewriter.java (11)
    M src/org/jitsi/service/neomedia/MediaStream.java (5)
    A src/org/jitsi/service/neomedia/RetransmissionRequester.java (48)

-- Patch Links --

https://github.com/jitsi/libjitsi/pull/129.patch
https://github.com/jitsi/libjitsi/pull/129.diff

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129


#2

hey boris, we've started doing some work on enabling retransmission and have found that h264 requires the rtx stream to be separate (different ssrc, different pt) all the way to the receiver (vp8 can do it in band). i'm wondering, is there a scenario where the sender says it wants to do retransmissions on a separate stream, but the receiver can handle them on the same stream? from what we've seen with vp8 and h264 it seems to be one way or another (either both sender and receiver can handle them on the same stream, or they need to be kept separate the entire way).

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-206534893


#3

one other comment: of course if the retransmission is done in time to mask the loss entirely from the downstream receiver, then this would work. but what if there is loss on the downlink and the receiver requests a retransmission?

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-206536266


#4

is the idea that the server would request rtx for loss on the uplink and mask them to look like the original stream, but still negotiate rtx with the receiver and, when the receiver reported loss, would retransmit them on the separate ssrc/payload type?

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-206575652


#5

Merged #129.

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#event-617548220


#6

Hey Brian,

Withour RTX webrtc.org does retransmissions "plain" -- by just sending the original packet again. The RTX format is the one which uses a separate stream (ssrc, pt, sequence numbers). I think that both can be used with h264, but there might be limitations in webrtc.org's h264 implementation that I'm not aware of.

Connecting endpoints which support RTX with ones that don't support it is a use case we built for, because firefox doesn't support RTX. The way this works is that the bridge terminates retransmissions.

With a sender: the bridge will request missing packets (with RTCP NACK). They may arrive either as copies of the lost packet or in RTX. If they arrive as RTX, the bridge will strip it and pass on a regular media packet.

With a receiver: the bridge will terminate incoming RTCP NACK packets. It will try to respond with a packet from its cache, with either RTX or a plain copy, depending on the receiver. If the packet is not in the cache, the bridge will do nothing: presumably the bridge already detected the loss and requested a retransmission from the sender, which will find its way to the receiver as a media packet (no RTX).

Does that answer your questions?

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-206641263


#7

@gpolitis why have you merged code that failed to pass the tests ?
Tests don't even compile for me

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-206889575


#8

@champtar, I'm sorry if this caused problems. It may have been opened and merged a bit too quickly, but I don't see any compilation failures. What exactly fails to compile for you?

We are looking into issues which might have been caused by this, but we don't know of any that would have an impact unless you enable RTX (which would not have worked prior to these PRs either).

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-206962072


#9

As @bgrozev said, this might have been merged a little too quickly. @champtar to which tests are you referring to? Unit tests? They compile fine here. The tests that jenkins is running are the jitsi-meet-torture tests but they're not very reliable and failure doesn't necessarily mean that something is wrong (we're working on fixing that, but that's a different issue).

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-206962913


#10

Actually there are problems with RTX and they have been captured by the tests, so we're temporarily disabling it. I apologize for the quick merge.

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-206980249


#11

I will paste compilation bug tomorrow, I'm out of office right now.

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-206980725


#12

https://github.com/jitsi/jicofo/commit/178d0a3edb7924f37ed0d54ea7ce920a503e034a

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-206983544


#13

Got it, thanks @champtar ! I think Pawel fixed this already. I wasn't seeing it in my local env for some reason (running "mvn install"). Apologies for the whole mess, I think it is all in a working state now, but we'll be testing more thoroughly today.

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-206991259


#14

Thanks Boris! I think so, let me see if I'm understanding it right.

Withour RTX webrtc.org does retransmissions "plain" -- by just sending the original packet again. The RTX format is the one which uses a separate stream (ssrc, pt, sequence numbers). I think that both can be used with h264, but there might be limitations in webrtc.org's h264 implementation that I'm not aware of.

We're still trying to figure out which configurations work and which don't work for h264 at this point. Will keep you posted on what we find there.

With a sender: the bridge will request missing packets (with RTCP NACK). They may arrive either as copies of the lost packet or in RTX. If they arrive as RTX, the bridge will strip it and pass on a regular media packet.

Makes sense and we've seen this. So on the far end it will just look like an out-of-order packet and be handled by the jitter buffer I assume?

With a receiver: the bridge will terminate incoming RTCP NACK packets. It will try to respond with a packet from its cache, with either RTX or a plain copy, depending on the receiver. If the packet is not in the cache, the bridge will do nothing: presumably the bridge already detected the loss and requested a retransmission from the sender, which will find its way to the receiver as a media packet (no RTX).

So does this mean that if a receiver has negotiated rtx and a separate payload type, when it sends a nack the bridge will send the retransmitted packet on the rtx stream using the rtx payload type?

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-207017324


#15

With a sender: the bridge will request missing packets (with RTCP NACK). They may arrive either as copies of the lost packet or in RTX. If they arrive as RTX, the bridge will strip it and pass on a regular media packet.

Makes sense and we've seen this. So on the far end it will just look like an out-of-order packet and be handled by the jitter buffer I assume?

Yes.

So does this mean that if a receiver has negotiated rtx and a separate payload type, when it sends a nack the bridge will send the retransmitted packet on the rtx stream using the rtx payload type?

Yes, exactly.

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-207033425


#16

@bgrozev @gpolitis i'm not using fixed dependencies and always rebuilding all projects, that's why jicofo was not compiling for me. Thanks for the fixes, I'll update tomorrow.

···

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jitsi/libjitsi/pull/129#issuecomment-207087945