[jitsi-dev] How to related multiple RTX codecs in Jitsi Videobridge?


#1

* How do I associate RTX extensions with codecs in Jitsi REST? For example 1, the bridge responds with 404.
  * Any codecs that the bridge can't process anyhow and which I may leave out?

Example 1 - fails:

[
  {
    "id": 96,
    "name": "VP8",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "x-google-start-bitrate=300"
      ]
    }
  },
  {
    "id": 97,
    "name": "rtx",
    "clockrate": 90000,
    "parameters": {
      "fmtp": [
        "apt=96"
      ]
    }
  },
  {
    "id": 98,
    "name": "VP9",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "x-google-start-bitrate=300"
     ]
    }
  },
  {
    "id": 99,
    "name": "rtx",
    "clockrate": 90000,
    "parameters": {
      "fmtp": [
        "apt=98"
      ]
    }
  },
  {
    "id": 100,
    "name": "H264",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f"
      ]
    }
  },
  {
    "id": 101,
    "name": "rtx",
    "clockrate": 90000,
    "parameters": {
      "fmtp": [
        "apt=100"
      ]
    }
  },
  {
    "id": 102,
    "name": "red",
    "clockrate": 90000
  },
  {
    "id": 124,
    "name": "rtx",
    "clockrate": 90000,
    "parameters": {
      "fmtp": [
        "apt=102"
      ]
    }
  },
  {
    "id": 127,
    "name": "ulpfec",
    "clockrate": 90000
  },
  {
    "id": 125,
    "name": "H264",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f"
      ]
    }
  },
  {
    "id": 123,
    "name": "rtx",
    "clockrate": 90000,
    "parameters": {
      "fmtp": [
        "apt=125"
      ]
    }
  }
]

Example 2 - works:

[
  {
    "id": 96,
    "name": "VP8",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "x-google-start-bitrate=300"
      ]
    }
  },
  {
    "id": 98,
    "name": "VP9",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "x-google-start-bitrate=300"
      ]
    }
  },
  {
    "id": 100,
    "name": "H264",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f"
      ]
    }
  },
  {
    "id": 102,
    "name": "red",
    "clockrate": 90000
  },
  {
    "id": 127,
    "name": "ulpfec",
    "clockrate": 90000
  },
  {
    "id": 125,
    "name": "H264",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f"
      ]
    }
  }
]


#2

Hi Oliver,

> * How do I associate RTX extensions with codecs in Jitsi REST? For
> example 1, the bridge responds with 404.
> * Any codecs that the bridge can't process anyhow and which I may
> leave out?
>
> Example 1 � fails:

See here for the format: https://github.com/jitsi/jitsi-videobridge/blob/master/doc/rest-colibri.md

>
> [
> {
> *"id"*: 96,
> *"name"*: *"VP8"*,
> *"clockrate"*: 90000,
> *"parameters"*: {
> *"rtcp-fb"*: [
> *"ccm fir"*,
> *"nack"*,
> *"nack pli"*,
> *"goog-remb"*,
> *"transport-cc"
> *],
> *"fmtp"*: [
> *"x-google-start-bitrate=300"
> *]

This should be something like this:

"parameters": {"x-google-start-bitrate": 300},
"rtcp-fbs": [
   {"type": "ccm", "subtype": "fir"},
   ...,
]

> }
> },
> {
> *"id"*: 97,
> *"name"*: *"rtx"*,
> *"clockrate"*: 90000,
> *"parameters"*: {
> *"fmtp"*: [
> *"apt=96"
> *]
Similar thing: "parameters": {"apt": 96}

Boris

···

On 05/12/2017 16:02, Oliver Hausler wrote:


#3

Thanks, Boris. I think I have an older format for rtcp-fb/rtcp-fbs, will update this. But that's not the issue.

Problem is that RTX codecs must be related with the main codec, and in SDP this is done using a= with a reference to the "parent" codec.

a=rtpmap:96 VP8/90000
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96

The documentation is missing an example how that's done, and the way how I do it is obviously wrong:

  {
    "id": 96,
    "name": "VP8",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "x-google-start-bitrate=300"
      ]
    }
  },
  {
    "id": 97,
    "name": "rtx",
    "clockrate": 90000,
    "parameters": {
      "fmtp": [
        "apt=96"
      ]
    }
  }

I would need the expected JSON for the fmtp parameters?

···

From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Oliver Hausler
Sent: Tuesday, December 5, 2017 14:02
To: Jitsi Dev Mailinglist (dev@jitsi.org) <dev@jitsi.org>
Subject: [jitsi-dev] How to related multiple RTX codecs in Jitsi Videobridge?

This sender failed our fraud detection checks and may not be who they appear to be. Learn about spoofing<http://aka.ms/LearnAboutSpoofing>

Feedback<http://aka.ms/SafetyTipsFeedback>

  * How do I associate RTX extensions with codecs in Jitsi REST? For example 1, the bridge responds with 404.
  * Any codecs that the bridge can't process anyhow and which I may leave out?

Example 1 - fails:

[
  {
    "id": 96,
    "name": "VP8",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "x-google-start-bitrate=300"
      ]
    }
  },
  {
    "id": 97,
    "name": "rtx",
    "clockrate": 90000,
    "parameters": {
      "fmtp": [
        "apt=96"
      ]
    }
  },
  {
    "id": 98,
    "name": "VP9",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "x-google-start-bitrate=300"
     ]
    }
  },
  {
    "id": 99,
    "name": "rtx",
    "clockrate": 90000,
    "parameters": {
      "fmtp": [
        "apt=98"
      ]
    }
  },
  {
    "id": 100,
    "name": "H264",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f"
      ]
    }
  },
  {
    "id": 101,
    "name": "rtx",
    "clockrate": 90000,
    "parameters": {
      "fmtp": [
        "apt=100"
      ]
    }
  },
  {
    "id": 102,
    "name": "red",
    "clockrate": 90000
  },
  {
    "id": 124,
    "name": "rtx",
    "clockrate": 90000,
    "parameters": {
      "fmtp": [
        "apt=102"
      ]
    }
  },
  {
    "id": 127,
    "name": "ulpfec",
    "clockrate": 90000
  },
  {
    "id": 125,
    "name": "H264",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f"
      ]
    }
  },
  {
    "id": 123,
    "name": "rtx",
    "clockrate": 90000,
    "parameters": {
      "fmtp": [
        "apt=125"
      ]
    }
  }
]

Example 2 - works:

[
  {
    "id": 96,
    "name": "VP8",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "x-google-start-bitrate=300"
      ]
    }
  },
  {
    "id": 98,
    "name": "VP9",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "x-google-start-bitrate=300"
      ]
    }
  },
  {
    "id": 100,
    "name": "H264",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f"
      ]
    }
  },
  {
    "id": 102,
    "name": "red",
    "clockrate": 90000
  },
  {
    "id": 127,
    "name": "ulpfec",
    "clockrate": 90000
  },
  {
    "id": 125,
    "name": "H264",
    "clockrate": 90000,
    "parameters": {
      "rtcp-fb": [
        "ccm fir",
        "nack",
        "nack pli",
        "goog-remb",
        "transport-cc"
      ],
      "fmtp": [
        "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f"
      ]
    }
  }
]


#4

Thanks, Boris. I think I have an older format for rtcp-fb/rtcp-fbs, will update this. But that's not the issue.

Problem is that RTX codecs must be related with the main codec, and in SDP this is done using a= with a reference to the "parent" codec.

Exactly, using the associated payload type (apt) format parameter (fmtp).

a=rtpmap:96 VP8/90000

a=rtcp-fb:96 ccm fir

a=rtcp-fb:96 nack

a=rtcp-fb:96 nack pli

a=rtcp-fb:96 goog-remb

a=rtcp-fb:96 transport-cc

a=rtpmap:97 rtx/90000

a=fmtp:97 apt=96

"parameters": {"apt": 96}

Boris

···

On 05/12/2017 16:38, Oliver Hausler wrote: