[jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?


#1

Emil,

so I assume your use of trickle simply introduces a delay that hides the actual problem

Are you saying I must synchronize patching the conference? If yes, is this per conference or in general? I assume I can make 2 parallel REST calls if a distinct conference is involved, correct?

Sounds like a race condition with your SSRC generation.

I was in fact making the REST call for adding ANSWER SDP in 2 different threads simultaneously. But even synchronizing these does not help. I am still seeing inconsistent results for SSRCs. And I even added a random time delay within the lock to make sure Jitsi has enough time for processing. When I do trickle, I get consistent results all the time (otherwise same code).

private static ReentrantLock addSdpLock = new ReentrantLock();

public void addSdp(@Nonnull String socketId, @Nonnull Sdp sdp) {
    addSdpLock.lock();
    try {
        log.info("TEST ENTER");

         response = JitsiApiBuilder.api().patchConference(id, conference).execute();

        Thread.sleep() // random delay 1000..2000 msec

        log.info("TEST EXIT");
    } finally {
        addSdpLock.unlock();
    }
}

2016-10-22 16:18:07,422 INFO Conference [nioEventLoopGroup-3-7] TEST ENTER
2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-7] TEST EXIT
2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-8] TEST ENTER
2016-10-22 16:18:10,325 INFO Conference [nioEventLoopGroup-3-8] TEST EXIT

···

From: Emil Ivov
Sent: Saturday, October 22, 2016 14:34
To: Oliver Hausler
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc<mailto:oliver@closeup.cc>> wrote:
Emil,

I am not using ICE trickle server -> client. Do you mean I can also switch it off in the other direction?

Yes. I think we only expect to see on address from the client in order to prevent a bug but it doesn't need to be a working address.

In other words: you don't meed to signal any actual addresses from client to server. Just send your offer as soon as you have it.

The SSRCs in the final colibri look better when trickle client -> server is on. I don't know if this is a bug in Jitsi, but when I don't trickle from client -> server, sometime/always (not sure) an SSRC is missing for one of the channels. If I trickle, it always looks fine.

Sounds like a race condition with your SSRC generation. SSRCs are in no way related to trickle ICE or addresses so I assume your use of trickle simply introduces a delay that hides the actual problem

Emil

Oliver.

From: Emil Ivov
Sent: Saturday, October 22, 2016 14:25
To: Jitsi Developers <dev@jitsi.org<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>;
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

Oliver,

From what you have said it doesn't sound like you need to trickle any candidates. As long as your videobridge is on a public network it will discover all client candidates as peer reflexive (this is still trickle ICE but with now actual client-side trickling).

This is what we do on all Jitsi Meet deployments including meet.jit.si<http://meet.jit.si>

Emil

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc<javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>> wrote:
Emil, Philipp, sorry for the late response.

@Philipp: It doesn't work for me with passive. Can you see any issues with my logs below? Might be entirely unrelated as I see DTLS warnings in the Jitsi log. I posted this separately: "Unknown DTLS handshake message and no streams from Jitsi (REST)"

@Emil: I assume we should be using ICE trickle as this is standard in WebRTC, so I tried to do that:

- I am creating an OFFER SDP which contains all Candidates discovered by Jitsi (no ICE trickle, makes no sense here as we have candidates in colibri already).

In Chrome I have 2 options:

1. ICE trickle: I listen to createAnswerOnSuccess(), send the ANSWER SDP to Jitsi, wait for onIceCandidate() and send each Candidate as it is discovered.

2. No ICE trickle: I listen to iceGatheringStateChange() == complete, then send the full ANSWER SDP with all Candidates.

I thought 1. would be the preferred way when using WebRTC/libjingle, so I followed what Philipp Seeser suggested and it looks like Jitsi accepts the "remote-candidate"(s) as it replays it:

"channel-bundles": [{
                                "id": "AKQQeYSlhEwltx69_ZWinO8|||||N-N77FcLReCCe5yoieF_7A",
                                "transport": {
                                                "xmlns": "urn:xmpp:jingle:transports:ice-udp:1",
                                                "rtcp-mux": true,
                                                "remote-candidate": {
                                                                "component": 1,
                                                                "protocol": "udp",
                                                                "priority": 1685987071,
                                                                "ip": "68.96.117.185",
                                                                "port": 56117,
                                                                "type": "srflx",
                                                                "generation": 0,
                                                                "foundation": "1855365925",
                                                                "network": 1
                                                },
                                                "setup": "passive"
                                }
                }]

I assume I must add each Candidate like that, and Jitsi picks the one which it will use. Because the last 2 candidates I am receiving use throw-away port 9 [why is Chrome generating these at all?], if Jitsi would use the last Candidate, it wouldn't work, but I believe it's smart enough and uses the best candidate, even though this isn't reflected in colibri (except for the immediate response, there is no change in colibri thereafter, so I cannot prove this).

Finally, I see a slightly distinct result between 1 and 2, though, when looking at "GET /colibri/conferences/{conferenceId}":

For 1. I see 2 audio channels per endpoint:

  "contents": [
    {
      "channels": [
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||AKPDvx2Egk8dqVFUHTgkE_A",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||AKPDvx2Egk8dqVFUHTgkE_A",
          "sources": [
3204522869
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            3204522869,
336257926
          ],
          "id": "61baaec9d48429b4",
          "direction": "sendrecv"
        },
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||APkpD2Im3kfzq2hM1o49z3E",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||APkpD2Im3kfzq2hM1o49z3E",
          "sources": [
1722705425
          ],
         "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            1722705425,
3673416731
          ],
          "id": "b41ab150c9739171",
          "direction": "sendrecv"
        }
      ],
     "name": "audio"
    },

For 2. one of the ssrcs seems to be missing for one of the channels (so Philipps approach with ICE trickle seems to be the better working):

  "contents": [
    {
      "channels": [
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||MEIiSaRqTGep7C2cDR42sQ",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||MEIiSaRqTGep7C2cDR42sQ",
          "sources": [
916465418
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
3268560502
          ],
          "id": "700d2d43eae73a28",
          "direction": "sendrecv"
        },
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||ANZL23OQrkVkqzUZ-XMyyPk",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||ANZL23OQrkVkqzUZ-XMyyPk",
          "sources": [
814905801
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            814905801,
2637946124
          ],
          "id": "ba07908e22e3b2a2",
          "direction": "sendrecv"
        }
      ],
      "name": "audio"
    },

From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Emil Ivov
Sent: Sunday, October 16, 2016 22:30
To: Jitsi Developers <dev@jitsi.org<mailto:dev@jitsi.org>>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

I am sorry I am chiming in so late but, why exactly do you need to send any additional candidates to the bridge in the first place?

On Sunday, 16 October 2016, Oliver Hausler <oliver@closeup.cc<mailto:oliver@closeup.cc>> wrote:
Thank you Philipp, I just noticed you sent a response to this.

Interesting, you patch each candidate into 'remote-candidate' separately. I will try this tomorrow.

From: dev [mailto:dev-bounces@jitsi.org<javascript:_e(%7B%7D,'cvml','dev-bounces@jitsi.org');>] On Behalf Of Philipp Seeser
Sent: Monday, October 10, 2016 03:02
To: Jitsi Developers <dev@jitsi.org<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

I am able to add candidates with trickle ice and REST.

e.g. candidate json body:

var candidatePatch = {
          'foundation': parsedCandidate.foundation,
          'component': parsedCandidate.component,
          'priority': parsedCandidate.priority,
          'ip': parsedCandidate.ip,
          'port': parsedCandidate.port,
          'generation': parsedCandidate.generation,
          'type': parsedCandidate.type,
          'rel-addr': parsedCandidate.raddr,
          'rel-port': parsedCandidate.rport,
          'protocol': parsedCandidate.protocol,
          'network' : parsedCandidate.network,
          'id' : parsedCandidate.id
        };
        var patch: any = {
          id: msg.streams.publisher,
          'channel-bundles': [{
            'id': msg.streams.channelBundleId,
            transport: {
              'xmlns': 'urn:xmpp:jingle:transports:ice-udp:1',
              'setup': 'passive',
              'remote-candidate': candidatePatch
            }
          }]
        };

Also look at https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/xmpp/JingleSessionPC.js#L330
and I recommend to replace the answerer offer with "passive"

          // XXX Videobridge is the (SDP) offerer and WebRTC (e.g. Chrome) is the
          // answerer (as orchestrated by Jicofo). In accord with
          // http://tools.ietf.org/html/rfc5245#section-5.2 and because both peers
          // are ICE FULL agents, Videobridge will take on the controlling role and
          // WebRTC will take on the controlled role. In accord with
          // https://tools.ietf.org/html/rfc5763#section-5, Videobridge will use the
          // setup attribute value of setup:actpass and WebRTC will be allowed to
          // choose either the setup attribute value of setup:active or
          // setup:passive. Chrome will by default choose setup:active because it is
          // RECOMMENDED by the respective RFC since setup:passive adds additional
          // latency. The case of setup:active allows WebRTC to send a DTLS
          // ClientHello as soon as an ICE connectivity check of its succeeds.
          // Unfortunately, Videobridge will be unable to respond immediately because
          // may not have WebRTC's answer or may have not completed the ICE
          // connectivity establishment. Even more unfortunate is that in the
          // described scenario Chrome's DTLS implementation will insist on
          // retransmitting its ClientHello after a second (the time is in accord
          // with the respective RFC) and will thus cause the whole connection
          // establishment to exceed at least 1 second. To work around Chrome's
          // idiosyncracy, don't allow it to send a ClientHello i.e. change its
          // default choice of setup:active to setup:passive.

2016-10-09 18:19 GMT+02:00 Oliver Hausler <oliver@closeup.cc<javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>>:
// Once again because the ICE Agent does not support adding
// candidates after the connectivity establishment has been started
// and because multiple transport-info JingleIQs may be used to send
// the whole set of transport candidates from the remote peer to the
// local peer, do not really start the connectivity establishment
// until we have at least one remote candidate per ICE Component.

This comment it ICE Agent makes me believe I am right. And Jitsi is sending streams when I send a complete SDP with all candidates.

Can anybody comment what's the status here, please.

_______________________________________________
dev mailing list
dev@jitsi.org<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile


#2

Emil,

> so I assume your use of trickle simply introduces a delay that hides the
actual problem

Are you saying I must synchronize patching the conference? If yes, is this
per conference or in general? I assume I can make 2 parallel REST calls if
a distinct conference is involved, correct?

No, I was not implying a race condition on the bridge but within your

client,
What I understood from your earlier e-mails is that:

1. When your clients trcikle your candidates, then send SSRCs, everything
looks good.
2. When they don't trickle then the SSRCs look wrong.

I have no idea how your client works so all of this is just speculation but
my suggestion was to make sure that you don't have a race condition on your
side.

As far as the bridge is concerned it only needs to get the right SSRCs and
at least one (whatever IP address).

I am afraid I won't be able to help debugging your code.

Emil

···

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc> wrote:

> Sounds like a race condition with your SSRC generation.

I was in fact making the REST call for adding ANSWER SDP in 2 different
threads simultaneously. But even synchronizing these does not help. I am
still seeing inconsistent results for SSRCs. And I even added a random time
delay within the lock to make sure Jitsi has enough time for processing.
When I do trickle, I get consistent results all the time (otherwise same
code).

*private static *ReentrantLock *addSdpLock *= *new *ReentrantLock();

*public *void addSdp(@Nonnull String socketId, @Nonnull Sdp sdp) {
    *addSdpLock*.lock();
    *try *{
        *log*.info(*"TEST ENTER"*);

         response = JitsiApiBuilder.*api*().patchConference(*id*, *conference*).execute();

        Thread.sleep() // random delay 1000..2000 msec

        *log*.info(*"TEST EXIT"*);
    } *finally *{
        *addSdpLock*.unlock();
    }

}

2016-10-22 16:18:07,422 INFO Conference [nioEventLoopGroup-3-7] TEST ENTER

2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-7] TEST EXIT

2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-8] TEST ENTER

2016-10-22 16:18:10,325 INFO Conference [nioEventLoopGroup-3-8] TEST EXIT

*From:* Emil Ivov
*Sent:* Saturday, October 22, 2016 14:34
*To:* Oliver Hausler
*Subject:* Re: [jitsi-dev] How to add remote-candidates to colibri with
trickle ICE using REST?

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc > <javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>> wrote:

Emil,

I am not using ICE trickle server -> client. Do you mean I can also switch
it off in the other direction?

Yes. I think we only expect to see on address from the client in order to
prevent a bug but it doesn't need to be a working address.

In other words: you don't meed to signal any actual addresses from client
to server. Just send your offer as soon as you have it.

The SSRCs in the final colibri look better when trickle client -> server
is on. I don't know if this is a bug in Jitsi, but when I don't trickle
from client -> server, sometime/always (not sure) an SSRC is missing for
one of the channels. If I trickle, it always looks fine.

Sounds like a race condition with your SSRC generation. SSRCs are in no
way related to trickle ICE or addresses so I assume your use of trickle
simply introduces a delay that hides the actual problem

Emil

Oliver.

*From:* Emil Ivov
*Sent:* Saturday, October 22, 2016 14:25
*To:* Jitsi Developers <dev@jitsi.org>;
*Subject:* Re: [jitsi-dev] How to add remote-candidates to colibri with
trickle ICE using REST?

Oliver,

From what you have said it doesn't sound like you need to trickle any
candidates. As long as your videobridge is on a public network it will
discover all client candidates as peer reflexive (this is still trickle ICE
but with now actual client-side trickling).

This is what we do on all Jitsi Meet deployments including meet.jit.si

Emil

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc> wrote:

Emil, Philipp, sorry for the late response.

@Philipp: It doesn't work for me with passive. Can you see any issues with
my logs below? Might be entirely unrelated as I see DTLS warnings in the
Jitsi log. I posted this separately: "Unknown DTLS handshake message and no
streams from Jitsi (REST)"

@Emil: I assume we should be using ICE trickle as this is standard in
WebRTC, so I tried to do that:

- I am creating an OFFER SDP which contains all Candidates
discovered by Jitsi (no ICE trickle, makes no sense here as we have
candidates in colibri already).

In Chrome I have 2 options:

1. ICE trickle: I listen to createAnswerOnSuccess(), send the
ANSWER SDP to Jitsi, wait for onIceCandidate() and send each Candidate as
it is discovered.

2. No ICE trickle: I listen to iceGatheringStateChange() ==
complete, then send the full ANSWER SDP with all Candidates.

I thought 1. would be the preferred way when using WebRTC/libjingle, so I
followed what Philipp Seeser suggested and it looks like Jitsi accepts the
"remote-candidate"(s) as it replays it:

"channel-bundles": [{

                                "id": "AKQQeYSlhEwltx69_ZWinO8|||||
N-N77FcLReCCe5yoieF_7A",

                                "transport": {

                                                "xmlns":
"urn:xmpp:jingle:transports:ice-udp:1",

                                                "rtcp-mux": true,

                                                "remote-candidate": {

"component": 1,

"protocol": "udp",

"priority": 1685987071,

                                                                "ip":
"68.96.117.185",

                                                                "port":
56117,

                                                                "type":
"srflx",

"generation": 0,

"foundation": "1855365925",

"network": 1

                                                },

                                                "setup": "passive"

                                }

                }]

I assume I must add each Candidate like that, and Jitsi picks the one
which it will use. Because the last 2 candidates I am receiving use
throw-away port 9 [why is Chrome generating these at all?], if Jitsi would
use the last Candidate, it wouldn't work, but I believe it's smart enough
and uses the best candidate, even though this isn't reflected in colibri
(except for the immediate response, there is no change in colibri
thereafter, so I cannot prove this).

Finally, I see a slightly distinct result between 1 and 2, though, when
looking at "GET /colibri/conferences/{conferenceId}":

For 1. I see 2 audio channels per endpoint:

  "contents": [

    {

      "channels": [

        {

          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||
AKPDvx2Egk8dqVFUHTgkE_A",

          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||
AKPDvx2Egk8dqVFUHTgkE_A",

          "sources": [

3204522869

          ],

          "rtp-level-relay-type": "mixer",

          "expire": 300,

          "initiator": true,

          "ssrcs": [

            3204522869,

336257926

          ],

          "id": "61baaec9d48429b4",

          "direction": "sendrecv"

        },

        {

          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||
APkpD2Im3kfzq2hM1o49z3E",

          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||
APkpD2Im3kfzq2hM1o49z3E",

          "sources": [

1722705425

          ],

         "rtp-level-relay-type": "mixer",

          "expire": 300,

          "initiator": true,

          "ssrcs": [

            1722705425,

3673416731

          ],

          "id": "b41ab150c9739171",

          "direction": "sendrecv"

        }

      ],

     "name": "audio"

    },

For 2. one of the ssrcs seems to be missing for one of the channels (so
Philipps approach with ICE trickle seems to be the better working):

  "contents": [

    {

      "channels": [

        {

          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||
MEIiSaRqTGep7C2cDR42sQ",

          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||
MEIiSaRqTGep7C2cDR42sQ",

          "sources": [

916465418

          ],

          "rtp-level-relay-type": "mixer",

          "expire": 300,

          "initiator": true,

          "ssrcs": [

3268560502

          ],

          "id": "700d2d43eae73a28",

          "direction": "sendrecv"

        },

        {

          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||
ANZL23OQrkVkqzUZ-XMyyPk",

          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||
ANZL23OQrkVkqzUZ-XMyyPk",

          "sources": [

814905801

          ],

          "rtp-level-relay-type": "mixer",

          "expire": 300,

          "initiator": true,

          "ssrcs": [

            814905801,

2637946124

          ],

          "id": "ba07908e22e3b2a2",

          "direction": "sendrecv"

        }

      ],

      "name": "audio"

    },

*From:* dev [mailto:dev-bounces@jitsi.org
<javascript:_e(%7B%7D,'cvml','dev-bounces@jitsi.org');>] *On Behalf Of *Emil
Ivov
*Sent:* Sunday, October 16, 2016 22:30
*To:* Jitsi Developers <dev@jitsi.org
<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>
*Subject:* Re: [jitsi-dev] How to add remote-candidates to colibri with
trickle ICE using REST?

I am sorry I am chiming in so late but, why exactly do you need to send
any additional candidates to the bridge in the first place?

On Sunday, 16 October 2016, Oliver Hausler <oliver@closeup.cc > <javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>> wrote:

Thank you Philipp, I just noticed you sent a response to this.

Interesting, you patch each candidate into 'remote-candidate' separately.
I will try this tomorrow.

*From:* dev [mailto:dev-bounces@jitsi.org] *On Behalf Of *Philipp Seeser
*Sent:* Monday, October 10, 2016 03:02
*To:* Jitsi Developers <dev@jitsi.org>
*Subject:* Re: [jitsi-dev] How to add remote-candidates to colibri with
trickle ICE using REST?

I am able to add candidates with trickle ice and REST.

e.g. candidate json body:

var candidatePatch = {

          'foundation': parsedCandidate.foundation,

          'component': parsedCandidate.component,

          'priority': parsedCandidate.priority,

          'ip': parsedCandidate.ip,

          'port': parsedCandidate.port,

          'generation': parsedCandidate.generation,

          'type': parsedCandidate.type,

          'rel-addr': parsedCandidate.raddr,

          'rel-port': parsedCandidate.rport,

          'protocol': parsedCandidate.protocol,

          'network' : parsedCandidate.network,

          'id' : parsedCandidate.id

        };

        var patch: any = {

          id: msg.streams.publisher,

          'channel-bundles': [{

            'id': msg.streams.channelBundleId,

            transport: {

              'xmlns': 'urn:xmpp:jingle:transports:ice-udp:1',

              'setup': 'passive',

              'remote-candidate': candidatePatch

            }

          }]

        };

Also look at https://github.com/jitsi/lib-jitsi-meet/blob/master/
modules/xmpp/JingleSessionPC.js#L330

and I recommend to replace the answerer offer with "passive"

          // XXX Videobridge is the (SDP) offerer and WebRTC (e.g. Chrome)
is the
          // answerer (as orchestrated by Jicofo). In accord with
          // http://tools.ietf.org/html/rfc5245#section-5.2 and because
both peers
          // are ICE FULL agents, Videobridge will take on the controlling
role and
          // WebRTC will take on the controlled role. In accord with
          // https://tools.ietf.org/html/rfc5763#section-5, Videobridge
will use the
          // setup attribute value of setup:actpass and WebRTC will be
allowed to
          // choose either the setup attribute value of setup:active or
          // setup:passive. Chrome will by default choose setup:active
because it is
          // RECOMMENDED by the respective RFC since setup:passive adds
additional
          // latency. The case of setup:active allows WebRTC to send a DTLS
          // ClientHello as soon as an ICE connectivity check of its
succeeds.
          // Unfortunately, Videobridge will be unable to respond
immediately because
          // may not have WebRTC's answer or may have not completed the ICE
          // connectivity establishment. Even more unfortunate is that in
the
          // described scenario Chrome's DTLS implementation will insist on
          // retransmitting its ClientHello after a second (the time is in
accord
          // with the respective RFC) and will thus cause the whole
connection
          // establishment to exceed at least 1 second. To work around
Chrome's
          // idiosyncracy, don't allow it to send a ClientHello i.e.
change its
          // default choice of setup:active to setup:passive.

2016-10-09 18:19 GMT+02:00 Oliver Hausler <oliver@closeup.cc>:

*// Once again because the ICE Agent does not support adding // candidates
after the connectivity establishment has been started // and because
multiple transport-info JingleIQs may be used to send // the whole set of
transport candidates from the remote peer to the // local peer, do not
really start the connectivity establishment // until we have at least one
remote candidate per ICE Component.*

This comment it ICE Agent makes me believe I am right. And Jitsi is
sending streams when I send a complete SDP with all candidates.

Can anybody comment what's the status here, please.

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile


#3

Emil,

I believe you are right with the synchronization. I was loading the conference outside the sync block. I think we can entirely remove loading and just patch what's required. Can't test right now but don't put any further work into this for now.

Oliver.

···

From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Emil Ivov
Sent: Saturday, October 22, 2016 16:45
To: Jitsi Developers <dev@jitsi.org>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc<mailto:oliver@closeup.cc>> wrote:
Emil,

so I assume your use of trickle simply introduces a delay that hides the actual problem

Are you saying I must synchronize patching the conference? If yes, is this per conference or in general? I assume I can make 2 parallel REST calls if a distinct conference is involved, correct?
No, I was not implying a race condition on the bridge but within your client,
What I understood from your earlier e-mails is that:

1. When your clients trcikle your candidates, then send SSRCs, everything looks good.
2. When they don't trickle then the SSRCs look wrong.

I have no idea how your client works so all of this is just speculation but my suggestion was to make sure that you don't have a race condition on your side.

As far as the bridge is concerned it only needs to get the right SSRCs and at least one (whatever IP address).

I am afraid I won't be able to help debugging your code.

Emil

Sounds like a race condition with your SSRC generation.

I was in fact making the REST call for adding ANSWER SDP in 2 different threads simultaneously. But even synchronizing these does not help. I am still seeing inconsistent results for SSRCs. And I even added a random time delay within the lock to make sure Jitsi has enough time for processing. When I do trickle, I get consistent results all the time (otherwise same code).

private static ReentrantLock addSdpLock = new ReentrantLock();

public void addSdp(@Nonnull String socketId, @Nonnull Sdp sdp) {
    addSdpLock.lock();
    try {
        log.info("TEST ENTER");

         response = JitsiApiBuilder.api().patchConference(id, conference).execute();

        Thread.sleep() // random delay 1000..2000 msec

        log.info("TEST EXIT");
    } finally {
        addSdpLock.unlock();
    }
}

2016-10-22 16:18:07,422 INFO Conference [nioEventLoopGroup-3-7] TEST ENTER
2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-7] TEST EXIT
2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-8] TEST ENTER
2016-10-22 16:18:10,325 INFO Conference [nioEventLoopGroup-3-8] TEST EXIT

From: Emil Ivov
Sent: Saturday, October 22, 2016 14:34
To: Oliver Hausler
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc<javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>> wrote:
Emil,

I am not using ICE trickle server -> client. Do you mean I can also switch it off in the other direction?

Yes. I think we only expect to see on address from the client in order to prevent a bug but it doesn't need to be a working address.

In other words: you don't meed to signal any actual addresses from client to server. Just send your offer as soon as you have it.

The SSRCs in the final colibri look better when trickle client -> server is on. I don't know if this is a bug in Jitsi, but when I don't trickle from client -> server, sometime/always (not sure) an SSRC is missing for one of the channels. If I trickle, it always looks fine.

Sounds like a race condition with your SSRC generation. SSRCs are in no way related to trickle ICE or addresses so I assume your use of trickle simply introduces a delay that hides the actual problem

Emil

Oliver.

From: Emil Ivov
Sent: Saturday, October 22, 2016 14:25
To: Jitsi Developers <dev@jitsi.org<mailto:dev@jitsi.org>>;
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

Oliver,

From what you have said it doesn't sound like you need to trickle any candidates. As long as your videobridge is on a public network it will discover all client candidates as peer reflexive (this is still trickle ICE but with now actual client-side trickling).

This is what we do on all Jitsi Meet deployments including meet.jit.si<http://meet.jit.si>

Emil

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc<mailto:oliver@closeup.cc>> wrote:
Emil, Philipp, sorry for the late response.

@Philipp: It doesn't work for me with passive. Can you see any issues with my logs below? Might be entirely unrelated as I see DTLS warnings in the Jitsi log. I posted this separately: "Unknown DTLS handshake message and no streams from Jitsi (REST)"

@Emil: I assume we should be using ICE trickle as this is standard in WebRTC, so I tried to do that:

- I am creating an OFFER SDP which contains all Candidates discovered by Jitsi (no ICE trickle, makes no sense here as we have candidates in colibri already).

In Chrome I have 2 options:

1. ICE trickle: I listen to createAnswerOnSuccess(), send the ANSWER SDP to Jitsi, wait for onIceCandidate() and send each Candidate as it is discovered.

2. No ICE trickle: I listen to iceGatheringStateChange() == complete, then send the full ANSWER SDP with all Candidates.

I thought 1. would be the preferred way when using WebRTC/libjingle, so I followed what Philipp Seeser suggested and it looks like Jitsi accepts the "remote-candidate"(s) as it replays it:

"channel-bundles": [{
                                "id": "AKQQeYSlhEwltx69_ZWinO8|||||N-N77FcLReCCe5yoieF_7A",
                                "transport": {
                                                "xmlns": "urn:xmpp:jingle:transports:ice-udp:1",
                                                "rtcp-mux": true,
                                                "remote-candidate": {
                                                                "component": 1,
                                                                "protocol": "udp",
                                                                "priority": 1685987071,
                                                                "ip": "68.96.117.185",
                                                                "port": 56117,
                                                                "type": "srflx",
                                                                "generation": 0,
                                                                "foundation": "1855365925",
                                                                "network": 1
                                                },
                                                "setup": "passive"
                                }
                }]

I assume I must add each Candidate like that, and Jitsi picks the one which it will use. Because the last 2 candidates I am receiving use throw-away port 9 [why is Chrome generating these at all?], if Jitsi would use the last Candidate, it wouldn't work, but I believe it's smart enough and uses the best candidate, even though this isn't reflected in colibri (except for the immediate response, there is no change in colibri thereafter, so I cannot prove this).

Finally, I see a slightly distinct result between 1 and 2, though, when looking at "GET /colibri/conferences/{conferenceId}":

For 1. I see 2 audio channels per endpoint:

  "contents": [
    {
      "channels": [
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||AKPDvx2Egk8dqVFUHTgkE_A",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||AKPDvx2Egk8dqVFUHTgkE_A",
          "sources": [
3204522869
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            3204522869,
336257926
          ],
          "id": "61baaec9d48429b4",
          "direction": "sendrecv"
        },
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||APkpD2Im3kfzq2hM1o49z3E",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||APkpD2Im3kfzq2hM1o49z3E",
          "sources": [
1722705425
          ],
         "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            1722705425,
3673416731
          ],
          "id": "b41ab150c9739171",
          "direction": "sendrecv"
        }
      ],
     "name": "audio"
    },

For 2. one of the ssrcs seems to be missing for one of the channels (so Philipps approach with ICE trickle seems to be the better working):

  "contents": [
    {
      "channels": [
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||MEIiSaRqTGep7C2cDR42sQ",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||MEIiSaRqTGep7C2cDR42sQ",
          "sources": [
916465418
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
3268560502
          ],
          "id": "700d2d43eae73a28",
          "direction": "sendrecv"
        },
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||ANZL23OQrkVkqzUZ-XMyyPk",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||ANZL23OQrkVkqzUZ-XMyyPk",
          "sources": [
814905801
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            814905801,
2637946124
          ],
          "id": "ba07908e22e3b2a2",
          "direction": "sendrecv"
        }
      ],
      "name": "audio"
    },

From: dev [mailto:dev-bounces@jitsi.org<javascript:_e(%7B%7D,'cvml','dev-bounces@jitsi.org');>] On Behalf Of Emil Ivov
Sent: Sunday, October 16, 2016 22:30
To: Jitsi Developers <dev@jitsi.org<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

I am sorry I am chiming in so late but, why exactly do you need to send any additional candidates to the bridge in the first place?

On Sunday, 16 October 2016, Oliver Hausler <oliver@closeup.cc<javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>> wrote:
Thank you Philipp, I just noticed you sent a response to this.

Interesting, you patch each candidate into 'remote-candidate' separately. I will try this tomorrow.

From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Philipp Seeser
Sent: Monday, October 10, 2016 03:02
To: Jitsi Developers <dev@jitsi.org<mailto:dev@jitsi.org>>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

I am able to add candidates with trickle ice and REST.

e.g. candidate json body:

var candidatePatch = {
          'foundation': parsedCandidate.foundation,
          'component': parsedCandidate.component,
          'priority': parsedCandidate.priority,
          'ip': parsedCandidate.ip,
          'port': parsedCandidate.port,
          'generation': parsedCandidate.generation,
          'type': parsedCandidate.type,
          'rel-addr': parsedCandidate.raddr,
          'rel-port': parsedCandidate.rport,
          'protocol': parsedCandidate.protocol,
          'network' : parsedCandidate.network,
          'id' : parsedCandidate.id
        };
        var patch: any = {
          id: msg.streams.publisher,
          'channel-bundles': [{
            'id': msg.streams.channelBundleId,
            transport: {
              'xmlns': 'urn:xmpp:jingle:transports:ice-udp:1',
              'setup': 'passive',
              'remote-candidate': candidatePatch
            }
          }]
        };

Also look at https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/xmpp/JingleSessionPC.js#L330
and I recommend to replace the answerer offer with "passive"

          // XXX Videobridge is the (SDP) offerer and WebRTC (e.g. Chrome) is the
          // answerer (as orchestrated by Jicofo). In accord with
          // http://tools.ietf.org/html/rfc5245#section-5.2 and because both peers
          // are ICE FULL agents, Videobridge will take on the controlling role and
          // WebRTC will take on the controlled role. In accord with
          // https://tools.ietf.org/html/rfc5763#section-5, Videobridge will use the
          // setup attribute value of setup:actpass and WebRTC will be allowed to
          // choose either the setup attribute value of setup:active or
          // setup:passive. Chrome will by default choose setup:active because it is
          // RECOMMENDED by the respective RFC since setup:passive adds additional
          // latency. The case of setup:active allows WebRTC to send a DTLS
          // ClientHello as soon as an ICE connectivity check of its succeeds.
          // Unfortunately, Videobridge will be unable to respond immediately because
          // may not have WebRTC's answer or may have not completed the ICE
          // connectivity establishment. Even more unfortunate is that in the
          // described scenario Chrome's DTLS implementation will insist on
          // retransmitting its ClientHello after a second (the time is in accord
          // with the respective RFC) and will thus cause the whole connection
          // establishment to exceed at least 1 second. To work around Chrome's
          // idiosyncracy, don't allow it to send a ClientHello i.e. change its
          // default choice of setup:active to setup:passive.

2016-10-09 18:19 GMT+02:00 Oliver Hausler <oliver@closeup.cc<mailto:oliver@closeup.cc>>:
// Once again because the ICE Agent does not support adding
// candidates after the connectivity establishment has been started
// and because multiple transport-info JingleIQs may be used to send
// the whole set of transport candidates from the remote peer to the
// local peer, do not really start the connectivity establishment
// until we have at least one remote candidate per ICE Component.

This comment it ICE Agent makes me believe I am right. And Jitsi is sending streams when I send a complete SDP with all candidates.

Can anybody comment what's the status here, please.

_______________________________________________
dev mailing list
dev@jitsi.org<mailto:dev@jitsi.org>
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile


#4

Emil,

You were right with the synchronization issue. SSRCs look fine now.

Oliver.

···

From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Oliver Hausler
Sent: Saturday, October 22, 2016 18:34
To: Jitsi Developers <dev@jitsi.org>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

Emil,

I believe you are right with the synchronization. I was loading the conference outside the sync block. I think we can entirely remove loading and just patch what's required. Can't test right now but don't put any further work into this for now.

Oliver.

From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Emil Ivov
Sent: Saturday, October 22, 2016 16:45
To: Jitsi Developers <dev@jitsi.org<mailto:dev@jitsi.org>>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc<mailto:oliver@closeup.cc>> wrote:
Emil,

so I assume your use of trickle simply introduces a delay that hides the actual problem

Are you saying I must synchronize patching the conference? If yes, is this per conference or in general? I assume I can make 2 parallel REST calls if a distinct conference is involved, correct?
No, I was not implying a race condition on the bridge but within your client,
What I understood from your earlier e-mails is that:

1. When your clients trcikle your candidates, then send SSRCs, everything looks good.
2. When they don't trickle then the SSRCs look wrong.

I have no idea how your client works so all of this is just speculation but my suggestion was to make sure that you don't have a race condition on your side.

As far as the bridge is concerned it only needs to get the right SSRCs and at least one (whatever IP address).

I am afraid I won't be able to help debugging your code.

Emil

Sounds like a race condition with your SSRC generation.

I was in fact making the REST call for adding ANSWER SDP in 2 different threads simultaneously. But even synchronizing these does not help. I am still seeing inconsistent results for SSRCs. And I even added a random time delay within the lock to make sure Jitsi has enough time for processing. When I do trickle, I get consistent results all the time (otherwise same code).

private static ReentrantLock addSdpLock = new ReentrantLock();

public void addSdp(@Nonnull String socketId, @Nonnull Sdp sdp) {
    addSdpLock.lock();
    try {
        log.info("TEST ENTER");

         response = JitsiApiBuilder.api().patchConference(id, conference).execute();

        Thread.sleep() // random delay 1000..2000 msec

        log.info("TEST EXIT");
    } finally {
        addSdpLock.unlock();
    }
}

2016-10-22 16:18:07,422 INFO Conference [nioEventLoopGroup-3-7] TEST ENTER
2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-7] TEST EXIT
2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-8] TEST ENTER
2016-10-22 16:18:10,325 INFO Conference [nioEventLoopGroup-3-8] TEST EXIT

From: Emil Ivov
Sent: Saturday, October 22, 2016 14:34
To: Oliver Hausler
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc<javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>> wrote:
Emil,

I am not using ICE trickle server -> client. Do you mean I can also switch it off in the other direction?

Yes. I think we only expect to see on address from the client in order to prevent a bug but it doesn't need to be a working address.

In other words: you don't meed to signal any actual addresses from client to server. Just send your offer as soon as you have it.

The SSRCs in the final colibri look better when trickle client -> server is on. I don't know if this is a bug in Jitsi, but when I don't trickle from client -> server, sometime/always (not sure) an SSRC is missing for one of the channels. If I trickle, it always looks fine.

Sounds like a race condition with your SSRC generation. SSRCs are in no way related to trickle ICE or addresses so I assume your use of trickle simply introduces a delay that hides the actual problem

Emil

Oliver.

From: Emil Ivov
Sent: Saturday, October 22, 2016 14:25
To: Jitsi Developers <dev@jitsi.org<mailto:dev@jitsi.org>>;
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

Oliver,

From what you have said it doesn't sound like you need to trickle any candidates. As long as your videobridge is on a public network it will discover all client candidates as peer reflexive (this is still trickle ICE but with now actual client-side trickling).

This is what we do on all Jitsi Meet deployments including meet.jit.si<http://meet.jit.si>

Emil

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc<mailto:oliver@closeup.cc>> wrote:
Emil, Philipp, sorry for the late response.

@Philipp: It doesn't work for me with passive. Can you see any issues with my logs below? Might be entirely unrelated as I see DTLS warnings in the Jitsi log. I posted this separately: "Unknown DTLS handshake message and no streams from Jitsi (REST)"

@Emil: I assume we should be using ICE trickle as this is standard in WebRTC, so I tried to do that:

- I am creating an OFFER SDP which contains all Candidates discovered by Jitsi (no ICE trickle, makes no sense here as we have candidates in colibri already).

In Chrome I have 2 options:

1. ICE trickle: I listen to createAnswerOnSuccess(), send the ANSWER SDP to Jitsi, wait for onIceCandidate() and send each Candidate as it is discovered.

2. No ICE trickle: I listen to iceGatheringStateChange() == complete, then send the full ANSWER SDP with all Candidates.

I thought 1. would be the preferred way when using WebRTC/libjingle, so I followed what Philipp Seeser suggested and it looks like Jitsi accepts the "remote-candidate"(s) as it replays it:

"channel-bundles": [{
                                "id": "AKQQeYSlhEwltx69_ZWinO8|||||N-N77FcLReCCe5yoieF_7A",
                                "transport": {
                                                "xmlns": "urn:xmpp:jingle:transports:ice-udp:1",
                                                "rtcp-mux": true,
                                                "remote-candidate": {
                                                                "component": 1,
                                                                "protocol": "udp",
                                                                "priority": 1685987071,
                                                                "ip": "68.96.117.185",
                                                                "port": 56117,
                                                                "type": "srflx",
                                                                "generation": 0,
                                                                "foundation": "1855365925",
                                                                "network": 1
                                                },
                                                "setup": "passive"
                                }
                }]

I assume I must add each Candidate like that, and Jitsi picks the one which it will use. Because the last 2 candidates I am receiving use throw-away port 9 [why is Chrome generating these at all?], if Jitsi would use the last Candidate, it wouldn't work, but I believe it's smart enough and uses the best candidate, even though this isn't reflected in colibri (except for the immediate response, there is no change in colibri thereafter, so I cannot prove this).

Finally, I see a slightly distinct result between 1 and 2, though, when looking at "GET /colibri/conferences/{conferenceId}":

For 1. I see 2 audio channels per endpoint:

  "contents": [
    {
      "channels": [
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||AKPDvx2Egk8dqVFUHTgkE_A",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||AKPDvx2Egk8dqVFUHTgkE_A",
          "sources": [
3204522869
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            3204522869,
336257926
          ],
          "id": "61baaec9d48429b4",
          "direction": "sendrecv"
        },
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||APkpD2Im3kfzq2hM1o49z3E",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||APkpD2Im3kfzq2hM1o49z3E",
          "sources": [
1722705425
          ],
         "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            1722705425,
3673416731
          ],
          "id": "b41ab150c9739171",
          "direction": "sendrecv"
        }
      ],
     "name": "audio"
    },

For 2. one of the ssrcs seems to be missing for one of the channels (so Philipps approach with ICE trickle seems to be the better working):

  "contents": [
    {
      "channels": [
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||MEIiSaRqTGep7C2cDR42sQ",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||MEIiSaRqTGep7C2cDR42sQ",
          "sources": [
916465418
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
3268560502
          ],
          "id": "700d2d43eae73a28",
          "direction": "sendrecv"
        },
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||ANZL23OQrkVkqzUZ-XMyyPk",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||ANZL23OQrkVkqzUZ-XMyyPk",
          "sources": [
814905801
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            814905801,
2637946124
          ],
          "id": "ba07908e22e3b2a2",
          "direction": "sendrecv"
        }
      ],
      "name": "audio"
    },

From: dev [mailto:dev-bounces@jitsi.org<javascript:_e(%7B%7D,'cvml','dev-bounces@jitsi.org');>] On Behalf Of Emil Ivov
Sent: Sunday, October 16, 2016 22:30
To: Jitsi Developers <dev@jitsi.org<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

I am sorry I am chiming in so late but, why exactly do you need to send any additional candidates to the bridge in the first place?

On Sunday, 16 October 2016, Oliver Hausler <oliver@closeup.cc<javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>> wrote:
Thank you Philipp, I just noticed you sent a response to this.

Interesting, you patch each candidate into 'remote-candidate' separately. I will try this tomorrow.

From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Philipp Seeser
Sent: Monday, October 10, 2016 03:02
To: Jitsi Developers <dev@jitsi.org<mailto:dev@jitsi.org>>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

I am able to add candidates with trickle ice and REST.

e.g. candidate json body:

var candidatePatch = {
          'foundation': parsedCandidate.foundation,
          'component': parsedCandidate.component,
          'priority': parsedCandidate.priority,
          'ip': parsedCandidate.ip,
          'port': parsedCandidate.port,
          'generation': parsedCandidate.generation,
          'type': parsedCandidate.type,
          'rel-addr': parsedCandidate.raddr,
          'rel-port': parsedCandidate.rport,
          'protocol': parsedCandidate.protocol,
          'network' : parsedCandidate.network,
          'id' : parsedCandidate.id
        };
        var patch: any = {
          id: msg.streams.publisher,
          'channel-bundles': [{
            'id': msg.streams.channelBundleId,
            transport: {
              'xmlns': 'urn:xmpp:jingle:transports:ice-udp:1',
              'setup': 'passive',
              'remote-candidate': candidatePatch
            }
          }]
        };

Also look at https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/xmpp/JingleSessionPC.js#L330
and I recommend to replace the answerer offer with "passive"

          // XXX Videobridge is the (SDP) offerer and WebRTC (e.g. Chrome) is the
          // answerer (as orchestrated by Jicofo). In accord with
          // http://tools.ietf.org/html/rfc5245#section-5.2 and because both peers
          // are ICE FULL agents, Videobridge will take on the controlling role and
          // WebRTC will take on the controlled role. In accord with
          // https://tools.ietf.org/html/rfc5763#section-5, Videobridge will use the
          // setup attribute value of setup:actpass and WebRTC will be allowed to
          // choose either the setup attribute value of setup:active or
          // setup:passive. Chrome will by default choose setup:active because it is
          // RECOMMENDED by the respective RFC since setup:passive adds additional
          // latency. The case of setup:active allows WebRTC to send a DTLS
          // ClientHello as soon as an ICE connectivity check of its succeeds.
          // Unfortunately, Videobridge will be unable to respond immediately because
          // may not have WebRTC's answer or may have not completed the ICE
          // connectivity establishment. Even more unfortunate is that in the
          // described scenario Chrome's DTLS implementation will insist on
          // retransmitting its ClientHello after a second (the time is in accord
          // with the respective RFC) and will thus cause the whole connection
          // establishment to exceed at least 1 second. To work around Chrome's
          // idiosyncracy, don't allow it to send a ClientHello i.e. change its
          // default choice of setup:active to setup:passive.

2016-10-09 18:19 GMT+02:00 Oliver Hausler <oliver@closeup.cc<mailto:oliver@closeup.cc>>:
// Once again because the ICE Agent does not support adding
// candidates after the connectivity establishment has been started
// and because multiple transport-info JingleIQs may be used to send
// the whole set of transport candidates from the remote peer to the
// local peer, do not really start the connectivity establishment
// until we have at least one remote candidate per ICE Component.

This comment it ICE Agent makes me believe I am right. And Jitsi is sending streams when I send a complete SDP with all candidates.

Can anybody comment what's the status here, please.

_______________________________________________
dev mailing list
dev@jitsi.org<mailto:dev@jitsi.org>
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile


#5

Awesome! Glad to know you've solved it!

···

On Sunday, 23 October 2016, Oliver Hausler <oliver@closeup.cc> wrote:

Emil,

You were right with the synchronization issue. SSRCs look fine now.

Oliver.

*From:* dev [mailto:dev-bounces@jitsi.org
<javascript:_e(%7B%7D,'cvml','dev-bounces@jitsi.org');>] *On Behalf Of *Oliver
Hausler
*Sent:* Saturday, October 22, 2016 18:34
*To:* Jitsi Developers <dev@jitsi.org
<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>
*Subject:* Re: [jitsi-dev] How to add remote-candidates to colibri with
trickle ICE using REST?

Emil,

I believe you are right with the synchronization. I was loading the
conference outside the sync block. I think we can entirely remove loading
and just patch what's required. Can't test right now but don't put any
further work into this for now.

Oliver.

*From:* dev [mailto:dev-bounces@jitsi.org
<javascript:_e(%7B%7D,'cvml','dev-bounces@jitsi.org');>] *On Behalf Of *Emil
Ivov
*Sent:* Saturday, October 22, 2016 16:45
*To:* Jitsi Developers <dev@jitsi.org
<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>
*Subject:* Re: [jitsi-dev] How to add remote-candidates to colibri with
trickle ICE using REST?

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc > <javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>> wrote:

Emil,

> so I assume your use of trickle simply introduces a delay that hides the
actual problem

Are you saying I must synchronize patching the conference? If yes, is this
per conference or in general? I assume I can make 2 parallel REST calls if
a distinct conference is involved, correct?

No, I was not implying a race condition on the bridge but within your
client,

What I understood from your earlier e-mails is that:

1. When your clients trcikle your candidates, then send SSRCs, everything
looks good.

2. When they don't trickle then the SSRCs look wrong.

I have no idea how your client works so all of this is just speculation
but my suggestion was to make sure that you don't have a race condition on
your side.

As far as the bridge is concerned it only needs to get the right SSRCs
and at least one (whatever IP address).

I am afraid I won't be able to help debugging your code.

Emil

> Sounds like a race condition with your SSRC generation.

I was in fact making the REST call for adding ANSWER SDP in 2 different
threads simultaneously. But even synchronizing these does not help. I am
still seeing inconsistent results for SSRCs. And I even added a random time
delay within the lock to make sure Jitsi has enough time for processing.
When I do trickle, I get consistent results all the time (otherwise same
code).

*private static *ReentrantLock *addSdpLock *= *new *ReentrantLock();

*public *void addSdp(@Nonnull String socketId, @Nonnull Sdp sdp) {
    *addSdpLock*.lock();
    *try *{
        *log*.info(*"TEST ENTER"*);

         response = JitsiApiBuilder.*api*().patchConference(*id*, *conference*).execute();

        Thread.sleep() // random delay 1000..2000 msec

        *log*.info(*"TEST EXIT"*);
    } *finally *{
        *addSdpLock*.unlock();
    }

}

2016-10-22 16:18:07,422 INFO Conference [nioEventLoopGroup-3-7] TEST ENTER

2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-7] TEST EXIT

2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-8] TEST ENTER

2016-10-22 16:18:10,325 INFO Conference [nioEventLoopGroup-3-8] TEST EXIT

*From:* Emil Ivov
*Sent:* Saturday, October 22, 2016 14:34
*To:* Oliver Hausler
*Subject:* Re: [jitsi-dev] How to add remote-candidates to colibri with
trickle ICE using REST?

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc> wrote:

Emil,

I am not using ICE trickle server -> client. Do you mean I can also switch
it off in the other direction?

Yes. I think we only expect to see on address from the client in order to
prevent a bug but it doesn't need to be a working address.

In other words: you don't meed to signal any actual addresses from client
to server. Just send your offer as soon as you have it.

The SSRCs in the final colibri look better when trickle client -> server
is on. I don't know if this is a bug in Jitsi, but when I don't trickle
from client -> server, sometime/always (not sure) an SSRC is missing for
one of the channels. If I trickle, it always looks fine.

Sounds like a race condition with your SSRC generation. SSRCs are in no
way related to trickle ICE or addresses so I assume your use of trickle
simply introduces a delay that hides the actual problem

Emil

Oliver.

*From:* Emil Ivov
*Sent:* Saturday, October 22, 2016 14:25
*To:* Jitsi Developers <dev@jitsi.org
<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>;
*Subject:* Re: [jitsi-dev] How to add remote-candidates to colibri with
trickle ICE using REST?

Oliver,

From what you have said it doesn't sound like you need to trickle any
candidates. As long as your videobridge is on a public network it will
discover all client candidates as peer reflexive (this is still trickle ICE
but with now actual client-side trickling).

This is what we do on all Jitsi Meet deployments including meet.jit.si

Emil

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc > <javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>> wrote:

Emil, Philipp, sorry for the late response.

@Philipp: It doesn't work for me with passive. Can you see any issues with
my logs below? Might be entirely unrelated as I see DTLS warnings in the
Jitsi log. I posted this separately: "Unknown DTLS handshake message and no
streams from Jitsi (REST)"

@Emil: I assume we should be using ICE trickle as this is standard in
WebRTC, so I tried to do that:

- I am creating an OFFER SDP which contains all Candidates
discovered by Jitsi (no ICE trickle, makes no sense here as we have
candidates in colibri already).

In Chrome I have 2 options:

1. ICE trickle: I listen to createAnswerOnSuccess(), send the
ANSWER SDP to Jitsi, wait for onIceCandidate() and send each Candidate as
it is discovered.

2. No ICE trickle: I listen to iceGatheringStateChange() ==
complete, then send the full ANSWER SDP with all Candidates.

I thought 1. would be the preferred way when using WebRTC/libjingle, so I
followed what Philipp Seeser suggested and it looks like Jitsi accepts the
"remote-candidate"(s) as it replays it:

"channel-bundles": [{

                                "id": "AKQQeYSlhEwltx69_ZWinO8|||||
N-N77FcLReCCe5yoieF_7A",

                                "transport": {

                                                "xmlns":
"urn:xmpp:jingle:transports:ice-udp:1",

                                                "rtcp-mux": true,

                                                "remote-candidate": {

"component": 1,

"protocol": "udp",

"priority": 1685987071,

                                                                "ip":
"68.96.117.185",

                                                                "port":
56117,

                                                                "type":
"srflx",

"generation": 0,

"foundation": "1855365925",

"network": 1

                                                },

                                                "setup": "passive"

                                }

                }]

I assume I must add each Candidate like that, and Jitsi picks the one
which it will use. Because the last 2 candidates I am receiving use
throw-away port 9 [why is Chrome generating these at all?], if Jitsi would
use the last Candidate, it wouldn't work, but I believe it's smart enough
and uses the best candidate, even though this isn't reflected in colibri
(except for the immediate response, there is no change in colibri
thereafter, so I cannot prove this).

Finally, I see a slightly distinct result between 1 and 2, though, when
looking at "GET /colibri/conferences/{conferenceId}":

For 1. I see 2 audio channels per endpoint:

  "contents": [

    {

      "channels": [

        {

          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||
AKPDvx2Egk8dqVFUHTgkE_A",

          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||
AKPDvx2Egk8dqVFUHTgkE_A",

          "sources": [

3204522869

          ],

          "rtp-level-relay-type": "mixer",

          "expire": 300,

          "initiator": true,

          "ssrcs": [

            3204522869,

336257926

          ],

          "id": "61baaec9d48429b4",

          "direction": "sendrecv"

        },

        {

          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||
APkpD2Im3kfzq2hM1o49z3E",

          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||
APkpD2Im3kfzq2hM1o49z3E",

          "sources": [

1722705425

          ],

         "rtp-level-relay-type": "mixer",

          "expire": 300,

          "initiator": true,

          "ssrcs": [

            1722705425,

3673416731

          ],

          "id": "b41ab150c9739171",

          "direction": "sendrecv"

        }

      ],

     "name": "audio"

    },

For 2. one of the ssrcs seems to be missing for one of the channels (so
Philipps approach with ICE trickle seems to be the better working):

  "contents": [

    {

      "channels": [

        {

          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||
MEIiSaRqTGep7C2cDR42sQ",

          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||
MEIiSaRqTGep7C2cDR42sQ",

          "sources": [

916465418

          ],

          "rtp-level-relay-type": "mixer",

          "expire": 300,

          "initiator": true,

          "ssrcs": [

3268560502

          ],

          "id": "700d2d43eae73a28",

          "direction": "sendrecv"

        },

        {

          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||
ANZL23OQrkVkqzUZ-XMyyPk",

          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||
ANZL23OQrkVkqzUZ-XMyyPk",

          "sources": [

814905801

          ],

          "rtp-level-relay-type": "mixer",

          "expire": 300,

          "initiator": true,

          "ssrcs": [

            814905801,

2637946124

          ],

          "id": "ba07908e22e3b2a2",

          "direction": "sendrecv"

        }

      ],

      "name": "audio"

    },

*From:* dev [mailto:dev-bounces@jitsi.org] *On Behalf Of *Emil Ivov
*Sent:* Sunday, October 16, 2016 22:30
*To:* Jitsi Developers <dev@jitsi.org>
*Subject:* Re: [jitsi-dev] How to add remote-candidates to colibri with
trickle ICE using REST?

I am sorry I am chiming in so late but, why exactly do you need to send
any additional candidates to the bridge in the first place?

On Sunday, 16 October 2016, Oliver Hausler <oliver@closeup.cc> wrote:

Thank you Philipp, I just noticed you sent a response to this.

Interesting, you patch each candidate into 'remote-candidate' separately.
I will try this tomorrow.

*From:* dev [mailto:dev-bounces@jitsi.org
<javascript:_e(%7B%7D,'cvml','dev-bounces@jitsi.org');>] *On Behalf Of *Philipp
Seeser
*Sent:* Monday, October 10, 2016 03:02
*To:* Jitsi Developers <dev@jitsi.org
<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>
*Subject:* Re: [jitsi-dev] How to add remote-candidates to colibri with
trickle ICE using REST?

I am able to add candidates with trickle ice and REST.

e.g. candidate json body:

var candidatePatch = {

          'foundation': parsedCandidate.foundation,

          'component': parsedCandidate.component,

          'priority': parsedCandidate.priority,

          'ip': parsedCandidate.ip,

          'port': parsedCandidate.port,

          'generation': parsedCandidate.generation,

          'type': parsedCandidate.type,

          'rel-addr': parsedCandidate.raddr,

          'rel-port': parsedCandidate.rport,

          'protocol': parsedCandidate.protocol,

          'network' : parsedCandidate.network,

          'id' : parsedCandidate.id

        };

        var patch: any = {

          id: msg.streams.publisher,

          'channel-bundles': [{

            'id': msg.streams.channelBundleId,

            transport: {

              'xmlns': 'urn:xmpp:jingle:transports:ice-udp:1',

              'setup': 'passive',

              'remote-candidate': candidatePatch

            }

          }]

        };

Also look at https://github.com/jitsi/lib-jitsi-meet/blob/master/
modules/xmpp/JingleSessionPC.js#L330

and I recommend to replace the answerer offer with "passive"

          // XXX Videobridge is the (SDP) offerer and WebRTC (e.g. Chrome)
is the
          // answerer (as orchestrated by Jicofo). In accord with
          // http://tools.ietf.org/html/rfc5245#section-5.2 and because
both peers
          // are ICE FULL agents, Videobridge will take on the controlling
role and
          // WebRTC will take on the controlled role. In accord with
          // https://tools.ietf.org/html/rfc5763#section-5, Videobridge
will use the
          // setup attribute value of setup:actpass and WebRTC will be
allowed to
          // choose either the setup attribute value of setup:active or
          // setup:passive. Chrome will by default choose setup:active
because it is
          // RECOMMENDED by the respective RFC since setup:passive adds
additional
          // latency. The case of setup:active allows WebRTC to send a DTLS
          // ClientHello as soon as an ICE connectivity check of its
succeeds.
          // Unfortunately, Videobridge will be unable to respond
immediately because
          // may not have WebRTC's answer or may have not completed the ICE
          // connectivity establishment. Even more unfortunate is that in
the
          // described scenario Chrome's DTLS implementation will insist on
          // retransmitting its ClientHello after a second (the time is in
accord
          // with the respective RFC) and will thus cause the whole
connection
          // establishment to exceed at least 1 second. To work around
Chrome's
          // idiosyncracy, don't allow it to send a ClientHello i.e.
change its
          // default choice of setup:active to setup:passive.

2016-10-09 18:19 GMT+02:00 Oliver Hausler <oliver@closeup.cc
<javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>>:

*// Once again because the ICE Agent does not support adding // candidates
after the connectivity establishment has been started // and because
multiple transport-info JingleIQs may be used to send // the whole set of
transport candidates from the remote peer to the // local peer, do not
really start the connectivity establishment // until we have at least one
remote candidate per ICE Component.*

This comment it ICE Agent makes me believe I am right. And Jitsi is
sending streams when I send a complete SDP with all candidates.

Can anybody comment what's the status here, please.

_______________________________________________
dev mailing list
dev@jitsi.org <javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile


#6

The issue was this:

- I was loading and existing conference.

- Then patched new channel info into that conference while leaving the other (existing) channels alone.

- Because patches into "sources" are transferred to "ssrc", existing channels had the local source still set, which was then moved to the external ssrc internally.

···

From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Emil Ivov
Sent: Sunday, October 23, 2016 17:59
To: Jitsi Developers <dev@jitsi.org>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

Awesome! Glad to know you've solved it!

On Sunday, 23 October 2016, Oliver Hausler <oliver@closeup.cc<mailto:oliver@closeup.cc>> wrote:
Emil,

You were right with the synchronization issue. SSRCs look fine now.

Oliver.

From: dev [mailto:dev-bounces@jitsi.org<javascript:_e(%7B%7D,'cvml','dev-bounces@jitsi.org');>] On Behalf Of Oliver Hausler
Sent: Saturday, October 22, 2016 18:34
To: Jitsi Developers <dev@jitsi.org<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

Emil,

I believe you are right with the synchronization. I was loading the conference outside the sync block. I think we can entirely remove loading and just patch what's required. Can't test right now but don't put any further work into this for now.

Oliver.

From: dev [mailto:dev-bounces@jitsi.org<javascript:_e(%7B%7D,'cvml','dev-bounces@jitsi.org');>] On Behalf Of Emil Ivov
Sent: Saturday, October 22, 2016 16:45
To: Jitsi Developers <dev@jitsi.org<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc<javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>> wrote:
Emil,

so I assume your use of trickle simply introduces a delay that hides the actual problem

Are you saying I must synchronize patching the conference? If yes, is this per conference or in general? I assume I can make 2 parallel REST calls if a distinct conference is involved, correct?
No, I was not implying a race condition on the bridge but within your client,
What I understood from your earlier e-mails is that:

1. When your clients trcikle your candidates, then send SSRCs, everything looks good.
2. When they don't trickle then the SSRCs look wrong.

I have no idea how your client works so all of this is just speculation but my suggestion was to make sure that you don't have a race condition on your side.

As far as the bridge is concerned it only needs to get the right SSRCs and at least one (whatever IP address).

I am afraid I won't be able to help debugging your code.

Emil

Sounds like a race condition with your SSRC generation.

I was in fact making the REST call for adding ANSWER SDP in 2 different threads simultaneously. But even synchronizing these does not help. I am still seeing inconsistent results for SSRCs. And I even added a random time delay within the lock to make sure Jitsi has enough time for processing. When I do trickle, I get consistent results all the time (otherwise same code).

private static ReentrantLock addSdpLock = new ReentrantLock();

public void addSdp(@Nonnull String socketId, @Nonnull Sdp sdp) {
    addSdpLock.lock();
    try {
        log.info("TEST ENTER");

         response = JitsiApiBuilder.api().patchConference(id, conference).execute();

        Thread.sleep() // random delay 1000..2000 msec

        log.info("TEST EXIT");
    } finally {
        addSdpLock.unlock();
    }
}

2016-10-22 16:18:07,422 INFO Conference [nioEventLoopGroup-3-7] TEST ENTER
2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-7] TEST EXIT
2016-10-22 16:18:08,947 INFO Conference [nioEventLoopGroup-3-8] TEST ENTER
2016-10-22 16:18:10,325 INFO Conference [nioEventLoopGroup-3-8] TEST EXIT

From: Emil Ivov
Sent: Saturday, October 22, 2016 14:34
To: Oliver Hausler
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc<mailto:oliver@closeup.cc>> wrote:
Emil,

I am not using ICE trickle server -> client. Do you mean I can also switch it off in the other direction?

Yes. I think we only expect to see on address from the client in order to prevent a bug but it doesn't need to be a working address.

In other words: you don't meed to signal any actual addresses from client to server. Just send your offer as soon as you have it.

The SSRCs in the final colibri look better when trickle client -> server is on. I don't know if this is a bug in Jitsi, but when I don't trickle from client -> server, sometime/always (not sure) an SSRC is missing for one of the channels. If I trickle, it always looks fine.

Sounds like a race condition with your SSRC generation. SSRCs are in no way related to trickle ICE or addresses so I assume your use of trickle simply introduces a delay that hides the actual problem

Emil

Oliver.

From: Emil Ivov
Sent: Saturday, October 22, 2016 14:25
To: Jitsi Developers <dev@jitsi.org<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>;
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

Oliver,

From what you have said it doesn't sound like you need to trickle any candidates. As long as your videobridge is on a public network it will discover all client candidates as peer reflexive (this is still trickle ICE but with now actual client-side trickling).

This is what we do on all Jitsi Meet deployments including meet.jit.si<http://meet.jit.si>

Emil

On Saturday, 22 October 2016, Oliver Hausler <oliver@closeup.cc<javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>> wrote:
Emil, Philipp, sorry for the late response.

@Philipp: It doesn't work for me with passive. Can you see any issues with my logs below? Might be entirely unrelated as I see DTLS warnings in the Jitsi log. I posted this separately: "Unknown DTLS handshake message and no streams from Jitsi (REST)"

@Emil: I assume we should be using ICE trickle as this is standard in WebRTC, so I tried to do that:

- I am creating an OFFER SDP which contains all Candidates discovered by Jitsi (no ICE trickle, makes no sense here as we have candidates in colibri already).

In Chrome I have 2 options:

1. ICE trickle: I listen to createAnswerOnSuccess(), send the ANSWER SDP to Jitsi, wait for onIceCandidate() and send each Candidate as it is discovered.

2. No ICE trickle: I listen to iceGatheringStateChange() == complete, then send the full ANSWER SDP with all Candidates.

I thought 1. would be the preferred way when using WebRTC/libjingle, so I followed what Philipp Seeser suggested and it looks like Jitsi accepts the "remote-candidate"(s) as it replays it:

"channel-bundles": [{
                                "id": "AKQQeYSlhEwltx69_ZWinO8|||||N-N77FcLReCCe5yoieF_7A",
                                "transport": {
                                                "xmlns": "urn:xmpp:jingle:transports:ice-udp:1",
                                                "rtcp-mux": true,
                                                "remote-candidate": {
                                                                "component": 1,
                                                                "protocol": "udp",
                                                                "priority": 1685987071,
                                                                "ip": "68.96.117.185",
                                                                "port": 56117,
                                                                "type": "srflx",
                                                                "generation": 0,
                                                                "foundation": "1855365925",
                                                                "network": 1
                                                },
                                                "setup": "passive"
                                }
                }]

I assume I must add each Candidate like that, and Jitsi picks the one which it will use. Because the last 2 candidates I am receiving use throw-away port 9 [why is Chrome generating these at all?], if Jitsi would use the last Candidate, it wouldn't work, but I believe it's smart enough and uses the best candidate, even though this isn't reflected in colibri (except for the immediate response, there is no change in colibri thereafter, so I cannot prove this).

Finally, I see a slightly distinct result between 1 and 2, though, when looking at "GET /colibri/conferences/{conferenceId}":

For 1. I see 2 audio channels per endpoint:

  "contents": [
    {
      "channels": [
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||AKPDvx2Egk8dqVFUHTgkE_A",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||AKPDvx2Egk8dqVFUHTgkE_A",
          "sources": [
3204522869
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            3204522869,
336257926
          ],
          "id": "61baaec9d48429b4",
          "direction": "sendrecv"
        },
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||APkpD2Im3kfzq2hM1o49z3E",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||APkpD2Im3kfzq2hM1o49z3E",
          "sources": [
1722705425
          ],
         "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            1722705425,
3673416731
          ],
          "id": "b41ab150c9739171",
          "direction": "sendrecv"
        }
      ],
     "name": "audio"
    },

For 2. one of the ssrcs seems to be missing for one of the channels (so Philipps approach with ICE trickle seems to be the better working):

  "contents": [
    {
      "channels": [
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||MEIiSaRqTGep7C2cDR42sQ",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||MEIiSaRqTGep7C2cDR42sQ",
          "sources": [
916465418
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
3268560502
          ],
          "id": "700d2d43eae73a28",
          "direction": "sendrecv"
        },
        {
          "endpoint": "AKQQeYSlhEwltx69_ZWinO8|||||ANZL23OQrkVkqzUZ-XMyyPk",
          "channel-bundle-id": "AKQQeYSlhEwltx69_ZWinO8|||||ANZL23OQrkVkqzUZ-XMyyPk",
          "sources": [
814905801
          ],
          "rtp-level-relay-type": "mixer",
          "expire": 300,
          "initiator": true,
          "ssrcs": [
            814905801,
2637946124
          ],
          "id": "ba07908e22e3b2a2",
          "direction": "sendrecv"
        }
      ],
      "name": "audio"
    },

From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of Emil Ivov
Sent: Sunday, October 16, 2016 22:30
To: Jitsi Developers <dev@jitsi.org<mailto:dev@jitsi.org>>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

I am sorry I am chiming in so late but, why exactly do you need to send any additional candidates to the bridge in the first place?

On Sunday, 16 October 2016, Oliver Hausler <oliver@closeup.cc<mailto:oliver@closeup.cc>> wrote:
Thank you Philipp, I just noticed you sent a response to this.

Interesting, you patch each candidate into 'remote-candidate' separately. I will try this tomorrow.

From: dev [mailto:dev-bounces@jitsi.org<javascript:_e(%7B%7D,'cvml','dev-bounces@jitsi.org');>] On Behalf Of Philipp Seeser
Sent: Monday, October 10, 2016 03:02
To: Jitsi Developers <dev@jitsi.org<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>>
Subject: Re: [jitsi-dev] How to add remote-candidates to colibri with trickle ICE using REST?

I am able to add candidates with trickle ice and REST.

e.g. candidate json body:

var candidatePatch = {
          'foundation': parsedCandidate.foundation,
          'component': parsedCandidate.component,
          'priority': parsedCandidate.priority,
          'ip': parsedCandidate.ip,
          'port': parsedCandidate.port,
          'generation': parsedCandidate.generation,
          'type': parsedCandidate.type,
          'rel-addr': parsedCandidate.raddr,
          'rel-port': parsedCandidate.rport,
          'protocol': parsedCandidate.protocol,
          'network' : parsedCandidate.network,
          'id' : parsedCandidate.id
        };
        var patch: any = {
          id: msg.streams.publisher,
          'channel-bundles': [{
            'id': msg.streams.channelBundleId,
            transport: {
              'xmlns': 'urn:xmpp:jingle:transports:ice-udp:1',
              'setup': 'passive',
              'remote-candidate': candidatePatch
            }
          }]
        };

Also look at https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/xmpp/JingleSessionPC.js#L330
and I recommend to replace the answerer offer with "passive"

          // XXX Videobridge is the (SDP) offerer and WebRTC (e.g. Chrome) is the
          // answerer (as orchestrated by Jicofo). In accord with
          // http://tools.ietf.org/html/rfc5245#section-5.2 and because both peers
          // are ICE FULL agents, Videobridge will take on the controlling role and
          // WebRTC will take on the controlled role. In accord with
          // https://tools.ietf.org/html/rfc5763#section-5, Videobridge will use the
          // setup attribute value of setup:actpass and WebRTC will be allowed to
          // choose either the setup attribute value of setup:active or
          // setup:passive. Chrome will by default choose setup:active because it is
          // RECOMMENDED by the respective RFC since setup:passive adds additional
          // latency. The case of setup:active allows WebRTC to send a DTLS
          // ClientHello as soon as an ICE connectivity check of its succeeds.
          // Unfortunately, Videobridge will be unable to respond immediately because
          // may not have WebRTC's answer or may have not completed the ICE
          // connectivity establishment. Even more unfortunate is that in the
          // described scenario Chrome's DTLS implementation will insist on
          // retransmitting its ClientHello after a second (the time is in accord
          // with the respective RFC) and will thus cause the whole connection
          // establishment to exceed at least 1 second. To work around Chrome's
          // idiosyncracy, don't allow it to send a ClientHello i.e. change its
          // default choice of setup:active to setup:passive.

2016-10-09 18:19 GMT+02:00 Oliver Hausler <oliver@closeup.cc<javascript:_e(%7B%7D,'cvml','oliver@closeup.cc');>>:
// Once again because the ICE Agent does not support adding
// candidates after the connectivity establishment has been started
// and because multiple transport-info JingleIQs may be used to send
// the whole set of transport candidates from the remote peer to the
// local peer, do not really start the connectivity establishment
// until we have at least one remote candidate per ICE Component.

This comment it ICE Agent makes me believe I am right. And Jitsi is sending streams when I send a complete SDP with all candidates.

Can anybody comment what's the status here, please.

_______________________________________________
dev mailing list
dev@jitsi.org<javascript:_e(%7B%7D,'cvml','dev@jitsi.org');>
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile

--
sent from my mobile