[jitsi-dev] Jitsi Meet media info in MUC presence


#1

Hi,

When new participant joins the conference the focus sends "addsource"
notifications over jingle session to each of the participants. When a
client receives this notification it needs to know to whom this new
SSRC belongs or JavaScript UI code will fail to display peer video.
Currently we advertise ownership information in MUC presence, that is
every participant adds his audio/video SSRCs to presence. Then other
clients store a map of MUC JID -> SSRC and uses it to display video
for particular peers.

Sometimes it happens that "addsource" notification arrives before
presence SSRC mapping. In this case video will never be displayed for
that peer. I wanted to propose two solutions for this problem:

1. Add JID SSRC "owner" parameter to
source[xmlns="urn:xmpp:jingle:apps:rtp:ssma:0"] element advertised in
INVITE and "addsource". Maybe with some custom namespace.

2. Add a waiting loop that will wait for JID to SSRC mapping like done here[1].

Which one would be better ? Or maybe something else ?

Regards,
Pawel

[1]: https://github.com/jitsi/jitsi-meet/blob/master/app.js#L235


#2

hey Pawel,

> When new participant joins the conference the focus sends "addsource"

notifications over jingle session to each of the participants. When a
client receives this notification it needs to know to whom this new
SSRC belongs or JavaScript UI code will fail to display peer video.
Currently we advertise ownership information in MUC presence, that is
every participant adds his audio/video SSRCs to presence. Then other
clients store a map of MUC JID -> SSRC and uses it to display video
for particular peers.

Sometimes it happens that "addsource" notification arrives before
presence SSRC mapping. In this case video will never be displayed for
that peer. I wanted to propose two solutions for this problem:

That's a bug. Stanzas are supposed to be delivered in-order.
Are they sent in the right order?

1. Add JID SSRC "owner" parameter to
source[xmlns="urn:xmpp:jingle:apps:rtp:ssma:0"] element advertised in
INVITE and "addsource". Maybe with some custom namespace.

You'll have to have a very strong argument to convince the author of xep-0339 :slight_smile:

2. Add a waiting loop that will wait for JID to SSRC mapping like done here[1].

Which one would be better ? Or maybe something else ?

Using media stream ids instead of SSRCs. Those are instantly available at the JS layer and can be sent earlier.

IIRC the ssrcs originate from a createoffer/answer callback so there might be timing behaviour involved. See
https://github.com/jitsi/jitsi-meet/blob/master/app.js#L615

···

Regards,
Pawel

[1]: https://github.com/jitsi/jitsi-meet/blob/master/app.js#L235

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#3

Hi Philipp,

hey Pawel,

When new participant joins the conference the focus sends "addsource"

notifications over jingle session to each of the participants. When a
client receives this notification it needs to know to whom this new
SSRC belongs or JavaScript UI code will fail to display peer video.
Currently we advertise ownership information in MUC presence, that is
every participant adds his audio/video SSRCs to presence. Then other
clients store a map of MUC JID -> SSRC and uses it to display video
for particular peers.

Sometimes it happens that "addsource" notification arrives before
presence SSRC mapping. In this case video will never be displayed for
that peer. I wanted to propose two solutions for this problem:

That's a bug. Stanzas are supposed to be delivered in-order.
Are they sent in the right order?

So prosody should guarantee stanzas order ? MUC presence is advertised
by peer and "addsource" comes from the focus, so they might not be
synchronized/ordered. Also first MUC presence sent by peer does not
contain media info yet. It's added there a moment later after set
remote/local description.

1. Add JID SSRC "owner" parameter to
source[xmlns="urn:xmpp:jingle:apps:rtp:ssma:0"] element advertised in
INVITE and "addsource". Maybe with some custom namespace.

You'll have to have a very strong argument to convince the author of
xep-0339 :slight_smile:

It was not my intention :wink: I don't feel like this should end up in any XEP.

2. Add a waiting loop that will wait for JID to SSRC mapping like done
here[1].

Which one would be better ? Or maybe something else ?

Using media stream ids instead of SSRCs. Those are instantly available at
the JS layer and can be sent earlier.

IIRC the ssrcs originate from a createoffer/answer callback so there might
be timing behaviour involved. See
https://github.com/jitsi/jitsi-meet/blob/master/app.js#L615

Thanks for suggestion ! We'll think about it.

Regards,
Pawel

···

On Tue, Aug 12, 2014 at 10:43 AM, Philipp Hancke <fippo@goodadvice.pages.de> wrote:


#4

Hi Philipp,

hey Pawel,

When new participant joins the conference the focus sends "addsource"

notifications over jingle session to each of the participants. When a
client receives this notification it needs to know to whom this new
SSRC belongs or JavaScript UI code will fail to display peer video.
Currently we advertise ownership information in MUC presence, that is
every participant adds his audio/video SSRCs to presence. Then other
clients store a map of MUC JID -> SSRC and uses it to display video
for particular peers.

Sometimes it happens that "addsource" notification arrives before
presence SSRC mapping. In this case video will never be displayed for
that peer. I wanted to propose two solutions for this problem:

That's a bug. Stanzas are supposed to be delivered in-order.
Are they sent in the right order?

So prosody should guarantee stanzas order ? MUC presence is advertised

yes. Any proper xmpp server does.

by peer and "addsource" comes from the focus, so they might not be
synchronized/ordered. Also first MUC presence sent by peer does not
contain media info yet. It's added there a moment later after set
remote/local description.

Right. So what should happen here is the following:
1) client joins, presence does not contain ssrcs
2) focus invites client with a session-initiate
3) client creates an answer
4) client updates presence, contains ssrcs (should happen in setlocaldescription.jingle)
5) client sends session-accept to focus
6) focus sends addsource jingle action to other clients

The problem you describe can happen when steps #4 and #5 are not in the right order. It might work most of the time (yay timing issues) even then. Can you check when setLocalDescription.jingle is executed relative to the session-accept?

I haven't had any problems with https://github.com/stpeter/mod_muc_focus but that sends out the presence in a sync way on the server side after receiving the session-accept so it is somewhat easier.

···

Am 12.08.2014 11:09, schrieb Paweł Domas:

On Tue, Aug 12, 2014 at 10:43 AM, Philipp Hancke > <fippo@goodadvice.pages.de> wrote: