[jitsi-dev] Only one videostream with mixed audio


#1

Hi, all.

I saw here that there are some attempts of making jitsi-meet work as an ios
app, but I'm pretty sure if there will be more than 3 participants in video
chat, the iphone/ipad app will crash.

What if we have an ability to set the bridge from jitsi-meet (with
config.js or just for the current user) to send only the
dominant videostream with mixed audio for the participant that requested
that?

That gives us the following directions:
1. The possibility to deploy an ios/android videoconference app (even with
cordova with phonertc).
2. Compatibility with Firefox (as @fippo suggested here
https://github.com/jitsi/jitsi-meet/issues/45).
3. Compatibility with Chrome mobile for android.
4. We get jitsi-meet even on IE and Safari (with Temasys plugin).
5. Participant invited from sip can see and hear other partcipants. So we
can participate to the conference directly from jitsy.
6. Participant with low internet connection can use the conference is such
way (connection quality indicator that @hristoterezov made, gives the
ability to implicitly set that).

Regards,
Zalmoxisus


#2

Yes, that is what i am trying to get to. if anyone else has been doing this as well and has made progress i could sure use some pointers.

Working with a jitsi-client created vidoebridge from a mac, i got to the point of getting jingle sesion-initiate but ran into a wall trying to create an sdp for webrtc. I have now changed tack to try to connect into ofmeet which should send what i need to make a connection.

ONCE i can at least make one connection and i have the concept proven, i want to do just that, i.e. have ONE video connection at a time but be able to choose from which participant i want to get video. For this app, audio is not important at all.

Peter Mycue
pmycue@us.ibm.com
pmycue@gmail.com
704-626-9772

···

On Oct 20, 2014, at 1:09 PM, Michael Zalmoxisus <zalmoxisus@gmail.com> wrote:

Hi, all.

I saw here that there are some attempts of making jitsi-meet work as an ios app, but I'm pretty sure if there will be more than 3 participants in video chat, the iphone/ipad app will crash.

What if we have an ability to set the bridge from jitsi-meet (with config.js or just for the current user) to send only the dominant videostream with mixed audio for the participant that requested that?

That gives us the following directions:
1. The possibility to deploy an ios/android videoconference app (even with cordova with phonertc).
2. Compatibility with Firefox (as @fippo suggested here https://github.com/jitsi/jitsi-meet/issues/45).
3. Compatibility with Chrome mobile for android.
4. We get jitsi-meet even on IE and Safari (with Temasys plugin).
5. Participant invited from sip can see and hear other partcipants. So we can participate to the conference directly from jitsy.
6. Participant with low internet connection can use the conference is such way (connection quality indicator that @hristoterezov made, gives the ability to implicitly set that).

Regards,
Zalmoxisus
_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#3

Hey Michael,

Hi, all.

I saw here that there are some attempts of making jitsi-meet work as an
ios app, but I'm pretty sure if there will be more than 3 participants
in video chat, the iphone/ipad app will crash.

You are very right indeed and I think you have actually missed many of our recent updates. Most of what you suggest here is already supported.

What if we have an ability to set the bridge from jitsi-meet (with
config.js or just for the current user) to send only the
dominant videostream with mixed audio for the participant that requested
that?

This is what we call Last N. Just set it to 1 in config.js :

channelLastN: 1

That gives us the following directions:
1. The possibility to deploy an ios/android videoconference app (even
with cordova with phonertc).
2. Compatibility with Firefox (as @fippo suggested here
https://github.com/jitsi/jitsi-meet/issues/45).

That one's trickier.. Actual compatibility with with firefox would currently require that it only receives one audio stream. Enabling mixing on Jitsi Meet really ruins performance. I am talking about something of the sort of a 10-fold increase in required resources.

Note that firefox are currently working on multistream support so we may have it as a full-rights citizen before the end of the year. Fingers crossed.

3. Compatibility with Chrome mobile for android.

We already have that and you'd be surprised how well it works with even 5 or 6 video streams.

4. We get jitsi-meet even on IE and Safari (with Temasys plugin).

Mmmm ... last time we tried this we stumbled upon a number of other issues. Multi stream support was not one of them.

5. Participant invited from sip can see and hear other partcipants. So
we can participate to the conference directly from jitsy.

Indeed. However these are two different things. Jitsi does not have a problem with receiving multiple streams. It was the first client to support Jitsi Videobridge. The bridge was actually part of Jitsi before we extracted it as a separate application.

The reason Jitsi does not currently work with Meet is mostly because we are missing transport components for VP8.

As for SIP: that's why we have Jigasi. It does mixing for audio and that's OK because it does it on a separate platform that can be scaled differently.

As for video, we can indeed make it relay a single video stream, but very few SIP services today would know what to do with VP8.

6. Participant with low internet connection can use the conference is
such way (connection quality indicator that @hristoterezov made, gives
the ability to implicitly set that).

This is the reason we have recently committed the Adaptive Last N functionality that adjusts this dynamically. We are almost done with simulcast as well which improves that in an even more dramatic way!

Emil

···

On 20.10.14, 19:09, Michael Zalmoxisus wrote:

--
https://jitsi.org


#4

Emil, thanks a lot for such a descriptive answer!

I've just tested the channelLastN=1, it works good. Doesn't this solve the
issue with firefox as we have only one active stream?

channelLastN is globally for the room. What do you think about an ability
to set it only for the users that require it (for example in case when a
user uses the mobile app to access the meet)? Adaptive Last N is a good
solution, but in some cases it can depend not only on bandwidth,

···

On Mon, Oct 20, 2014 at 10:17 PM, Emil Ivov <emcho@jitsi.org> wrote:

Hey Michael,

On 20.10.14, 19:09, Michael Zalmoxisus wrote:

Hi, all.

I saw here that there are some attempts of making jitsi-meet work as an
ios app, but I'm pretty sure if there will be more than 3 participants
in video chat, the iphone/ipad app will crash.

You are very right indeed and I think you have actually missed many of our
recent updates. Most of what you suggest here is already supported.

What if we have an ability to set the bridge from jitsi-meet (with

config.js or just for the current user) to send only the
dominant videostream with mixed audio for the participant that requested
that?

This is what we call Last N. Just set it to 1 in config.js :

channelLastN: 1

That gives us the following directions:

1. The possibility to deploy an ios/android videoconference app (even
with cordova with phonertc).
2. Compatibility with Firefox (as @fippo suggested here
https://github.com/jitsi/jitsi-meet/issues/45).

That one's trickier.. Actual compatibility with with firefox would
currently require that it only receives one audio stream. Enabling mixing
on Jitsi Meet really ruins performance. I am talking about something of the
sort of a 10-fold increase in required resources.

Note that firefox are currently working on multistream support so we may
have it as a full-rights citizen before the end of the year. Fingers
crossed.

3. Compatibility with Chrome mobile for android.

We already have that and you'd be surprised how well it works with even 5
or 6 video streams.

4. We get jitsi-meet even on IE and Safari (with Temasys plugin).

Mmmm ... last time we tried this we stumbled upon a number of other
issues. Multi stream support was not one of them.

5. Participant invited from sip can see and hear other partcipants. So

we can participate to the conference directly from jitsy.

Indeed. However these are two different things. Jitsi does not have a
problem with receiving multiple streams. It was the first client to support
Jitsi Videobridge. The bridge was actually part of Jitsi before we
extracted it as a separate application.

The reason Jitsi does not currently work with Meet is mostly because we
are missing transport components for VP8.

As for SIP: that's why we have Jigasi. It does mixing for audio and that's
OK because it does it on a separate platform that can be scaled differently.

As for video, we can indeed make it relay a single video stream, but very
few SIP services today would know what to do with VP8.

6. Participant with low internet connection can use the conference is

such way (connection quality indicator that @hristoterezov made, gives
the ability to implicitly set that).

This is the reason we have recently committed the Adaptive Last N
functionality that adjusts this dynamically. We are almost done with
simulcast as well which improves that in an even more dramatic way!

Emil

--
https://jitsi.org

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#5

The last N feature is developed as a channel property so that's already
possible and used by the adaptivity feature.

Firefox also requires a single SSRC for audio though. Mixing would help but
it would kill scalability for the bridge. So we'd rather wait for them to
be done woth the multistream support.

--sent from my mobile

···

On 20 Oct 2014 10:01 PM, "Michael Zalmoxisus" <zalmoxisus@gmail.com> wrote:

Emil, thanks a lot for such a descriptive answer!

I've just tested the channelLastN=1, it works good. Doesn't this solve the
issue with firefox as we have only one active stream?

channelLastN is globally for the room. What do you think about an ability
to set it only for the users that require it (for example in case when a
user uses the mobile app to access the meet)? Adaptive Last N is a good
solution, but in some cases it can depend not only on bandwidth,

On Mon, Oct 20, 2014 at 10:17 PM, Emil Ivov <emcho@jitsi.org> wrote:

Hey Michael,

On 20.10.14, 19:09, Michael Zalmoxisus wrote:

Hi, all.

I saw here that there are some attempts of making jitsi-meet work as an
ios app, but I'm pretty sure if there will be more than 3 participants
in video chat, the iphone/ipad app will crash.

You are very right indeed and I think you have actually missed many of
our recent updates. Most of what you suggest here is already supported.

What if we have an ability to set the bridge from jitsi-meet (with

config.js or just for the current user) to send only the
dominant videostream with mixed audio for the participant that requested
that?

This is what we call Last N. Just set it to 1 in config.js :

channelLastN: 1

That gives us the following directions:

1. The possibility to deploy an ios/android videoconference app (even
with cordova with phonertc).
2. Compatibility with Firefox (as @fippo suggested here
https://github.com/jitsi/jitsi-meet/issues/45).

That one's trickier.. Actual compatibility with with firefox would
currently require that it only receives one audio stream. Enabling mixing
on Jitsi Meet really ruins performance. I am talking about something of the
sort of a 10-fold increase in required resources.

Note that firefox are currently working on multistream support so we may
have it as a full-rights citizen before the end of the year. Fingers
crossed.

3. Compatibility with Chrome mobile for android.

We already have that and you'd be surprised how well it works with even 5
or 6 video streams.

4. We get jitsi-meet even on IE and Safari (with Temasys plugin).

Mmmm ... last time we tried this we stumbled upon a number of other
issues. Multi stream support was not one of them.

5. Participant invited from sip can see and hear other partcipants. So

we can participate to the conference directly from jitsy.

Indeed. However these are two different things. Jitsi does not have a
problem with receiving multiple streams. It was the first client to support
Jitsi Videobridge. The bridge was actually part of Jitsi before we
extracted it as a separate application.

The reason Jitsi does not currently work with Meet is mostly because we
are missing transport components for VP8.

As for SIP: that's why we have Jigasi. It does mixing for audio and
that's OK because it does it on a separate platform that can be scaled
differently.

As for video, we can indeed make it relay a single video stream, but very
few SIP services today would know what to do with VP8.

6. Participant with low internet connection can use the conference is

such way (connection quality indicator that @hristoterezov made, gives
the ability to implicitly set that).

This is the reason we have recently committed the Adaptive Last N
functionality that adjusts this dynamically. We are almost done with
simulcast as well which improves that in an even more dramatic way!

Emil

--
https://jitsi.org

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev