[jitsi-dev] Strategy for connecting/disconnecting current device


#1

Hi guys,

Imagine the situation when you're using a media device like a webcam with microphone in jitsi-meet and you unplug your device during the conference.
For example, you are using webcam for video, webcam microphone for audio input and headset for audio output. You don't have any other webcams, however you have additional audio input device in form of headset's microphone and also one more output device in form of speakers. For some reason webcam gets unplugged, so you loose your current both microphone and webcam.
What should happen in this case?
Should video and audio tracks just stop and show some error dialog?
Or should the conference try to switch to another (probably, default) devices if available?
What should happen after you plug your webcam back?
Should conference switch back to your webcam and microphone?

Regards,
Kostia


#2

Hi Kostya,

Hi guys,

Imagine the situation when you're using a media device like a webcam with microphone in jitsi-meet and you unplug your device during the conference.
For example, you are using webcam for video, webcam microphone for audio input and headset for audio output. You don't have any other webcams, however you have additional audio input device in form of headset's microphone and also one more output device in form of speakers. For some reason webcam gets unplugged, so you loose your current both microphone and webcam.
What should happen in this case?
Should video and audio tracks just stop and show some error dialog?
Or should the conference try to switch to another (probably, default) devices if available?
What should happen after you plug your webcam back?
Should conference switch back to your webcam and microphone?

IMO ideally it would be good to show a dialog indicating to the user that the webcam has been unplugged and let him choose the new devices to use from drop-downs. We should definitely have the default choice there as well, so that the user can still just click OK. Does this make sense?

Regards,
Yana

···

On 11 May 2016, at 8:00 AM, Kostiantyn Tsaregradskyi <tsaregradskyi@gmail.com> wrote:

Regards,
Kostia
_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#3

Hi,

I tried today how conferences work in Hangouts and Facebook. FB is so buggy that I even won't mention it at all. So let's look on Hangouts.
Here is a number of cases I've tried with Hangouts in Chrome by calling to Hangouts on iOS:

1) two connected cameras
When I unplug active (first) camera, second camera is selected, but video becomes muted. Then we have two cases:
  a) If I unmute video, video is shown from second camera. If I plug first camera back, video is switched to first camera.
  b) If I don't unmute video and just plug first camera back, nothing happens. But when I'll unmute video, it shows video from the first camera.

2) one connected camera
When I unplug camera, video button disappears. When I plug it back, video (and corresponding button) appears automatically.

3) no camera initially connected
When I plug camera in during the call, video button appears and it's in muted state. When I unmute it, video from camera is shown.

4) two connected microphones (completely the same as for 1) except for bugs).
When I unplug active (first) mic, second mic is selected, but audio input becomes muted. Then we have two cases:
  a) If I unmute audio input, audio input is done from second mic. If I plug first mic back, audio input is switched to first mic (though there might no sound come to peer - seems like a bug)
  b) If I don't unmute audio input and just plug first mic back, nothing happens. But when I'll unmute audio input, it takes sound from the first mic (though there might no sound come to peer - seems like a bug)

5) one connected mic (same as for 2))
When I unplug microphone, audio input button disappears and "microphone muted" sign is shown. When I plug mic back, audio input (and corresponding button) works automatically.

6) no microphone initially connected (different from 3))
When I plug microphone in during the call, audio button appears and sound input starts. Audio input is not muted!

7) several audio outputs connected
When I unplug active audio output device, default audio device is selected. Nothing is muted. When I plug first device back, nothing happens.

Yesterday we discussed Hangouts approach with Emil and are going to do their way. WDYT?

Also I was wondering what is the reason of implicitly adding 'Auto' devices (https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/RTC/RTCUtils.js#L309) even if we have no devices of requested type? E.g. right now if you even don't have a camera, you still get one 'videoinput' device. The same for 'audioinput', and I've already did it the same for 'audiooutput' just to make it consistent. Maybe there was some historical reasons for that? I suggest not to add those 'Auto' devices if we have no real devices. In this case we can just skip some of device dropdowns in settings menu if we have no devices. WDYT?
Also should we probable hide microphone and camera buttons in toolbar when those devices are not present?

Thanks,
Kostia

···

11 мая 2016, в 19:07, Yana Stamcheva <yana@jitsi.org> написал(а):

Hi Kostya,

On 11 May 2016, at 8:00 AM, Kostiantyn Tsaregradskyi <tsaregradskyi@gmail.com> wrote:

Hi guys,

Imagine the situation when you're using a media device like a webcam with microphone in jitsi-meet and you unplug your device during the conference.
For example, you are using webcam for video, webcam microphone for audio input and headset for audio output. You don't have any other webcams, however you have additional audio input device in form of headset's microphone and also one more output device in form of speakers. For some reason webcam gets unplugged, so you loose your current both microphone and webcam.
What should happen in this case?
Should video and audio tracks just stop and show some error dialog?
Or should the conference try to switch to another (probably, default) devices if available?
What should happen after you plug your webcam back?
Should conference switch back to your webcam and microphone?

IMO ideally it would be good to show a dialog indicating to the user that the webcam has been unplugged and let him choose the new devices to use from drop-downs. We should definitely have the default choice there as well, so that the user can still just click OK. Does this make sense?

Regards,
Yana

Regards,
Kostia
_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#4

Hi Kostya,

Hi,

I tried today how conferences work in Hangouts and Facebook. FB is so buggy that I even won't mention it at all. So let's look on Hangouts.
Here is a number of cases I've tried with Hangouts in Chrome by calling to Hangouts on iOS:

1) two connected cameras
When I unplug active (first) camera, second camera is selected, but video becomes muted. Then we have two cases:
  a) If I unmute video, video is shown from second camera. If I plug first camera back, video is switched to first camera.
  b) If I don't unmute video and just plug first camera back, nothing happens. But when I'll unmute video, it shows video from the first camera.

This sounds confusing, but I guess the whole use case is kind of complex and confusing, so it as well maybe the easiest way to implement it.

2) one connected camera
When I unplug camera, video button disappears. When I plug it back, video (and corresponding button) appears automatically.

Ideally I would suggest to keep the video button with a muted icon and disable it with a tooltip message explaining that we don’t currently have any webcam connected. I find it more user friendly.

3) no camera initially connected
When I plug camera in during the call, video button appears and it's in muted state. When I unmute it, video from camera is shown.

That sounds good, except that as above I would suggest we enable the button at this point.

4) two connected microphones (completely the same as for 1) except for bugs).
When I unplug active (first) mic, second mic is selected, but audio input becomes muted. Then we have two cases:
  a) If I unmute audio input, audio input is done from second mic. If I plug first mic back, audio input is switched to first mic (though there might no sound come to peer - seems like a bug)
  b) If I don't unmute audio input and just plug first mic back, nothing happens. But when I'll unmute audio input, it takes sound from the first mic (though there might no sound come to peer - seems like a bug)

5) one connected mic (same as for 2))
When I unplug microphone, audio input button disappears and "microphone muted" sign is shown. When I plug mic back, audio input (and corresponding button) works automatically.

I would suggest we do it the same way as for camera, i.e. disable the button and show muted icon with the appropriate tooltip explanation.

6) no microphone initially connected (different from 3))
When I plug microphone in during the call, audio button appears and sound input starts. Audio input is not muted!

It’s strange audio starts unmuted and not the video, but then I guess that seeing something inappropriate by mistake is worse than hearing something.

7) several audio outputs connected
When I unplug active audio output device, default audio device is selected. Nothing is muted. When I plug first device back, nothing happens.

Yesterday we discussed Hangouts approach with Emil and are going to do their way. WDYT?

In general it sounds good. A little bit messy and inconsistent at points, but we can adjust thing along the way.

Also I was wondering what is the reason of implicitly adding 'Auto' devices (https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/RTC/RTCUtils.js#L309) even if we have no devices of requested type? E.g. right now if you even don't have a camera, you still get one 'videoinput' device. The same for 'audioinput', and I've already did it the same for 'audiooutput' just to make it consistent. Maybe there was some historical reasons for that? I suggest not to add those 'Auto' devices if we have no real devices. In this case we can just skip some of device dropdowns in settings menu if we have no devices. WDYT?

After discussing this a little bit with Pawel Domas off-list we figured out that it was implemented this way just because it was simpler to not support “None” device option. I don’t think it’s a good idea to skip the whole dropdown when we don’t have any devices. The user has no way to know the difference between having no devices and not having this setting option. So what would be really nice is to handle “None”.

I can imagine a use case where the user would like to disable camera permanently by selecting None device and this can be also useful for us while testing. On the other hand this complicates the interface a little bit and together with Auto and Default is maybe too much. So I would also agree if we add “None” as an indication that there’re no devices of this type and not add it to the dropdown in general if you see what I mean.

Also should we probable hide microphone and camera buttons in toolbar when those devices are not present?

See my replies above :slight_smile:

Thanks,
Yana

···

On 12 May 2016, at 6:51 AM, Kostiantyn Tsaregradskyi <tsaregradskyi@gmail.com> wrote:

Thanks,
Kostia

11 мая 2016, в 19:07, Yana Stamcheva <yana@jitsi.org> написал(а):

Hi Kostya,

On 11 May 2016, at 8:00 AM, Kostiantyn Tsaregradskyi <tsaregradskyi@gmail.com> wrote:

Hi guys,

Imagine the situation when you're using a media device like a webcam with microphone in jitsi-meet and you unplug your device during the conference.
For example, you are using webcam for video, webcam microphone for audio input and headset for audio output. You don't have any other webcams, however you have additional audio input device in form of headset's microphone and also one more output device in form of speakers. For some reason webcam gets unplugged, so you loose your current both microphone and webcam.
What should happen in this case?
Should video and audio tracks just stop and show some error dialog?
Or should the conference try to switch to another (probably, default) devices if available?
What should happen after you plug your webcam back?
Should conference switch back to your webcam and microphone?

IMO ideally it would be good to show a dialog indicating to the user that the webcam has been unplugged and let him choose the new devices to use from drop-downs. We should definitely have the default choice there as well, so that the user can still just click OK. Does this make sense?

Regards,
Yana

Regards,
Kostia
_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#5

Hi,

Well, it's seems we can't get rid from 'Auto' devices for now, as it's not currently possible to determine which of you devices is default, e.g. if you have two webcams, it's impossible to say which is default one (when you use empty string for deviceId).
As for having 'None' for devices - it doesn't make much sense for me, as you can always mute your microphone or webcam and achieve the result.

So I suggest to do the following:
- on the library side we remove automatically adding of 'Auto' devices to the list of devices returned by enumrateDevices()
- on the jitsi-meet side we add 'Auto' option to dropdowns in Settings, and if there are no devices for particular category - we just disable the <select> with "None" caption.

WDYT?

Also I found an interesting case, which I want to discuss.
Imagine that you have no microphones and no cameras (or you did not give permission for browser to use them). But you have some different output devices and you still can connect to the conference and you want to change your audio output device (we can still get IDs of audio output devices, but not the labels). But in this case you cannot change audio output due to SecurityError: No permission to use requested device.
This seems to be something wrong with spec or browser: granting permissions for microphone gives you permission to change audio output. It doesn't seem very logical. Why should I ask for microphone or webcam permissions if I just want to re-route audio to another device.
In any case, how should we handle the situation above? The easiest way is just not allow to change audio output device if we don't have access to mic or webcam.

Thanks,
Kostia

···

12 мая 2016, в 22:31, Yana Stamcheva <yana@jitsi.org> написал(а):

Hi Kostya,

On 12 May 2016, at 6:51 AM, Kostiantyn Tsaregradskyi <tsaregradskyi@gmail.com <mailto:tsaregradskyi@gmail.com>> wrote:

Hi,

I tried today how conferences work in Hangouts and Facebook. FB is so buggy that I even won't mention it at all. So let's look on Hangouts.
Here is a number of cases I've tried with Hangouts in Chrome by calling to Hangouts on iOS:

1) two connected cameras
When I unplug active (first) camera, second camera is selected, but video becomes muted. Then we have two cases:
  a) If I unmute video, video is shown from second camera. If I plug first camera back, video is switched to first camera.
  b) If I don't unmute video and just plug first camera back, nothing happens. But when I'll unmute video, it shows video from the first camera.

This sounds confusing, but I guess the whole use case is kind of complex and confusing, so it as well maybe the easiest way to implement it.

2) one connected camera
When I unplug camera, video button disappears. When I plug it back, video (and corresponding button) appears automatically.

Ideally I would suggest to keep the video button with a muted icon and disable it with a tooltip message explaining that we don’t currently have any webcam connected. I find it more user friendly.

3) no camera initially connected
When I plug camera in during the call, video button appears and it's in muted state. When I unmute it, video from camera is shown.

That sounds good, except that as above I would suggest we enable the button at this point.

4) two connected microphones (completely the same as for 1) except for bugs).
When I unplug active (first) mic, second mic is selected, but audio input becomes muted. Then we have two cases:
  a) If I unmute audio input, audio input is done from second mic. If I plug first mic back, audio input is switched to first mic (though there might no sound come to peer - seems like a bug)
  b) If I don't unmute audio input and just plug first mic back, nothing happens. But when I'll unmute audio input, it takes sound from the first mic (though there might no sound come to peer - seems like a bug)

5) one connected mic (same as for 2))
When I unplug microphone, audio input button disappears and "microphone muted" sign is shown. When I plug mic back, audio input (and corresponding button) works automatically.

I would suggest we do it the same way as for camera, i.e. disable the button and show muted icon with the appropriate tooltip explanation.

6) no microphone initially connected (different from 3))
When I plug microphone in during the call, audio button appears and sound input starts. Audio input is not muted!

It’s strange audio starts unmuted and not the video, but then I guess that seeing something inappropriate by mistake is worse than hearing something.

7) several audio outputs connected
When I unplug active audio output device, default audio device is selected. Nothing is muted. When I plug first device back, nothing happens.

Yesterday we discussed Hangouts approach with Emil and are going to do their way. WDYT?

In general it sounds good. A little bit messy and inconsistent at points, but we can adjust thing along the way.

Also I was wondering what is the reason of implicitly adding 'Auto' devices (https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/RTC/RTCUtils.js#L309) even if we have no devices of requested type? E.g. right now if you even don't have a camera, you still get one 'videoinput' device. The same for 'audioinput', and I've already did it the same for 'audiooutput' just to make it consistent. Maybe there was some historical reasons for that? I suggest not to add those 'Auto' devices if we have no real devices. In this case we can just skip some of device dropdowns in settings menu if we have no devices. WDYT?

After discussing this a little bit with Pawel Domas off-list we figured out that it was implemented this way just because it was simpler to not support “None” device option. I don’t think it’s a good idea to skip the whole dropdown when we don’t have any devices. The user has no way to know the difference between having no devices and not having this setting option. So what would be really nice is to handle “None”.

I can imagine a use case where the user would like to disable camera permanently by selecting None device and this can be also useful for us while testing. On the other hand this complicates the interface a little bit and together with Auto and Default is maybe too much. So I would also agree if we add “None” as an indication that there’re no devices of this type and not add it to the dropdown in general if you see what I mean.

Also should we probable hide microphone and camera buttons in toolbar when those devices are not present?

See my replies above :slight_smile:

Thanks,
Yana

Thanks,
Kostia

11 мая 2016, в 19:07, Yana Stamcheva <yana@jitsi.org> написал(а):

Hi Kostya,

On 11 May 2016, at 8:00 AM, Kostiantyn Tsaregradskyi <tsaregradskyi@gmail.com> wrote:

Hi guys,

Imagine the situation when you're using a media device like a webcam with microphone in jitsi-meet and you unplug your device during the conference.
For example, you are using webcam for video, webcam microphone for audio input and headset for audio output. You don't have any other webcams, however you have additional audio input device in form of headset's microphone and also one more output device in form of speakers. For some reason webcam gets unplugged, so you loose your current both microphone and webcam.
What should happen in this case?
Should video and audio tracks just stop and show some error dialog?
Or should the conference try to switch to another (probably, default) devices if available?
What should happen after you plug your webcam back?
Should conference switch back to your webcam and microphone?

IMO ideally it would be good to show a dialog indicating to the user that the webcam has been unplugged and let him choose the new devices to use from drop-downs. We should definitely have the default choice there as well, so that the user can still just click OK. Does this make sense?

Regards,
Yana

Regards,
Kostia
_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org <mailto:dev@jitsi.org>
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org <mailto:dev@jitsi.org>
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev