Unable to use non-user facing camera

#1

Using the Jitsi SDK for Android version 1.16.0, I seem to be unable to obtain video on a device that has a camera, but not a camera where the facingMode is ‘user’.

The same issue occurs when I build the SDK from the current tip of master.

This is the log from my application, show that there is a camera (facingmode ‘back’), but failing to use that camera for some reason.

06-20 16:08:15.444 4308-4364/nl.goodbytes.demo I/ReactNativeJS: '[modules/RTC/RTCUtils.js]', '<n>: ', 'Get media constraints', { audio: false,
      video: 
       { mandatory: {},
         optional: [ { facingMode: 'user' } ],
         facingMode: 'user' } }
06-20 16:08:15.470 4308-4308/nl.goodbytes.demo I/com.oney.WebRTCModule.WebRTCModule: getUserMedia(video): { NativeMap: {"facingMode":"user","optional":[{"facingMode":"user"}],"mandatory":{}} }
06-20 16:08:15.472 4308-4308/nl.goodbytes.demo I/CameraManagerGlobal: Connecting to camera service
06-20 16:08:15.477 4308-4308/nl.goodbytes.demo D/com.oney.WebRTCModule.WebRTCModule: Creating video capturer using Camera2 API.
06-20 16:08:15.481 4308-4364/nl.goodbytes.demo I/ReactNativeJS: '[modules/RTC/RTCUtils.js]', '<n>: ', 'Get media constraints', { audio: true, video: false }
06-20 16:08:15.488 4308-4364/nl.goodbytes.demo I/ReactNativeJS: '[modules/xmpp/xmpp.js]', '<n>: ', 'P2P STUN servers: ', [ { urls: 'stun:stun.l.google.com:19302' },
      { urls: 'stun:stun1.l.google.com:19302' },
      { urls: 'stun:stun2.l.google.com:19302' } ]
06-20 16:08:15.499 4308-4308/nl.goodbytes.demo I/com.oney.WebRTCModule.WebRTCModule: getUserMedia(audio): mandatory: [], optional: [googNoiseSuppression: true, googEchoCancellation: true, echoCancellation: true, googEchoCancellation2: true, googDAEchoCancellation: true]
06-20 16:08:15.503 4308-4308/nl.goodbytes.demo D/com.oney.WebRTCModule.WebRTCModule: MediaStream id: d45df408-f615-43e8-896b-a972885ecddc
06-20 16:08:15.513 4308-4364/nl.goodbytes.demo I/ReactNativeJS: '[modules/xmpp/xmpp.js]', '<n>: ', '(TIME) Strophe connecting:\t', 1529507295512
06-20 16:08:15.598 4308-4364/nl.goodbytes.demo W/ReactNativeJS: '[__filename]', '<s>: ', 'Calendar access not granted.'
06-20 16:08:15.601 4308-4364/nl.goodbytes.demo I/ReactNativeJS: '[modules/RTC/RTCUtils.js]', '<n>: ', 'Available devices: ', [ { facing: 'back',
        label: 'Camera 0, Facing back, Orientation 0',
        kind: 'videoinput',
        deviceId: '0',
        groupId: null },
      { facing: null,
        label: 'Audio',
        kind: 'audioinput',
        deviceId: 'audio-1',
        groupId: null } ]
06-20 16:08:15.602 4308-4364/nl.goodbytes.demo W/ReactNativeJS: '[modules/statistics/AnalyticsAdapter.js]', '<n>: ', 'Not sending an event, disposed.'
    '[modules/statistics/AnalyticsAdapter.js]', '<n>: ', 'Not sending an event, disposed.'
06-20 16:08:15.605 4308-4364/nl.goodbytes.demo W/ReactNativeJS: '[__filename]', '<s>: ', 'Calendar access not granted.'
06-20 16:08:15.607 4308-4364/nl.goodbytes.demo W/ReactNativeJS: '[modules/RTC/RTCUtils.js]', '<n>: ', 'Failed to get access to local media. Error ', { name: null,
      message: 'Failed to create new track',
      constraintName: undefined }, { audio: false,
      video: 
       { mandatory: {},
         optional: [ { facingMode: 'user' } ],
         facingMode: 'user' } }
06-20 16:08:15.609 4308-4364/nl.goodbytes.demo E/ReactNativeJS: '[modules/RTC/RTCUtils.js]', '<n>: ', 'failed to obtain video stream - stop', [gum.general: Failed to create new track]
06-20 16:08:15.618 4308-4364/nl.goodbytes.demo W/ReactNativeJS: '[modules/statistics/AnalyticsAdapter.js]', '<n>: ', 'Not sending an event, disposed.'
06-20 16:08:15.619 4308-4365/nl.goodbytes.demo E/unknown:ReactNative: console.error: "[modules/RTC/RTCUtils.js]", "<n>: ", "failed to obtain video stream - stop", {"gum":{"error":{"name":null,"message":"Failed to create new track"},"constraints":{"audio":false,"video":{"mandatory":{},"optional":[{"facingMode":"user"}],"facingMode":"user"}},"devices":["video"]},"name":"gum.general","message":"Failed to create new track","stack":"r@index.android.bundle:719:85856\nindex.android.bundle:719:214612\nu@index.android.bundle:719:199789\nindex.android.bundle:719:200157\nindex.android.bundle:705:1290\nvalue@index.android.bundle:35:3467\nindex.android.bundle:35:1240\nvalue@index.android.bundle:35:2606\nvalue@index.android.bundle:35:1210\n[native code]"}, stack:
    <unknown>@332:2579
    n@719:302307
    <unknown>@719:218994
    <unknown>@719:214624
    u@719:199789
    <unknown>@719:200157
    <unknown>@705:1290
    value@35:3467
    <unknown>@35:1240
    value@35:2606
    value@35:1210
06-20 16:08:15.619 4308-4364/nl.goodbytes.demo E/ReactNativeJS: '[__filename]', '<s>: ', 'Failed to create local tracks', [ 'video' ], [gum.general: Failed to create new track]
06-20 16:08:15.624 4308-4365/nl.goodbytes.demo E/unknown:ReactNative: console.error: "[__filename]", "<s>: ", "Failed to create local tracks", ["video"], {"gum":{"error":{"name":null,"message":"Failed to create new track"},"constraints":{"audio":false,"video":{"mandatory":{},"optional":[{"facingMode":"user"}],"facingMode":"user"}},"devices":["video"]},"name":"gum.general","message":"Failed to create new track","stack":"r@index.android.bundle:719:85856\nindex.android.bundle:719:214612\nu@index.android.bundle:719:199789\nindex.android.bundle:719:200157\nindex.android.bundle:705:1290\nvalue@index.android.bundle:35:3467\nindex.android.bundle:35:1240\nvalue@index.android.bundle:35:2606\nvalue@index.android.bundle:35:1210\n[native code]"}, stack:
    <unknown>@332:2579
    s@676:833
    <unknown>@820:991
    <unknown>@66:629
    <unknown>@66:657
    <unknown>@40:1772
    b@40:628
    w@40:992
    callImmediates@40:3173
    value@35:2778
    <unknown>@35:1360
    value@35:2606
    value@35:1330
    value@35:1265
Meet SDK for Android fails after React Native upgrade to 0.55.4
#2

I’m surprised by the duplicate ‘facingMode’ definition in the constraint, but I’m having trouble understanding the meaning of these constraints in detail. Indentation is mine:

{  
  audio:false,
  video:{  
    mandatory:{  

    },
    optional:[  
      {  
        facingMode:'user'
      }
    ],
    facingMode:'user'
  }
}
#3

We’ve recently fixed a very similar issue here: https://github.com/jitsi/react-native-webrtc/pull/24

I have been using that solution successfully for a while, until I rebuild the SDK for Android binary artifact earlier today.

#4

From this log it looks like the WebRTC stack only detects a device facing ‘back’.

#5

That’s correct: the device has only one camera, and its facing mode is ‘back’. What is troubling me is that Meet does not use that camera at all.

#6

I see. Then I guess you’re right that we’re not treating it as optional and it’s weird that the constraint appears in both optional and ‘video’ scope (not sure if it’s related to “new” vs “old” constraints syntaxes which I recall being worked on some time ago). Do you know if we have a Github issue for that already ? If not could you please create one ? Thanks !

#7

I’ve modified lib-jitsi-meet to drop those constraints (it now reports Get media constraints', { audio: false, video: { mandatory: {}, optional: [] } }, but the camera is still not used with this.

#8

Yes, it looks like a bug

#9

Even when I modify lib-jitsi-meet, to make the default / preferred facingMode ‘back’ instead of ‘user’, the camera fails to be used.

#10

I think I might be onto something now: if I change the default facingMode to ‘environment’, the back camera is used (even though its facingMode is reported as ‘back’).

What exactly defines the facingMode of the camera? I’m particularly interested in finding out why my camera is reporting ‘back’ instead of ‘environment’. Is that reported by … some driver in the hardware?

The React WebRTC module uses this code
while determining what camera to use (see com.oney.WebRTCModule.GetUserMediaImpl#createVideoCapturer):

    boolean isFrontFacing;
    if (facingMode == null) {
        facingMode = "user";
        isFrontFacing = true;
    } else {
        isFrontFacing = !facingMode.equals("environment");
    }

As my facingMode is ‘back’, instead of ‘environment’, it’d still try to find a front facing camera (that’s not available). Maybe the prudent thing here is to check against ‘user’, instead of !environment (and reduce the amount of hard-coded identifiers by half).

I’m still unsure why this is happening now, while it was not a problem before. I’m starting to wonder if some kind of over-the-air update modified the facingMode as reported by my device.

#11

On second thought, that’s probably not the best idea. Of the four defined facing modes, “environment” is the only one that is defined as not ‘facing the user’.

#12

The “duplication” is necessary because there are different versions of the WebRTC standard and implementations which deal with these constraints. I don’t remember who supported and expected what right now. It’s probably going to be very useful to track the details down and put them as source code comments.

#14

Give current master a go. We (literraly) just updated to M67 and I shifted the code around for camera handling. The logic is now contained in a VideoCaptureController: https://github.com/jitsi/react-native-webrtc/blob/master/android/src/main/java/com/oney/WebRTCModule/VideoCaptureController.java

AFAICT you should be hitting the last resort case: no camera was matched so we pick the first one available.

#15

I’m upgrading my code now. I’ll post again when I’ve run some tests with it.

I was puzzled that that didn’t happen. I’ve tried to recompile the native react webrtc module (to add more debug logging), but I failed to hook that up into the SDK. Any pointers on that would be very welcome.

#16

Sadly, no joy. I’m adding the logs from the device.

Can you explain how I can de-obfuscate those stack traces? That might help figuring out where things go wrong.

I/cr_LibraryLoader: Expected native library version number "52.0.2743.100", actual native library version number "52.0.2743.100"
W/ReactNativeJS: '[JitsiMeetJS.js]', '<n>: ', 'Analytics disabled, disposing.'
W/ReactNativeJS: '[modules/statistics/AnalyticsAdapter.js]', '<n>: ', 'Disposing of analytics adapter.'
I/ReactNativeJS: '[modules/RTC/ScreenObtainer.js]', '<n>: ', 'Screen sharing not supported by the current browser: ', 'react-native'
I/ReactNativeJS: '[modules/RTC/ScreenObtainer.js]', '<n>: ', 'Desktop sharing disabled'
I/ReactNativeJS: '[__filename]', '<i>: ', 'Start muted: '
I/ReactNativeJS: '[__filename]', '<i>: ', 'Start audio only set to false'
I/ReactNativeJS: '[__filename]', '<i>: ', 'Audio-only disabled'
I/ReactNativeJS: '[modules/RTC/RTCUtils.js]', '<n>: ', 'Get media constraints', { audio: true, video: false }
I/ReactNativeJS: '[modules/RTC/RTCUtils.js]', '<n>: ', 'Get media constraints', { audio: false,
                   video: 
                    { mandatory: {},
                      optional: [ { facingMode: 'user' } ],
                      facingMode: 'user' } }
I/com.oney.WebRTCModule.WebRTCModule: getUserMedia(audio): mandatory: [], optional: [googNoiseSuppression: true, googEchoCancellation: true, echoCancellation: true, googEchoCancellation2: true, googDAEchoCancellation: true]
D/com.oney.WebRTCModule.WebRTCModule: MediaStream id: 48669a21-5c77-4b2d-836b-c8b30b2dd7f7
I/ReactNativeJS: '[modules/xmpp/xmpp.js]', '<n>: ', 'P2P STUN servers: ', [ { urls: 'stun:stun.l.google.com:19302' },
                   { urls: 'stun:stun1.l.google.com:19302' },
                   { urls: 'stun:stun2.l.google.com:19302' } ]
I/com.oney.WebRTCModule.WebRTCModule: getUserMedia(video): { NativeMap: {"facingMode":"user","optional":[{"facingMode":"user"}],"mandatory":{}} }
I/CameraManagerGlobal: Connecting to camera service
D/com.oney.WebRTCModule.WebRTCModule: Creating video capturer using Camera2 API.
I/ReactNativeJS: '[modules/xmpp/xmpp.js]', '<n>: ', '(TIME) Strophe connecting:\t', 1530007134529
W/ReactNativeJS: '[__filename]', '<i>: ', 'Calendar access not granted.'
I/ReactNativeJS: '[modules/RTC/RTCUtils.js]', '<n>: ', 'Available devices: ', [ { facing: 'back',
                     label: 'Camera 0, Facing back, Orientation 0',
                     kind: 'videoinput',
                     deviceId: '0',
                     groupId: null },
                   { facing: null,
                     label: 'Audio',
                     kind: 'audioinput',
                     deviceId: 'audio-1',
                     groupId: null } ]
W/ReactNativeJS: '[__filename]', '<i>: ', 'Calendar access not granted.'
W/ReactNativeJS: '[__filename]', '<i>: ', 'Calendar access not granted.'
I/ReactNativeJS: '[modules/RTC/RTCUtils.js]', '<n>: ', 'onUserMediaSuccess'
D/ReactNativeJS: '[modules/RTC/JitsiLocalTrack.js]', '<n>: ', 'Setting new MSID: 48669a21-5c77-4b2d-836b-c8b30b2dd7f7 b88c2bea-7aee-4a07-a9b6-feb8e787a845 on LocalTrack[undefined,audio]'
W/ReactNativeJS: '[modules/RTC/RTCUtils.js]', '<n>: ', 'Failed to get access to local media. Error ', { name: null,
                   message: 'Failed to create new track',
                   constraintName: undefined }, { audio: false,
                   video: 
                    { mandatory: {},
                      optional: [ { facingMode: 'user' } ],
                      facingMode: 'user' } }
E/ReactNativeJS: '[modules/RTC/RTCUtils.js]', '<n>: ', 'failed to obtain video stream - stop', [gum.general: Failed to create new track]
E/ReactNativeJS: '[__filename]', '<i>: ', 'Failed to create local tracks', [ 'video' ], [gum.general: Failed to create new track]
E/unknown:ReactNative: console.error: "[modules/RTC/RTCUtils.js]", "<n>: ", "failed to obtain video stream - stop", {"gum":{"error":{"name":null,"message":"Failed to create new track"},"constraints":{"audio":false,"video":{"mandatory":{},"optional":[{"facingMode":"user"}],"facingMode":"user"}},"devices":["video"]},"name":"gum.general","message":"Failed to create new track","stack":"r@index.android.bundle:728:86318\nindex.android.bundle:728:241191\nu@index.android.bundle:728:233068\nindex.android.bundle:728:233436\nindex.android.bundle:714:1292\nvalue@index.android.bundle:45:3872\nindex.android.bundle:45:1396\nvalue@index.android.bundle:45:3009\nvalue@index.android.bundle:45:1366\n[native code]"}, stack:
                       <unknown>@342:2586
                       n@728:218626
                       <unknown>@728:245573
                       <unknown>@728:241203
                       u@728:233068
                       <unknown>@728:233436
                       <unknown>@714:1292
                       value@45:3872
                       <unknown>@45:1396
                       value@45:3009
                       value@45:1366
E/unknown:ReactNative: console.error: "[__filename]", "<i>: ", "Failed to create local tracks", ["video"], {"gum":{"error":{"name":null,"message":"Failed to create new track"},"constraints":{"audio":false,"video":{"mandatory":{},"optional":[{"facingMode":"user"}],"facingMode":"user"}},"devices":["video"]},"name":"gum.general","message":"Failed to create new track","stack":"r@index.android.bundle:728:86318\nindex.android.bundle:728:241191\nu@index.android.bundle:728:233068\nindex.android.bundle:728:233436\nindex.android.bundle:714:1292\nvalue@index.android.bundle:45:3872\nindex.android.bundle:45:1396\nvalue@index.android.bundle:45:3009\nvalue@index.android.bundle:45:1366\n[native code]"}, stack:
                       <unknown>@342:2586
                       i@685:836
                       <unknown>@829:993
                       _@63:151
                       <unknown>@63:878
                       <unknown>@49:1787
                       x@49:643
                       k@49:1007
                       callImmediates@49:3188
                       value@45:3183
                       <unknown>@45:1516
                       value@45:3009
                       value@45:1486
                       value@45:1421