Error while building jitsi-meet with custom lib-jitsi-meet


#1

I’m trying to use a local build of lib-jitsi-meet in a build of jitsi-meet. I’m following the instructions as provided here: https://github.com/jitsi/jitsi-meet#working-with-the-library-sources-lib-jitsi-meet

Sadly, the ‘make’ build fails after updating the lib-jitsi-meet location.

From what I can tell, the npm install lib-jitsi-meet --force command removes at least one module (jQuery-Impromptu) while it’s still needed by the build.

I’m unsure if this is intentional, or how to fix this properly. Any help would be greatly appreciated.

This is what I’m doing exactly, on today’s master branches of both projects

First, I attempt to clean any leftovers from previous iterations:

$ rm -rf node_modules/
$ make clean

Next, I build jitsi-meet (I’m not sure if the ‘make’ is needed at this point)

$ npm install
$ make

So far, so good.

As things change later, I’m recording the number of node_modules here:

$ ls -l node_modules/ | wc -l
1036

jQuery-Impromptu is still in node_modules at this point.

Next, I edit package.json, and replace the lib-jitsi-meet entry with this line:

"lib-jitsi-meet": "file:///home/guus/github/lib-jitsi-meet"

As instructed by the documentation, I (re)install the dependency like so:

$ npm install lib-jitsi-meet --force

This appears to be successful. However, 10 entries appear to be removed from node_modules:

$ ls -l node_modules/ | wc -l
1026

jQuery-Impromptu is one of the entries that disappeared.

Now, ‘make’ fails:

$ make
./node_modules/.bin/webpack -p
Hash: ac13550d6dce8e01c2b197cbffd1d8eb54a21fa2
Version: webpack 3.9.1
Child
    Hash: ac13550d6dce8e01c2b1
    Time: 153884ms
                                           Asset       Size  Chunks                    Chunk Names
              dial_in_info_bundle.min.js.LICENSE    1.23 kB          [emitted]         
                               app.bundle.min.js     3.3 MB       0  [emitted]  [big]  app.bundle
                      dial_in_info_bundle.min.js     455 kB       2  [emitted]  [big]  dial_in_info_bundle
                              alwaysontop.min.js     286 kB       3  [emitted]  [big]  alwaysontop
                      do_external_connect.min.js    11.1 kB       4  [emitted]         do_external_connect
                         flacEncodeWorker.min.js    8.28 kB       5  [emitted]         flacEncodeWorker
                       app.bundle.min.js.LICENSE    9.32 kB          [emitted]         
    device_selection_popup_bundle.min.js.LICENSE    9.16 kB          [emitted]         
            device_selection_popup_bundle.min.js    3.07 MB       1  [emitted]  [big]  device_selection_popup_bundle
                      alwaysontop.min.js.LICENSE  529 bytes          [emitted]         
                              app.bundle.min.map    15.7 MB       0  [emitted]         app.bundle
           device_selection_popup_bundle.min.map    14.6 MB       1  [emitted]         device_selection_popup_bundle
                     dial_in_info_bundle.min.map    1.88 MB       2  [emitted]         dial_in_info_bundle
                             alwaysontop.min.map    1.32 MB       3  [emitted]         alwaysontop
                     do_external_connect.min.map      77 kB       4  [emitted]         do_external_connect
                        flacEncodeWorker.min.map      54 kB       5  [emitted]         flacEncodeWorker
     [453] ./react/features/local-recording/recording/flac/messageTypes.js 1.06 kB {0} {1} {5} [built]
     [472] ./react/features/base/config/getRoomName.js 761 bytes {0} {1} {4} [built]
     [696] ./app.js + 2 modules 4.49 kB {0} [built]
     [701] ./conference.js 98.9 kB {0} [built]
     [711] ./modules/keyboardshortcut/keyboardshortcut.js 7.44 kB {0} [built]
     [712] ./modules/remotecontrol/RemoteControl.js 4.83 kB {0} [built]
     [719] ./modules/UI/UI.js 30.6 kB {0} [built]
     [724] ./react/index.web.js 1.93 kB {0} [built]
     [725] ./react/features/settings/popup.js 444 bytes {1} [built]
     [726] ./react/features/settings/DeviceSelectionPopup.js 13.4 kB {1} [built]
     [727] ./react/features/always-on-top/index.js + 3 modules 15.3 kB {3} [built]
     [730] multi ./react/features/base/react/prop-types-polyfill.js ./react/features/invite/components/dial-in-info-page 40 bytes {2} [built]
     [731] ./react/features/invite/components/dial-in-info-page/index.js + 2 modules 4.02 kB {2} [built]
     [732] ./connection_optimization/do_external_connect.js 2.51 kB {4} [built]
     [733] ./react/features/local-recording/recording/flac/flacEncodeWorker.js 11.8 kB {5} [built]
        + 719 hidden modules
    
    ERROR in ./app.js
    Module not found: Error: Can't resolve 'jQuery-Impromptu' in '/home/guus/github/jitsi-meet-copy'
     @ ./app.js 11:0-26
Child
    Hash: 97cbffd1d8eb54a21fa2
    Time: 6992ms
                   Asset     Size  Chunks             Chunk Names
     external_api.min.js  26.6 kB       0  [emitted]  external_api
    external_api.min.map   126 kB       0  [emitted]  external_api
       [1] ./react/features/base/util/uri.js 15.5 kB {0} [built]
       [3] ./modules/API/external/index.js 171 bytes {0} [built]
       [4] ./modules/API/external/external_api.js 33.1 kB {0} [built]
       [7] ./modules/transport/index.js + 8 modules 24 kB {0} [built]
       [8] ./react/features/base/util/helpers.js 2.27 kB {0} [built]
       [9] ./react/features/base/util/httpUtils.js 758 bytes {0} [built]
      [11] ./modules/API/external/electronPopupsConfig.json 100 bytes {0} [built]
        + 5 hidden modules
Makefile:18: recipe for target 'compile' failed
make: *** [compile] Error 2

#2

Hum, not sure what is wrong. But you can try adding: "lib-jitsi-meet": "github: guus/lib-jitsi-meet#your_commit_hash_from_branch_goes_here" and proceed with normal make.

You can also do a watch on lib-jitsi-meet, which will be compiling non-minified version of lib-jitsi-meet on every change, make a link from the resulting file to the jitsi-meet/libs/lib-jitsi-meet.min.js and use webpack-dev-server and whenever you make a change in lib-jitsi-meet, wait for it to compile (with watch it is quicker) and just reload the https://localhost:8080. But to run watch in lib-jitsi-meet you need some small modifications in the package.json there.


#3

Wow, this has been, and continues to be, a frustrating effort (despite the excellent help that you’re giving me).

Part of my build failure was my own stupid fault: I was using a version of NPM that was to old.

I am tasked with applying a small change to a version of the SDK that’s is not the current version, but only a couple of months old (from May this year). As we only want to change a default (reduce the video resolution as used by the SDK), we’d like to prevent making other changes that will require us to retest the entire setup.

I checked out the corresponding tag (jitsi-meet_3088) for jitsi-meet and lib-jitsi-meet (having a uniform tag across all repos is really helpful here, thanks for that!), only to find that we can no longer build. Turns out that a dependency is simply no longer available (@saghul tackled that in https://github.com/jitsi/jitsi-meet/commit/3d6e18394e2bc3e2a95063847b076c20d007b698)

With all of this resolved/backported, I still can’t build jitsi-meet with a custom lib-jitsi-meet, using the --force option as described in the documentation (I’ve not written down what was going wrong this time, it was something different than before though).

Instead of the --force option, I used the npm link route, as you suggested. That finally got me a good build! Yeah!

I wanted to quickly rebuild the SDK and get that into my project, only to be stopped by yet another surprise:

> Task :sdk:bundleReleaseJsAndAssets 
Scanning folders for symlinks in /home/guus/github/jitsi-meet/node_modules (14ms)
Scanning folders for symlinks in /home/guus/github/jitsi-meet/node_modules (11ms)
Loading dependency graph, done.
warning: the transform cache was reset.

Expected path `/home/guus/github/lib-jitsi-meet/lib-jitsi-meet.min.js` to be relative to one of the project roots

If I read this correctly, then the Android build doesn’t like the symlink created by npm link (update: probably caused by this issue in the JavaScript bundler for React Native)

:cold_sweat:

I am the first to admit that I’m an not an front-end / node.js / Android expert - but I do think that given the experience that I have, things shouldn’t be this hard. I’m hoping that, based on my experience above, we can make some improvements:

  1. Figure out a way to make a build fail early on when an incompatible version of NPM is used
  2. Make it hard for people to use the wrong version of NPM in the first place (by facilitating/mandating something like NVM that pins NPM to a specific version).
  3. Mandate that all dependencies come from a stable package repository (like npmjs.com) and abolish linking to repositories that are out of our control, like things on Github. If we must use Github repositories, lets only use stuff in our own repository.

I considered adding a suggestion to add a daily/weekly CI builds for every stable releases that checks if the build remains reproducible over time. Although it would probably not be to hard to set that up, I’m not sure what the process should be after such a build fails.

I’m hoping that you all can add to this list. There are probably more, and better, ideas to improve the stability and reproducibility of our projects over time.

The above still leaves me with the issue of not being able to build the Android SDK, that has a single-line change, but at least it’d gotten me to this stage a lot faster!