Customizing Jitsi / Jibri Live Streaming Layout

BingeWave · July 25, 2022, 5:57pm

There have been a few topics on how to customize the UI/UIX for the Jibri living streaming of Jitsi. Namely these two topics here:

I don’t think neither give a clear how. To start, using Jitsi Low Level API and assuming you have Jibri setup right, you can start a live stream like this:

          room.startRecording({
                broadcastId: [some_id],
                mode: JitsiRecordingConstants.mode.STREAM,
                streamId: [an_rtmp_url_you_want_to_send_the_stream_to]
            }).then(function (res) {
                //If Successful, you get a stream id
                restreamingSessionID = res._sessionID;

            }).catch(function (err) {
                    //If it didn't work, you get an error
                    console.log(err)
            })

That will start a stream. Now the stream will show users where there is a main video and other video will be smaller on the site.

One very weird thing is in the Jitsi Config at /usr/share/jitsi-meet/interface_config.js, if I modify the SHOW_JITSI_WATERMARK: false, the watermark will no longer show. It is weird because I have a custom build app with no watermark, so the Watermaterk is being placed in the video by the Jitsi code and is not a direct X11 Screen Grab by Chrome. To there is a layout being created somewhere.

Looking through some of Jibri’s code, I see that its doing an X11Grab here: jibri/Commands.kt at master · jitsi/jibri · GitHub

fun getFfmpegCommandLinux(ffmpegExecutorParams: FfmpegExecutorParams, sink: Sink): List<String> {
    return listOf(
        "ffmpeg", "-y", "-v", "info",
        "-f", "x11grab",
        "-draw_mouse", "0",
        "-r", ffmpegExecutorParams.framerate.toString(),
        "-s", ffmpegExecutorParams.resolution,
        "-thread_queue_size", ffmpegExecutorParams.queueSize.toString(),
        "-i", ":0.0+0,0",
        "-f", ffmpegExecutorParams.audioSource,
        "-thread_queue_size", ffmpegExecutorParams.queueSize.toString(),
        "-i", ffmpegExecutorParams.audioDevice,
        "-acodec", "aac", "-strict", "-2", "-ar", "44100", "-b:a", "128k",
        "-af", "aresample=async=1",
        "-c:v", "libx264", "-preset", ffmpegExecutorParams.videoEncodePreset,
        *sink.options, "-pix_fmt", "yuv420p", "-r", ffmpegExecutorParams.framerate.toString(),
        "-crf", ffmpegExecutorParams.h264ConstantRateFactor.toString(),
        "-g", ffmpegExecutorParams.gopSize.toString(), "-tune", "zerolatency",
        "-f", sink.format, sink.path
    )
}

Where in either Jibri or Jitsi can I modify what is being grabbed and outputted to the user for live streams for a custom live streaming experience or layout?

emrah · July 25, 2022, 6:30pm

What do you mean by “I have a custom build app with no watermark”? Do you see a customized web page when you join a meeting through Chrome using Jitsi base address?

BingeWave · July 25, 2022, 8:03pm

@emrah See screenshot below, its a custom built video app that is using jitsi-low-level api to power the video conferencing. Notice there is no Jitsi Logo in any of the videos. But if I were to live stream the video, the Jitsi Logo would appear unless I set it to false in the config. And it disregards everything else on the screen and only outputs the videos.

This means that Jitsi is somehow creating its own layout and not using the one on-screen. So how could I live stream the below layout where its actually grabbing whats on screening and live streaming it?

emrah · July 26, 2022, 7:15am

Jibri uses Chrome to join the meeting, not the custom app unless the base addresses are the same.

BingeWave · July 26, 2022, 10:07am

@emrah but how is Chrome deciding what to grab on the screen? Chrome only captures the videos currently.

emrah · July 26, 2022, 10:40am

What do you see when you open the https://___JITSI_HOST___/roomname page using Chrome?

You can get the value of ___JITSI_HOST___ from your jibri.conf

github.com

jitsi-contrib/installers/blob/main/templates/jibri/etc/jitsi/jibri/jibri.conf#L44


      
          api {
            http {
              external-api-port = 2222
              internal-api-port = 3333
            }
          
          
  xmpp {
              environments = [{
                name = "my-environment"
                xmpp-server-hosts = ["___JITSI_HOST___"]
                xmpp-domain = "___JITSI_HOST___"
          
          
      control-muc {
                  domain = "internal.auth.___JITSI_HOST___"
                  room-name = "JibriBrewery"
                  nickname = "jibri-nickname"
                }
          
          
      control-login {
                  domain = "auth.___JITSI_HOST___"
                  username = "jibri"

BingeWave · July 26, 2022, 1:17pm

My Jitsi config is hard-coded like below. Should I be using the “JITSI_HOST” variable? @emrah

xmpp {
      // See example_xmpp_envs.conf for an example of what is expected here
       environments = [
              {
                name = "prod environment"
                xmpp-server-hosts = ["meet.example.com"]
                xmpp-domain = "meet.example.com"

                control-muc {
                    domain = "internal.auth.meet.example.com"
                    room-name = "JibriBrewery"
                    nickname = "jibri1"
                }

                control-login {
                    domain = "auth.meet.example.com"
                    username = "jibri"
                    password = "xxxxxx"
                }

                call-login {
                    domain = "recorder.meet.example.com"
                    username = "recorder"
                    password = "xxxxxxxx"
                }

                strip-from-room-domain = "conference."
                usage-timeout = 0
                trust-all-xmpp-certs = true
            }]
    }
  }

BingeWave · July 26, 2022, 1:27pm

@emrah Also let me give you a real example. This a video app created that has a TicTacToe game and a Broadcast button on the button:

Video App: BingeWave Venues

This is where the video is live streamed too:

Live Stream: BingeWave Venues

To test click the Join button to load the video and TicTacToe game and broadcast button at the bottom., Click the Start Broadcast at the bottom and in the other link it will start the stream. Its only grabbing the video and nothing else on screen.

BingeWave · July 26, 2022, 1:31pm

Also I made this youtube video to show as well: - YouTube

emrah · July 26, 2022, 1:39pm

Your application address is https://widgets.bingewave.com but your Jitsi server address is https://meet.bingewave.com.

Jibri connects to the meeting using the Jitsi server address (something like https://meet.bingewave.com/roomname?jwt=...) and publishes its Chrome view which is not the same with your app page.

BingeWave · July 27, 2022, 2:41pm

@emrah I think I understand what you are saying. My issue is the the app being called by Jibri is the one at meet.bingewave.com, but the the users are using is widgets.bingewave.com. What I probably need to do is modify the nginx external configurations the same (Bosh, Colibri, etc) but change how the application is being served.

For example:

server_names_hash_bucket_size 64;

types {
# nginx's default mime.types doesn't include a mapping for wasm
    application/wasm     wasm;
}
server {
    listen 80;
    listen [::]:80;
    server_name meet.example.com;

    location ^~ /.well-known/acme-challenge/ {
        default_type "text/plain";
        root         /usr/share/jitsi-meet;
    }
    location = /.well-known/acme-challenge/ {
        return 404;
    }
    location / {
        return 301 https://$host$request_uri;
    }
}
server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name meet.example.com;

    # Mozilla Guideline v5.4, nginx 1.17.7, OpenSSL 1.1.1d, intermediate configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
    ssl_prefer_server_ciphers off;

    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:10m;  # about 40000 sessions
    ssl_session_tickets off;

    add_header Strict-Transport-Security "max-age=63072000" always;

    ssl_certificate /etc/letsencrypt/live/meet.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/meet.example.com/privkey.pem;

    root /usr/share/jitsi-meet;

    # ssi on with javascript for multidomain variables in config.js
    ssi on;
    ssi_types application/x-javascript application/javascript;

    index index.html index.htm;
    error_page 404 /static/404.html;

    gzip on;
    gzip_types text/plain text/css application/javascript application/json image/x-icon application/octet-stream application/wasm;
    gzip_vary on;
    gzip_proxied no-cache no-store private expired auth;
    gzip_min_length 512;

    location = /config.js {
        alias /etc/jitsi/meet/meet.example.com-config.js;
    }

    location = /external_api.js {
        alias /usr/share/jitsi-meet/libs/external_api.min.js;
    }

    # ensure all static content can always be found first
    location ~ ^/(libs|css|static|images|fonts|lang|sounds|connection_optimization|.well-known)/(.*)$
    {
        add_header 'Access-Control-Allow-Origin' '*';
        alias /usr/share/jitsi-meet/$1/$2;

        # cache all versioned files
        if ($arg_v) {
            expires 1y;
        }
    }

    # BOSH
    location = /http-bind {
        proxy_pass       http://xxxxxxxxx:5280/http-bind;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $http_host;
        add_header 'Access-Control-Allow-Origin' '*';
        add_header 'Access-Control-Allow-Headers' 'Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken,Keep-Alive,X-Requested-With,If-Modified-Since';
    }

    # xmpp websockets
    location = /xmpp-websocket {
        proxy_pass http://xxxxxxxxx:5280/xmpp-websocket?prefix=$prefix&$args;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $http_host;
        tcp_nodelay on;
    }

    # colibri (JVB) websockets for jvb1
    location ~ ^/colibri-ws/default-id/(.*) {
	proxy_pass http://xxxxxxxxx:9090/colibri-ws/default-id/$1$is_args$args;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        tcp_nodelay on;
    }

    # load test minimal client, uncomment when used
    #location ~ ^/_load-test/([^/?&:'"]+)$ {
    #    rewrite ^/_load-test/(.*)$ /load-test/index.html break;
    #}
    #location ~ ^/_load-test/libs/(.*)$ {
    #    add_header 'Access-Control-Allow-Origin' '*';
    #    alias /usr/share/jitsi-meet/load-test/libs/$1;
    #}

    location ~ ^/([^/?&:'"]+)$ {
        ##This is root domain of the jitsi app, I need to change this to my app
    }

    location @root_path {
        ##This is also root domain of the jitsi app, I need to change this to my app
    }

    location ~ ^/([^/?&:'"]+)/config.js$
    {
        set $subdomain "$1.";
        set $subdir "$1/";

        alias /etc/jitsi/meet/meet.example.com-config.js;
    }

    # Anything that didn't match above, and isn't a real file, assume it's a room name and redirect to /
    location ~ ^/([^/?&:'"]+)/(.*)$ {
        set $subdomain "$1.";
        set $subdir "$1/";
        rewrite ^/([^/?&:'"]+)/(.*)$ /$2;
    }

    # BOSH for subdomains
    location ~ ^/([^/?&:'"]+)/http-bind {
        set $subdomain "$1.";
        set $subdir "$1/";
        set $prefix "$1";

        rewrite ^/(.*)$ /http-bind;
    }

    # websockets for subdomains
    location ~ ^/([^/?&:'"]+)/xmpp-websocket {
        set $subdomain "$1.";
        set $subdir "$1/";
        set $prefix "$1";

        rewrite ^/(.*)$ /xmpp-websocket;
    }
}