'Listen' to Jitsi video stream from external app

Developers
#1

Hi everyone,

I hope you won’t mind me posting this as my first post, but I’m sure with what I’m currently working on it won’t be my last.

I was wondering if anyone could help me out with understanding how to approach my problem.

I would like to connect the Jitsi video feed to an external barcode reader app for the purpose of identifying barcoded items in a physical room. i.e If I am talking to my gf on Jitsi, and she decides we need shampoo on our shopping list, I’d like her to be able to hold a (custom) barcoded item up on screen, and have a log of that barcode stored in a csv file later for retrieval as a list.

I have a fully working Jitsi using this guide: How To: Install Jitsi Server - YouTube

The Barcode app I would like to use is this one: GitHub - gautamkumarjaiswal/BarCodeReader: Bar code reader using webcam

I currently have it running on my local machine and it works fine. I would need it to not look for my internal webcam, but instead search for it video source from Jitsi. I am open to ideas for better ways to do this.

For now, I would like to know:

  • Can I connect JVB to the BCR app via websockets? If so, is this the best method?
  • Is it an issue that the app is Python based, or is it better if it is written in Java?
  • Would I just be editing the jvb.config file with my websocket setup, and then setting up a client websocket in my BCR app? Or are there more files I need to edit in JVB?

Once I know I am on the right path, I will learn the code and protocols to implement it, I just want to make sure my time is spent learning the right thing.

Thanks for taking time to read this. I am not used to posting on tech forums, so if it is useful for me to provide any more detail or code, please do let me know. I will get to it asap.

Thanks,

CRN.

#2

Does the barcode reader check the stream all the times or do you trigger it when it’s needed?
Checking all the times consumes a lot of CPU

#3

Hi emrah,

Thanks for the reply. Yes the scanner does check constantly. It could be an option to add a button inside Jitsi that launches it to save on cpu, but I would prefer it to just notices a barcode seamlessly if poss.

CRN.

#4

You can convert the meeting to the png frames using jibri and a daemon continously checks the png frames for a barcode.

#5

That sounds like a great option. I am part way installing Jibri now, so I will get back to that. Could you elaborate on the ‘daemon’ bit? I’m quite new to coding, as my technical background is in live events prior to covid. I’m still catching up on some terminology. Thank you!

#6

jibri will create the PNG frames in a folder and a daemon (a program which run as a service, as a background process) will continously check this folder for new frames and if there is any, search on it for a barcode

#7

Thank you emrah. I will get Jibri up and running first and then check back in. I appreciate the help.

#8

Hi emrah,

I now have Jibri configured on it’s own server following the guide on here including jibri.config. I have it running on 4 cores with 8gb ram and it records fine.

Would you be able to point me in the direction of some resources on how to configure Jibri? Especially for converting to png as you recommend.

Thank you!

CRN.

Edit:

I’ve done some digging around. I found a jibri.jar file which when I Googled told me it was a class library built on zip format, so I ran unzip jibri.jar and got a list of classes. Is this by any chance relating to the config file, in that I can invoke these classes in said file?

#9
  • Create a fake ffmpeg in your jibri server

/usr/local/bin/ffmpeg

#!/bin/bash

ARGS=$@

[[ "$(whoami)" != "jibri" ]] && exec /usr/bin/ffmpeg $ARGS

MP4=$(echo $ARGS | rev | awk '{print $1}' | rev)
DEST=$(dirname $MP4)

echo $ARGS >> /tmp/ffmpeg.log
exec /usr/bin/ffmpeg -y -v info -f x11grab -draw_mouse 0 -r 30 -s 1280x720 \
    -thread_queue_size 4096 -i :0.0+0,0 -an -vf fps=5 $DEST/frame-%06d.png
  • set the execute permission
chmod 755 /usr/local/bin/ffmpeg

  • Start recording on Jitsi

  • Check the recording folder on jibri server

Tip: how to customize ffmpeg without changing Jibri’s code

1 Like
#10

Hi emrah,

That works! :slight_smile:

Now I need to analyze the png images. My understanding is that Jibri uses Google Chrome as a sort of virtual screen to capture from.

I have found this: Webcam Barcode Scanner with HTML5 and Web Browser - CodeProject

The above appears to be an open source way to scan barcodes in any html5 browser, so I was wondering if you think I am on to something with this? Is it likely to work with the Chrome I installed with Jibri?

Best,

CRN.

#11

It seems that this is only for webcam

#12

Hi emrah,

Thank you for your help! I was able to get this working by running a python script with zbar to handle the png files and sort them accordingly.

A great learning curve, but I’m slowly figuring everything out.

CRN.

1 Like