Jitsi API for accessing streaming data


#1

Hello guys. I wonder if there is a possibility to access streaming data for a processing purpose. Please share


#2

Could you please provide more details about the use case you’re interested in and the constraints that you’re trying to satisfy?

For example, which one of the many projects of the Jitsi organization are you asking about? The Jitsi desktop client written primarily in Java, the Jitsi Meet web client, or the Jitsi Meet for Android and iOS clients? Or maybe you’re trying to access streaming data inside Jitsi Videobridge?


#3

Hi Lyubo! Thanks for your reply.
My goal is to record and transcribe of what has been told in a video/audio conference.
My subgoals are of 2:

  1. To send audio data to Speech To Text Engines such as Google Speech.
  2. To know to which user (that is, username/nickname) current audio/video stream belongs.
    From how I see probably the best place is Jitsi Videobridge to remove concerns regarding clients platforms.

Please give me some directions where I should look at or dig in.
Thanks in advance!


#4

I forgot to mention in my last reply that in the best scenario I need to do it in realtime.


#5
  1. To send audio data to Speech To Text Engines such as Google Speech.

@Nik_V and @Boris_Grozev did that already! Guys, would you care to give pointers to @Alexey_Makarov?


#6

@Alexey_Makarov this is already implemented in jigasi https://github.com/jitsi/jigasi.

All you need to do is:

  • install jigasi
  • add org.jitsi.jigasi.ENABLE_TRANSCRIPTION=true to the /etc/jitsi/jigasi/sip-communicator.properties
  • add export GOOGLE_APPLICATION_CREDENTIALS=... to /etc/jitsi/jigasi/config pointing to the json file with credentials for Google Cloud Speech API.
  • and also enable the transcription in the UI, adding transcribingEnabled: true, to /etc/jitsi/meet/yourdomain.com-config.js
    This will give you a real-time transcription of the conference. All the code(non-UI of course) is in jigasi under src/main/java/org/jitsi/jigasi/transcription.

All credits to @Nik_V, @Boris_Grozev and Praveen Gupta, they had worked on this project.
https://nikvaessen.github.io/jekyll/update/2017/08/24/gsoc2017_work_product_submission.html


#7

Awesome! Thanks for this.