Jigasi transcription call sendRequest non stop! (with 48000/1920 audio input sizes)

drali · April 14, 2021, 10:50pm

hi,
i am trying connect a transcription service to Jigasi.
when i compile jigasi i have some errors in test phase. but anyway it gives me jigasi.jar and other libs
for example something like this:

2021-04-14 22:20:31.711 SEVERE: [15] org.jitsi.impl.neomedia.device.DeviceConfiguration.log() Failed to register custom Renderer org.jitsi.impl.neomedia.jmfext.media.renderer.audio.PulseAudioRenderer with JMF.java.lang.IllegalStateException: audioSystem
i didn’t change jigasi codes, i just add ONE file like vosk.
my java version:
openjdk 11.0.10 2021-01-19
OpenJDK Runtime Environment (build 11.0.10+9-Ubuntu-0ubuntu1.18.04)
OpenJDK 64-Bit Server VM (build 11.0.10+9-Ubuntu-0ubuntu1.18.04, mixed mode, sharing)
my compile code: mvn install -Dassembly.skipAssembly=false
i am using master branch (last commit: feat: Adjust some default config values and examples.
)

the main problem is: when jigasi joins a room. in my StreamingRecognitionSession class sendRequest method calls non stop! one time with 48k bytes and one time with 1920!!!

can anyone helps me please
thanks

the last thing!! when jigasi joins the room for transcription i hear an unnormal sound for about 0.5 seconds (like someone is breathing)

jigasi.log (190.8 KB) compile_test.log (18.9 KB)

Jigasi 2021-04-15 02:53:47.355 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.375 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.375 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.394 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.394 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.415 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.415 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.434 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

damencho · April 14, 2021, 11:35pm

Like this one? jitsi-meet/joined.wav at master · jitsi/jitsi-meet · GitHub
That is strange sound notifications are played only when there is a sip session

What is the problem? It is a stream with audio coming from the conference.

drali · April 14, 2021, 11:50pm

thanks for your reply
look at logs time. in this senario my transcription gives an audio every 10 ms!
later when i was testing Jigasi it sends me 48k byte audio every 500 ms. and ALWASY audio size was 48k. i think that was it’s normal behavior.
but now sendRequest function is calling so so much fast with 48k and 1920 byte size audio.
look at this logs and their times: (i attached complete log in first post)
i am sure this is not my bug! i don’t have any for loop for recursive call of this function. i just write a log and return

Jigasi 2021-04-15 02:53:45.627 INFO: [78] org.jitsi.jigasi.TranscriptionGatewaySession.log() Transcriber: Media Device Audio

Jigasi 2021-04-15 02:53:45.678 INFO: [78] service.protocol.media.MediaHandler.registerDynamicPTsWithStream().1003 Dynamic PT map: 126=rtpmap:-1 telephone-event/8000; 111=rtpmap:-1 opus/48000/2 fmtp:useinbandfec=1;minptime=10; 103=rtpmap:-1 unknown/90000;

Jigasi 2021-04-15 02:53:45.679 INFO: [78] service.protocol.media.MediaHandler.registerDynamicPTsWithStream().1020 PT overrides [103->104 ]

Jigasi 2021-04-15 02:53:45.699 INFO: [78] service.protocol.media.CallPeerMediaHandler.start().1961 Starting

Jigasi 2021-04-15 02:53:45.875 INFO: [78] org.jitsi.jigasi.JvbConference.callStateChanged().1485 [ctx=1618439024264151426727] JVB conference call IN_PROGRESS.

Jigasi 2021-04-15 02:53:45.879 WARNING: [78] org.jitsi.jigasi.stats.StatsHandler.startConferencePeriodicRunnable().329 [ctx=1618439024264151426727] Stats handler missing for call:Call: id=1618439024716510962790 peers=1

Jigasi 2021-04-15 02:53:46.802 INFO: [87] org.jitsi.srtp.crypto.OpenSslWrapperLoader.log() jitsisrtp successfully loaded

Jigasi 2021-04-15 02:53:47.355 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.375 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.375 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.394 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.394 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.415 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.415 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.434 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.434 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.454 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.454 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.474 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.475 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.495 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.495 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.514 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.514 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.535 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.535 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.554 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.555 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.575 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.575 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.594 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.594 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.614 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.615 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.635 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.635 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.654 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.654 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.674 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.675 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.695 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.695 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.714 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.714 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.734 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.735 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.755 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.755 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.774 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.774 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.794 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.795 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.815 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.815 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.834 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.834 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.854 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.854 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.874 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.874 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.895 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.895 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.914 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.914 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.934 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.934 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.954 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.955 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.975 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.975 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:47.994 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:47.994 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:48.014 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:48.014 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:48.035 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:48.035 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:48.054 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:48.054 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

Jigasi 2021-04-15 02:53:48.075 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 1920

Jigasi 2021-04-15 02:53:48.075 SEVERE: [126] org.jitsi.jigasi.transcription.MyTranscriptionService.log() sendRequest called with request.getAudio().length: 48000

damencho · April 15, 2021, 12:56am

Maybe it depend whether you are translating or decoding the audio

drali · April 15, 2021, 1:22am

no. chaning this option didn’t change it.
org.jitsi.jigasi.transcription.ENABLE_TRANSLATION=false

anyway in both case getting 48k data in every 20ms is not ok! (when there is only one participant)

i see many strange behaviour :’( should i try Oracle java instead of openjdk?

which version is using for making jitsi/jigasi docker image?

thanks alot

damencho · April 15, 2021, 2:39am

Not sure, maybe java 11 … Or java 8 …

drali · April 15, 2021, 12:44pm

it is ok on java 8 (call sendRequest every 500ms with 48k data)
but with java 11 we have problem (call sendRequest every 10 ms with 48k and 1920)

same code, same config!

my java 8:
openjdk version “1.8.0_282”
OpenJDK Runtime Environment (build 1.8.0_282-8u282-b08-0ubuntu1~18.04-b08)
OpenJDK 64-Bit Server VM (build 25.282-b08, mixed mode)

my java 11:
openjdk version “11.0.10” 2021-01-19
OpenJDK Runtime Environment (build 11.0.10+9-Ubuntu-0ubuntu1.18.04)
OpenJDK 64-Bit Server VM (build 11.0.10+9-Ubuntu-0ubuntu1.18.04, mixed mode, sharing)

damencho · April 15, 2021, 9:58pm

Yep, we run jigasi with java 8, not sure what can be the problem with java11.