Jingle DTMF from Jicofo to Jigasi

Aditya123 · February 25, 2023, 6:48am

Hello Team,
I have a use case where i need to forward the jingle DTMF from Jicofo to Jigasi. Here is the scenario:

Client (Jingle DTMF) --> Jicofo --> Jigasi(RFC 2833) --> SIP B2BUA

Client (Jitsi-meet) sends Jingle DTMF (XEP-0181) to Jicofo and Jicofo needs to forward the Jingle DTMF to Jigasi…
I was able to format the jingle DTMF from lib-jitsi-meet and added code in jitsi-xmpp-extensions to support jingle DTMF. Now, Jicofo is able to receive the DTMF tone. I also figured out how to send the DTMF from Jigasi (inband RTP) to B2BUA.
Now, i am trying to figure out how to forward the jingle message from Jicofo to Jigasi. Looks like, the session-info messages are abstracted in jitsi communicator package and Jigasi is not having any interface to receive jingle session-info messages.

May i know if there is a way to forward the DTMF message without having to touch jitsi communicator package?

Let me know if i can share you any further details.

Thanks in advance

damencho · February 25, 2023, 12:36pm

Why jingle DTMF?
We have used in the past this

github.com

jitsi/lib-jitsi-meet/blob/931ca368b79004a103451dde9d6f89dbe1415e2c/modules/RTC/TraceablePeerConnection.js#L2722


      
          
          
/**
           * Sends DTMF tones if possible.
           *
           * @param {string} tones - The DTMF tones string as defined by {@code RTCDTMFSender.insertDTMF}, 'tones' argument.
           * @param {number} duration - The amount of time in milliseconds that each DTMF should last. It's 200ms by default.
           * @param {number} interToneGap - The length of time in miliseconds to wait between tones. It's 200ms by default.
           *
           * @returns {void}
           */
          TraceablePeerConnection.prototype.sendTones = function(tones, duration = 200, interToneGap = 200) {
              if (!this._dtmfSender) {
                  if (this.peerconnection.getSenders) {
                      const rtpSender = this.peerconnection.getSenders().find(s => s.dtmf);
          
          
            this._dtmfSender = rtpSender && rtpSender.dtmf;
                      this._dtmfSender && logger.info(`${this} initialized DTMFSender using getSenders`);
                  }
          
          
        if (!this._dtmfSender) {
                      const localAudioTrack = Array.from(this.localTracks.values()).find(t => t.isAudioTrack());

Just make sure you set this in jigasi

github.com

jitsi/infra-configuration/blob/8977c4fc72c39d7e7028e7465217084c7893a192/ansible/roles/jigasi/templates/sip-communicator.properties.j2#L15


      
          {% if jigasi_bosh_host_override %}
          org.jitsi.jigasi.BOSH_HOST_OVERRIDE={{ jigasi_bosh_host_override }}
          {% endif %}
          
          
net.java.sip.communicator.impl.protocol.SingleCallInProgressPolicy.enabled=false
          
          
# Adjust opus encoder complexity
          net.java.sip.communicator.impl.neomedia.codec.audio.opus.encoder.COMPLEXITY=10
          
          
# allow dtmf pass through to the sip side
          org.jitsi.service.neomedia.AudioMediaStream.DISABLE_DTMF_HANDLING=true
          
          
{% if jigasi_hole_punch_pkt_count %}
          net.java.sip.communicator.impl.protocol.HOLE_PUNCH_PKT_COUNT={{ jigasi_hole_punch_pkt_count }}
          {% endif %}
          
          
# Disables packet logging
          net.java.sip.communicator.packetlogging.PACKET_LOGGING_ENABLED=false
          
          
# Increase incoming UDP buffer size
          org.jitsi.impl.neomedia.RTPConnectorInputStream.SO_RCVBUF={{ jigasi_udp_buffer_size }}

Aditya123 · February 25, 2023, 1:18pm

Thank you so much for this. If i understand correctly, the code that is present in lib-jitsi-meet sends the DTMF tone as an in-band RTP to JVB and then JVB relays it to Jigasi (jigasi receives it via toneReceivedEvent in SipGatewaySession.java). Is my understanding correct?

Regarding the use case, we have a SIP B2BUA that has some ivr related functionality to be executed.
As long as Jigasi can receive the DTMF sent from web client, we are good. Jingle DTMF is not mandatory.

damencho · February 25, 2023, 2:20pm

It is not inband, but rtp dtmf (rfc4733 DTMFs). Yep, and from SipGatewaySession those are forwarded then to the sip side. How they are sent, depends on the sip account settings in jigasi. The default one is rfc4733 if support is signaled on the sip side.

Aditya123 · February 26, 2023, 2:59pm

This is amazing. Thank you for sharing.

Aditya123 · March 3, 2023, 4:51am

Hi. I have tried this method and see that DTMF is received by Jigasi. But Jigasi is not forwarding the DTMF to the other leg. I have tried all the DTMF methods on the SIP side (AUTO_DTMF, RTP_DTMF, SIP_INFO_DTMF and INBAND_DTMF). I see the following error in Jigasi logs

2023-03-02 20:00:57.625 FINE: [113] org.jitsi.impl.neomedia.transform.DiscardTransformEngine.log() passing through  RTP ssrc=2482250868, seqnum=11673, ts=999519582, streamHashCode=417864897
2023-03-02 20:00:57.625 WARNING: [117] net.sf.fmj.media.Log.warning() Stopping stream because of payload type mismatch: expecting pt=111, got pt=126

2023-03-02 20:00:57.705 FINE: [113] org.jitsi.impl.neomedia.transform.DiscardTransformEngine.log() passing through  RTP ssrc=2482250868, seqnum=11679, ts=999524382, streamHashCode=417864897
2023-03-02 20:00:57.706 WARNING: [117] net.sf.fmj.media.Log.warning() Stopping stream because of payload type mismatch: expecting pt=126, got pt=111
2023-03-02 20:00:57.706 INFO: [117] net.sf.fmj.media.Log.info() Stopping RTPSourceStream.
2023-03-02 20:00:57.706 FINE: [117] net.sf.fmj.media.Log.comment() RTP socket receive buffer size: -1 bytes.

OPUS was negotiated as 111 and telephony-event was negotiated as 126 on the XMPP side. Not sure why jigasi is expecting the OPUS for telephone-event and vice versa.

Could you please let me know what could be the issue? Let me know if you need any further information

Thank you

damencho · March 3, 2023, 12:13pm

You have added org.jitsi.service.neomedia.AudioMediaStream.DISABLE_DTMF_HANDLING=true, right?

damencho · March 3, 2023, 12:15pm

And audio is not working through jigasi in this call?

Aditya123 · March 3, 2023, 12:46pm

audio is working through jigasi call. I have added the DISABLE_DTMF_HANDLING to true

org.jitsi.service.neomedia.AudioMediaStream.DISABLE_DTMF_HANDLING=true

and then tried all the possible DTMF methods on the SIP side

net.java.sip.communicator.impl.protocol.sip.acc1403273890647.DTMF_METHOD=AUTO_DTMF
net.java.sip.communicator.impl.protocol.sip.acc1403273890647.DTMF_METHOD=RTP_DTMF
net.java.sip.communicator.impl.protocol.sip.acc1403273890647.DTMF_METHOD=SIP_INFO_DTMF
net.java.sip.communicator.impl.protocol.sip.acc1403273890647.DTMF_METHOD=INBAND_DTMF

I have tried DTMF handler from SIP side (entering DTMFs from SIP Phone. ). Those are working as well.
Only issue I see is: DTMFs are not forwarded from Jigasi XMPP leg to Jigasi SIP leg.

damencho · March 3, 2023, 2:00pm

Can you try adding a print here: jigasi/SipGatewaySession.java at d35bfb208a0f8f28d17477c932b32445e73b58a8 · jitsi/jigasi · GitHub
And see whether dtmfs are reaching jigasi code on the xmpp side and if that is true means they are not handled on the sip side …

Aditya123 · March 3, 2023, 2:22pm

yes i have added some logs around toneReceived event (net.java.sip.communicator.service.protocol.event) but those were not logged. Looks like they got discarded even before reaching the jigasi code.
Alternatively, I have also tried, implementing DTMFListener interface from org.jitsi.service.ndeomedia.event as well. Those were also not getting logged.

Is there something i’m missing here?

damencho · March 3, 2023, 2:31pm

Can you check what does this method returns null or DtmfTransformEngine?

github.com

jitsi/libjitsi/blob/99118c1e78a941feb43149cc9ec9a0645a07046f/src/main/java/org/jitsi/impl/neomedia/AudioMediaStreamImpl.java#L460


      
          {
              if (dtmfTransformEngine == null)
              {
                  ConfigurationService cfg = LibJitsi.getConfigurationService();
                  if (cfg == null || !cfg.getBoolean(
                          AudioMediaStream.DISABLE_DTMF_HANDLING_PNAME, false))
                  {
                      dtmfTransformEngine = new DtmfTransformEngine(this);
                  }
              }
              return dtmfTransformEngine;
          }
          
          
/**
           * {@inheritDoc}
           */
          @Override
          protected SsrcTransformEngine createSsrcTransformEngine()
          {
              if (ssrcTransformEngine == null)
                  ssrcTransformEngine = new SsrcTransformEngine(this);

Hum wait … you are not using the translator mode right?

github.com

jitsi/jigasi/blob/eec6f197e59f073cbb1bd6704e9655865f9f549b/jigasi-home/sip-communicator.properties#L13


      
          # as well as other account properties
          
          
# Name of default JVB room that will be joined if no special header is included
          # in SIP invite
          #org.jitsi.jigasi.DEFAULT_JVB_ROOM_NAME=siptest@conference.<<DOMAIN_BASE>>
          
          
org.jitsi.jigasi.MUC_SERVICE_ADDRESS=conference.<<DOMAIN_BASE>>
          
          
net.java.sip.communicator.impl.protocol.SingleCallInProgressPolicy.enabled=false
          
          
# Should be enabled when using translator mode
          #net.java.sip.communicator.impl.neomedia.audioSystem.audiosilence.captureDevice_list=["AudioSilenceCaptureDevice:noTransferData"]
          
          
# Adjust opus encoder complexity
          net.java.sip.communicator.impl.neomedia.codec.audio.opus.encoder.COMPLEXITY=10
          
          
# Disables packet logging
          net.java.sip.communicator.packetlogging.PACKET_LOGGING_ENABLED=false
          
          
net.java.sip.communicator.impl.protocol.sip.acc1403273890647=acc1403273890647
          net.java.sip.communicator.impl.protocol.sip.acc1403273890647.ACCOUNT_UID=SIP\:<<JIGASI_SIPUSER>>

Translator mode is where jigasi is resending rtp and not doing any decode - encode, in translator mode the sip side needs to be able to handle multiple streams and doing mixing. When in that mode you need AudioMediaStream.DISABLE_DTMF_HANDLING=true so dtmfs are not handled and send to the listeners and they just pass through the chain.

But if jigasi is decoding and then encoding you need:
AudioMediaStream.DISABLE_DTMF_HANDLING=false, which is the default value. So dtmfs are signalled to the listeners so you can re-send them.

Sorry, I forgot about that detail.

Can you try running with AudioMediaStream.DISABLE_DTMF_HANDLING=false or commenting it. Then you should get the tones inside the SipGatewaySession.java. Do you get those?

Aditya123 · March 3, 2023, 2:43pm

will check and let you know about dtmfTransformEngine inside libjitsi.

Regarding disable_dtmf_handling, I tried both the options. The only difference i see when I set DISABLE_DTMF_HANDLING=true is:

2023-03-02 19:45:07.950 FINE: [112] org.jitsi.impl.neomedia.transform.DiscardTransformEngine.log() passing through  RTP ssrc=563216014, seqnum=2746, ts=499591366, streamHashCode=711776117
2023-03-02 19:45:07.951 WARNING: [116] net.sf.fmj.media.Log.warning() Stopping stream because of payload type mismatch: expecting pt=111, got pt=126
2023-03-02 19:45:07.951 INFO: [116] net.sf.fmj.media.Log.info() Stopping RTPSourceStream.
2023-03-02 19:45:07.951 FINE: [116] net.sf.fmj.media.Log.comment() RTP socket receive buffer size: -1 bytes.

2023-03-02 19:45:07.951 INFO: [111] net.sf.fmj.media.Log.info() Stopping RTPSourceStream.
2023-03-02 19:45:07.951 INFO: [182] net.sf.fmj.media.Log.info() Stopping RTPSourceStream.
2023-03-02 19:45:07.952 INFO: [111] net.sf.fmj.media.Log.info() Stopping RTPSourceStream.

2023-03-02 19:45:08.036 FINE: [112] org.jitsi.impl.neomedia.transform.DiscardTransformEngine.log() passing through  RTP ssrc=563216014, seqnum=2752, ts=499596166, streamHashCode=711776117
2023-03-02 19:45:08.036 WARNING: [116] net.sf.fmj.media.Log.warning() Stopping stream because of payload type mismatch: expecting pt=126, got pt=111
2023-03-02 19:45:08.036 INFO: [116] net.sf.fmj.media.Log.info() Stopping RTPSourceStream.
2023-03-02 19:45:08.036 FINE: [105] org.jitsi.srtp.SrtpCryptoContext.log() Updated replay window with 7215. maxIdx=7215, window=0xffffffffffffffff: [7152, 7153, 7154, 7155, 7156, 7157, 7158, 7159, 7160, 7161, 7162, 7163, 7164, 7165, 7166, 7167, 7168, 7169, 7170, 7171, 7172, 7173, 7174, 7175, 7176, 7177, 7178, 7179, 7180, 7181, 7182, 7183, 7184, 7185, 7186, 7187, 7188, 7189, 7190, 7191, 7192, 7193, 7194, 7195, 7196, 7197, 7198, 7199, 7200, 7201, 7202, 7203, 7204, 7205, 7206, 7207, 7208, 7209, 7210, 7211, 7212, 7213, 7214, 7215]
2023-03-02 19:45:08.036 FINE: [116] net.sf.fmj.media.Log.comment() RTP socket receive buffer size: -1 bytes.

2023-03-02 19:45:08.037 FINE: [113] org.jitsi.impl.neomedia.transform.DiscardTransformEngine.log() passing through  RTP ssrc=2550712886, seqnum=61465, ts=106080, streamHashCode=1766026962
2023-03-02 19:45:08.037 INFO: [111] net.sf.fmj.media.Log.info() Stopping RTPSourceStream.
2023-03-02 19:45:08.037 INFO: [213] net.sf.fmj.media.Log.info() Stopping RTPSourceStream.
2023-03-02 19:45:08.037 INFO: [111] net.sf.fmj.media.Log.info() Stopping RTPSourceStream.

The above logs are not appearing when i set the DISABLE_DTMF_HANDLING=false.

damencho · March 3, 2023, 3:21pm

what about the mode you run? is it re-encoding media or you are passing it through and mixing on the sip side?

Aditya123 · March 3, 2023, 4:01pm

When DISABLE_DTMF_HANDLING=false, I have used RTP_DTMF

net.java.sip.communicator.impl.protocol.sip.acc1403273890647.DTMF_METHOD=RTP_DTMF

damencho · March 3, 2023, 4:07pm

So you don’t have these set:

net.java.sip.communicator.impl.neomedia.audioSystem.audiosilence.captureDevice_list=["AudioSilenceCaptureDevice:noTransferData"]
net.java.sip.communicator.impl.protocol.sip.acc1403273890647.USE_TRANSLATOR_IN_CONFERENCE=true
org.jitsi.jigasi.xmpp.acc.USE_TRANSLATOR_IN_CONFERENCE=true

?

Aditya123 · March 5, 2023, 7:16am

No. First one was commented and i don’t see the other in the sip-communicator.properties