[jitsi-dev] Support for Real Time Text


#1

Hi,

Does jitsi (or libjitsi) have support for Real Time Text as specified in
https://tools.ietf.org/html/rfc4103 and if not what would it take for me to
implement that, i.e. what parts of the source codes should I start looking
at and get familiar with? Thanks!

···

--
Martti von Hertzen
  martti.vonhertzen@gmail.com


#2

Hello Martti,

Hi,

Does jitsi (or libjitsi) have support for Real Time Text as specified in https://tools.ietf.org/html/rfc4103 and if not what would it take for me to implement that, i.e. what parts of the source codes should I start looking at and get familiar with? Thanks!

There's currently no support for rfc4103 as far as I know. Nowadays we implement most everything with a TransformEngine, so I would start looking at this part of the code. See for example DtmfTransformEngine, where in reverseTransform we parse DTMF tones from incoming packets and trigger an API. For sending DTMF is not a good example. Now it can be implemented more easily with MediaStream#injectPacket. It would be easier to use an existing audio or video stream (or at least to start this way).

Regards,
Boris

···

On 20/09/2017 09:00, Martti von Hertzen wrote:

--
Martti von Hertzen
martti.vonhertzen@gmail.com <mailto:martti.vonhertzen@gmail.com>

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#3

Hi Boris and thank you for your answer!

I've been playing with this now and I think I should be able to test this,
but I have the following problems (I'm testing my libjitsi by using Jitsi
desktop and replacing my jar with the one in the sc-bundles):
- Not only does libjitsi not reply with the t140 media attribute that I've
added, video doesn't work anymore (so there's no video answer button in
jitsi desktop with my libjitsi.jar). I've not touched anything concerning
video (like VideoMediaStreamImpl or anything, just AudioMediaStreamImpl).
Is there a compile flag I have to give to maven to get video back?
- Where should I add support for t140? I've checked where dtmf
("telephone-event") is used and I've tried to add t140 in a similar way,
but how does libjitsi advertise support for media types?
- I noticed that there is already support for TEXT (rfc4103) in
MediaTypes.java, but it's not used anywhere.
- Do I have to do something special for redundancy support ("red"
attribute) or is that handled automatically?

Here's the SDP from the INFO and the 200OK as taken from wireshark:

Session Description Protocol
    Session Description Protocol Version (v): 0
    Owner/Creator, Session Id (o): 100 3466 618 IN IP4 10.102.15.34
    Session Name (s): Talk
    Connection Information (c): IN IP4 10.102.15.34
    Bandwidth Information (b): AS:2000
    Time Description, active time (t): 0 0
    Media Description, name and address (m): audio 21638 RTP/AVP 8 0 9 101
    Media Attribute (a): rtpmap:101 telephone-event/8000
    Media Description, name and address (m): video 26718 RTP/AVP 96 97
    Media Attribute (a): rtpmap:96 VP8/90000
    Media Attribute (a): rtpmap:97 H264/90000
    Media Attribute (a): fmtp:97 profile-level-id=428014
    Media Description, name and address (m): text 27176 RTP/AVP 96 97
    Media Attribute (a): rtpmap:96 red/1000
    Media Attribute (a): fmtp:96 97/97/97
    Media Attribute (a): rtpmap:97 t140/1000

Session Description Protocol
    Session Description Protocol Version (v): 0
    Owner/Creator, Session Id (o): 198-jitsi.org 0 0 IN IP4 10.102.15.33
    Session Name (s): -
    Connection Information (c): IN IP4 10.102.15.33
    Time Description, active time (t): 0 0
    Media Description, name and address (m): audio 5000 RTP/AVP 8 0 9 101
    Media Attribute (a): rtpmap:8 PCMA/8000
    Media Attribute (a): rtpmap:0 PCMU/8000
    Media Attribute (a): rtpmap:9 G722/8000
    Media Attribute (a): rtpmap:101 telephone-event/8000
    Media Description, name and address (m): video 0 RTP/AVP 96 97
    Media Description, name and address (m): text 0 RTP/AVP 96 97

Thank you for your help! Best regards,

···

--
Martti von Hertzen


#4

You should place the jar in lib/installer-exclude instead.

Boris

···

On 26/09/2017 07:24, Martti von Hertzen wrote:

Hi Boris and thank you for your answer!

I've been playing with this now and I think I should be able to test this, but I have the following problems (I'm testing my libjitsi by using Jitsi desktop and replacing my jar with the one in the sc-bundles):


#5

Hi Boris and thank you for your answer!

I've been playing with this now and I think I should be able to test this,
but I have the following problems (I'm testing my libjitsi by using Jitsi
desktop and replacing my jar with the one in the sc-bundles):
- Not only does libjitsi not reply with the t140 media attribute that I've
added, video doesn't work anymore (so there's no video answer button in jitsi
desktop with my libjitsi.jar). I've not touched anything concerning video
(like VideoMediaStreamImpl or anything, just AudioMediaStreamImpl). Is there
a compile flag I have to give to maven to get video back?

Executing "mvn package", then copying to Jitsi Desktop should be enough. Make sure the correct version ends up in the sc-bundles folder. I haven't updated libjitsi in Jitsi Desktop in a while. It's possible that some recent work for Meet broke something Jitsi Desktop is relying on. Try a self-built libjitsi without any modifications first, if that doesn't work either, try a self-built one based on the currently committed binary (which is from https://github.com/jitsi/libjitsi/commit/4c31a7cd3f48ca7711b5aa2b3a8ef5192a9feeb0)

- Where should I add support for t140? I've checked where dtmf ("telephone-
event") is used and I've tried to add t140 in a similar way, but how does
libjitsi advertise support for media types?

libjitsi doesn't, the individual protocols in Jitsi Desktop do that.

- I noticed that there is already support for TEXT (rfc4103) in
MediaTypes.java, but it's not used anywhere.

That's coming from https://github.com/jitsi/jitsi/pull/3, which I unfortunately still haven't gotten around to actually merge.

- Do I have to do something special for redundancy support ("red" attribute)
or is that handled automatically?

Here's the SDP from the INFO and the 200OK as taken from wireshark:

Session Description Protocol
    [...]

Session Description Protocol
    [...]
Thank you for your help! Best regards,

Ingo