[jitsi-dev] Msrp beta


#1

Hi,

Attached, please find a patch for Jitsi to support basic IM with MSRP.
Note that this is a first attempt and should not be used to commit into the
repo.

Instead, I'd like the Jitsi developers to review this work and tell me
whether I'm on the right track or way off base.
Any other hints, tips etc. will also be greatly appreciated.

Thanks in advance,
    Tom Uijldert.

Description:

jitsiMsrpPatch.txt (33.5 KB)

libjitsiMsrpPatch.txt (1.72 KB)

msrp.jar (117 KB)

···

-----------
The patch introduces a property that activates MSRP for simple chat when set
(see the protocol provider service).

This creates the MSRP operation set for basic chat which uses the
call-handler to negotiate the connection.

So:
--
- on pressing "Send a message" in a SIP contact, an MSRP session is set up
as soon as the first message is typed (this'll also trigger the call window
btw.).
- likewise, when receiving a call with an "m=message" session description,
an MSRP session is set up and presence triggered.

If you want to test:
-------------------
- apply the patches and get the 3 libraries (see build.xml diff).
The slf4j libraries can be downloaded from their site, I've included the
last msrp.jar here, although that can also be downloaded from the Sonatype
site.
- do the rebuild
- make sure you have the MSRP_MESSAGING_PREFERRED property set to true.

For further study/Todo:
----------------------
- implement nicknames
- implement typing indicator
- hook into file transfer
- do something intelligent with status reports, aborted messages etc.

- desktop sharing?
- parallel voice/video/chat?
- chat-conferencing/multi-chat?


#2

Hey Tom,

Very sorry for the delay. A few notes inline:

Hi,

Attached, please find a patch for Jitsi to support basic IM with MSRP.

Very glad to know there's progress on the matter.

Note that this is a first attempt and should not be used to commit into the
repo.

Instead, I'd like the Jitsi developers to review this work and tell me
whether I'm on the right track or way off base.
Any other hints, tips etc. will also be greatly appreciated.

Thanks in advance,
    Tom Uijldert.

Description:
-----------
The patch introduces a property that activates MSRP for simple chat when set
(see the protocol provider service).

Could you please send us an overall description of your design and
reasoning?

This creates the MSRP operation set for basic chat which uses the
call-handler to negotiate the connection.

So:
--
- on pressing "Send a message" in a SIP contact, an MSRP session is set up
as soon as the first message is typed (this'll also trigger the call window
btw.).
- likewise, when receiving a call with an "m=message" session description,
an MSRP session is set up and presence triggered.

What kind of presence is triggered exactly?

If you want to test:
-------------------
- apply the patches and get the 3 libraries (see build.xml diff).
The slf4j libraries can be downloaded from their site, I've included the
last msrp.jar here, although that can also be downloaded from the Sonatype
site.
- do the rebuild
- make sure you have the MSRP_MESSAGING_PREFERRED property set to true.

I haven't tried it yet but have the following overall comments:

* I think that having a new implementation of the basic telephony opset
that is also a field in the existing implementation sounds a bit too heavy.

* I am not sure why you need a new MediaHandler implementation. This is
just a new media type so it should require almost no changes to the
media handler. The new one does seem to reuse a lot of code from the old
version.

* Your OperationSetBasicInstantMessaging does make sense. I suppose it
should be the entry point (rather than basic telephony).

* I know that this is just an early draft but I thought I'd mention
these: I noticed several javadocs with @see or @return clauses only. I
also saw some braces that were not alone on a line (e.g. "} catch")

Looking forward to seeing the rest.

Cheers,
Emil

···

On 25.03.13, 17:35, Tom Uijldert wrote:

For further study/Todo:
----------------------
- implement nicknames
- implement typing indicator
- hook into file transfer
- do something intelligent with status reports, aborted messages etc.

- desktop sharing?
- parallel voice/video/chat?
- chat-conferencing/multi-chat?

--
https://jitsi.org


#3

From: Emil Ivov [mailto:emcho@jitsi.org]

Hey Tom,

Very sorry for the delay. A few notes inline:

As they say, better late than never. Glad you could spare the time.

> Hi,
>
> Attached, please find a patch for Jitsi to support basic IM with
MSRP.

Very glad to know there's progress on the matter.

[snip]

> The patch introduces a property that activates MSRP for simple chat
> when set (see the protocol provider service).

Could you please send us an overall description of your design and
reasoning?

Eehhm, there is none.
Basically, I'm still finding my way around Jitsi and do some experimenting
so this is more a proof of concept, but the goal is clear:
  - to integrate MSRP capabilities into Jitsi in the smoothest way
possible (without changing too much in the user interfacing/experience).

So what I'd like to achieve is be able to handle an incoming call that asks
for messaging in its sdp to open up an msrp chat.
Likewise, I'd like to be able to call someone else and ask for an msrp chat.

So some configuration property like "enable msrp" should be part of a SIP
account that would unlock this functionality.
At the same time, I'd like Jitsi to be able to still handle SIP MESSAGE
im's.

>
> This creates the MSRP operation set for basic chat which uses the
> call-handler to negotiate the connection.
>
> So:
> --
> - on pressing "Send a message" in a SIP contact, an MSRP session is
> set up as soon as the first message is typed (this'll also trigger
the
> call window btw.).
> - likewise, when receiving a call with an "m=message" session
> description, an MSRP session is set up and presence triggered.

What kind of presence is triggered exactly?

Sorry, I meant contact-window.

> If you want to test:
> -------------------
> - apply the patches and get the 3 libraries (see build.xml diff).
> The slf4j libraries can be downloaded from their site, I've included
> the last msrp.jar here, although that can also be downloaded from the
> Sonatype site.
> - do the rebuild
> - make sure you have the MSRP_MESSAGING_PREFERRED property set to
true.

I haven't tried it yet but have the following overall comments:

* I think that having a new implementation of the basic telephony opset
that is also a field in the existing implementation sounds a bit too
heavy.

Not sure I follow. Don't think I have a new implementation of telephony(?)

* I am not sure why you need a new MediaHandler implementation. This is
just a new media type so it should require almost no changes to the
media handler. The new one does seem to reuse a lot of code from the
old version.

I created or wrapped the Handler in order to not change too much in existing
code (as a subclass/superclass?) but granted, it can be integrated.

* Your OperationSetBasicInstantMessaging does make sense. I suppose it
should be the entry point (rather than basic telephony).

Again, not sure I follow. Imo. I just hook into the telephony-part because I
need the rendez-vous capabilities (?)

* I know that this is just an early draft but I thought I'd mention
these: I noticed several javadocs with @see or @return clauses only. I
also saw some braces that were not alone on a line (e.g. "} catch")

Ofcourse, probably Eclipse code generation here and there but -as I said-
proof of concept :slight_smile:

Looking forward to seeing the rest.

Thanks.

···

-----Original Message-----
On 25.03.13, 17:35, Tom Uijldert wrote:

Cheers,
Emil

>
> For further study/Todo:
> ----------------------
> - implement nicknames
> - implement typing indicator
> - hook into file transfer
> - do something intelligent with status reports, aborted messages etc.
>
> - desktop sharing?
> - parallel voice/video/chat?
> - chat-conferencing/multi-chat?
>

--
https://jitsi.org


#4

Hey Tom,

From: Emil Ivov [mailto:emcho@jitsi.org]

Hey Tom,

Very sorry for the delay. A few notes inline:

As they say, better late than never. Glad you could spare the time.

Hi,

Attached, please find a patch for Jitsi to support basic IM with

MSRP.

Very glad to know there's progress on the matter.

[snip]

The patch introduces a property that activates MSRP for simple chat
when set (see the protocol provider service).

Could you please send us an overall description of your design and
reasoning?

Eehhm, there is none.

Oh, I wasn't asking for an UML diagram or something. Just your thoughts
and the design you had in mind. Basically just explain it as if we were
on the phone :slight_smile:

Basically, I'm still finding my way around Jitsi and do some experimenting
so this is more a proof of concept, but the goal is clear:
  - to integrate MSRP capabilities into Jitsi in the smoothest way
possible (without changing too much in the user interfacing/experience).

OK, that's a good plan.

So what I'd like to achieve is be able to handle an incoming call that asks
for messaging in its sdp to open up an msrp chat.

OK.

Likewise, I'd like to be able to call someone else and ask for an msrp chat.

OK.

Note that chats would be complicated because we don't currently have
conversation management within Jitsi's GUI and you would have to manage
this in the implementation.

That's why I thought that starting with the file transfer use case would
be more straight forward, as you have clear begin and end points there.

So some configuration property like "enable msrp" should be part of a SIP
account that would unlock this functionality.

Yup, definitely.

At the same time, I'd like Jitsi to be able to still handle SIP MESSAGE
im's.

Ah well ... I think that's too much of a hassle. I'd suggest making the
choice during account creation. Either you want one or the other.
Depending on that choice you either register one implementation or the
other (I am talking about the messaging opset).

This creates the MSRP operation set for basic chat which uses the
call-handler to negotiate the connection.

So:
--
- on pressing "Send a message" in a SIP contact, an MSRP session is
set up as soon as the first message is typed (this'll also trigger

the

call window btw.).
- likewise, when receiving a call with an "m=message" session
description, an MSRP session is set up and presence triggered.

What kind of presence is triggered exactly?

Sorry, I meant contact-window.

If you want to test:
-------------------
- apply the patches and get the 3 libraries (see build.xml diff).
The slf4j libraries can be downloaded from their site, I've included
the last msrp.jar here, although that can also be downloaded from the
Sonatype site.
- do the rebuild
- make sure you have the MSRP_MESSAGING_PREFERRED property set to

true.

I haven't tried it yet but have the following overall comments:

* I think that having a new implementation of the basic telephony opset
that is also a field in the existing implementation sounds a bit too
heavy.

Not sure I follow. Don't think I have a new implementation of telephony(?)

I might have been confused by the basic instant messaging impl then. Sorry.

* I am not sure why you need a new MediaHandler implementation. This is
just a new media type so it should require almost no changes to the
media handler. The new one does seem to reuse a lot of code from the
old version.

I created or wrapped the Handler in order to not change too much in existing
code (as a subclass/superclass?) but granted, it can be integrated.

* Your OperationSetBasicInstantMessaging does make sense. I suppose it
should be the entry point (rather than basic telephony).

Again, not sure I follow. Imo. I just hook into the telephony-part because I
need the rendez-vous capabilities (?)

I was thinking that you could have the INVITE passed to the IM op set
right from the provider ... but I guess I don't mind it this way either.

* I know that this is just an early draft but I thought I'd mention
these: I noticed several javadocs with @see or @return clauses only. I
also saw some braces that were not alone on a line (e.g. "} catch")

Ofcourse, probably Eclipse code generation here and there but -as I said-
proof of concept :slight_smile:

Sure :).

Cheers,
Emil

···

On 09.04.13, 18:03, Tom Uijldert wrote:

-----Original Message-----
On 25.03.13, 17:35, Tom Uijldert wrote:

Looking forward to seeing the rest.

Thanks.

Cheers,
Emil

For further study/Todo:
----------------------
- implement nicknames
- implement typing indicator
- hook into file transfer
- do something intelligent with status reports, aborted messages etc.

- desktop sharing?
- parallel voice/video/chat?
- chat-conferencing/multi-chat?

--
https://jitsi.org

--
https://jitsi.org


#5

From: Emil Ivov [mailto:emcho@jitsi.org]

Hey Tom,

>> From: Emil Ivov [mailto:emcho@jitsi.org]
>>

[SNIP]

Note that chats would be complicated because we don't currently have
conversation management within Jitsi's GUI and you would have to manage
this in the implementation.

Please elaborate, not sure I follow and need some design-info:

Came more-and-more to the conclusion that I need to drop any tie-in to
call-handling.
Handling INVITE's that both contain audio-sdp *and* message-sdp is Utopia.
Showing both the call-window and the chat window (like in the beta) is
confusing for users.
So when message-sdp comes in (or sent out), no call-window is/should be
set-up, just the chat-window (mind you, as a SIP dialog/call it should still
be administered as some session/callpeer).

But then an incoming INVITE must be accepted through another (non-call)
pop-up along the lines of "accept <contact> chat [Y/N]?".

Now I have that already if the INVITE is from a chatroom (yes, when the
contact-header indicates ";isfocus" - this lead to a complete conference
call window in the beta where it should really open some chatroom window),
accomplished by firing an InvitationEvent for an ad-hoc chatroom.
Question is, how do I subsequently (on the join) activate the chatroom
window and give it focus within Jitsi? Looks like Jitsi has that already but
have problems tracking how that's done (have problems mimicking such
behaviour with IRC/XMPP).

Likewise, is there an acceptance pop-up already for a 1-to-1 chat and how do
I activate that?

Also, the message-window only pops-up when the first message is received.
Ideally, it should pop-up the moment it is accepted.
Is this what you meant with "conversation management"?

Btw.: is there some form of design/UML of the software? Because I find
myself spending an inordinate amount of time figuring out how some stuff
interacts (don't get me wrong, the software seems/is quite structured but
the sheer volume gives it a steep learning-curve).

That's why I thought that starting with the file transfer use case
would be more straight forward, as you have clear begin and end points
there.

My guess is, the same issues will apply. I need the SIP dialog handling
there but not the call-stuff.
As soon as I want to send/receive a file to a contact, acceptance pop-ups
are needed again and possibly auto-popup any file transfer window?
So maybe I need more separation between SIP handling a call and/or chat
handling.

As usual, any further hints: much appreciated.

Regards,
    Tom Uijldert.

.

···

-----Original Message-----
On 09.04.13, 18:03, Tom Uijldert wrote:
>> -----Original Message-----


#6

--sent from my mobile

> From: Emil Ivov [mailto:emcho@jitsi.org]
>
> Hey Tom,
>
> >> From: Emil Ivov [mailto:emcho@jitsi.org]
> >>

[SNIP]

> Note that chats would be complicated because we don't currently have
> conversation management within Jitsi's GUI and you would have to manage
> this in the implementation.

Please elaborate, not sure I follow and need some design-info:

The GUI does not have the concept of a conversation with a start and an
end. It just deals with individual messages. For example, it will be up to
you to decide, within the protocol, when to close a chat. There will be no
indication from the GUI about this.

File transfers are however session oriented and a first implementation and
integration would much simpler there.

Came more-and-more to the conclusion that I need to drop any tie-in to
call-handling.
Handling INVITE's that both contain audio-sdp *and* message-sdp is Utopia.

That's reasonable and we had the same goal when Joao was first taking a
stab at this a few years ago. Back then we decided we would simply reject
INVITEs that have both a msrp and rtp.

Showing both the call-window and the chat window (like in the beta) is
confusing for users.
So when message-sdp comes in (or sent out), no call-window is/should be
set-up, just the chat-window (mind you, as a SIP dialog/call it should

still

be administered as some session/callpeer).

Makes sense.

But then an incoming INVITE must be accepted through another (non-call)
pop-up along the lines of "accept <contact> chat [Y/N]?".

I am not sure about this. Have you ever seen any popular 1-2-1 chat
application do this? People don,t ask you permission every time they want
to send you messages on skype, gtalk or facebook. They just do.

We may however want to add something like this for people not on your
roster .. But that would have to be a new mechanism.

Now I have that already if the INVITE is from a chatroom (yes, when the
contact-header indicates ";isfocus" - this lead to a complete conference
call window in the beta where it should really open some chatroom window),
accomplished by firing an InvitationEvent for an ad-hoc chatroom.

Oh my ... You also want to handle MUCs with your first impl? This is
shaping to be quite of a patch and the integration effort may prove to be
quite substantial.

Question is, how do I subsequently (on the join) activate the chatroom
window and give it focus within Jitsi? Looks like Jitsi has that already

but

have problems tracking how that's done (have problems mimicking such
behaviour with IRC/XMPP).

Should happen on the first message IIRC ... But I might be wrong.

Likewise, is there an acceptance pop-up already for a 1-to-1 chat and how

do

I activate that?

No there isn't. We may consider this for out of roster senders but you can
just accept everyone for starters.

Also, the message-window only pops-up when the first message is received.
Ideally, it should pop-up the moment it is accepted.
Is this what you meant with "conversation management"?

That too yes. Note however, that I wouldn't want empty chatwindows popping
up. I'd rather see them only when I have something to read. We can still
issue a toaster notification like we do for pro-active typing notifications.

Btw.: is there some form of design/UML of the software? Because I find
myself spending an inordinate amount of time figuring out how some stuff
interacts (don't get me wrong, the software seems/is quite structured but
the sheer volume gives it a steep learning-curve).

Well, there's the AOSA book ...but that's all. We are very much open to
contributions in that regard.

> That's why I thought that starting with the file transfer use case
> would be more straight forward, as you have clear begin and end points
> there.

My guess is, the same issues will apply. I need the SIP dialog handling
there but not the call-stuff.
As soon as I want to send/receive a file to a contact, acceptance pop-ups
are needed again and possibly auto-popup any file transfer window?

Yes, and all this already exists for file transfers! So I very strongly
believe that this will be easier to write and then easier for is to review
and integrate in Jitsi.

So maybe I need more separation between SIP handling a call and/or chat
handling.

Yes, agreed.

As usual, any further hints: much appreciated.

Well, I've been trying to hint ... as hard as I can toward trying FTs first
.. :slight_smile:

But basically opset telephony can do the demuxing. It will handle one way
anything that has audio and video (potentially cancelling chats and FTs in
the same request), passing message media to the msrp op set and handling
FTs a third way.

Cheers,
Emil

--sent from my mobile

···

On May 1, 2013 12:32 PM, "Tom Uijldert" <tom.uijldert@gmail.com> wrote:

> -----Original Message-----
> On 09.04.13, 18:03, Tom Uijldert wrote:
> >> -----Original Message-----

Regards,
    Tom Uijldert.