[sip-comm-dev] handling vnc within sip communicator


#1

Hi all,

Excuse me, I might be a bit clueless with concern to sip communicator.
I'm interested in negotiating vnc sessions, from within voip client.

The scenario I have in mind goes something like this:

A calls B - I want to talk with you, I want to do some_audio, some_video
and vnc chatting with you. the URLS of the media sessions are:
B to A - OK, I can do that .....

Basically something SIP was designed to do. My initial idea is to hook
into sip communicator so that it knows how to handle vnc.

the client part can be adopted from the VNC java applets around the net.
The VNC server part should be native. sip-communicator should be
configured to start or use the appropriate VNC server. Desktop sharing
implemented in java is too much, if you want to stay cross-platform.

From protocol point of view VNC is just another type of media stream,

similar to rtp. It does its own screen (video-stream) encoding, plus
mouse and keyboard event streams. In the SIP family, it is a RTP cousin,
I suppose.

I wonder how to go about implementing this for sip-communicator.
Is ProtocolProvider the appropriate starting point?

If I understand the roadmap correctly, there is planned support for
jabber/xmpp as well. It would be good, if vnc support is independent
from the negotiation mechanism. SIP/Jabber/ something else used to
negotiate the session between the parties, and the RTP & VNC clients
take over afterwards.

Cheers,
Vlado

···

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#2

Hey Vlado,

Excuse me, I might be a bit clueless with concern to sip
communicator. I'm interested in negotiating vnc sessions, from within
voip client.

Excellent idea! Personally, I am very keen on seeing a VNC plugin in SC
and believe many would use it!

I wonder how to go about implementing this for sip-communicator. Is
ProtocolProvider the appropriate starting point?

Yes that would be it. The ProtocolProvider is still under dev and design
and I happen to be the one doing that part. I'll try to put up extensive
documentation as soon as I have a draft for the service that's ready to
be discussed and a at least one working implementation (the first one
would most probably be ICQ/AIM - oscar). Let me try and give you a few
pointers until this happens:

An important thing about the ProtocolProvider service is the notion of
OperationSets. OperationSets are the way we handle the fact that
different protocols offer different features. Every ProtocolProvider
implementation would implement at least one, and generally a few of
these operation sets. An ICQ implementation for example would be
implementing the IM, Presence and FileTransfer operation sets. A SIP
implementation could offer implementations of the Telephony, IM and
Presence OperationSets.

Once instantiated, a ProtocolProvider implementation can be queried for
the OperationSet-s it implements (through the getSupportedOperationSets
method).

If I understand the roadmap correctly, there is planned support for jabber/xmpp as well. It would be good, if vnc support is independent from the negotiation mechanism. SIP/Jabber/ something else used to negotiate the session between the parties, and the RTP & VNC clients take over afterwards.

Indeed. The way I see it, VNC functionality would make for a separate
OperationSet with proposed implementations by the SIP and Jabber
packages. If you are willing to work on this one, you'd have to craft
the OperationSet interface and make it include methods that you believe
necessary for the client part. Once this has been done, you'd have to
extend the sip implementation and provide implementations for these
methods. The same goes for Jabber.

What troubles me in this scheme however is the fact VNC would only be
implementable by modifying an existing PP impl and redistributing the
whole service implementation. I can already see all sorts of parallel
implementations all of which offer intersecting sets of features. (I
believe Miranda for windows suffers similar problems). We'd have to
think of a way for making sure that one could register additional
operation sets to existing PP implementations.

I'll try to take all this into account before I propose a draft of the
PP service for discussion. I hope to have the PP service and its first
implementation ready for Christmas (but people will tell you that I tend
to set utopic deadlines :slight_smile: ). Will do my best anyway!

Cheers
Emil

···

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#3

Hi again,

Once instantiated, a ProtocolProvider implementation can be queried for
the OperationSet-s it implements (through the getSupportedOperationSets
method).

> If I understand the roadmap correctly, there is planned support for
> jabber/xmpp as well. It would be good, if vnc support is independent
> from the negotiation mechanism. SIP/Jabber/ something else used to
> negotiate the session between the parties, and the RTP & VNC clients
> take over afterwards.

Let's say I craft a desktop sharing operation set, having in mind what VNC
can provide. That is fine. The problem I have with this is that we have
several different concerns in the protocols. SIP, Jabber and some of the
rest provide different layered services.

For example:
  session negotiation and establishment (SIP/SDP, jabber (not sure which
    JEPs)
  message exchange SIMPLE
  bidirectional media pipes RTP/RTSP
  some other out-of band data

The media part is handled by JMF and the RTP (not sure how's that done
in java, probably somewhere in java.net, but that is my least concern),
where the SIP PP implements/should implement the wrapping of all related
parts.

Can we construct layered ProtocolProviders -
SIP&Jabber will be able to use and RTP PP for rtp media streams, the VNC
PP for vnc streams, maybe http streams can be used as a firewall
friendly alternative as well.

It could well be that TransportProvider is a better word for this
concept, since the encapsulation of PPs can get really hairy.

What troubles me in this scheme however is the fact VNC would only be
implementable by modifying an existing PP impl and redistributing the
whole service implementation. I can already see all sorts of parallel
implementations all of which offer intersecting sets of features. (I
believe Miranda for windows suffers similar problems). We'd have to
think of a way for making sure that one could register additional
operation sets to existing PP implementations.

What I suggest might provide a work around then, so that if a protocol is
Has an addTransport (or similar) operation in some operation set, we
will be able to add arbitrary transports to SIP, Jabber, or indeed any
other protocol, which provides facilities for negotiating arbitrary
media sessions.

A few more details

for SIP/SDP based:
an sdp description like:
c=IN IP4 224.5.6.7
a=type:H332
m=audio 49230 RTP/AVP 0
m=video 49232 RTP/AVP 31
m=application 12349 udp vnc // the VNC transport mime=application/x-vnc

For Jabber using the out of band JEP-0066:
<iq type='set'
    from='romeo@montague.net/orchard'
    to='juliet@capulet.com/chamber'>
    id='send1'>
  <query xmlns='jabber:iq:oob'
         sid='a0'>
    <url>vnc://romeo.montague.net:12345</url>
    <url>rtp://romeo.montague.net:49230</url>
    <url>rtp://romeo.montague.net:49232</url>
  </query>
</iq>

You can see the similarities between the two. Both the vnc and rtp
streams are of no concerns to the SIP and Jabber stacks, they could
be handled by a separate, reusable sub-systems.

How that translates into sc terms, I don't have an idea, too new to this
and still reading, but hope it gives the general picture from where I am
standing.

Cheers,
Vlado

···

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#4

Hey Vlado,

vlado wrote:

Let's say I craft a desktop sharing operation set, having in mind
what VNC can provide. That is fine. The problem I have with this is
that we have several different concerns in the protocols. SIP, Jabber
and some of the rest provide different layered services.

OperationSet-s are interfaces designed to be independent of the
underlying protocol so this shouldn't be a problem.

session negotiation and establishment (SIP/SDP, jabber (not sure
which JEPs) message exchange SIMPLE bidirectional media
pipes RTP/RTSP some other out-of band data

The media part is handled by JMF and the RTP (not sure how's that
done in java, probably somewhere in java.net, but that is my least
concern), where the SIP PP implements/should implement the wrapping
of all related parts.

The protocol provider is supposed to encapsulate signalling protocols
only (I agree that the name may leed to confusion). RTP and RTSP are
taken care of by implementations of the media service.

Can we construct layered ProtocolProviders - SIP&Jabber will be able
to use and RTP PP for rtp media streams, the VNC PP for vnc streams,
maybe http streams can be used as a firewall friendly alternative as
well.

Well that's actually how things are currently intended to be.
ProtocolProviders only take care of the signalling and instant messaging
part of a protocol. Media Service implementations, notified by protocol
provider originating events and details would be handling media
accordingly. The part of a VNC implementation that would have to handle
the actual data exchange would have to place itself at the same level as
the media service. The VNC operation set implementation would only take
care of generating the right javax.sdp and javax.sip objects (i.e. sip messages and SDP descriptions) based
on parameters such as CallID and CallParticipantID, port number, ip
address and whatever else is necessary.

An implementation of an OperationSetDesktopSharing over SIP would therefore generate the corresponding SDP string while its Jabber impl would produce the JEP-0066 description like the one you quote.

You can see the similarities between the two. Both the vnc and rtp streams are of no concerns to the SIP and Jabber stacks, they could be handled by a separate, reusable sub-systems.

How that translates into sc terms, I don't have an idea, too new to
this and still reading, but hope it gives the general picture from
where I am standing.

Yes that's a good vision and it is quite similar to the way things are (or should I say will be, once finished) done in SC-1.0. With the only difference that data transport protocols do not appear as subsystems of a protocol provider but as parallel OSGI services directly accessible to the GUI (and to any other plugin/bundle for that matter). The reason to have things this way is that various data transportation protocols would require that their own programming interfaces be made available to others (such as the GUI) so there's no straightforward way (if any) of simply hiding them in the PP impl and exporting a common interface.

I hope I am making sense. Let me try an example for the VoIP case. Whenever an incoming call is received, the GUI (and any other interested plugins) would receive an event through the BasicTelephony operation set notifying them of the orignating call. At that point the GUI asks the MediaService to open a session corresponding to the received CallID object. The MediaService would return a MediaDescription object which the GUI, would pass back to the ProtocolProvider through a method like

     OperationSetBasicTelephony
         .answerCall(CallID id, MediaDescription desc)

so that it could extract the necessary details out of the description and generate the corresponding protocol dependant description. For SIP that would be an SDP string and for Jabber (I believe) an XMPP string (I am not familiar with jabber so translate that in whatever it really has to be :wink: ).

I don't know how things work in VNC but a wild guess produces the following analogy:

When receiving an invitation for desktop sharing, the GUI (or some other plugin) would receive the event through the OperationSetDesktopSharing implementation and pass the corresponding CallID (or some other session identification if necessary) to a DesktopSharing service of some kind (analogical to the MediaService). That service would prepare to send and receive data through the network and generate on its own turn a DeskSharingDescription which would then be forwarded back by the GUI to the DesktopSharing operation set in a method like

     OperationSetDesktopSharing
         .acceptSession(CallID id, DeskSharingDescription desc)

Does that or something similar make sense? If I am not being clear (which is probably the case) I may try to draw a diagram, let me know if so.

Cheers
Emil

···

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net