[sip-comm-dev] Another presence issue


#1

Hey guys,

there is another issue concerning SIMPLE presence. The published state
is not explicitely removed from the presence server once we set our
state to offline (i.e., unregister). The subscriptions are removed in
this case, but the published state remains on the server until it
expires (it is correctly set to "closed" in the pidf presence document,
though).

A problem arises if SC is closed and restarted thereafter, and publishes
its new presence (once connected to the SIP server again). In this case,
we get a new etag assigned by the server, so the presence server
contains two contraditory tuples for the same contact address (which
itself is not necessarily a problem yet, and is allowed according to RFC
3863). However, the presence server now (until the old state expires)
sends these two tuples in the notify messages for the same contact
address, one containing the old closed state and another one containing
the open status of our newly started SC instance.

In my tests this results in my SIP account being displayed offline in
other SC clients (in my own client as well, since I've got myself in my
contact list, too). I'm certain though that whether I am shown as online
or offline here only depends on the order in which these tuples occur in
the pidf document.

So now I'm wondering if there shouldn't be an explicit removal of our
publication state (PUBLISH with Expires:0, which AFAIK currently never
happens). For testing purposes, I modified the code so that this
unpublication is done every time we set our status to offline. In case
you're interested, I attached a patch containing this modification. I'm
not sure what you think of this solution though, because this makes the
"closed" state in the pidf <basic> element almost obsolete. On the other
hand, the "closed" state could still make sense for something like an
"invisible" mode, in which our pidf state is set to closed but we're
still registered to the SIP server...

However this is dealt with, I would vote for at least sending an
explicit PUBLISH removal request whenever SC is closed (given that we
did publish our state with a presence server before and this publication
has not expired yet).

Apart from this, the real problem IMO is once more the pidf processing
(hope I'm not bugging you, Ben ;)). The processing there should be
adapted so that the order of (conflicting) tuples doesn't matter
anymore. I think the best solution to deal with this is to collect the
state information of all contacts, tuples, etc. contained in a pidf
document *before* actually promoting any state changes on to the Contact
objects. By this way, we could easily first sort the resulting state
list accordingly (e.g., to prefer an online over an offline status for
conflicting contacts). As a side effect, this would make it very easy to
additionally implement support for the priority attribute of the contact
element.

I did not make any changes to the code reflecting this yet, because I
first wanted to get another opinion.

Ben, WDYT?

@Joel: if you read this - I think that the presence problem you
described earlier today stems from this issue.

Cheers,
Ralph

sip-presence-unpublish.patch (1.83 KB)

···

###########################################
CONFIDENTIALITY: This e-mail and any attachments are confidential and may also be privileged.
If you are not the designated recipient, please notify the sender immediately by reply e-mail and destroy all copies (digital and paper).
Any unauthorized disclosure, distribution, copying, storage or use of the information contained in this e-mail or any attachments is strictly prohibited and may be unlawful.


#2

Hi Ralph.
I think you're right. The type of behavior that you describe is what
I've been experience.
I'm not going to apply your patch right now, I will wait until someone
more experienced than me comment your solution to know if it is the best
way to resolve the problem.

BR,

Joel Silva
Analyst
........................................................................
.....................................
Novabase
Av. Eng. Duarte Pacheco, 15F . 1099-078 Lisboa - Portugal
Tel. (+351) 213 836 300 . Fax (+351) 213 836 301 .
mailto:joel.silva@novabase.pt
www.novabase.pt

···

-----Original Message-----

From: ralph.weires@hitec.lu [mailto:ralph.weires@hitec.lu]

Sent: quarta-feira, 17 de Outubro de 2007 13:31
To: dev@sip-communicator.dev.java.net
Subject: [sip-comm-dev] Another presence issue

Hey guys,

there is another issue concerning SIMPLE presence. The published state
is not explicitely removed from the presence server once we set our
state to offline (i.e., unregister). The subscriptions are removed in
this case, but the published state remains on the server until it
expires (it is correctly set to "closed" in the pidf presence document,
though).

A problem arises if SC is closed and restarted thereafter, and publishes
its new presence (once connected to the SIP server again). In this case,
we get a new etag assigned by the server, so the presence server
contains two contraditory tuples for the same contact address (which
itself is not necessarily a problem yet, and is allowed according to RFC
3863). However, the presence server now (until the old state expires)
sends these two tuples in the notify messages for the same contact
address, one containing the old closed state and another one containing
the open status of our newly started SC instance.

In my tests this results in my SIP account being displayed offline in
other SC clients (in my own client as well, since I've got myself in my
contact list, too). I'm certain though that whether I am shown as online
or offline here only depends on the order in which these tuples occur in
the pidf document.

So now I'm wondering if there shouldn't be an explicit removal of our
publication state (PUBLISH with Expires:0, which AFAIK currently never
happens). For testing purposes, I modified the code so that this
unpublication is done every time we set our status to offline. In case
you're interested, I attached a patch containing this modification. I'm
not sure what you think of this solution though, because this makes the
"closed" state in the pidf <basic> element almost obsolete. On the other
hand, the "closed" state could still make sense for something like an
"invisible" mode, in which our pidf state is set to closed but we're
still registered to the SIP server...

However this is dealt with, I would vote for at least sending an
explicit PUBLISH removal request whenever SC is closed (given that we
did publish our state with a presence server before and this publication
has not expired yet).

Apart from this, the real problem IMO is once more the pidf processing
(hope I'm not bugging you, Ben ;)). The processing there should be
adapted so that the order of (conflicting) tuples doesn't matter
anymore. I think the best solution to deal with this is to collect the
state information of all contacts, tuples, etc. contained in a pidf
document *before* actually promoting any state changes on to the Contact
objects. By this way, we could easily first sort the resulting state
list accordingly (e.g., to prefer an online over an offline status for
conflicting contacts). As a side effect, this would make it very easy to
additionally implement support for the priority attribute of the contact
element.

I did not make any changes to the code reflecting this yet, because I
first wanted to get another opinion.

Ben, WDYT?

@Joel: if you read this - I think that the presence problem you
described earlier today stems from this issue.

Cheers,
Ralph

###########################################
CONFIDENTIALITY: This e-mail and any attachments are confidential and
may also be privileged.
If you are not the designated recipient, please notify the sender
immediately by reply e-mail and destroy all copies (digital and paper).
Any unauthorized disclosure, distribution, copying, storage or use of
the information contained in this e-mail or any attachments is strictly
prohibited and may be unlawful.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#3

Hi Ralph,

Before answering to you, I wanted to tell you that I really love all your mails because you're doing such a wonderful investigation and coding job that you save me probably many hours of work :), so thank you once again for all your work.

ralph.weires@hitec.lu a �crit :

Hey guys,

there is another issue concerning SIMPLE presence. The published state
is not explicitely removed from the presence server once we set our
state to offline (i.e., unregister). The subscriptions are removed in
this case, but the published state remains on the server until it
expires (it is correctly set to "closed" in the pidf presence document,
though).

A problem arises if SC is closed and restarted thereafter, and publishes
its new presence (once connected to the SIP server again). In this case,
we get a new etag assigned by the server, so the presence server
contains two contraditory tuples for the same contact address (which
itself is not necessarily a problem yet, and is allowed according to RFC
3863). However, the presence server now (until the old state expires)
sends these two tuples in the notify messages for the same contact
address, one containing the old closed state and another one containing
the open status of our newly started SC instance.

In my tests this results in my SIP account being displayed offline in
other SC clients (in my own client as well, since I've got myself in my
contact list, too). I'm certain though that whether I am shown as online
or offline here only depends on the order in which these tuples occur in
the pidf document.

So now I'm wondering if there shouldn't be an explicit removal of our
publication state (PUBLISH with Expires:0, which AFAIK currently never
happens). For testing purposes, I modified the code so that this
unpublication is done every time we set our status to offline. In case
you're interested, I attached a patch containing this modification. I'm
not sure what you think of this solution though, because this makes the
"closed" state in the pidf <basic> element almost obsolete. On the other
hand, the "closed" state could still make sense for something like an
"invisible" mode, in which our pidf state is set to closed but we're
still registered to the SIP server...

However this is dealt with, I would vote for at least sending an
explicit PUBLISH removal request whenever SC is closed (given that we
did publish our state with a presence server before and this publication
has not expired yet).
  
This behavior is the result of a thought I had when coding SIMPLE. AFAIK, there is no clear specification of what should happen when we go offline while using a distant PA.
I understand that there is effectively a problem when we disconnect and reconnect as you described above. However something intellectually bother me with the "set an expires to 0" solution:
When a subscription expires, the contact enter in an undermined state which is not really offline, I don't know exactly how a PA can handle this but I'm not sure that all of the implementations will always consider the user as offline. So I decided to simply set the user in the offline state and keep the subscription running because this state must have a time limit.
However there is effectively the drawback you described above which is clearly unacceptable...

As far as I'm writing this, I'm re-reading this: http://tools.ietf.org/html/rfc3903#section-4.5 and it seems that the solution you suggested is more coherent with the rfc philosophy.

WDYT ?

(more inline)

Apart from this, the real problem IMO is once more the pidf processing
(hope I'm not bugging you, Ben ;)).

Absolutely not :slight_smile:

The processing there should be
adapted so that the order of (conflicting) tuples doesn't matter
anymore. I think the best solution to deal with this is to collect the
state information of all contacts, tuples, etc. contained in a pidf
document *before* actually promoting any state changes on to the Contact
objects. By this way, we could easily first sort the resulting state
list accordingly (e.g., to prefer an online over an offline status for
conflicting contacts). As a side effect, this would make it very easy to
additionally implement support for the priority attribute of the contact
element.

I did not make any changes to the code reflecting this yet, because I
first wanted to get another opinion.

Ben, WDYT?
  
It could be cool, effectively, to handle this in a smarter way. However there is the question of what to do when a contact pretends to be offline and online in the same time knowing that a state can be used to describe something absolutely not related to an IM state.
I think that more than preferring a state on an other we should create a new special state, something like "probably online" and use this as a presence state for a contact in a case of many conflicting status.

WDYT ?

Cheers,
Ben.

···

@Joel: if you read this - I think that the presence problem you
described earlier today stems from this issue.

Cheers,
Ralph

###########################################
CONFIDENTIALITY: This e-mail and any attachments are confidential and may also be privileged.
If you are not the designated recipient, please notify the sender immediately by reply e-mail and destroy all copies (digital and paper).
Any unauthorized disclosure, distribution, copying, storage or use of the information contained in this e-mail or any attachments is strictly prohibited and may be unlawful.
------------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#4

Hi Ben,

Before answering to you, I wanted to tell you that I really
love all your mails because you're doing such a wonderful
investigation and coding job that you save me probably many
hours of work :), so thank you once again for all your work.

You're welcome, I'm glad that all the stuff I wrote is understandable
*g*

This behavior is the result of a thought I had when coding SIMPLE.
AFAIK, there is no clear specification of what should happen
when we go offline while using a distant PA.
I understand that there is effectively a problem when we
disconnect and reconnect as you described above. However
something intellectually bother me with the "set an expires
to 0" solution:
When a subscription expires, the contact enter in an
undermined state which is not really offline, I don't know
exactly how a PA can handle this but I'm not sure that all of
the implementations will always consider the user as offline.
So I decided to simply set the user in the offline state and
keep the subscription running because this state must have a
time limit.
However there is effectively the drawback you described above
which is clearly unacceptable...

As far as I'm writing this, I'm re-reading this:
http://tools.ietf.org/html/rfc3903#section-4.5 and it seems
that the solution you suggested is more coherent with the rfc
philosophy.

WDYT ?

I would say since expiration and explicit removal (which is also an
expiration in the end) of an event state is part of the RFC, a proper PA
implementation must be able to deal with this. However, I do not know
how current implementations in fact handle this, with the exception of
OpenSER which I'm using as PA. OpenSER generates an appropriate pidf
document by itself (including a "closed" state) for use in the NOTIFY
messages whenever a publication expires or is explicitely removed.

BTW, the way it currently works also causes another problem which I did
not recognize up to now. Short description:
- If we change our state in SC to offline, this change is published to
the PA (as "closed" state in pidf). The PA stores this state connected
to the current ETag.
- The issued republish task for this publication will be cancelled by
the RegistrationStateChangeListener, so our "closed" presence state will
expire on the PA after the specified expiration time
- If we now set our state to online again *after* our publish expired on
the PA, the PA has already invalidated our last ETag, yet SC sends a
publish modification request to the PA (including the SIP-If-Match
header with the old ETag and a pidf body)
- The PA responds with a 412 Conditional request failed error because
the ETag is not valid anymore
=> This response isn't handled in the code, causing SC to automatically
switch to P2P presence mode

The more I think about it, the more I'm convinced that an explicit
request for removing our state (PUBLISH with Expires: 0) is indeed be
best way whenever we also unregister from the server.

In any case, handling a 412 response to a PUBLISH request should be
added, according to the definition in RFC 3903, section 5 (discard the
old ETag and issue a complete new publication request without Etag
again).

It could be cool, effectively, to handle this in a smarter
way. However there is the question of what to do when a
contact pretends to be offline and online in the same time
knowing that a state can be used to describe something
absolutely not related to an IM state.
I think that more than preferring a state on an other we
should create a new special state, something like "probably
online" and use this as a presence state for a contact in a
case of many conflicting status.

WDYT ?

If there are conflicting tuples, I would actually always prefer any
other given state which is not "closed", no matter which communication
means it specifies. BTW, by "conflicting tuples" I only mean tuples
whose contact elements refer to the same URI *and* have the same
priority. How to handle contacts with the same priority is up to
implementations anyway, according to RFC 3863/4.1.5. I'm not sure if an
additional state would be required to deal with this.

If there should be multiple conflicting tuples which are not "closed"
(e.g. containing different RPID states) - not sure, probably just pick
one of them.

Cheers,
Ralph

···

###########################################
CONFIDENTIALITY: This e-mail and any attachments are confidential and may also be privileged.
If you are not the designated recipient, please notify the sender immediately by reply e-mail and destroy all copies (digital and paper).
Any unauthorized disclosure, distribution, copying, storage or use of the information contained in this e-mail or any attachments is strictly prohibited and may be unlawful.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#5

Hi everyone.
I need a SIP/SIMPLE client source code to perform some tests.
I just want a very simple client that can see other contacts status.
Anyone has one that can share? I know that sip-communicator do this but
I need a simpler client to test a combination of changes to the code
rapidly.

Thanks,
Joel.

···

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#6

Hi Ralph, all,

I commited 3 fixes concerning our previous discussions :
- your patch modified (it discards the ETag even if the OK received concerning a PUBLISH isn't the response to a PUBLISH with an Expires at 0)
- finally, as you reported problems with Wengo and the namespaces in the pidf documents, I decided to implement the fallback to any namespace if no data is found in the pidf namespace
- I added the support of the 412 responses to be more compliant with the rfc.

I'll work on the rewrite of the setPidfPresence method ASAP.

Thanks again Ralph for all your suggestions and for your patch.
Ben.

ralph.weires@hitec.lu a �crit :

···

Hi Ben,

Before answering to you, I wanted to tell you that I really love all your mails because you're doing such a wonderful investigation and coding job that you save me probably many hours of work :), so thank you once again for all your work.
    
You're welcome, I'm glad that all the stuff I wrote is understandable
*g*

This behavior is the result of a thought I had when coding SIMPLE. AFAIK, there is no clear specification of what should happen when we go offline while using a distant PA.
I understand that there is effectively a problem when we disconnect and reconnect as you described above. However something intellectually bother me with the "set an expires to 0" solution:
When a subscription expires, the contact enter in an undermined state which is not really offline, I don't know exactly how a PA can handle this but I'm not sure that all of the implementations will always consider the user as offline. So I decided to simply set the user in the offline state and keep the subscription running because this state must have a time limit.
However there is effectively the drawback you described above which is clearly unacceptable...

As far as I'm writing this, I'm re-reading this: http://tools.ietf.org/html/rfc3903#section-4.5 and it seems that the solution you suggested is more coherent with the rfc philosophy.

WDYT ?
    
I would say since expiration and explicit removal (which is also an
expiration in the end) of an event state is part of the RFC, a proper PA
implementation must be able to deal with this. However, I do not know
how current implementations in fact handle this, with the exception of
OpenSER which I'm using as PA. OpenSER generates an appropriate pidf
document by itself (including a "closed" state) for use in the NOTIFY
messages whenever a publication expires or is explicitely removed.

BTW, the way it currently works also causes another problem which I did
not recognize up to now. Short description:
- If we change our state in SC to offline, this change is published to
the PA (as "closed" state in pidf). The PA stores this state connected
to the current ETag.
- The issued republish task for this publication will be cancelled by
the RegistrationStateChangeListener, so our "closed" presence state will
expire on the PA after the specified expiration time
- If we now set our state to online again *after* our publish expired on
the PA, the PA has already invalidated our last ETag, yet SC sends a
publish modification request to the PA (including the SIP-If-Match
header with the old ETag and a pidf body)
- The PA responds with a 412 Conditional request failed error because
the ETag is not valid anymore
=> This response isn't handled in the code, causing SC to automatically
switch to P2P presence mode

The more I think about it, the more I'm convinced that an explicit
request for removing our state (PUBLISH with Expires: 0) is indeed be
best way whenever we also unregister from the server.

In any case, handling a 412 response to a PUBLISH request should be
added, according to the definition in RFC 3903, section 5 (discard the
old ETag and issue a complete new publication request without Etag
again).

It could be cool, effectively, to handle this in a smarter way. However there is the question of what to do when a contact pretends to be offline and online in the same time knowing that a state can be used to describe something absolutely not related to an IM state.
I think that more than preferring a state on an other we should create a new special state, something like "probably online" and use this as a presence state for a contact in a case of many conflicting status.

WDYT ?
    
If there are conflicting tuples, I would actually always prefer any
other given state which is not "closed", no matter which communication
means it specifies. BTW, by "conflicting tuples" I only mean tuples
whose contact elements refer to the same URI *and* have the same
priority. How to handle contacts with the same priority is up to
implementations anyway, according to RFC 3863/4.1.5. I'm not sure if an
additional state would be required to deal with this.

If there should be multiple conflicting tuples which are not "closed"
(e.g. containing different RPID states) - not sure, probably just pick
one of them.

Cheers,
Ralph

###########################################
CONFIDENTIALITY: This e-mail and any attachments are confidential and may also be privileged.
If you are not the designated recipient, please notify the sender immediately by reply e-mail and destroy all copies (digital and paper).
Any unauthorized disclosure, distribution, copying, storage or use of the information contained in this e-mail or any attachments is strictly prohibited and may be unlawful.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#7

Hello Joel,

Since no one answered this, here's a tip that you may find useful. It
would really be very easy to strip all the things you don't need off of
SIP Communicator. You only need to edit lib/felix.client.run.properties
in order to achieve this, and remove bundles that you are not going to
use. (Some might cause dependency conflicts which would be easy to fix,
but just in case, you may start with removing all protocol
implementations apart from SIP)

Code you need to look at in order to do your tests is located inside

net.java.sip.communicator.impl.protocol.sip.OperationSetPresenceSipImpl.java

and

net.java.sip.communicator.impl.protocol.sip.OperationSetBasicInstantMessagingSipImpl.java

Don't hesitate to come back with any questions that might arise during
your modifications.

Cheers
Emil

Joel Silva wrote:

···

Hi everyone.
I need a SIP/SIMPLE client source code to perform some tests.
I just want a very simple client that can see other contacts status.
Anyone has one that can share? I know that sip-communicator do this but
I need a simpler client to test a combination of changes to the code
rapidly.

Thanks,
Joel.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#8

Thanks Emil!
:wink:

Joel Silva
Analyst
.............................................................................................................
Novabase
Av. Eng. Duarte Pacheco, 15F . 1099-078 Lisboa - Portugal
Tel. (+351) 213 836 300 . Fax (+351) 213 836 301 .
mailto:joel.silva@novabase.pt
www.novabase.pt

···

-----Original Message-----

From: Emil Ivov [mailto:emcho@sip-communicator.org]

Sent: terça-feira, 23 de Outubro de 2007 18:25
To: dev@sip-communicator.dev.java.net
Subject: Re: [sip-comm-dev] SIP/SIMPLE client

Hello Joel,

Since no one answered this, here's a tip that you may find useful. It would really be very easy to strip all the things you don't need off of SIP Communicator. You only need to edit lib/felix.client.run.properties in order to achieve this, and remove bundles that you are not going to use. (Some might cause dependency conflicts which would be easy to fix, but just in case, you may start with removing all protocol implementations apart from SIP)

Code you need to look at in order to do your tests is located inside

net.java.sip.communicator.impl.protocol.sip.OperationSetPresenceSipImpl.java

and

net.java.sip.communicator.impl.protocol.sip.OperationSetBasicInstantMessagingSipImpl.java

Don't hesitate to come back with any questions that might arise during your modifications.

Cheers
Emil

Joel Silva wrote:

Hi everyone.
I need a SIP/SIMPLE client source code to perform some tests.
I just want a very simple client that can see other contacts status.
Anyone has one that can share? I know that sip-communicator do this
but I need a simpler client to test a combination of changes to the
code rapidly.

Thanks,
Joel.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail:
dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net