[sip-comm-dev] Portaudio unstable (was: Some problems when using secure calls)


#1

Damain, all,

during the weekend I did some more tests to get information about
the problems with portaudio.

First my system configuration:
- OpenSuse 11.2 64 bit, Java 1.6 64 bit. I did the tests with both Sun and
  openJDK environment

- AMD dual core processor (may have an effect on thread timing)

Some other results:
Javasound works with the Sun Java environment (not fully tested inside
openJDK). In general no problems using Javasound / Sun Java.

Here the notes regarding the portaudio tests:

When running with ZRTP/SRTP enabled I had no success with portaudio because
of this message:

    [java] Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1186
    [java] Expression 'PaAlsaStreamComponent_Initialize( &self->playback, alsaApi, outParams, StreamDirection_Out, NULL != callback )' failed in
'src/hostapi/alsa/pa_linux_alsa.c', line: 1417
    [java] Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags,
userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1994
    [java] 19:17:56.657 SCHWERWIEGEND: impl.neomedia.notify.PortAudioClipImpl.run().153 Cannot open portaudio device for notifications

This gave me some hint: ZRTP plays a notification sound if the handshake was
successful. This notification sound is of course in parallel to the existing
audio data stream.

To overcome this problem I switched of the notification sound. After that
I was able to have _some_ secured calls under very specific circumstances:
- first set up a call to iptel "music" and listen to the fine music for
  a short time :slight_smile:
- the next call to a Zfone enabled SIP client - I got a secure call

There is no way yet to setup secure calls otherwise, other calls result
in the "device unavailable" error from portaudio C code.

Analysing some more:
When calling iptel's music sets up the audio device with a sample rate
of 8000.0 immediately.
Calling any other SIP client the first setup of the audio device has
a sample rate of 22050.0 (half of 44100 which is a CD sample rate). Only
after receiving the 200 OK the new sample rate 8000.0 is used.
- the main difference here: iptel's music does not send a 180 Ringing but
  a 200 OK immediately (no ringing at the server). It seems that 180 Ringing
  triggers a device setup already.

Looking at the various exceptions and error messages to me it seems to be a
timing problem (thread execution/race condition?). Some thread blocks resources
(audio resources?) because SC never shuts down gracefully but "forced".

During my tests sometimes the whole Java JVM crashes, sometimes with error log,
sometime without. I attach a dump log to this mail.

Regards,
Werner

hs_err_pid27995.log (94.8 KB)


#2

In addition to my previous mail here some more observations:

when I switch off the ringing sound completely then I'm able to
create secure calls. However, usually just one per start of SC
because after the first call some thread blocks further calls

looking a bit more into the Sound events (ringing is one of them)
I saw that at the very end the sound clips use either JavaSoundAudioClip
or SunAudioClip to play the sound. Maybe these implementations reserve
to audio resource for too long (or they block it somehow) so that
portaudio cannot open the audio resource again.

As said, I've a fast dual core CPU :slight_smile: .

I'll keep you updated if I find more.

Regards,
Werner

···

Am 21.12.2009 08:20, schrieb Werner Dittmann:

Damain, all,

during the weekend I did some more tests to get information about
the problems with portaudio.

First my system configuration:
- OpenSuse 11.2 64 bit, Java 1.6 64 bit. I did the tests with both Sun and
  openJDK environment

- AMD dual core processor (may have an effect on thread timing)

Some other results:
Javasound works with the Sun Java environment (not fully tested inside
openJDK). In general no problems using Javasound / Sun Java.

Here the notes regarding the portaudio tests:

When running with ZRTP/SRTP enabled I had no success with portaudio because
of this message:

    [java] Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1186
    [java] Expression 'PaAlsaStreamComponent_Initialize( &self->playback, alsaApi, outParams, StreamDirection_Out, NULL != callback )' failed in
'src/hostapi/alsa/pa_linux_alsa.c', line: 1417
    [java] Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags,
userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1994
    [java] 19:17:56.657 SCHWERWIEGEND: impl.neomedia.notify.PortAudioClipImpl.run().153 Cannot open portaudio device for notifications

This gave me some hint: ZRTP plays a notification sound if the handshake was
successful. This notification sound is of course in parallel to the existing
audio data stream.

To overcome this problem I switched of the notification sound. After that
I was able to have _some_ secured calls under very specific circumstances:
- first set up a call to iptel "music" and listen to the fine music for
  a short time :slight_smile:
- the next call to a Zfone enabled SIP client - I got a secure call

There is no way yet to setup secure calls otherwise, other calls result
in the "device unavailable" error from portaudio C code.

Analysing some more:
When calling iptel's music sets up the audio device with a sample rate
of 8000.0 immediately.
Calling any other SIP client the first setup of the audio device has
a sample rate of 22050.0 (half of 44100 which is a CD sample rate). Only
after receiving the 200 OK the new sample rate 8000.0 is used.
- the main difference here: iptel's music does not send a 180 Ringing but
  a 200 OK immediately (no ringing at the server). It seems that 180 Ringing
  triggers a device setup already.

Looking at the various exceptions and error messages to me it seems to be a
timing problem (thread execution/race condition?). Some thread blocks resources
(audio resources?) because SC never shuts down gracefully but "forced".

During my tests sometimes the whole Java JVM crashes, sometimes with error log,
sometime without. I attach a dump log to this mail.

Regards,
Werner

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#3

Well, I played a bit more with portaudio. This time some tests with the
settings of the portadio devices in the media configuration.

Setting the device to "deafault" often gives a _very_ bad sound quality, bearly
understandable, lot of heavy "vibrato". Also only one "ring" is played,
then playing of the ringtone is shortened to just a very short "bip", sometimes
the microphone part is not switched on (the audio level is always at null)

The good part: I can setup secure call without getting exceptions :slight_smile:

Switching from "default" to another setting (selecting the DAC/ADC channel
of my sound card) gives real good sound, very clear, good overall quality.
However, then I get these nasty exceptions.

One part of these exceptions in case I select the direct channel can be
found at CallPeerSipImpl, processInviteOK(...):

This method first analyses the SDP data (if not early media) and sets up
dev media device. This processing also includes initialization of portaudio.

At the end the method fires the state change to "CONNECTED". Only this state
changes calls the method to stop the ringtone audio clip. Until this call
the Javasound audio clip is in "play" mode and may block audio resources.
As a quick hack I put the state change SDP processing and now I can place
secure calls all the time. But it is not possible to play audio clips
during the call with this setting.

What I can say for sure: playing of audio clips using JavaSoundAudioClip and
at the same time using portaudio with direct channel does not work, at least
not at my system. Using the "default" setting is not yet an option because
of bad sound quality.

In both cases only one or two calls per SC invocation then some thread blocks
(not yet found where).

Regards,
Werner

···

Am 21.12.2009 11:25, schrieb Werner Dittmann:

In addition to my previous mail here some more observations:

when I switch off the ringing sound completely then I'm able to
create secure calls. However, usually just one per start of SC
because after the first call some thread blocks further calls

looking a bit more into the Sound events (ringing is one of them)
I saw that at the very end the sound clips use either JavaSoundAudioClip
or SunAudioClip to play the sound. Maybe these implementations reserve
to audio resource for too long (or they block it somehow) so that
portaudio cannot open the audio resource again.

As said, I've a fast dual core CPU :slight_smile: .

I'll keep you updated if I find more.

Regards,
Werner

Am 21.12.2009 08:20, schrieb Werner Dittmann:

Damain, all,

during the weekend I did some more tests to get information about
the problems with portaudio.

First my system configuration:
- OpenSuse 11.2 64 bit, Java 1.6 64 bit. I did the tests with both Sun and
  openJDK environment

- AMD dual core processor (may have an effect on thread timing)

Some other results:
Javasound works with the Sun Java environment (not fully tested inside
openJDK). In general no problems using Javasound / Sun Java.

Here the notes regarding the portaudio tests:

When running with ZRTP/SRTP enabled I had no success with portaudio because
of this message:

    [java] Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1186
    [java] Expression 'PaAlsaStreamComponent_Initialize( &self->playback, alsaApi, outParams, StreamDirection_Out, NULL != callback )' failed in
'src/hostapi/alsa/pa_linux_alsa.c', line: 1417
    [java] Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags,
userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1994
    [java] 19:17:56.657 SCHWERWIEGEND: impl.neomedia.notify.PortAudioClipImpl.run().153 Cannot open portaudio device for notifications

This gave me some hint: ZRTP plays a notification sound if the handshake was
successful. This notification sound is of course in parallel to the existing
audio data stream.

To overcome this problem I switched of the notification sound. After that
I was able to have _some_ secured calls under very specific circumstances:
- first set up a call to iptel "music" and listen to the fine music for
  a short time :slight_smile:
- the next call to a Zfone enabled SIP client - I got a secure call

There is no way yet to setup secure calls otherwise, other calls result
in the "device unavailable" error from portaudio C code.

Analysing some more:
When calling iptel's music sets up the audio device with a sample rate
of 8000.0 immediately.
Calling any other SIP client the first setup of the audio device has
a sample rate of 22050.0 (half of 44100 which is a CD sample rate). Only
after receiving the 200 OK the new sample rate 8000.0 is used.
- the main difference here: iptel's music does not send a 180 Ringing but
  a 200 OK immediately (no ringing at the server). It seems that 180 Ringing
  triggers a device setup already.

Looking at the various exceptions and error messages to me it seems to be a
timing problem (thread execution/race condition?). Some thread blocks resources
(audio resources?) because SC never shuts down gracefully but "forced".

During my tests sometimes the whole Java JVM crashes, sometimes with error log,
sometime without. I attach a dump log to this mail.

Regards,
Werner

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#4

Hi Werner,

However, usually just one per start of SC
because after the first call some thread blocks further calls

Could I please ask you to provide thread dumps when you report
blocking? I usually create them with jstack <PID> after I look up the
PID of SC with jps.

I saw that at the very end the sound clips use either JavaSoundAudioClip
or SunAudioClip to play the sound.

Well... our intention is to use PortAudioClipImpl for the
notifications when PortAudio is used for capture and playback for the
same reason for using JavaSound and PortAudio at one and the same time
isn't possible. In other words, if it does happen that JavaSound and
PortAudio get simultaneously used, it is a bug and we have to fix it.

Best regards,
Lubomir

···

On Mon, Dec 21, 2009 at 12:25 PM, Werner Dittmann <Werner.Dittmann@t-online.de> wrote:

In addition to my previous mail here some more observations:

when I switch off the ringing sound completely then I'm able to
create secure calls. However, usually just one per start of SC
because after the first call some thread blocks further calls

looking a bit more into the Sound events (ringing is one of them)
I saw that at the very end the sound clips use either JavaSoundAudioClip
or SunAudioClip to play the sound. Maybe these implementations reserve
to audio resource for too long (or they block it somehow) so that
portaudio cannot open the audio resource again.

As said, I've a fast dual core CPU :slight_smile: .

I'll keep you updated if I find more.

Regards,
Werner

Am 21.12.2009 08:20, schrieb Werner Dittmann:

Damain, all,

during the weekend I did some more tests to get information about
the problems with portaudio.

First my system configuration:
- OpenSuse 11.2 64 bit, Java 1.6 64 bit. I did the tests with both Sun and
openJDK environment

- AMD dual core processor (may have an effect on thread timing)

Some other results:
Javasound works with the Sun Java environment (not fully tested inside
openJDK). In general no problems using Javasound / Sun Java.

Here the notes regarding the portaudio tests:

When running with ZRTP/SRTP enabled I had no success with portaudio because
of this message:

\[java\] Expression &#39;AlsaOpen\( &amp;alsaApi\-&gt;baseHostApiRep, params, streamDir, &amp;self\-&gt;pcm \)&#39; failed in &#39;src/hostapi/alsa/pa\_linux\_alsa\.c&#39;, line: 1186
\[java\] Expression &#39;PaAlsaStreamComponent\_Initialize\( &amp;self\-&gt;playback, alsaApi, outParams, StreamDirection\_Out, NULL \!= callback \)&#39; failed in

'src/hostapi/alsa/pa_linux_alsa.c', line: 1417
[java] Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags,
userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1994
[java] 19:17:56.657 SCHWERWIEGEND: impl.neomedia.notify.PortAudioClipImpl.run().153 Cannot open portaudio device for notifications

This gave me some hint: ZRTP plays a notification sound if the handshake was
successful. This notification sound is of course in parallel to the existing
audio data stream.

To overcome this problem I switched of the notification sound. After that
I was able to have _some_ secured calls under very specific circumstances:
- first set up a call to iptel "music" and listen to the fine music for
a short time :slight_smile:
- the next call to a Zfone enabled SIP client - I got a secure call

There is no way yet to setup secure calls otherwise, other calls result
in the "device unavailable" error from portaudio C code.

Analysing some more:
When calling iptel's music sets up the audio device with a sample rate
of 8000.0 immediately.
Calling any other SIP client the first setup of the audio device has
a sample rate of 22050.0 (half of 44100 which is a CD sample rate). Only
after receiving the 200 OK the new sample rate 8000.0 is used.
- the main difference here: iptel's music does not send a 180 Ringing but
a 200 OK immediately (no ringing at the server). It seems that 180 Ringing
triggers a device setup already.

Looking at the various exceptions and error messages to me it seems to be a
timing problem (thread execution/race condition?). Some thread blocks resources
(audio resources?) because SC never shuts down gracefully but "forced".

During my tests sometimes the whole Java JVM crashes, sometimes with error log,
sometime without. I attach a dump log to this mail.

Regards,
Werner

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#5

Lubo,

seems that this is a bug. Looking at SC's source tree we have 2
independent audionotification implementations and the Notification
service found in "net.java.sip.communicator.impl.notification " uses
the old media stuff as far as I can see. The neomedia audio notifier
implementation is not registered via an activator in the same
way as the old audio notifier is. Thus the notification service
finds the old one only.

Regards,
Werner

···

Am 21.12.2009 13:15, schrieb Lubomir Marinov:

Hi Werner,

However, usually just one per start of SC
because after the first call some thread blocks further calls

Could I please ask you to provide thread dumps when you report
blocking? I usually create them with jstack <PID> after I look up the
PID of SC with jps.

I saw that at the very end the sound clips use either JavaSoundAudioClip
or SunAudioClip to play the sound.

Well... our intention is to use PortAudioClipImpl for the
notifications when PortAudio is used for capture and playback for the
same reason for using JavaSound and PortAudio at one and the same time
isn't possible. In other words, if it does happen that JavaSound and
PortAudio get simultaneously used, it is a bug and we have to fix it.

Best regards,
Lubomir

On Mon, Dec 21, 2009 at 12:25 PM, Werner Dittmann > <Werner.Dittmann@t-online.de> wrote:

In addition to my previous mail here some more observations:

when I switch off the ringing sound completely then I'm able to
create secure calls. However, usually just one per start of SC
because after the first call some thread blocks further calls

looking a bit more into the Sound events (ringing is one of them)
I saw that at the very end the sound clips use either JavaSoundAudioClip
or SunAudioClip to play the sound. Maybe these implementations reserve
to audio resource for too long (or they block it somehow) so that
portaudio cannot open the audio resource again.

As said, I've a fast dual core CPU :slight_smile: .

I'll keep you updated if I find more.

Regards,
Werner

Am 21.12.2009 08:20, schrieb Werner Dittmann:

Damain, all,

during the weekend I did some more tests to get information about
the problems with portaudio.

First my system configuration:
- OpenSuse 11.2 64 bit, Java 1.6 64 bit. I did the tests with both Sun and
  openJDK environment

- AMD dual core processor (may have an effect on thread timing)

Some other results:
Javasound works with the Sun Java environment (not fully tested inside
openJDK). In general no problems using Javasound / Sun Java.

Here the notes regarding the portaudio tests:

When running with ZRTP/SRTP enabled I had no success with portaudio because
of this message:

    [java] Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1186
    [java] Expression 'PaAlsaStreamComponent_Initialize( &self->playback, alsaApi, outParams, StreamDirection_Out, NULL != callback )' failed in
'src/hostapi/alsa/pa_linux_alsa.c', line: 1417
    [java] Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags,
userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1994
    [java] 19:17:56.657 SCHWERWIEGEND: impl.neomedia.notify.PortAudioClipImpl.run().153 Cannot open portaudio device for notifications

This gave me some hint: ZRTP plays a notification sound if the handshake was
successful. This notification sound is of course in parallel to the existing
audio data stream.

To overcome this problem I switched of the notification sound. After that
I was able to have _some_ secured calls under very specific circumstances:
- first set up a call to iptel "music" and listen to the fine music for
  a short time :slight_smile:
- the next call to a Zfone enabled SIP client - I got a secure call

There is no way yet to setup secure calls otherwise, other calls result
in the "device unavailable" error from portaudio C code.

Analysing some more:
When calling iptel's music sets up the audio device with a sample rate
of 8000.0 immediately.
Calling any other SIP client the first setup of the audio device has
a sample rate of 22050.0 (half of 44100 which is a CD sample rate). Only
after receiving the 200 OK the new sample rate 8000.0 is used.
- the main difference here: iptel's music does not send a 180 Ringing but
  a 200 OK immediately (no ringing at the server). It seems that 180 Ringing
  triggers a device setup already.

Looking at the various exceptions and error messages to me it seems to be a
timing problem (thread execution/race condition?). Some thread blocks resources
(audio resources?) because SC never shuts down gracefully but "forced".

During my tests sometimes the whole Java JVM crashes, sometimes with error log,
sometime without. I attach a dump log to this mail.

Regards,
Werner

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#6

Hi,

The old notify service(audionotifier.jar) is no longer build, run and
no longer distributed with the application. Maybe its in your felix
cache and although its missing from felix.client.run.properties its
loaded from felix(sounds like a bug) and as there are two notify
services are registered it seems the old one is the one that is used.
You can try cleaning your felix cache: delete folder
"~/.sip-communicator/sip-communicator.bin" and try again.

damencho

···

On Mon, Dec 21, 2009 at 2:51 PM, Werner Dittmann <Werner.Dittmann@t-online.de> wrote:

Lubo,

seems that this is a bug. Looking at SC's source tree we have 2
independent audionotification implementations and the Notification
service found in "net.java.sip.communicator.impl.notification " uses
the old media stuff as far as I can see. The neomedia audio notifier
implementation is not registered via an activator in the same
way as the old audio notifier is. Thus the notification service
finds the old one only.

Regards,
Werner

Am 21.12.2009 13:15, schrieb Lubomir Marinov:

Hi Werner,

However, usually just one per start of SC
because after the first call some thread blocks further calls

Could I please ask you to provide thread dumps when you report
blocking? I usually create them with jstack <PID> after I look up the
PID of SC with jps.

I saw that at the very end the sound clips use either JavaSoundAudioClip
or SunAudioClip to play the sound.

Well... our intention is to use PortAudioClipImpl for the
notifications when PortAudio is used for capture and playback for the
same reason for using JavaSound and PortAudio at one and the same time
isn't possible. In other words, if it does happen that JavaSound and
PortAudio get simultaneously used, it is a bug and we have to fix it.

Best regards,
Lubomir

On Mon, Dec 21, 2009 at 12:25 PM, Werner Dittmann >> <Werner.Dittmann@t-online.de> wrote:

In addition to my previous mail here some more observations:

when I switch off the ringing sound completely then I'm able to
create secure calls. However, usually just one per start of SC
because after the first call some thread blocks further calls

looking a bit more into the Sound events (ringing is one of them)
I saw that at the very end the sound clips use either JavaSoundAudioClip
or SunAudioClip to play the sound. Maybe these implementations reserve
to audio resource for too long (or they block it somehow) so that
portaudio cannot open the audio resource again.

As said, I've a fast dual core CPU :slight_smile: .

I'll keep you updated if I find more.

Regards,
Werner

Am 21.12.2009 08:20, schrieb Werner Dittmann:

Damain, all,

during the weekend I did some more tests to get information about
the problems with portaudio.

First my system configuration:
- OpenSuse 11.2 64 bit, Java 1.6 64 bit. I did the tests with both Sun and
openJDK environment

- AMD dual core processor (may have an effect on thread timing)

Some other results:
Javasound works with the Sun Java environment (not fully tested inside
openJDK). In general no problems using Javasound / Sun Java.

Here the notes regarding the portaudio tests:

When running with ZRTP/SRTP enabled I had no success with portaudio because
of this message:

\[java\] Expression &#39;AlsaOpen\( &amp;alsaApi\-&gt;baseHostApiRep, params, streamDir, &amp;self\-&gt;pcm \)&#39; failed in &#39;src/hostapi/alsa/pa\_linux\_alsa\.c&#39;, line: 1186
\[java\] Expression &#39;PaAlsaStreamComponent\_Initialize\( &amp;self\-&gt;playback, alsaApi, outParams, StreamDirection\_Out, NULL \!= callback \)&#39; failed in

'src/hostapi/alsa/pa_linux_alsa.c', line: 1417
[java] Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags,
userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1994
[java] 19:17:56.657 SCHWERWIEGEND: impl.neomedia.notify.PortAudioClipImpl.run().153 Cannot open portaudio device for notifications

This gave me some hint: ZRTP plays a notification sound if the handshake was
successful. This notification sound is of course in parallel to the existing
audio data stream.

To overcome this problem I switched of the notification sound. After that
I was able to have _some_ secured calls under very specific circumstances:
- first set up a call to iptel "music" and listen to the fine music for
a short time :slight_smile:
- the next call to a Zfone enabled SIP client - I got a secure call

There is no way yet to setup secure calls otherwise, other calls result
in the "device unavailable" error from portaudio C code.

Analysing some more:
When calling iptel's music sets up the audio device with a sample rate
of 8000.0 immediately.
Calling any other SIP client the first setup of the audio device has
a sample rate of 22050.0 (half of 44100 which is a CD sample rate). Only
after receiving the 200 OK the new sample rate 8000.0 is used.
- the main difference here: iptel's music does not send a 180 Ringing but
a 200 OK immediately (no ringing at the server). It seems that 180 Ringing
triggers a device setup already.

Looking at the various exceptions and error messages to me it seems to be a
timing problem (thread execution/race condition?). Some thread blocks resources
(audio resources?) because SC never shuts down gracefully but "forced".

During my tests sometimes the whole Java JVM crashes, sometimes with error log,
sometime without. I attach a dump log to this mail.

Regards,
Werner

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#7

Damian,

you are right - I was misguided on that.

However, that doesn't change anything. I cross-checked that portaudio
is used (the neomedia notification) but the symptoms remain.

I can do some more tests when I exchange the processing in "processInviteOK"
as decribed in an earlier post. Thus first stop the clip processing,
then process the SDP and setup the audio device with new parameters.
Somehow even the PortAudioClipImpl blocks the device in case I select
the direct seeting (not default) on my sound card.
Portaudio default setting still has some problems with sound, ringtone,
and micro.

Regards,
Werner

PS: thanks for the hint with jstack/jps - I already created a dump, will
verify it and send to you guys.

W.

···

Am 21.12.2009 14:04, schrieb Damian Minkov:

Hi,

The old notify service(audionotifier.jar) is no longer build, run and
no longer distributed with the application. Maybe its in your felix
cache and although its missing from felix.client.run.properties its
loaded from felix(sounds like a bug) and as there are two notify
services are registered it seems the old one is the one that is used.
You can try cleaning your felix cache: delete folder
"~/.sip-communicator/sip-communicator.bin" and try again.

damencho

On Mon, Dec 21, 2009 at 2:51 PM, Werner Dittmann > <Werner.Dittmann@t-online.de> wrote:

Lubo,

seems that this is a bug. Looking at SC's source tree we have 2
independent audionotification implementations and the Notification
service found in "net.java.sip.communicator.impl.notification " uses
the old media stuff as far as I can see. The neomedia audio notifier
implementation is not registered via an activator in the same
way as the old audio notifier is. Thus the notification service
finds the old one only.

Regards,
Werner

Am 21.12.2009 13:15, schrieb Lubomir Marinov:

Hi Werner,

However, usually just one per start of SC
because after the first call some thread blocks further calls

Could I please ask you to provide thread dumps when you report
blocking? I usually create them with jstack <PID> after I look up the
PID of SC with jps.

I saw that at the very end the sound clips use either JavaSoundAudioClip
or SunAudioClip to play the sound.

Well... our intention is to use PortAudioClipImpl for the
notifications when PortAudio is used for capture and playback for the
same reason for using JavaSound and PortAudio at one and the same time
isn't possible. In other words, if it does happen that JavaSound and
PortAudio get simultaneously used, it is a bug and we have to fix it.

Best regards,
Lubomir

On Mon, Dec 21, 2009 at 12:25 PM, Werner Dittmann >>> <Werner.Dittmann@t-online.de> wrote:

In addition to my previous mail here some more observations:

when I switch off the ringing sound completely then I'm able to
create secure calls. However, usually just one per start of SC
because after the first call some thread blocks further calls

looking a bit more into the Sound events (ringing is one of them)
I saw that at the very end the sound clips use either JavaSoundAudioClip
or SunAudioClip to play the sound. Maybe these implementations reserve
to audio resource for too long (or they block it somehow) so that
portaudio cannot open the audio resource again.

As said, I've a fast dual core CPU :slight_smile: .

I'll keep you updated if I find more.

Regards,
Werner

Am 21.12.2009 08:20, schrieb Werner Dittmann:

Damain, all,

during the weekend I did some more tests to get information about
the problems with portaudio.

First my system configuration:
- OpenSuse 11.2 64 bit, Java 1.6 64 bit. I did the tests with both Sun and
  openJDK environment

- AMD dual core processor (may have an effect on thread timing)

Some other results:
Javasound works with the Sun Java environment (not fully tested inside
openJDK). In general no problems using Javasound / Sun Java.

Here the notes regarding the portaudio tests:

When running with ZRTP/SRTP enabled I had no success with portaudio because
of this message:

    [java] Expression 'AlsaOpen( &alsaApi->baseHostApiRep, params, streamDir, &self->pcm )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1186
    [java] Expression 'PaAlsaStreamComponent_Initialize( &self->playback, alsaApi, outParams, StreamDirection_Out, NULL != callback )' failed in
'src/hostapi/alsa/pa_linux_alsa.c', line: 1417
    [java] Expression 'PaAlsaStream_Initialize( stream, alsaHostApi, inputParameters, outputParameters, sampleRate, framesPerBuffer, callback, streamFlags,
userData )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1994
    [java] 19:17:56.657 SCHWERWIEGEND: impl.neomedia.notify.PortAudioClipImpl.run().153 Cannot open portaudio device for notifications

This gave me some hint: ZRTP plays a notification sound if the handshake was
successful. This notification sound is of course in parallel to the existing
audio data stream.

To overcome this problem I switched of the notification sound. After that
I was able to have _some_ secured calls under very specific circumstances:
- first set up a call to iptel "music" and listen to the fine music for
  a short time :slight_smile:
- the next call to a Zfone enabled SIP client - I got a secure call

There is no way yet to setup secure calls otherwise, other calls result
in the "device unavailable" error from portaudio C code.

Analysing some more:
When calling iptel's music sets up the audio device with a sample rate
of 8000.0 immediately.
Calling any other SIP client the first setup of the audio device has
a sample rate of 22050.0 (half of 44100 which is a CD sample rate). Only
after receiving the 200 OK the new sample rate 8000.0 is used.
- the main difference here: iptel's music does not send a 180 Ringing but
  a 200 OK immediately (no ringing at the server). It seems that 180 Ringing
  triggers a device setup already.

Looking at the various exceptions and error messages to me it seems to be a
timing problem (thread execution/race condition?). Some thread blocks resources
(audio resources?) because SC never shuts down gracefully but "forced".

During my tests sometimes the whole Java JVM crashes, sometimes with error log,
sometime without. I attach a dump log to this mail.

Regards,
Werner

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#8

Attached are two dumps produced with jstack that show thread block.
Two synchronized methods in
net.java.sip.communicator.impl.neomedia.portaudio.streams.MasterPortAudioStream
lock each other (stop() and read() )

I've looked at the circustances: in both case it seams that not input
data is produced (from the capture device, microphone) and thus the read
hangs. I've checked this behaviour with wireshark: during the first
several second SC send audio dat to the other client, then no data is
sent anymore, seems SC does not capture any data anymore. This happens
with default setting as well as for direct settings.

Haven't seen yet which thread switches off the microphone after some seconds.

Regards,
Werner

block-with-direct.dat (32.2 KB)

block-with-default.dat (31.5 KB)

···

Am 21.12.2009 14:53, schrieb Werner Dittmann:

Damian,

you are right - I was misguided on that.

However, that doesn't change anything. I cross-checked that portaudio
is used (the neomedia notification) but the symptoms remain.

I can do some more tests when I exchange the processing in "processInviteOK"
as decribed in an earlier post. Thus first stop the clip processing,
then process the SDP and setup the audio device with new parameters.
Somehow even the PortAudioClipImpl blocks the device in case I select
the direct seeting (not default) on my sound card.
Portaudio default setting still has some problems with sound, ringtone,
and micro.

Regards,
Werner

PS: thanks for the hint with jstack/jps - I already created a dump, will
verify it and send to you guys.

W.

Am 21.12.2009 14:04, schrieb Damian Minkov:

Hi,

<SNIP ---- SNAP>


#9

Hi,

can you try testing the latest build(2305) or latest revision does the
problem with the hang still exists.

Thanks
damencho

···

On Mon, Dec 21, 2009 at 4:57 PM, Werner Dittmann <Werner.Dittmann@t-online.de> wrote:

Attached are two dumps produced with jstack that show thread block.
Two synchronized methods in
net.java.sip.communicator.impl.neomedia.portaudio.streams.MasterPortAudioStream
lock each other (stop() and read() )

I've looked at the circustances: in both case it seams that not input
data is produced (from the capture device, microphone) and thus the read
hangs. I've checked this behaviour with wireshark: during the first
several second SC send audio dat to the other client, then no data is
sent anymore, seems SC does not capture any data anymore. This happens
with default setting as well as for direct settings.

Haven't seen yet which thread switches off the microphone after some seconds.

Regards,
Werner

Am 21.12.2009 14:53, schrieb Werner Dittmann:

Damian,

you are right - I was misguided on that.

However, that doesn't change anything. I cross-checked that portaudio
is used (the neomedia notification) but the symptoms remain.

I can do some more tests when I exchange the processing in "processInviteOK"
as decribed in an earlier post. Thus first stop the clip processing,
then process the SDP and setup the audio device with new parameters.
Somehow even the PortAudioClipImpl blocks the device in case I select
the direct seeting (not default) on my sound card.
Portaudio default setting still has some problems with sound, ringtone,
and micro.

Regards,
Werner

PS: thanks for the hint with jstack/jps - I already created a dump, will
verify it and send to you guys.

W.

Am 21.12.2009 14:04, schrieb Damian Minkov:

Hi,

<SNIP ---- SNAP>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#10

Damian,

I'm just testing the whole stuff again. First results: nothing
changed so far :frowning: - ringtone plays just once, the short "bips"
every 3-4 seconds. Audio quality in portaudio default setting
is not sufficent. Also the thread blocking stays. Currently I'm
trying to get a grip on this when using "default" setting. Also
the capturing stops after some seconds - thus the thread block.

Another SIP client that uses Alsa default setting works ok.

Using direct settings (seleting the DAC/ADC devices) result in
even worse effects, maybe because of the following exception:

2:01:39.945 SCHWERWIEGEND: util.UtilActivator.uncaughtException().80 An uncaught exception occurred in thread=Thread[JMF thread:
com.sun.media.ProcessEngine@1599f755 ( prefetchThread),9,system] and message was: null
java.lang.NullPointerException
        at net.java.sip.communicator.impl.neomedia.jmfext.media.renderer.audio.PortAudioRenderer.stop(PortAudioRenderer.java:119)
        at net.java.sip.communicator.impl.neomedia.jmfext.media.renderer.audio.PortAudioRenderer.close(PortAudioRenderer.java:195)
        at com.sun.media.BasicRendererModule.doFailedPrefetch(BasicRendererModule.java:186)
        at com.sun.media.PlaybackEngine.doFailedPrefetch(PlaybackEngine.java:760)
        at com.sun.media.PrefetchWorkThread.failed(BasicController.java:1442)
        at com.sun.media.StateTransitionWorkThread.run(BasicController.java:1346)
12:03:52.235 SCHWERWIEGEND: impl.shutdowntimeout.ShutdownTimeout.run().78 Failed to gently shutdown. Forcin

As I've seen you modified "PortAudioClipImpl" to call
                         portAudioStream.stop();
+ portAudioStream.close();

and you modified OutputPortAudioStream.close() to call stop also. Maybe this "double" stop
causes to above exception.

Regards,
Werner

···

Am 21.12.2009 17:03, schrieb Damian Minkov:

Hi,

can you try testing the latest build(2305) or latest revision does the
problem with the hang still exists.

Thanks
damencho

On Mon, Dec 21, 2009 at 4:57 PM, Werner Dittmann > <Werner.Dittmann@t-online.de> wrote:

Attached are two dumps produced with jstack that show thread block.
Two synchronized methods in
net.java.sip.communicator.impl.neomedia.portaudio.streams.MasterPortAudioStream
lock each other (stop() and read() )

I've looked at the circustances: in both case it seams that not input
data is produced (from the capture device, microphone) and thus the read
hangs. I've checked this behaviour with wireshark: during the first
several second SC send audio dat to the other client, then no data is
sent anymore, seems SC does not capture any data anymore. This happens
with default setting as well as for direct settings.

Haven't seen yet which thread switches off the microphone after some seconds.

Regards,
Werner

Am 21.12.2009 14:53, schrieb Werner Dittmann:

Damian,

you are right - I was misguided on that.

However, that doesn't change anything. I cross-checked that portaudio
is used (the neomedia notification) but the symptoms remain.

I can do some more tests when I exchange the processing in "processInviteOK"
as decribed in an earlier post. Thus first stop the clip processing,
then process the SDP and setup the audio device with new parameters.
Somehow even the PortAudioClipImpl blocks the device in case I select
the direct seeting (not default) on my sound card.
Portaudio default setting still has some problems with sound, ringtone,
and micro.

Regards,
Werner

PS: thanks for the hint with jstack/jps - I already created a dump, will
verify it and send to you guys.

W.

Am 21.12.2009 14:04, schrieb Damian Minkov:

Hi,

<SNIP ---- SNAP>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#11

Hello to all

I tried the 1309 build with debian, in order to try out the portaudio.

Here are my brief comments:

Using "default" devices results in very poor sound quality.
There are several other combinations that provide better/good sound
quality but I am still looking for the best combination. Ill give u
feedback on this later.

Never the less, I can only place one single SIP call. SC blocks after
hanging up (only the call button), and sometimes it even crashes.

The good news are that the huge time delay in sound
processing/transmission during a call was eliminated.

Carlos

···

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#12

All,

IMHO I found the root cause for the portaudio problems. It is
located in the C source (the JNI part) that causes a buffer
overflow. The offending code in ReadStream:

...
        if(inStream->inputResampleFactor != 1)
        {
            spx_uint32_t in_len;
            in_len = lrint(
                frames
                *inStream->inputChannelCount
                *inStream->inputResampleFactor);

            short res[in_len];

            errorCode = Pa_ReadStream(
                inStream->stream,
                res,
                frames*inStream->inputResampleFactor);

...

The resample factor is >5 (assuming a sample of 8000). the number
of frames to read is multiplied with this number. The Java code
creates a too small buffer:

new byte[PortAudioManager.NUM_SAMPLES*frameSize];

where frameSize is 2, NUM_SAMPLES is 160, resulting in 320 bytes.
The C code multiplies NUM_SAMPLES (frames in the C code) with
the resampleFactore which results in 160 * 5.5 which is ~1000.

The same problem is also in writeStream. The Java code hands over
a too small buffer, also writeStream multiplies to buffersize with
a resampleFactor. This results in the bad audio quality.

Regards,
Werner

···

Am 22.12.2009 13:27, schrieb Werner Dittmann:

Damian,

I'm just testing the whole stuff again. First results: nothing
changed so far :frowning: - ringtone plays just once, the short "bips"
every 3-4 seconds. Audio quality in portaudio default setting
is not sufficent. Also the thread blocking stays. Currently I'm
trying to get a grip on this when using "default" setting. Also
the capturing stops after some seconds - thus the thread block.

Another SIP client that uses Alsa default setting works ok.

Using direct settings (seleting the DAC/ADC devices) result in
even worse effects, maybe because of the following exception:

2:01:39.945 SCHWERWIEGEND: util.UtilActivator.uncaughtException().80 An uncaught exception occurred in thread=Thread[JMF thread:
com.sun.media.ProcessEngine@1599f755 ( prefetchThread),9,system] and message was: null
java.lang.NullPointerException
        at net.java.sip.communicator.impl.neomedia.jmfext.media.renderer.audio.PortAudioRenderer.stop(PortAudioRenderer.java:119)
        at net.java.sip.communicator.impl.neomedia.jmfext.media.renderer.audio.PortAudioRenderer.close(PortAudioRenderer.java:195)
        at com.sun.media.BasicRendererModule.doFailedPrefetch(BasicRendererModule.java:186)
        at com.sun.media.PlaybackEngine.doFailedPrefetch(PlaybackEngine.java:760)
        at com.sun.media.PrefetchWorkThread.failed(BasicController.java:1442)
        at com.sun.media.StateTransitionWorkThread.run(BasicController.java:1346)
12:03:52.235 SCHWERWIEGEND: impl.shutdowntimeout.ShutdownTimeout.run().78 Failed to gently shutdown. Forcin

As I've seen you modified "PortAudioClipImpl" to call
                         portAudioStream.stop();
+ portAudioStream.close();

and you modified OutputPortAudioStream.close() to call stop also. Maybe this "double" stop
causes to above exception.

Regards,
Werner

<SNIP --- SNAP>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#13

All,

just before going to X-mas I have tracked down this issue (at least for
my OpenSuse 11.2 system). This is a problem in the portaudio <-> ALSA
interface, not in SC.

I also had very poor sound here. After fixing the problem in portaudio
the sound quality is now very good.

portaudio uses a specific call to synchronize the Alsa buffer with its
own buffers. This call did only snychronize the internal Alsa buffer,
not goin down to the HW. This gave a wrong counter of "frames available
to play". Somehow the internal Alsa buffers are out of sync with the
HW.

I replaced this call with the Alsa call that also gets the HW buffer
counters and synchronizes all counters.

Regards,
Werner

···

Am 22.12.2009 18:13, schrieb Carlos Alexandre:

Hello to all

I tried the 1309 build with debian, in order to try out the portaudio.

Here are my brief comments:

Using "default" devices results in very poor sound quality.
There are several other combinations that provide better/good sound
quality but I am still looking for the best combination. Ill give u
feedback on this later.

Never the less, I can only place one single SIP call. SC blocks after
hanging up (only the call button), and sometimes it even crashes.

The good news are that the huge time delay in sound
processing/transmission during a call was eliminated.

Carlos

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#14

Damian, all,

Damian,

I'm just testing the whole stuff again. First results: nothing
changed so far :frowning: - ringtone plays just once, the short "bips"
every 3-4 seconds. Audio quality in portaudio default setting
is not sufficent.

This is solved on my system after fixing a problem in the portaudio
library (see my mail just sent a minute ago)

Also the thread blocking stays. Currently I'm

trying to get a grip on this when using "default" setting. Also
the capturing stops after some seconds - thus the thread block.

Not yet solved, but this seems to be a portaudio problem too because
also the portaudio test program (patest_record) stops to get input
from the capture device.

I'll try to track that down after the holidays :slight_smile:

Best regards,
Werner

···

Am 22.12.2009 13:27, schrieb Werner Dittmann:

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#15

All,

Sorry - was a bit too early :frowning: . I've overlooked the allocation
of temporary buffers to hold the data.

However, testing with portaudio very often also results in
real crashes, always segmentation vialotaion (SIGSEGV) detected
in libasound.so. Thus somehow memory was corrupted some time
before. This happens mostly when I hangup a call after the
audio input (reading from microphone) stopped.

Regards,
Werner

···

Am 23.12.2009 10:12, schrieb Werner Dittmann:

All,

IMHO I found the root cause for the portaudio problems. It is
located in the C source (the JNI part) that causes a buffer
overflow. The offending code in ReadStream:

...
        if(inStream->inputResampleFactor != 1)
        {
            spx_uint32_t in_len;
            in_len = lrint(
                frames
                *inStream->inputChannelCount
                *inStream->inputResampleFactor);

            short res[in_len];

            errorCode = Pa_ReadStream(
                inStream->stream,
                res,
                frames*inStream->inputResampleFactor);

...

The resample factor is >5 (assuming a sample of 8000). the number
of frames to read is multiplied with this number. The Java code
creates a too small buffer:

new byte[PortAudioManager.NUM_SAMPLES*frameSize];

where frameSize is 2, NUM_SAMPLES is 160, resulting in 320 bytes.
The C code multiplies NUM_SAMPLES (frames in the C code) with
the resampleFactore which results in 160 * 5.5 which is ~1000.

The same problem is also in writeStream. The Java code hands over
a too small buffer, also writeStream multiplies to buffersize with
a resampleFactor. This results in the bad audio quality.

Regards,
Werner

Am 22.12.2009 13:27, schrieb Werner Dittmann:

Damian,

I'm just testing the whole stuff again. First results: nothing
changed so far :frowning: - ringtone plays just once, the short "bips"
every 3-4 seconds. Audio quality in portaudio default setting
is not sufficent. Also the thread blocking stays. Currently I'm
trying to get a grip on this when using "default" setting. Also
the capturing stops after some seconds - thus the thread block.

Another SIP client that uses Alsa default setting works ok.

Using direct settings (seleting the DAC/ADC devices) result in
even worse effects, maybe because of the following exception:

2:01:39.945 SCHWERWIEGEND: util.UtilActivator.uncaughtException().80 An uncaught exception occurred in thread=Thread[JMF thread:
com.sun.media.ProcessEngine@1599f755 ( prefetchThread),9,system] and message was: null
java.lang.NullPointerException
        at net.java.sip.communicator.impl.neomedia.jmfext.media.renderer.audio.PortAudioRenderer.stop(PortAudioRenderer.java:119)
        at net.java.sip.communicator.impl.neomedia.jmfext.media.renderer.audio.PortAudioRenderer.close(PortAudioRenderer.java:195)
        at com.sun.media.BasicRendererModule.doFailedPrefetch(BasicRendererModule.java:186)
        at com.sun.media.PlaybackEngine.doFailedPrefetch(PlaybackEngine.java:760)
        at com.sun.media.PrefetchWorkThread.failed(BasicController.java:1442)
        at com.sun.media.StateTransitionWorkThread.run(BasicController.java:1346)
12:03:52.235 SCHWERWIEGEND: impl.shutdowntimeout.ShutdownTimeout.run().78 Failed to gently shutdown. Forcin

As I've seen you modified "PortAudioClipImpl" to call
                         portAudioStream.stop();
+ portAudioStream.close();

and you modified OutputPortAudioStream.close() to call stop also. Maybe this "double" stop
causes to above exception.

Regards,
Werner

<SNIP --- SNAP>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#16

Hi,

first of all to make echo cancel work we had to make reading and
writing in equal portions: it is 20ms.
We open the device in the native code in 44.1kHz, cause there are
systems in which only this sample rate is supported. But from the java
side we see the all the devices as they work on the sample rate we use
8kHz, and all the resampling is done in the native part using
speexdsp.
In your mail you missed some code which is also essential to what is
going on. So the java code gives a buffer with length 20ms. and
expects it to be in 8kHz (320 bytes). The code you show estimates what
will be the corresponding buffer before resampling, in 44.1kHz. So the
buffer will be in_len = 44100/8000*160( = 882). Then we read that
buffer and use the resampler to resample the data into the byte buffer
comming from java and we return it. So there is no loss of data,
everything is fine.
Here I will answer you and to the other mail, cause both are
connected. Currently when opening stream we specify
FRAMES_PER_BUFFER_UNSPECIFIED for the framesPerBuffer. We can add an
extra parameter saying what will be the reading/writing period, which
for now is 20ms and make the calculation and open the stream with the
size of framesPerBuffer what we will use, or we can make this
calculations in java as we know that the JNI code always opens the
devices in 44100, I think the first approach is better cause currently
the java code uses those devices as they work on 8kHz.

And so the problem with the quality lies somewhere else. Till now we
have tested on several different machines on mac,windows and linux,
but most of them are using hda-intel sound device but none of them has
produced quality problems when using the default devices. I suppose
its some incompatibility in the properties of the media we use and the
media that comes from the device, we currently expect media 8kHz,
sample and frame size 16, little endian, signed. Although if one of
that parameter is wrong we won't here anything but garbage I think.
Maybe there is a problem with the media comming from the device and
after resmapling the qulity is low. What you can do if your device is
support opening in 8kHz mode you can change in the natve part the
value of DEFAULT_SAMPLE_RATE to be 8000.0, this will skip the
resampler and you can test will this have an impact of the quality.
Or you can make opening from java to be 44100 just for the tests, but
this will be harder cause other things may fail in the media part.

Hope this helps :slight_smile:

Thanks
damencho

···

On Wed, Dec 23, 2009 at 11:12 AM, Werner Dittmann <Werner.Dittmann@t-online.de> wrote:

All,

IMHO I found the root cause for the portaudio problems. It is
located in the C source (the JNI part) that causes a buffer
overflow. The offending code in ReadStream:

...
if(inStream->inputResampleFactor != 1)
{
spx_uint32_t in_len;
in_len = lrint(
frames
*inStream->inputChannelCount
*inStream->inputResampleFactor);

       short res\[in\_len\];

       errorCode = Pa\_ReadStream\(
           inStream\-&gt;stream,
           res,
           frames\*inStream\-&gt;inputResampleFactor\);

...

The resample factor is >5 (assuming a sample of 8000). the number
of frames to read is multiplied with this number. The Java code
creates a too small buffer:

new byte[PortAudioManager.NUM_SAMPLES*frameSize];

where frameSize is 2, NUM_SAMPLES is 160, resulting in 320 bytes.
The C code multiplies NUM_SAMPLES (frames in the C code) with
the resampleFactore which results in 160 * 5.5 which is ~1000.

The same problem is also in writeStream. The Java code hands over
a too small buffer, also writeStream multiplies to buffersize with
a resampleFactor. This results in the bad audio quality.

Regards,
Werner

Am 22.12.2009 13:27, schrieb Werner Dittmann:

Damian,

I'm just testing the whole stuff again. First results: nothing
changed so far :frowning: - ringtone plays just once, the short "bips"
every 3-4 seconds. Audio quality in portaudio default setting
is not sufficent. Also the thread blocking stays. Currently I'm
trying to get a grip on this when using "default" setting. Also
the capturing stops after some seconds - thus the thread block.

Another SIP client that uses Alsa default setting works ok.

Using direct settings (seleting the DAC/ADC devices) result in
even worse effects, maybe because of the following exception:

2:01:39.945 SCHWERWIEGEND: util.UtilActivator.uncaughtException().80 An uncaught exception occurred in thread=Thread[JMF thread:
com.sun.media.ProcessEngine@1599f755 ( prefetchThread),9,system] and message was: null
java.lang.NullPointerException
at net.java.sip.communicator.impl.neomedia.jmfext.media.renderer.audio.PortAudioRenderer.stop(PortAudioRenderer.java:119)
at net.java.sip.communicator.impl.neomedia.jmfext.media.renderer.audio.PortAudioRenderer.close(PortAudioRenderer.java:195)
at com.sun.media.BasicRendererModule.doFailedPrefetch(BasicRendererModule.java:186)
at com.sun.media.PlaybackEngine.doFailedPrefetch(PlaybackEngine.java:760)
at com.sun.media.PrefetchWorkThread.failed(BasicController.java:1442)
at com.sun.media.StateTransitionWorkThread.run(BasicController.java:1346)
12:03:52.235 SCHWERWIEGEND: impl.shutdowntimeout.ShutdownTimeout.run().78 Failed to gently shutdown. Forcin

As I've seen you modified "PortAudioClipImpl" to call
portAudioStream.stop();
+ portAudioStream.close();

and you modified OutputPortAudioStream.close() to call stop also. Maybe this "double" stop
causes to above exception.

Regards,
Werner

<SNIP --- SNAP>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#17

Hi again :slight_smile:

yes we saw that SIGSEGV its produced on hangup cause we hangup both of
the streams which are served from different threads and those closes
seems to clear some resources which seems to be the problem.

Thanks
damencho

···

On Wed, Dec 23, 2009 at 12:18 PM, Werner Dittmann <Werner.Dittmann@t-online.de> wrote:

All,

Sorry - was a bit too early :frowning: . I've overlooked the allocation
of temporary buffers to hold the data.

However, testing with portaudio very often also results in
real crashes, always segmentation vialotaion (SIGSEGV) detected
in libasound.so. Thus somehow memory was corrupted some time
before. This happens mostly when I hangup a call after the
audio input (reading from microphone) stopped.

Regards,
Werner

Am 23.12.2009 10:12, schrieb Werner Dittmann:

All,

IMHO I found the root cause for the portaudio problems. It is
located in the C source (the JNI part) that causes a buffer
overflow. The offending code in ReadStream:

...
if(inStream->inputResampleFactor != 1)
{
spx_uint32_t in_len;
in_len = lrint(
frames
*inStream->inputChannelCount
*inStream->inputResampleFactor);

        short res\[in\_len\];

        errorCode = Pa\_ReadStream\(
            inStream\-&gt;stream,
            res,
            frames\*inStream\-&gt;inputResampleFactor\);

...

The resample factor is >5 (assuming a sample of 8000). the number
of frames to read is multiplied with this number. The Java code
creates a too small buffer:

new byte[PortAudioManager.NUM_SAMPLES*frameSize];

where frameSize is 2, NUM_SAMPLES is 160, resulting in 320 bytes.
The C code multiplies NUM_SAMPLES (frames in the C code) with
the resampleFactore which results in 160 * 5.5 which is ~1000.

The same problem is also in writeStream. The Java code hands over
a too small buffer, also writeStream multiplies to buffersize with
a resampleFactor. This results in the bad audio quality.

Regards,
Werner

Am 22.12.2009 13:27, schrieb Werner Dittmann:

Damian,

I'm just testing the whole stuff again. First results: nothing
changed so far :frowning: - ringtone plays just once, the short "bips"
every 3-4 seconds. Audio quality in portaudio default setting
is not sufficent. Also the thread blocking stays. Currently I'm
trying to get a grip on this when using "default" setting. Also
the capturing stops after some seconds - thus the thread block.

Another SIP client that uses Alsa default setting works ok.

Using direct settings (seleting the DAC/ADC devices) result in
even worse effects, maybe because of the following exception:

2:01:39.945 SCHWERWIEGEND: util.UtilActivator.uncaughtException().80 An uncaught exception occurred in thread=Thread[JMF thread:
com.sun.media.ProcessEngine@1599f755 ( prefetchThread),9,system] and message was: null
java.lang.NullPointerException
at net.java.sip.communicator.impl.neomedia.jmfext.media.renderer.audio.PortAudioRenderer.stop(PortAudioRenderer.java:119)
at net.java.sip.communicator.impl.neomedia.jmfext.media.renderer.audio.PortAudioRenderer.close(PortAudioRenderer.java:195)
at com.sun.media.BasicRendererModule.doFailedPrefetch(BasicRendererModule.java:186)
at com.sun.media.PlaybackEngine.doFailedPrefetch(PlaybackEngine.java:760)
at com.sun.media.PrefetchWorkThread.failed(BasicController.java:1442)
at com.sun.media.StateTransitionWorkThread.run(BasicController.java:1346)
12:03:52.235 SCHWERWIEGEND: impl.shutdowntimeout.ShutdownTimeout.run().78 Failed to gently shutdown. Forcin

As I've seen you modified "PortAudioClipImpl" to call
portAudioStream.stop();
+ portAudioStream.close();

and you modified OutputPortAudioStream.close() to call stop also. Maybe this "double" stop
causes to above exception.

Regards,
Werner

<SNIP --- SNAP>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#18

Hi Damian,

regarding the SIGSEGV in jportaudio: this happens because the two
streams are conneted with each other. For example close output first:

in that case the close function performs a PortAudioStream_free for the
output audio stream. If echo cancel is enabled (usually it is) then
the output port audio stream is connected to the input audio stream. Thus
closing output first results in an invalid connected stream. The read
function uses the connected output stream to cancel the echo - but the
output stream is gone. This results in spurious SIGSEGV, depending if
the contents of free memory block was already overwritten or not.

Hope this helps to fix the problem in SC.

Have fun in fixing this problem :wink: .

Regards,
Werner

···

Am 23.12.2009 11:26, schrieb Damian Minkov:

Hi again :slight_smile:

yes we saw that SIGSEGV its produced on hangup cause we hangup both of
the streams which are served from different threads and those closes
seems to clear some resources which seems to be the problem.

Thanks
damencho

On Wed, Dec 23, 2009 at 12:18 PM, Werner Dittmann > <Werner.Dittmann@t-online.de> wrote:

All,

Sorry - was a bit too early :frowning: . I've overlooked the allocation
of temporary buffers to hold the data.

However, testing with portaudio very often also results in
real crashes, always segmentation vialotaion (SIGSEGV) detected
in libasound.so. Thus somehow memory was corrupted some time
before. This happens mostly when I hangup a call after the
audio input (reading from microphone) stopped.

Regards,
Werner

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#19

Hi Werner,

I think that a big part of the team (including Damian and Lubomir) is now on vacation for the holidays, so they'll probably answer these only next week.

Cheers,
Yana

···

On Dec 28, 2009, at 1:18 PM, Werner Dittmann wrote:

Hi Damian,

regarding the SIGSEGV in jportaudio: this happens because the two
streams are conneted with each other. For example close output first:

in that case the close function performs a PortAudioStream_free for the
output audio stream. If echo cancel is enabled (usually it is) then
the output port audio stream is connected to the input audio stream. Thus
closing output first results in an invalid connected stream. The read
function uses the connected output stream to cancel the echo - but the
output stream is gone. This results in spurious SIGSEGV, depending if
the contents of free memory block was already overwritten or not.

Hope this helps to fix the problem in SC.

Have fun in fixing this problem :wink: .

Regards,
Werner

Am 23.12.2009 11:26, schrieb Damian Minkov:

Hi again :slight_smile:

yes we saw that SIGSEGV its produced on hangup cause we hangup both of
the streams which are served from different threads and those closes
seems to clear some resources which seems to be the problem.

Thanks
damencho

On Wed, Dec 23, 2009 at 12:18 PM, Werner Dittmann >> <Werner.Dittmann@t-online.de> wrote:

All,

Sorry - was a bit too early :frowning: . I've overlooked the allocation
of temporary buffers to hold the data.

However, testing with portaudio very often also results in
real crashes, always segmentation vialotaion (SIGSEGV) detected
in libasound.so. Thus somehow memory was corrupted some time
before. This happens mostly when I hangup a call after the
audio input (reading from microphone) stopped.

Regards,
Werner

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#20

Hi,

I see what you mean, I will check that, I thought those situations are
handled (freeing re-samplers and echo cancellers). But I will check
that with the connected streams. Good catch thanks.
It wont be hard to handle this as I see that those streams are
connected with double link.

Thanks
damencho

···

On Mon, Dec 28, 2009 at 1:18 PM, Werner Dittmann <Werner.Dittmann@t-online.de> wrote:

Hi Damian,

regarding the SIGSEGV in jportaudio: this happens because the two
streams are conneted with each other. For example close output first:

in that case the close function performs a PortAudioStream_free for the
output audio stream. If echo cancel is enabled (usually it is) then
the output port audio stream is connected to the input audio stream. Thus
closing output first results in an invalid connected stream. The read
function uses the connected output stream to cancel the echo - but the
output stream is gone. This results in spurious SIGSEGV, depending if
the contents of free memory block was already overwritten or not.

Hope this helps to fix the problem in SC.

Have fun in fixing this problem :wink: .

Regards,
Werner

Am 23.12.2009 11:26, schrieb Damian Minkov:

Hi again :slight_smile:

yes we saw that SIGSEGV its produced on hangup cause we hangup both of
the streams which are served from different threads and those closes
seems to clear some resources which seems to be the problem.

Thanks
damencho

On Wed, Dec 23, 2009 at 12:18 PM, Werner Dittmann >> <Werner.Dittmann@t-online.de> wrote:

All,

Sorry - was a bit too early :frowning: . I've overlooked the allocation
of temporary buffers to hold the data.

However, testing with portaudio very often also results in
real crashes, always segmentation vialotaion (SIGSEGV) detected
in libasound.so. Thus somehow memory was corrupted some time
before. This happens mostly when I hangup a call after the
audio input (reading from microphone) stopped.

Regards,
Werner

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net