[sip-comm-dev] Audio sample rate - questions and remarks


#1

Damian, Lubo,

I had a look to Lubo's patch that enhances the handling of play out
data. While looking at the patch I again noticed that SC uses a fixed
frame buffer size at the moment. For example we always allocate buffers
for 160 frames (covering 20ms at 8000Hz). Internally this will be
re sampled to match the sample rate of the device, usually 44.100.
libjportaudio allocates 882 frames when using 44.100Hz.
882/160 = 44100/8000

Some observations:

The ring tone and other clips have a sample rate of 22050 but SC always
allocates 160 to play audio. For ring tone/clips this results to
320 frames after up-sampling to 44.100 which is ~7ms instead of 20ms.

Similar cases are the better speex codecs. SC's media configuration
offers speex/16000 and speex/32000. With speex/32000 a 160 frame buffer
is just worth ~4ms - this would be critical to handle for the RTP streams.

Are there any plans to introduce an adaptive frame buffer size that
depends on the sample rate and stick at about 20ms per frame buffer:
sampleRate / 50 = frameBufferSize.

In the MasterPortAudioStream and OutputPortAudioStream constructors this
would be something like this:

framesPerBuffer = (int)(sampleRate/50.0);

A test version of SC with this simple modification works quite well on my
system :wink: - not yet fully tested with speex/16000 or speex/32000.

Regards,
Werner

···

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#2

framesPerBuffer = (int)(sampleRate/50.0);

Please excuse my ignorance but... if sampleRate denotes "the number of
samples played or recorded per second" and the result is supposed to
be the number of frames that represent 20ms of audio, shouldn't there
be the number of channels involved?

···

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#3

Hi Werner,

the reason we make such fixed size was we have particular size for reading
and writing, to and from the devices so we can easily plug the echo
canceller. You are right about the other sample rates we can change this
size for them, but we must be sure we use the same size for reading and
writing and echo canceller is aware of that :).
About the higher sample rates of speex there is more work to be done for the
re-sampling cause currently we open the devices from the java side as 8000
and then from the jni part the real opening is made on the default for the
device, for example 44.1 kHz. So if we use speex 16 kHz it reading will be
44.1kHz -> 8kHz -> 16kHz. But this are things we know about and if there
are volunteers to work on this there are more than welcome :slight_smile:

Thanks
damencho

···

On Wed, Jan 20, 2010 at 7:41 PM, Werner Dittmann < Werner.Dittmann@t-online.de> wrote:

Damian, Lubo,

I had a look to Lubo's patch that enhances the handling of play out
data. While looking at the patch I again noticed that SC uses a fixed
frame buffer size at the moment. For example we always allocate buffers
for 160 frames (covering 20ms at 8000Hz). Internally this will be
re sampled to match the sample rate of the device, usually 44.100.
libjportaudio allocates 882 frames when using 44.100Hz.
882/160 = 44100/8000

Some observations:

The ring tone and other clips have a sample rate of 22050 but SC always
allocates 160 to play audio. For ring tone/clips this results to
320 frames after up-sampling to 44.100 which is ~7ms instead of 20ms.

Similar cases are the better speex codecs. SC's media configuration
offers speex/16000 and speex/32000. With speex/32000 a 160 frame buffer
is just worth ~4ms - this would be critical to handle for the RTP streams.

Are there any plans to introduce an adaptive frame buffer size that
depends on the sample rate and stick at about 20ms per frame buffer:
sampleRate / 50 = frameBufferSize.

In the MasterPortAudioStream and OutputPortAudioStream constructors this
would be something like this:

framesPerBuffer = (int)(sampleRate/50.0);

A test version of SC with this simple modification works quite well on my
system :wink: - not yet fully tested with speex/16000 or speex/32000.

Regards,
Werner

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#4

Lubo,

see inline

Regards,
Werner

framesPerBuffer = (int)(sampleRate/50.0);

Please excuse my ignorance but... if sampleRate denotes "the number of
samples played or recorded per second" and the result is supposed to
be the number of frames that represent 20ms of audio, shouldn't there
be the number of channels involved?

Not for the "framesPerBuffer". When you compute the frameBufferSize (in bytes)
then this is frameBufferSize = framesPerBuffer * sampleSize * channels.

A 'frame' contains the complete data for all channels.

···

Am 20.01.2010 20:09, schrieb Lubomir Marinov:

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#5

Hi Damian,

without knowing too much about the speex resample processing and
echo cancellation, but just as a thought:

the JNI part always samples up to the device sample rate for
playback and samples down for reading. Wouldn't it make sense to
perform the echo cancel at the device sample rate if both devices
have the same sample rate:

writing:
- re-sample to device sample rate
- write data to device
- store the re-sampled data in buffer with timestamp (not the
  original data)

reading:
- read data at device sample rate
- use this data to perform echo cancellation
- resample to required SC sample rate (8kHz, 16kHz, 32kHz)

In case the devices have different sample rates one additonal re-sample
step is necessary to re-sample to the sample rate of the device with
the lowest sample rate (usually 44.1kHz)

WDYT? Would it make sense to test this? If so I would give it a try
during the weekend.

BTW, the portaudio clip implementation already opens the stream with
a sample rate of 22050 Hz. I've not checked what the speex codecs do.

Regards,
Werner

···

Am 21.01.2010 09:05, schrieb Damian Minkov:

Hi Werner,

the reason we make such fixed size was we have particular size for reading
and writing, to and from the devices so we can easily plug the echo
canceller. You are right about the other sample rates we can change this
size for them, but we must be sure we use the same size for reading and
writing and echo canceller is aware of that :).
About the higher sample rates of speex there is more work to be done for the
re-sampling cause currently we open the devices from the java side as 8000
and then from the jni part the real opening is made on the default for the
device, for example 44.1 kHz. So if we use speex 16 kHz it reading will be
44.1kHz -> 8kHz -> 16kHz. But this are things we know about and if there
are volunteers to work on this there are more than welcome :slight_smile:

Thanks
damencho

On Wed, Jan 20, 2010 at 7:41 PM, Werner Dittmann < > Werner.Dittmann@t-online.de> wrote:

Damian, Lubo,

I had a look to Lubo's patch that enhances the handling of play out
data. While looking at the patch I again noticed that SC uses a fixed
frame buffer size at the moment. For example we always allocate buffers
for 160 frames (covering 20ms at 8000Hz). Internally this will be
re sampled to match the sample rate of the device, usually 44.100.
libjportaudio allocates 882 frames when using 44.100Hz.
882/160 = 44100/8000

Some observations:

The ring tone and other clips have a sample rate of 22050 but SC always
allocates 160 to play audio. For ring tone/clips this results to
320 frames after up-sampling to 44.100 which is ~7ms instead of 20ms.

Similar cases are the better speex codecs. SC's media configuration
offers speex/16000 and speex/32000. With speex/32000 a 160 frame buffer
is just worth ~4ms - this would be critical to handle for the RTP streams.

Are there any plans to introduce an adaptive frame buffer size that
depends on the sample rate and stick at about 20ms per frame buffer:
sampleRate / 50 = frameBufferSize.

In the MasterPortAudioStream and OutputPortAudioStream constructors this
would be something like this:

framesPerBuffer = (int)(sampleRate/50.0);

A test version of SC with this simple modification works quite well on my
system :wink: - not yet fully tested with speex/16000 or speex/32000.

Regards,
Werner

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#6

Hi,

I had to mention this earlier but recommended sample rates for echo cancel
is the sample rate of the codec and the recommended rates are 8,16 and 32
kHz. Although I've seen comments that working with 32kHz is not so good. So
far I have tried and tested and I got best results when it is working on 8
kHz.

Thanks
damencho

···

On Thu, Jan 21, 2010 at 6:49 PM, Werner Dittmann < Werner.Dittmann@t-online.de> wrote:

Hi Damian,

without knowing too much about the speex resample processing and
echo cancellation, but just as a thought:

the JNI part always samples up to the device sample rate for
playback and samples down for reading. Wouldn't it make sense to
perform the echo cancel at the device sample rate if both devices
have the same sample rate:

writing:
- re-sample to device sample rate
- write data to device
- store the re-sampled data in buffer with timestamp (not the
original data)

reading:
- read data at device sample rate
- use this data to perform echo cancellation
- resample to required SC sample rate (8kHz, 16kHz, 32kHz)

In case the devices have different sample rates one additonal re-sample
step is necessary to re-sample to the sample rate of the device with
the lowest sample rate (usually 44.1kHz)

WDYT? Would it make sense to test this? If so I would give it a try
during the weekend.

BTW, the portaudio clip implementation already opens the stream with
a sample rate of 22050 Hz. I've not checked what the speex codecs do.

Regards,
Werner

Am 21.01.2010 09:05, schrieb Damian Minkov:
> Hi Werner,
>
> the reason we make such fixed size was we have particular size for
reading
> and writing, to and from the devices so we can easily plug the echo
> canceller. You are right about the other sample rates we can change this
> size for them, but we must be sure we use the same size for reading and
> writing and echo canceller is aware of that :).
> About the higher sample rates of speex there is more work to be done for
the
> re-sampling cause currently we open the devices from the java side as
8000
> and then from the jni part the real opening is made on the default for
the
> device, for example 44.1 kHz. So if we use speex 16 kHz it reading will
be
> 44.1kHz -> 8kHz -> 16kHz. But this are things we know about and if there
> are volunteers to work on this there are more than welcome :slight_smile:
>
> Thanks
> damencho
>
>
> On Wed, Jan 20, 2010 at 7:41 PM, Werner Dittmann < > > Werner.Dittmann@t-online.de> wrote:
>
>> Damian, Lubo,
>>
>> I had a look to Lubo's patch that enhances the handling of play out
>> data. While looking at the patch I again noticed that SC uses a fixed
>> frame buffer size at the moment. For example we always allocate buffers
>> for 160 frames (covering 20ms at 8000Hz). Internally this will be
>> re sampled to match the sample rate of the device, usually 44.100.
>> libjportaudio allocates 882 frames when using 44.100Hz.
>> 882/160 = 44100/8000
>>
>> Some observations:
>>
>> The ring tone and other clips have a sample rate of 22050 but SC always
>> allocates 160 to play audio. For ring tone/clips this results to
>> 320 frames after up-sampling to 44.100 which is ~7ms instead of 20ms.
>>
>> Similar cases are the better speex codecs. SC's media configuration
>> offers speex/16000 and speex/32000. With speex/32000 a 160 frame buffer
>> is just worth ~4ms - this would be critical to handle for the RTP
streams.
>>
>> Are there any plans to introduce an adaptive frame buffer size that
>> depends on the sample rate and stick at about 20ms per frame buffer:
>> sampleRate / 50 = frameBufferSize.
>>
>> In the MasterPortAudioStream and OutputPortAudioStream constructors this
>> would be something like this:
>>
>> framesPerBuffer = (int)(sampleRate/50.0);
>>
>> A test version of SC with this simple modification works quite well on
my
>> system :wink: - not yet fully tested with speex/16000 or speex/32000.
>>
>>
>> Regards,
>> Werner
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
>> For additional commands, e-mail: dev-help@sip-communicator.dev.java.net
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#7

Damian Minkov написа:

Hi,

I had to mention this earlier but recommended sample rates for echo
cancel is the sample rate of the codec and the recommended rates are
8,16 and 32 kHz. Although I've seen comments that working with 32kHz is
not so good. So far I have tried and tested and I got best results when
it is working on 8 kHz.

Well it's probably worth noting that while it is definitely a good thing
to make sure that echo cancellation always works it is in no way a deal
breaker. Having support for wide band codecs is quite important even if
it means sacrificing echo cancellation for wide band calls.

Cheers,
Emil

···

Thanks
damencho

On Thu, Jan 21, 2010 at 6:49 PM, Werner Dittmann > <Werner.Dittmann@t-online.de <mailto:Werner.Dittmann@t-online.de>> wrote:

    Hi Damian,

    without knowing too much about the speex resample processing and
    echo cancellation, but just as a thought:

    the JNI part always samples up to the device sample rate for
    playback and samples down for reading. Wouldn't it make sense to
    perform the echo cancel at the device sample rate if both devices
    have the same sample rate:

    writing:
    - re-sample to device sample rate
    - write data to device
    - store the re-sampled data in buffer with timestamp (not the
     original data)

    reading:
    - read data at device sample rate
    - use this data to perform echo cancellation
    - resample to required SC sample rate (8kHz, 16kHz, 32kHz)

    In case the devices have different sample rates one additonal re-sample
    step is necessary to re-sample to the sample rate of the device with
    the lowest sample rate (usually 44.1kHz)

    WDYT? Would it make sense to test this? If so I would give it a try
    during the weekend.

    BTW, the portaudio clip implementation already opens the stream with
    a sample rate of 22050 Hz. I've not checked what the speex codecs do.

    Regards,
    Werner

    Am 21.01.2010 09:05, schrieb Damian Minkov:
    > Hi Werner,
    >
    > the reason we make such fixed size was we have particular size for
    reading
    > and writing, to and from the devices so we can easily plug the echo
    > canceller. You are right about the other sample rates we can
    change this
    > size for them, but we must be sure we use the same size for
    reading and
    > writing and echo canceller is aware of that :).
    > About the higher sample rates of speex there is more work to be
    done for the
    > re-sampling cause currently we open the devices from the java side
    as 8000
    > and then from the jni part the real opening is made on the default
    for the
    > device, for example 44.1 kHz. So if we use speex 16 kHz it reading
    will be
    > 44.1kHz -> 8kHz -> 16kHz. But this are things we know about and
    if there
    > are volunteers to work on this there are more than welcome :slight_smile:
    >
    > Thanks
    > damencho
    >
    >
    > On Wed, Jan 20, 2010 at 7:41 PM, Werner Dittmann < > > Werner.Dittmann@t-online.de <mailto:Werner.Dittmann@t-online.de>> > wrote:
    >
    >> Damian, Lubo,
    >>
    >> I had a look to Lubo's patch that enhances the handling of play out
    >> data. While looking at the patch I again noticed that SC uses a fixed
    >> frame buffer size at the moment. For example we always allocate
    buffers
    >> for 160 frames (covering 20ms at 8000Hz). Internally this will be
    >> re sampled to match the sample rate of the device, usually 44.100.
    >> libjportaudio allocates 882 frames when using 44.100Hz.
    >> 882/160 = 44100/8000
    >>
    >> Some observations:
    >>
    >> The ring tone and other clips have a sample rate of 22050 but SC
    always
    >> allocates 160 to play audio. For ring tone/clips this results to
    >> 320 frames after up-sampling to 44.100 which is ~7ms instead of 20ms.
    >>
    >> Similar cases are the better speex codecs. SC's media configuration
    >> offers speex/16000 and speex/32000. With speex/32000 a 160 frame
    buffer
    >> is just worth ~4ms - this would be critical to handle for the RTP
    streams.
    >>
    >> Are there any plans to introduce an adaptive frame buffer size that
    >> depends on the sample rate and stick at about 20ms per frame buffer:
    >> sampleRate / 50 = frameBufferSize.
    >>
    >> In the MasterPortAudioStream and OutputPortAudioStream
    constructors this
    >> would be something like this:
    >>
    >> framesPerBuffer = (int)(sampleRate/50.0);
    >>
    >> A test version of SC with this simple modification works quite
    well on my
    >> system :wink: - not yet fully tested with speex/16000 or speex/32000.
    >>
    >>
    >> Regards,
    >> Werner
    >>
    >> ---------------------------------------------------------------------
    >> To unsubscribe, e-mail:
    dev-unsubscribe@sip-communicator.dev.java.net
    <mailto:dev-unsubscribe@sip-communicator.dev.java.net>
    >> For additional commands, e-mail:
    dev-help@sip-communicator.dev.java.net
    <mailto:dev-help@sip-communicator.dev.java.net>
    >>
    >>
    >

    ---------------------------------------------------------------------
    To unsubscribe, e-mail:
    dev-unsubscribe@sip-communicator.dev.java.net
    <mailto:dev-unsubscribe@sip-communicator.dev.java.net>
    For additional commands, e-mail:
    dev-help@sip-communicator.dev.java.net
    <mailto:dev-help@sip-communicator.dev.java.net>

--
Emil Ivov, Ph.D. 67000 Strasbourg,
Project Lead France
SIP Communicator
emcho@sip-communicator.org PHONE: +33.1.77.62.43.30
http://sip-communicator.org FAX: +33.1.77.62.47.31

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net