[sip-comm-dev]questions about JMF to intergrate celt


#1

Hi again, Lubo, Jean, Emil,

I am now trying to implement JavaEncoder and JavaDecoder based on
JMF(extended from com.ibm.media.codec.audio.AudioCodec), and here are my
questions:

1. Except for the method "int process(Buffer input, Buffer output)", which
methods are needed to be overridden in JavaEncoder?

2. In CELT, the decoder is dependent on encoder, e.g. (1). both the
encoder(CELTEncoder) and decoder(CELTDecoder) need to be initialized with
exactly the same mode(CELTMode), otherwise the quality will be very bad.
(2). the number of bytes to read from "data" by clet_decode() method MUST be
exactly the number of bytes returned by the encoder. However, in
JMF,JavaEncoder and JavaDecoder seem to be unrelated with each other. How to
solve these problems?

3. celt_encode() encodes one frame once, but i am not very sure about the
process unit of process() in JavaEncoder. Should i process one frame in
process() at a time just like celt_encode() or shoud i process the whole
audio stream in process()? In my opinion, i think the former sounds
reasonable.

4. Before celt_encode can work, a CELTMode and a CELTEncoder should be
constructed and parameters such as sample rate, frame size, channels should
be supplied. This information is specified in the audio format of the input
stream. Since the input buffer is only accessible in process(Buffer input,
Buffer output), how to get this information prior to the call of process() ?

5. AudioFormat. The requirement of CELT is sample rate, frame size,
channels, bytes per frame, but frame size and bytes per frame don't have
correspondence member in AudioFormat. So how to set the input format and
supported format in JavaEncoder? And in the constructor AudioFormat(String
encoding), what's the proper value of encoding for CELT?

Thanks in advance.
Jing


#2

I am now trying to implement JavaEncoder and JavaDecoder based on
JMF(extended from com.ibm.media.codec.audio.AudioCodec), and here are my

The package com.ibm should be considered internal to JMF in my opinion
so I'd rather go with AbstractCodecExt. Besides, since I'm going to
review your code, you'll likely want to look at my Codec
implementations as well.

1. Except for the method "int process(Buffer input, Buffer output)", which
methods are needed to be overridden in JavaEncoder?

Do use AbstractCodecExt and see the extenders I've implemented.

2. In CELT, the decoder is dependent on encoder, e.g. (1). both the
encoder(CELTEncoder) and decoder(CELTDecoder) need to be initialized with
exactly the same mode(CELTMode), otherwise the quality will be very bad.
(2). the number of bytes to read from "data" by clet_decode() method MUST be
exactly the number of bytes returned by the encoder. However, in
JMF,JavaEncoder and JavaDecoder seem to be unrelated with each other. How to
solve these problems?

The decoder and encoder cannot possibly be instance-dependent on one
another because one may encode on one machine and decode on another.
If CELTMode needs to be value equal, I guess you have to look at how
it relates to the input and output AudioFormat properties of the
Codec.

3. celt_encode() encodes one frame once, but i am not very sure about the
process unit of process() in JavaEncoder. Should i process one frame in
process() at a time just like celt_encode() or shoud i process the whole
audio stream in process()? In my opinion, i think the former sounds
reasonable.

Refer to the javadocs of the return values of the #process method and
see how we've already used them in our Codec implementations to
achieve what you're describing.

4. Before celt_encode can work, a CELTMode and a CELTEncoder should be
constructed and parameters such as sample rate, frame size, channels should
be supplied. This information is specified in the audio format of the input
stream. Since the input buffer is only accessible in process(Buffer input,
Buffer output), how to get this information prior to the call of process() ?

Please look at the javadocs of the Codec interface and examine our
Codec implementations. Each Codec has inputFormat and outputFormat
properties which are configured prior to calling #process on it.
Additionally, each Buffer supplied to #process may have a value for
its format property as well.

5. AudioFormat. The requirement of CELT is sample rate, frame size,
channels, bytes per frame, but frame size and bytes per frame don't have
correspondence member in AudioFormat. So how to set the input format and
supported format in JavaEncoder? And in the constructor AudioFormat(String
encoding), what's the proper value of encoding for CELT?

If frame size and bytes per frame don't deviate from the common case,
they're likely sample size * channels and bytes per sample * channels,
respectively.

···

On Tue, Jul 13, 2010 at 2:39 PM, Dai Jing <daijing09@gmail.com> wrote:

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#3

Hi guys,

Just my 2 cents about encoder/decoder interdependence.

If I understood correctly, the encoder needs a decoder instance because it uses prediction and needs to evaluate some parameters?

If that's the case then we should consider putting both encoder and decoder (for celt) in JavaEncoder. Or to put it another way, there is not necessarily a one-to-one mapping between the codec encoder and the wrapped encoder.

However one thing I guess you shouldn't do is call JavaDecoder from within JavaEncoder. Calling celt_decode would be the way.

Hope this helps,
Jean

···

Le 13 juil. 2010 à 21:17, Lubomir Marinov <lubo@sip-communicator.org> a écrit :

On Tue, Jul 13, 2010 at 2:39 PM, Dai Jing <daijing09@gmail.com> wrote:

I am now trying to implement JavaEncoder and JavaDecoder based on
JMF(extended from com.ibm.media.codec.audio.AudioCodec), and here are my

The package com.ibm should be considered internal to JMF in my opinion
so I'd rather go with AbstractCodecExt. Besides, since I'm going to
review your code, you'll likely want to look at my Codec
implementations as well.

1. Except for the method "int process(Buffer input, Buffer output)", which
methods are needed to be overridden in JavaEncoder?

Do use AbstractCodecExt and see the extenders I've implemented.

2. In CELT, the decoder is dependent on encoder, e.g. (1). both the
encoder(CELTEncoder) and decoder(CELTDecoder) need to be initialized with
exactly the same mode(CELTMode), otherwise the quality will be very bad.
(2). the number of bytes to read from "data" by clet_decode() method MUST be
exactly the number of bytes returned by the encoder. However, in
JMF,JavaEncoder and JavaDecoder seem to be unrelated with each other. How to
solve these problems?

The decoder and encoder cannot possibly be instance-dependent on one
another because one may encode on one machine and decode on another.
If CELTMode needs to be value equal, I guess you have to look at how
it relates to the input and output AudioFormat properties of the
Codec.

3. celt_encode() encodes one frame once, but i am not very sure about the
process unit of process() in JavaEncoder. Should i process one frame in
process() at a time just like celt_encode() or shoud i process the whole
audio stream in process()? In my opinion, i think the former sounds
reasonable.

Refer to the javadocs of the return values of the #process method and
see how we've already used them in our Codec implementations to
achieve what you're describing.

4. Before celt_encode can work, a CELTMode and a CELTEncoder should be
constructed and parameters such as sample rate, frame size, channels should
be supplied. This information is specified in the audio format of the input
stream. Since the input buffer is only accessible in process(Buffer input,
Buffer output), how to get this information prior to the call of process() ?

Please look at the javadocs of the Codec interface and examine our
Codec implementations. Each Codec has inputFormat and outputFormat
properties which are configured prior to calling #process on it.
Additionally, each Buffer supplied to #process may have a value for
its format property as well.

5. AudioFormat. The requirement of CELT is sample rate, frame size,
channels, bytes per frame, but frame size and bytes per frame don't have
correspondence member in AudioFormat. So how to set the input format and
supported format in JavaEncoder? And in the constructor AudioFormat(String
encoding), what's the proper value of encoding for CELT?

If frame size and bytes per frame don't deviate from the common case,
they're likely sample size * channels and bytes per sample * channels,
respectively.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#4

Hi guys,

Just my 2 cents about encoder/decoder interdependence.

If I understood correctly, the encoder needs a decoder instance because it
uses prediction and needs to evaluate some parameters?

No, it's about the parameters like rate, framesize, bytes per packet, etc.
Are these kinds of parameters constant when sip use this codec? If so, my
question turns to: how do I get them from FMJ?

If that's the case then we should consider putting both encoder and decoder
(for celt) in JavaEncoder. Or to put it another way, there is not
necessarily a one-to-one mapping between the codec encoder and the wrapped
encoder.

Thanks, this is also helpful for me anyways. In fact, the celt encoder and
decoder have already considered these prediction enc and dec things.

However one thing I guess you shouldn't do is call JavaDecoder from within
JavaEncoder. Calling celt_decode would be the way.

Nice reminding :slight_smile:

···

2010/7/14 Jean Lorchat <jean.lorchat@gmail.com>