[jitsi-dev] Echo cancellation with multiple playback streams


#1

Hi,

As you may know, Jitsi doesn't support echo cancellation for PulseAudio, which I believe Jitsi defaults to on Linux. The PortAudio backend does support it, though it doesn't perform very well.

It turns out PulseAudio includes a WebRTC-based echo canceller since version 2.0, which was released over two years ago. Enabling it programmatically would be a one-liner. Unfortunately, two years later Ubuntu still has it disabled at compile time [1].

Therefore, I was thinking about implementing a WebRTC-based echo canceller right inside Jitsi, like it's currently done for the Mac CoreAudio backend.

It seems more or less straightforward to do, except when you have more than one playback stream, which I presume would be the case in a conference mode. Looking at how the Mac CoreAudio backend handles this, it seems plain wrong. The way I understood the source code, it feeds audio samples from different playback streams into the same echo canceller, as if those samples came from the same playback stream, sequentially. That doesn't sound right to me and that's what I want to discuss. What's the proper way of handling this? Mix your playback streams and feed that into the echo canceller? Sounds too hard to implement. In fact, that would be like reimplementing a good part of PulseAudio, given that you will have to deal with timings and possibly different sampling rates. Have a separate echo canceller for each combination of capture and playback stream and then let each echo canceller associated with the given capture stream modify the data coming from a microphone? Sounds like an overkill, and I've no idea if it's going to work at all. Any other options?

[1]: https://bugs.launchpad.net/ubuntu/+source/pulseaudio/+bug/1261666


#2

На 31.05.14, 10:25, Joseph Artsimovich написа:

As you may know, Jitsi doesn't support echo cancellation for PulseAudio, which I believe Jitsi defaults to on Linux. The PortAudio backend does support it, though it doesn't perform very well.

It turns out PulseAudio includes a WebRTC-based echo canceller since version 2.0, which was released over two years ago. Enabling it programmatically would be a one-liner.

I think we/Jitsi signal to PulseAudio that we would like echo cancellation if available.

Unfortunately, two years later Ubuntu still has it disabled at compile time [1].

Therefore, I was thinking about implementing a WebRTC-based echo canceller right inside Jitsi, like it's currently done for the Mac CoreAudio backend.

It seems more or less straightforward to do, except when you have more than one playback stream, which I presume would be the case in a conference mode.

The multiple remote strems to be played back in a conference in Jitsi are mixed into a single local playback stream.

Looking at how the Mac CoreAudio backend handles this, it seems plain wrong. The way I understood the source code, it feeds audio samples from different playback streams into the same echo canceller, as if those samples came from the same playback stream, sequentially. That doesn't sound right to me and that's what I want to discuss. What's the proper way of handling this? Mix your playback streams and feed that into the echo canceller? Sounds too hard to implement. In fact, that would be like reimplementing a good part of PulseAudio, given that you will have to deal with timings and possibly different sampling rates. Have a separate echo canceller for each combination of capture and playback stream and then let each echo canceller associated with the given capture stream modify the data coming from a microphone? Sounds like an overkill, and I've no idea if it's going to work at all. Any other options?

Well, these above are exactly the reason why the WebRTC-based echo canceller in Mac CoreAudio is implemented the way it is. Which is the same as the Speex-based echo canceller in PortAudio, by the way. These implementations cover the most common cases in which Jitsi finds itself.

···

[1]: https://bugs.launchpad.net/ubuntu/+source/pulseaudio/+bug/1261666


#3

На 31.05.14, 10:25, Joseph Artsimovich написа:

As you may know, Jitsi doesn't support echo cancellation for PulseAudio, which I believe Jitsi defaults to on Linux. The PortAudio backend does support it, though it doesn't perform very well.

It turns out PulseAudio includes a WebRTC-based echo canceller since version 2.0, which was released over two years ago. Enabling it programmatically would be a one-liner.

I think we/Jitsi signal to PulseAudio that we would like echo cancellation if available.

I don't think Jitsi does. Google tells me the way to tell PulseAudio to enable echo cancellation is:

setenv("PULSE_PROP", "filter.want=echo-cancel", 1);

I did a search for PULSE_PROP in both jitsi and libjitsi repositories but couldn't find anything.

Unfortunately, two years later Ubuntu still has it disabled at compile time [1].

Therefore, I was thinking about implementing a WebRTC-based echo canceller right inside Jitsi, like it's currently done for the Mac CoreAudio backend.

It seems more or less straightforward to do, except when you have more than one playback stream, which I presume would be the case in a conference mode.

The multiple remote strems to be played back in a conference in Jitsi are mixed into a single local playback stream.

Does that mean I can assume there will only be a single (active) PulseAudioRenderer instance with mediaRole == MEDIA_ROLE_PHONE? That would certainly help.

Another question then:
What's the best way to make PulseAudioRenderer and DataSource.PulseAudioStream share an instance of an echo canceller? In Mac CoreAudio backend the echo canceller is encapsulated in native code and uses a static list of capture streams. In case of PulseAudio, encapsulation in native code is probably not an option, as Jitsi pretty much exports the native PulseAudio API via JNI, and then works with that from Java. Any better approach to link the capture and render streams together on the Java side?

···

On 31/05/14 11:42, Lyubomir Marinov wrote:

--
Joseph


#4

На 31.05.14, 14:17, Joseph Artsimovich написа:

···

On 31/05/14 11:42, Lyubomir Marinov wrote:

I think we/Jitsi signal to PulseAudio that we would like echo cancellation if available.

I don't think Jitsi does. Google tells me the way to tell PulseAudio to enable echo cancellation is:

setenv("PULSE_PROP", "filter.want=echo-cancel", 1);

I did a search for PULSE_PROP in both jitsi and libjitsi repositories but couldn't find anything.

Well, when we were writing the PulseAudio support in Jitsi, the PulseAudio promise was that acoustic echo cancellation would be enabled by default if available for streams with media role phone.


#5

I remember reading they had plans to make it a default, so I had to check whether they did so. The way to check it is by doing the following while being in the middle of a call:

pacmd list-modules | grep echo

If echo cancellation is enabled, you'll see output like this:

name: <module-echo-cancel>

The results on my machine (Ubuntu 14.04) show that echo cancellation *is not* enabled by default. If you set:

export PULSE_PROP="filter.want=echo-cancel"

then the echo cancellation module is loaded, though without webrtc being compiled in, it hardly makes any difference :frowning:

Now, going back to my previous question about the possibility of having multiple active PulseAudioRenderer instances. While it may be true it won't happen in a conference call (can anyone confirm?), I noticed Jitsi supports having multiple calls active at the same time. Will that scenario have multiple PulseAudioRenderer instances? If so, I am correct to assume the Mac CoreAudio WebRTC-based echo canceller won't work (well) in this case?

···

On 31/05/14 12:28, Lyubomir Marinov wrote:

На 31.05.14, 14:17, Joseph Artsimovich написа:

On 31/05/14 11:42, Lyubomir Marinov wrote:

I think we/Jitsi signal to PulseAudio that we would like echo cancellation if available.

I don't think Jitsi does. Google tells me the way to tell PulseAudio to enable echo cancellation is:

setenv("PULSE_PROP", "filter.want=echo-cancel", 1);

I did a search for PULSE_PROP in both jitsi and libjitsi repositories but couldn't find anything.

Well, when we were writing the PulseAudio support in Jitsi, the PulseAudio promise was that acoustic echo cancellation would be enabled by default if available for streams with media role phone.

--
Joseph


#6

Hi , echo cancellation from pulse applied to a softphone should be this filter.want=echo-cancel media.role=phone and coud be used in a shell or in a launcher adding export PULSE_PROP="filter.want=echo-cancel media.role=phone"

Hope this helps - John

···

on 01/06/2014 09:45, Joseph Artsimovich wrote:

On 31/05/14 12:28, Lyubomir Marinov wrote:

На 31.05.14, 14:17, Joseph Artsimovich написа:

On 31/05/14 11:42, Lyubomir Marinov wrote:

I think we/Jitsi signal to PulseAudio that we would like echo cancellation if available.

I don't think Jitsi does. Google tells me the way to tell PulseAudio to enable echo cancellation is:

setenv("PULSE_PROP", "filter.want=echo-cancel", 1);

I did a search for PULSE_PROP in both jitsi and libjitsi repositories but couldn't find anything.

Well, when we were writing the PulseAudio support in Jitsi, the PulseAudio promise was that acoustic echo cancellation would be enabled by default if available for streams with media role phone.

I remember reading they had plans to make it a default, so I had to check whether they did so. The way to check it is by doing the following while being in the middle of a call:

pacmd list-modules | grep echo

If echo cancellation is enabled, you'll see output like this:

name: <module-echo-cancel>

The results on my machine (Ubuntu 14.04) show that echo cancellation *is not* enabled by default. If you set:

export PULSE_PROP="filter.want=echo-cancel"

then the echo cancellation module is loaded, though without webrtc being compiled in, it hardly makes any difference :frowning:

Now, going back to my previous question about the possibility of having multiple active PulseAudioRenderer instances. While it may be true it won't happen in a conference call (can anyone confirm?), I noticed Jitsi supports having multiple calls active at the same time. Will that scenario have multiple PulseAudioRenderer instances? If so, I am correct to assume the Mac CoreAudio WebRTC-based echo canceller won't work (well) in this case?

--
Joseph

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#7

Yes, that's correct. Echo cancellation will not work for two separate simultaneous calls currently ... but this is not a use case we consider as highly prioritary.

Emil

···

On 01.06.14, 10:45, Joseph Artsimovich wrote:

Now, going back to my previous question about the possibility of having
multiple active PulseAudioRenderer instances. While it may be true it
won't happen in a conference call (can anyone confirm?), I noticed Jitsi
supports having multiple calls active at the same time. Will that
scenario have multiple PulseAudioRenderer instances? If so, I am correct
to assume the Mac CoreAudio WebRTC-based echo canceller won't work
(well) in this case?

--
https://jitsi.org


#8

In case of Jitsi, media.role=phone is not necessary, as Jitsi sets that one using PulseAudio API.

···

On 01/06/14 09:47, john wrote:

Hi , echo cancellation from pulse applied to a softphone should be this filter.want=echo-cancel media.role=phone and coud be used in a shell or in a launcher adding export PULSE_PROP="filter.want=echo-cancel media.role=phone"

Hope this helps - John

on 01/06/2014 09:45, Joseph Artsimovich wrote:

On 31/05/14 12:28, Lyubomir Marinov wrote:

На 31.05.14, 14:17, Joseph Artsimovich написа:

On 31/05/14 11:42, Lyubomir Marinov wrote:

I think we/Jitsi signal to PulseAudio that we would like echo cancellation if available.

I don't think Jitsi does. Google tells me the way to tell PulseAudio to enable echo cancellation is:

setenv("PULSE_PROP", "filter.want=echo-cancel", 1);

I did a search for PULSE_PROP in both jitsi and libjitsi repositories but couldn't find anything.

Well, when we were writing the PulseAudio support in Jitsi, the PulseAudio promise was that acoustic echo cancellation would be enabled by default if available for streams with media role phone.

I remember reading they had plans to make it a default, so I had to check whether they did so. The way to check it is by doing the following while being in the middle of a call:

pacmd list-modules | grep echo

If echo cancellation is enabled, you'll see output like this:

name: <module-echo-cancel>

The results on my machine (Ubuntu 14.04) show that echo cancellation *is not* enabled by default. If you set:

export PULSE_PROP="filter.want=echo-cancel"

then the echo cancellation module is loaded, though without webrtc being compiled in, it hardly makes any difference :frowning:

Now, going back to my previous question about the possibility of having multiple active PulseAudioRenderer instances. While it may be true it won't happen in a conference call (can anyone confirm?), I noticed Jitsi supports having multiple calls active at the same time. Will that scenario have multiple PulseAudioRenderer instances? If so, I am correct to assume the Mac CoreAudio WebRTC-based echo canceller won't work (well) in this case?

--
Joseph

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev