[sip-comm-dev] Memory usage with desktop streaming


#1

Hi,

Use desktop streaming features with a big screen consumes a lot of memory. Lubomir suggests offline to use a PullBufferDataSource/Stream rather than a PushBufferDataSource/Stream to avoid more big allocations.

Here some statistics about CPU and memory usage on a Debian GNU/Linux x86-64 running latest trunk with only one peer that stream its 1280x800 desktop.
CPU: between 74% and 80%
RAM: ~533 Mo

A prototype of "Pull" desktop streaming device has been implemented, and here the results:
CPU: between 70% and 80%
RAM: ~269 Mo then drops to ~229,6 Mo 2-3 minutes after

To compare with webcam (lti-civil linux):
CPU: between 64% and 72%
RAM: ~310,4 Mo

The major changes are in ImageStream class which not longer spawn a thread (to grab screen), in fact all is done in the read(Buffer buf) method.
We check if buf contains a valid data object (getData()) and if its length is sufficient to contains screen (otherwise we use our own allocated buffer). Normally first time read is called, buf's data has a size of 500, but in the next call data has the right size (I think it reuse our allocated buffer). We try to respect a 10 frames per second (hardcoded for the moment) so we sleep time needed to respect frame rate after capture (approximately the read method is called every 70 - 80 milliseconds on Linux). Note with no sleep java process eat arround 90-92% (in SIP session with H264 encoding).

Other changes includes addition of AbstractPullBufferCaptureDevice and AbstractPullBufferStream classes but I simply copy/paste/adapt from AbstractPushBufferCaptureDevice/Stream, maybe it is not good (copy/paste is evil) but the Pull/Push* interfaces looks very similar. Maybe implement interfaces needed in DataSource and ImageStream like in PortAudio DataSource would be another solution. WDYT

Other thing, I remark local video is missing (error when creating the SourceCloneable).

Regards,

···

--
Seb

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#2

Hi Seb,

Please excuse my asking in case it's clear from your e-mail but did
you change the JNI interface/the native counterpart to capture into
the byte[] array of the Buffer? I'm not sure what is best here given
that GetByteArrayElements is likely to make a copy of the Java byte[].
An alternative approach I guess is to cache a native buffer to capture
into and then SetByteArrayRegion. Anyway, the idea of both approaches
is to remove the allocations of native buffers on each screen capture.

Regards,
Lubo

···

On Wed, Feb 10, 2010 at 4:55 PM, Sebastien Vincent <seb@sip-communicator.org> wrote:

Use desktop streaming features with a big screen consumes a lot of memory.
Lubomir suggests offline to use a PullBufferDataSource/Stream rather than a
PushBufferDataSource/Stream to avoid more big allocations.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#3

Lubomir Marinov a �crit :

  

Use desktop streaming features with a big screen consumes a lot of memory.
Lubomir suggests offline to use a PullBufferDataSource/Stream rather than a
PushBufferDataSource/Stream to avoid more big allocations.
    
Hi Seb,

Please excuse my asking in case it's clear from your e-mail but did
you change the JNI interface/the native counterpart to capture into
the byte[] array of the Buffer?

No I have not changed JNI stuff yet. For the moment JNI still creates bytes array (for each call). But instead of creating another arrays for converting ARGB images to bytes, I use directly Buffer's data.

I'm not sure what is best here given
that GetByteArrayElements is likely to make a copy of the Java byte[].
An alternative approach I guess is to cache a native buffer to capture
into and then SetByteArrayRegion. Anyway, the idea of both approaches
is to remove the allocations of native buffers on each screen capture.

Yes I will see this this week, to optimize JNI part.

Thanks,

···

On Wed, Feb 10, 2010 at 4:55 PM, Sebastien Vincent > <seb@sip-communicator.org> wrote:

--
Seb

Regards,
Lubo

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#4

Thank you! It's important because you're currently holding at least
9+9Mbytes on the Java side for Emil's big display.

Lubomir Marinov a écrit :

...

No I have not changed JNI stuff yet. For the moment JNI still creates bytes
array (for each call). But instead of creating another arrays for converting
ARGB images to bytes, I use directly Buffer's data.

I'm not sure what is best here given
that GetByteArrayElements is likely to make a copy of the...

Yes I will see this this week, to optimize JNI part.

Thanks,

···

On 10 Feb 2010 18:31, "Sebastien Vincent" <seb@sip-communicator.org> wrote:

On Wed, Feb 10, 2010 at 4:55 PM, Sebastien Vincent > <seb@sip-communicator.org> wrote:

--
Seb

Regards,

Lubo

>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:...

---------------------------------------------------------------------
To unsubscribe, e-mail: dev...


#5

Hi,

I change a little bit the JNI interfaces for screen capture. Now the native screen grabbers write bytes directly in a byte array we pass in.

So I have tested three methods each ones is used in a (screen's width x screen's height) for loop:
- Get a pointer to byte array (most of times it is a copy) with GetByteArrayElements and then write directly the four-bytes integer pixel;
- Use SetByteArrayRegion with the four-bytes integer pixel;
- Get the real pointer to byte array with GetPrimitiveArrayCritical then write directly the four-bytes integer pixel.

For more information about GetPrimitiveArrayCritical see http://java.sun.com/j2se/1.3/docs/guide/jni/jni-12.html#GetPrimitiveArrayCritical

In order to have just the amount of memory used for grabbing, I show the result in _preview_ panel and _not_ in a SIP session. The laptop screen is 1280x800 running Linux x86-64. Values are from `top` command and gnome-system-monitor.

GetByteArrayElements
CPU: 22%
RAM: 164,2 mo

SetByteArrayRegion:
CPU: 45% (I think this function is great for a large amount of data set in one pass rather than just 4 bytes 1280 x 800 times).
RAM: 159 mo

GetPrimitiveArrayCritical:
CPU: 18%
RAM: 156,5 mo

Sometimes memory consumption are much higher or not (maybe it is because of Java internal cache...). But CPU charge is the approximately the same for every test.

For information, in a SIP session with one desktop streaming, sender's CPU charge are ~60% for GetByteArrayElements, ~71% for SetByteArrayRegion and ~55% for GetPrimitiveArrayCriticial.

Regards,

···

--
Seb

Lubomir Marinov a �crit :

Thank you! It's important because you're currently holding at least 9+9Mbytes on the Java side for Emil's big display.

On 10 Feb 2010 18:31, "Sebastien Vincent" <seb@sip-communicator.org >> <mailto:seb@sip-communicator.org>> wrote:

Lubomir Marinov a �crit :

>
> On Wed, Feb 10, 2010 at 4:55 PM, Sebastien Vincent >> > <seb@sip-communicator.org <mailto:seb@sip-communicator.org>> wrote:
> >>
>...

No I have not changed JNI stuff yet. For the moment JNI still creates bytes array (for each call). But instead of creating another arrays for converting ARGB images to bytes, I use directly Buffer's data.

> I'm not sure what is best here given
> that GetByteArrayElements is likely to make a copy of the...

Yes I will see this this week, to optimize JNI part.

Thanks,

--
Seb

    Regards,
    Lubo

    >
    ---------------------------------------------------------------------
    > To unsubscribe, e-mail:...

---------------------------------------------------------------------
To unsubscribe, e-mail: dev...

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net