[sip-comm-dev] Screenshot of a "source resized screen" with desktop streaming


#1

Hi,

Yesterday I made some test and try to find a way to tell our brand new scaler (which use libswscale) to resize at a "user" desired resolution (instead to output at the capture resolution).

And I found one :slight_smile: . In MediaDeviceSession::setFormat(Processor processor, Format format), format has just the encoding set (H264/RTP) and we can add the desired size in it (I can made a less intrusive way to do that by override this method in VideoMediaDeviceSession). But (yes there is a but) the results in really bad (see desktop_1280x1024_resized_to_1000x800_rgbconverter.png). The fact that the images is first resized from RGB 1280x1024 to RGB 1000x800 via JMF's RGBConverter/JavaRGBConverter and then change format via our new SwScaler.

I try to remove JavaRGBConverter/RGBConverter from JMF plugin and play with supportedInput/outputFormats but filter graph failed. So today I will look at the way that RGBConverter works and try to resize/scale in one call.

So that is the bad thing, now a better new for our SwScaler :slight_smile: . If I hardcode output format in SwScaler (let's say 1000x800 to have right ratio with 1280x1024) I have a wonderful resized image :slight_smile: . See desktop_1280x1024_resized_to_1000x800.png, desktop_1280x1024_resized_to_1000x800_video.png and desktop_1280x1024_resized_to_1000x800_video_fullscreen.png. (Note I use the FullScreenLayout's overlay set to true to have image fit all to the screen).

A few word in case I don't manage to make RGB to YUV rescaling with desired resolution via JMF/filtergraph stuff, we can made an accessor for setting SwScaler's outputFormat but yes, I know it is pretty bad and I will do my best to find another way.
BTW for negocating resolution in SIP/SDP we can inspire us of this ietf-draft: http://tools.ietf.org/html/draft-ietf-mmusic-image-attributes-04

Regards,

路路路

--
Seb


#2

I accidentally bumped into Comparing Colour Space Conversion Libraries
at http://www.bluishcoder.co.nz/2010/02/19/comparing-colour-space-conversion-libraries.html
and I thought it was kind of related to what we're doing or at least a
somewhat interesting read.

路路路

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#3

Hi all,

Last friday, I still didn't find a way to make rescaling/change format with specified size in one call work. But like I said another solution is to set SwScaler's output size.

Today I experiment this in Video/MediaDeviceSession by creating a new SwScaler object, set its output size and finally set the codec chain (JMF's setCodecChain) for all video TrackControl objects and it is working fine. BTW if we haven't any output size to set, we do the classic way (because the SwScaler will just change format and no rescale).

WDYT, Is it an acceptable solution ?

Regards,

路路路

--
Seb

Sebastien Vincent a 锟絚rit :

Hi,

Yesterday I made some test and try to find a way to tell our brand new scaler (which use libswscale) to resize at a "user" desired resolution (instead to output at the capture resolution).

And I found one :slight_smile: . In MediaDeviceSession::setFormat(Processor processor, Format format), format has just the encoding set (H264/RTP) and we can add the desired size in it (I can made a less intrusive way to do that by override this method in VideoMediaDeviceSession). But (yes there is a but) the results in really bad (see desktop_1280x1024_resized_to_1000x800_rgbconverter.png). The fact that the images is first resized from RGB 1280x1024 to RGB 1000x800 via JMF's RGBConverter/JavaRGBConverter and then change format via our new SwScaler.

I try to remove JavaRGBConverter/RGBConverter from JMF plugin and play with supportedInput/outputFormats but filter graph failed. So today I will look at the way that RGBConverter works and try to resize/scale in one call.

So that is the bad thing, now a better new for our SwScaler :slight_smile: . If I hardcode output format in SwScaler (let's say 1000x800 to have right ratio with 1280x1024) I have a wonderful resized image :slight_smile: . See desktop_1280x1024_resized_to_1000x800.png, desktop_1280x1024_resized_to_1000x800_video.png and desktop_1280x1024_resized_to_1000x800_video_fullscreen.png. (Note I use the FullScreenLayout's overlay set to true to have image fit all to the screen).

A few word in case I don't manage to make RGB to YUV rescaling with desired resolution via JMF/filtergraph stuff, we can made an accessor for setting SwScaler's outputFormat but yes, I know it is pretty bad and I will do my best to find another way.
BTW for negocating resolution in SIP/SDP we can inspire us of this ietf-draft: http://tools.ietf.org/html/draft-ietf-mmusic-image-attributes-04

Regards,

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#4

WDYT, Is it an acceptable solution ?

Hello Seb,

Presuming that setting the negotiated video output size on the input
and the output DataSource doesn't tell it to the libswscaler as well,
I think your solution is acceptable because there is no other idea and
it's the only working approach.

Regards,
Lubo

路路路

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#5

Lubomir Marinov a 锟絚rit :

WDYT, Is it an acceptable solution ?
聽聽聽聽
Hello Seb,

Presuming that setting the negotiated video output size on the input
and the output DataSource doesn't tell it to the libswscaler as well,
I think your solution is acceptable because there is no other idea and
it's the only working approach.
聽聽
OK thanks for your answer.

I will test a little bit more and clean up before commiting my changes (and recompile libffmpeg library for all of our supported platforms).

路路路

--
Seb

Regards,
Lubo

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net