[jitsi-dev] On Bitrate Adaptation for GSoC


#1

Update and Questions!
Update from Last time: Video call worked outside school network.
Research Update On the bitrate adaptation - task 3:
I have read and digested these papers - "Rate Adaptation for Adaptive HTTP Streaming" - http://dl.acm.org/citation.cfm?id=1943575 and "SARA" ieeexplore.ieee.org/iel7/7226899/7247062/07247436.pdf and I have a question with regards implementing adaptive bitrate streaming in jitsi.
In the 1st paper, they identified two types of rate adaptation techniques, Sender driven rate adaptation and Receiver driven rate adaptation.
The basic difference is that in the sender driven rate adaptation, the sender (the client end generating the video stream or a server) estimates network condition and determines which bitrate to use to encode the video stream.
In the receiver driven rate adaptation, the receiver(the client end requesting the video stream) computes the required bitrate and sends the bitrate to the server or client generating the video stream. In this model, it is assumed that the client/server generating the videostream encodes the video segments in multiple bitrates such that it can match requested bitrates to the closest available video segment with that bitrate. This model also has the advantage of scalability - am thinking of conference calling were different callers have different network conditions.
I will save the maths for now. I will be adapting and implementing the algorithm described in the "Rate Adaptation for Adaptive HTTP streaming" paper for task 3. SARA uses video segment size, buffer and throughput to estimate bitrate, I think it will be "bulky" for live streaming.
Questions:

From my understanding so far, I think am expected to the rate adaptation logic for the sending side, that is implementing the sender driven rate adaptation. Is this correct?

Is it possible that during a call, both clients encode in multiple bitrates to enable a receiver driven rate adaptation. I know this will consume memory and resources but is this an option.?
I have also been going through the libjitsi file and noticed that for every implementation in the impl folder, there seems to be an abstract class or interface in the "service" folder. I am not clearly sure why it so and would like to know.
Thanks,Regards,Julian.

Hi Boris,
Thanks for your response.I used regular XMPP accounts and it worked, video calls and desktop sharing became enabled although if I try to initiate a call(voice/video/desktop sharing) from any of the two ends (On windows - downloaded jitsi msi and Ubuntu running compiled jitsi version)
The caller displays:
Call ended by remote side. Reason: failed-application. Error: Could not establish connection (ICE failed and no relay found)
and callee displays - Error: Could not establish connection (ICE failed and no relay found).
As for the google talks, voice calls work. I don't know what might cause the error though I don't feel its the jitsi client. I will still re-try outside school network.

With regards this,
   
   - "The Jitsi client should just tell libjitsi (i.e. the MediaStream instance) to use adaptive rate
control, and the library should handle it from there."

This makes sense. I will look more into this. Thanks.

Thanks,Regards,Julian.

Hi Julian,

Hi Boris,
Thanks, got it. mvn install worked.

For this task;

* 3. Enable the use of bandwidth estimation and set the encoder's target

bitrate accordingly

I think I figured out were changes need to be made -
jitsi/src/net/java/sip/communicatior/impl/neomedia/MediaConfigurationImpl.java
in createVideoAdvancedSettings() method

1st if I implement the interface from the
libjitsi/src/org/jitsi/service/neomedia/rtp/bandwidthEstimator.java in
the jitsi project, I can be able to register a listener through the
BandwidthEstimator's "addListener(Listener listener)" which retrieves
the current estimated bandwidth using the BandwidthEstimator's
"getLatestEstimate()";,

Then I can use some "bitrate estimation equation/formular" which am
currently looking in to, to set the new bitrate in the JSpinner
videoBitrate. Already, I see a changeListener has been added to the
JSpinner to pick up any bit rate changes.

Q1. Please is the above reasoning sound?

Good job researching the problem! I think what you describe could work
(I'm a bit rusty with my knowledge of the jitsi client itself). However,
I would like to do this this in a different way. The Jitsi client should
just tell libjitsi (i.e. the MediaStream instance) to use adaptive rate
control, and the library should handle it from there.

I think that right now we don't support setting a target bitrate on the
encoder dynamically, so that would need to be implemented.

One other thing, I wanted to test the video call with another contact on
google talk (hangout) using the jitsi client. But the video call button
wasn't enabled (Both the libjitsi library I built and the old one worked
but the video wasn't enabled). Using jitsi client for chatting via works.
my webcam works when viewed under devices.(ie. from jitsi chat console >
Tools >options>device)

Q2) A quick question, What happens when all encoding options are checked
(> Tools >options> encoding)?

I'm not sure I understand the question. Is it is about which encoding
ends up being used? When a call is initialized there is an offer-answer
procedure, in which the two sides exchange their lists of supported
formats. They then use the highest priority format which is supported by
both.

Q3) How do I enable the video button ("Call contact")

I think this might have to do with hangouts. Can you try with some
regular XMPP accounts? There is a list of public XMPP servers here:
https://xmpp.net/directory.php

Regards,
Boris

···

On Friday, March 17, 2017 10:29 PM, CHIMAOBI JULIAN <jcchuks@ymail.com> wrote:
    On Friday, March 17, 2017 5:00 PM, Boris Grozev <boris@jitsi.org> wrote:
On 17/03/2017 14:55, CHIMAOBI JULIAN wrote:


#2

Hi Julian,

Update and Questions!

Update from Last time: Video call worked outside school network.

Research Update On the bitrate adaptation - task 3:

I have read and digested these papers - "Rate Adaptation for Adaptive
HTTP Streaming" - http://dl.acm.org/citation.cfm?id=1943575 and
"SARA" ieeexplore.ieee.org/iel7/7226899/7247062/07247436.pdf and I have
a question with regards implementing adaptive bitrate streaming in jitsi.

Great initiative!

In the 1st paper, they identified two types of rate adaptation
techniques, Sender driven rate adaptation and Receiver driven rate
adaptation.

The basic difference is that in the sender driven rate adaptation, the
sender (the client end generating the video stream or a server)
estimates network condition and determines which bitrate to use to
encode the video stream.

In the receiver driven rate adaptation, the receiver(the client end
requesting the video stream) computes the required bitrate and sends the
bitrate to the server or client generating the video stream. In this
model, it is assumed that the client/server generating the videostream
encodes the video segments in multiple bitrates such that it can match
requested bitrates to the closest available video segment with that
bitrate. This model also has the advantage of scalability - am thinking
of conference calling were different callers have different network
conditions.

I will save the maths for now. I will be adapting and implementing the
algorithm described in the "Rate Adaptation for Adaptive HTTP streaming"
paper for task 3. SARA uses video segment size, buffer and throughput
to estimate bitrate, I think it will be "bulky" for live streaming.

I'm afraid these papers are discussing a use-case which is different from ours. The term "live streaming" usually refers to a non-interactive scenario (e.g. streaming a video on youtube) as opposed to the real-time or interactive one (e.g. an audio/video conference).

Live streaming can tolerate a slight delay (say, a few seconds), which is why it is possible to divide the stream in segments and have clients download said segments over HTTP.

For real-time conferences we have to use a different approach.

Questions:

From my understanding so far, I think am expected to the rate adaptation
logic for the sending side, that is implementing the sender driven rate
adaptation.
Is this correct?

Well, as far as the original GSoC project idea goes, the actual implementation of the algorithm already exists, and you are expected to integrate it in the client.

Since you seem to be interested in the nitty-gritty, I can recommend a paper titled "Performance Analysis of Receive-Side Real-Time Congestion Control for WebRTC" by Varun Singh et al.

If such lower-level tasks interest you, we might be able to come up with another more suitable GSoC project in this area.

Is it possible that during a call, both clients encode in multiple
bitrates to enable a receiver driven rate adaptation. I know this will
consume memory and resources but is this an option.?

Yes! This is called simulcast, and we use it for conferences in jitsi-meet (it's not very useful for 1-to-1 calls, really).

I have also been going through the libjitsi file and noticed that for
every implementation in the impl folder, there seems to be an abstract
class or interface in the "service" folder. I am not clearly sure why it
so and would like to know.

It is an architecture decision, it separates the public API from the implementation.

Regards,
Boris

···

On 24/03/2017 20:13, CHIMAOBI JULIAN wrote: