[jitsi-dev] [libjitsi] general questions


#1

Hi all,

Just a few general questions about using the libjitsi and the conference focus concept correctly...

Let's say there is a video session with a conference focus and 3 connected peers.
Every peer has exactly one video stream with the conference focus so the conference focus has a total of 3 different video streams.

- On which of the 3 streams would the conference focus create the local visual component (always thinking that streams can be removed or added dynamically)?

- adding a new peer to a running session doesn't seem to be a problem, but closing a stream to a specific peer stops a sending cam on conference focus site and no video is sent anymore to all the other peers. So how is it supposed to realize adding and removing peers to/from a running session?

- how can an audio playback device be changed (before receiving stream and during running audio session) (portaudio)?

- when a new peer connects to a running video session and 2 H264 video streams are coming in at the same time, there is often a problem with the
FFmpeg.avcodec_open(avctx, avcodec) as it seems that this function isn't thread safe. Function returns -1 for the 2nd received stream and player can't be realized.
I could avoid this for testing purposes inserting the following in JNIDecoder.open() so I guess it's a threading problem:
if (++instanceCounter > 1)
    try {Thread.sleep(2000);} catch (InterruptedException e) {}
int value = FFmpeg.avcodec_open(avctx, avcodec);
System.err.println("Jitsi.JNIDecoder.open() FFmpeg.avcodec_open("+avctx+", "+avcodec+")="+value+" -> " + ((value<0) ? "NOK" : "OK"));
if (value < 0)
    throw new RuntimeException("Could not open codec CODEC_ID_H264");

Hope anyone can bring some light into that.
Thanks.


#2

Thank you for the feedback!

- On which of the 3 streams would the conference focus create the local visual component (always thinking that streams can be removed or added dynamically)?

Since the 3 video streams share one and the same MediaDevice and RTPTranslator, the intended behaviour is to not matter on which video stream the local visual Component is created. We're currently actively working on fixing video conference-related issues so, if this one does not work as expected, we'll try to address it soon.

- adding a new peer to a running session doesn't seem to be a problem, but closing a stream to a specific peer stops a sending cam on conference focus site and no video is sent anymore to all the other peers. So how is it supposed to realize adding and removing peers to/from a running session?

We've also noticed such problems and, as stated above, we'll try to address them soon.

- how can an audio playback device be changed (before receiving stream and during running audio session) (port audio)?

Jitsi is the reference implementation on the subject. Generally, just set the new MediaDevice on the MediaStream(s). On the conference focus, create a mixer on the new MediaDevice and set it on the MediaStream(s).

- when a new peer connects to a running video session and 2 H264 video streams are coming in at the same time, there is often a problem with the
FFmpeg.avcodec_open(avctx, avcodec) as it seems that this function isn't thread safe. Function returns -1 for the 2nd received stream and player can't be realized.
I could avoid this for testing purposes inserting the following in JNIDecoder.open() so I guess it's a threading problem:
if (++instanceCounter > 1)
    try {Thread.sleep(2000);} catch (InterruptedException e) {}
int value = FFmpeg.avcodec_open(avctx, avcodec);
System.err.println("Jitsi.JNIDecoder.open() FFmpeg.avcodec_open("+avctx+", "+avcodec+")="+value+" -> " + ((value<0) ? "NOK" : "OK"));
if (value < 0)
    throw new RuntimeException("Could not open codec CODEC_ID_H264");

Please report it in the form of a new issue at https://jitsi.org/index.php/Development/NewIssue.

···

On Jul 29, 2012, at 7:07 PM, Rene Lamotte <chummer25@gmx.net> wrote:


#3

- On which of the 3 streams would the conference focus create the local visual component (always thinking that streams can be removed or added dynamically)?

Since the 3 video streams share one and the same MediaDevice and RTPTranslator, the intended behaviour is to not matter on which video stream the local visual Component is created. We're currently actively working on fixing video conference-related issues so, if this one does not work as expected, we'll try to address it soon.

It seems to matter on which stream I create the local visual component. When I don't create it on the last initialized stream, the component stays gray and isn't painted (but with the correct size). Once initialized, a new stream can be added without any problem.

- how can an audio playback device be changed (before receiving stream and during running audio session) (port audio)?

Jitsi is the reference implementation on the subject. Generally, just set the new MediaDevice on the MediaStream(s). On the conference focus, create a mixer on the new MediaDevice and set it on the MediaStream(s).

So I would have to call two time MediaStream.setDevice()? One time with the capture device and one time with the playback device?

- when a new peer connects to a running video session and 2 H264 video streams are coming in at the same time, there is often a problem with the
FFmpeg.avcodec_open(avctx, avcodec) as it seems that this function isn't thread safe. Function returns -1 for the 2nd received stream and player can't be realized.
I could avoid this for testing purposes inserting the following in JNIDecoder.open() so I guess it's a threading problem:
if (++instanceCounter > 1)
    try {Thread.sleep(2000);} catch (InterruptedException e) {}
int value = FFmpeg.avcodec_open(avctx, avcodec);
System.err.println("Jitsi.JNIDecoder.open() FFmpeg.avcodec_open("+avctx+", "+avcodec+")="+value+" -> " + ((value<0) ? "NOK" : "OK"));
if (value < 0)
    throw new RuntimeException("Could not open codec CODEC_ID_H264");

Please report it in the form of a new issue at https://jitsi.org/index.php/Development/NewIssue.

Will open that issue this evening.


#4

I'm sorry, I answered about the capture device.

We don't have a public .service.neomedia API to change the audio playback device. We support the ConfigurationService properties net.java.sip.communicator.impl.neomedia.audioSystem and net.java.sip.communicator.impl.neomedia.audioSystem.<audioSystem>.playbackDevice i.e. for the case that you're asking about you should have net.java.sip.communicator.impl.neomedia.audioSystem equal to portaudio and net.java.sip.communicator.impl.neomedia.audioSystem.portaudio.playbackDevice equal to the locator of the playback device of your choice.

···

On Jul 30, 2012, at 3:48 PM, "Rene Lamotte" <chummer25@gmx.net> wrote:

- how can an audio playback device be changed (before receiving stream and during running audio session) (port audio)?

So I would have to call two time MediaStream.setDevice()? One time with the capture device and one time with the playback device?


#5

Could you please give me an example of how I could change this playback device during a running audio session?
Sorry about the stupid question but I can't find any reference in the jitsi code.

public static void setPlaybackAudioDevice(CaptureDeviceInfo cdi) {
        audioPlaybackDevice = cdi;
        ConfigurationService cs = LibJitsi.getConfigurationService();
        cs.setProperty("net.java.sip.communicator.impl.neomedia.audioSystem", "portaudio");
        cs.setProperty("net.java.sip.communicator.impl.neomedia.audioSystem.portaudio.playbackDevice",
                (cdi == null) ? "" : cdi.getLocator().toString());

        /*
          * I guess I must somehow reinitialize the AudioSystem or even the streams???
          * Just trying below as this is not working. :wink:
          */
        DeviceConfiguration conf = ((MediaServiceImpl) LibJitsi.getMediaService()).getDeviceConfiguration();
        AudioSystem sys = conf.getAudioSystem();
        sys.setPlaybackDevice(cdi, true);
        Renderer renderer = sys.createRenderer(true);
        if (cdi != null)
            renderer.start();
        else
            renderer.stop();
    }

Thanks in advance.

-----Urspr�ngliche Nachricht-----

···

From: Lyubomir Marinov

Sent: Monday, July 30, 2012 9:32 PM
To: dev@jitsi.java.net
Subject: [jitsi-dev] Re: [libjitsi] general questions

On Jul 30, 2012, at 3:48 PM, "Rene Lamotte" <chummer25@gmx.net> wrote:

- how can an audio playback device be changed (before receiving stream and during running audio session) (port audio)?

So I would have to call two time MediaStream.setDevice()? One time with the capture device and one time with the playback device?

I'm sorry, I answered about the capture device.

We don't have a public .service.neomedia API to change the audio playback device. We support the ConfigurationService properties net.java.sip.communicator.impl.neomedia.audioSystem and net.java.sip.communicator.impl.neomedia.audioSystem.<audioSystem>.playbackDevice i.e. for the case that you're asking about you should have net.java.sip.communicator.impl.neomedia.audioSystem equal to portaudio and net.java.sip.communicator.impl.neomedia.audioSystem.portaudio.playbackDevice equal to the locator of the playback device of your choice.


#6

Could you please give me an example of how I could change this playback device during a running audio session?

We do not have a public API to change the audio playback device.

      ConfigurationService cs = LibJitsi.getConfigurationService();
      cs.setProperty("net.java.sip.communicator.impl.neomedia.audioSystem", "portaudio");
      cs.setProperty("net.java.sip.communicator.impl.neomedia.audioSystem.portaudio.playbackDevice",
              (cdi == null) ? "" : cdi.getLocator().toString());

Setting the ConfigurationService properties will not cause a running audio session to pick up the changes in their values.

We'll work on these.

      Renderer renderer = sys.createRenderer(true);
      if (cdi != null)
          renderer.start();
      else
          renderer.stop();

DeviceSystem#createRenderer(boolean) is internal neomedia API, please do not use it.

Additionally, javax.media.Renderer is a JMF/FMJ plug-in and it cannot be manipulated by #start() and #stop() alone. The official JMF documentation should be a good starting point.

Anyway, I presume the source code has been written for the purposes of testing and figuring out how to switch the playback device only.

···

On Aug 7, 2012, at 4:46 PM, Rene Lamotte <chummer25@gmx.net> wrote:


#7

Could you please give me an example of how I could change this playback device during a running audio session?

We do not have a public API to change the audio playback device.

      ConfigurationService cs = LibJitsi.getConfigurationService();

cs.setProperty("net.java.sip.communicator.impl.neomedia.audioSystem", "portaudio");

cs.setProperty("net.java.sip.communicator.impl.neomedia.audioSystem.portaudio.playbackDevice",
              (cdi == null) ? "" : cdi.getLocator().toString());

Setting the ConfigurationService properties will not cause a running audio session to pick up the changes in their values.
We'll work on these.

Ok, this means I can define the playback device BEFORE creating the streams via the ConfigurationService.
It would be great to have this feature in future releases as sometime routing the playback to another output is important.

      Renderer renderer = sys.createRenderer(true);
      if (cdi != null)
          renderer.start();
      else
          renderer.stop();

DeviceSystem#createRenderer(boolean) is internal neomedia API, please do not use it.

Additionally, javax.media.Renderer is a JMF/FMJ plug-in and it cannot be manipulated by #start() and #stop() alone. The official JMF documentation should > be a good starting point.

Anyway, I presume the source code has been written for the purposes of testing and figuring out how to switch the playback device only.

Yes, this was for testing only.

Thanks for your answer Lyubomir.