[jitsi-dev] [jitsi-videobridge] Missing remote downstream RTP media (audio/video) - Jitsi-Videobridge BUG ???


#1

Hi Boris Grozev,

After I terminated the videobridge call, I observed the following debug messages log from JVB.

JVB 2016-02-12 13:16:36.561 INFO: [65] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10004/udp/host -> 92.168.1.100:5001/udp/host (audio.RTCP), failing.
JVB 2016-02-12 13:16:36.562 INFO: [63] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10003/udp/host -> 192.168.1.100:5000/udp/host (audio.RTP), failing.
JVB 2016-02-12 13:16:49.041 INFO: [125] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10002/udp/host -> 118.201.189.22:5001/udp/srflx (audio.RTCP), failing.
JVB 2016-02-12 13:16:49.041 INFO: [124] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10001/udp/host -> 118.201.189.22:5000/udp/srflx (audio.RTP), failing.
JVB 2016-02-12 13:16:51.557 INFO: [127] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10004/udp/host -> 192.168.1.100:5001/udp/host (audio.RTCP), failing.
JVB 2016-02-12 13:16:51.558 INFO: [126] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10003/udp/host -> 192.168.1.100:5000/udp/host (audio.RTP), failing.
JVB 2016-02-12 13:17:04.040 INFO: [63] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10001/udp/host -> 118.201.189.22:5000/udp/srflx (audio.RTP), failing.
JVB 2016-02-12 13:17:04.041 INFO: [65] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10002/udp/host -> 118.201.189.22:5001/udp/srflx (audio.RTCP), failing.

Look like JVB was actually sending UDP data out during the active call session???
But why none is being observed on wireshark or received by both jitsi-android and jitsi-windows clients???

···

=======================================

Hi Boris Grozev,

Many thanks for the clarification - the timeout is only the result of the proposed candidates validity check, and they have nothing to do with the streaming.

On further checking/experiment using wireshark running on the xmpp/jvb server:
I observed that there are continuous UDP data streaming from both the
Jitsi-android (192.168.1.100) and Jitsi-windows ( 118.201.189.22)
to the JVB server 192.168.1.8 (port map from 115.66.221.38 )

==== wireshark log ========
.....
12403 0.000000000 192.168.1.100 192.168.1.8 UDP 148 5000 → 10003 Len=104
12404 0.014788975 118.201.189.22 192.168.1.8 UDP 62 5000 → 10001 Len=15
12405 0.014788975 118.201.189.22 192.168.1.8 UDP 62 5000 → 10001 Len=15
12406 0.022708100 192.168.1.100 192.168.1.8 UDP 144 5000 → 10003 Len=100
12407 0.040265327 192.168.1.100 192.168.1.8 UDP 141 5000 → 10003 Len=97
.....

However I did not see any UDP data streaming from JVB (115.66.221.38) to either jitsi-android or jitsi-windows.
Any advice on what else can I check to understand why there is no udp streaming data sent from jvb?
What can be the possible reasons (option not set?) that cause jvb failed to streaming UDP data?

Thanks & Regards,
CM Eng

-----Original Message-----
From: dev [mailto:dev-bounces@jitsi.org] On Behalf Of dev-request@jitsi.org
Sent: Friday, February 12, 2016 11:52 AM
To: dev@jitsi.org
Subject: dev Digest, Vol 35, Issue 53

Send dev mailing list submissions to
  dev@jitsi.org

To subscribe or unsubscribe via the World Wide Web, visit
  http://lists.jitsi.org/mailman/listinfo/dev
or, via email, send a message with subject or body 'help' to
  dev-request@jitsi.org

You can reach the person managing the list at
  dev-owner@jitsi.org

When replying, please edit your Subject line so it is more specific than "Re: Contents of dev digest..."

Today's Topics:

   1. Re: [jitsi-videobridge] Missing remote downstream RTP media
      (audio/video) - Jitsi-Videobridge BUG ??? (Boris Grozev)
   2. [jitsi-android] What are future plans for this project? (cmeng.gm)

----------------------------------------------------------------------

Message: 1
Date: Thu, 11 Feb 2016 21:43:54 -0600
From: Boris Grozev <boris@jitsi.org>
To: Jitsi Developers <dev@jitsi.org>
Subject: Re: [jitsi-dev] [jitsi-videobridge] Missing remote downstream
  RTP media (audio/video) - Jitsi-Videobridge BUG ???
Message-ID: <56BD54FA.3030401@jitsi.org>
Content-Type: text/plain; charset=utf-8; format=flowed

On 11/02/16 21:18, cmeng.gm wrote:

Hi,

After an in depth investigation on the problem, following are my findings:

(Note: timestamp may not teddy with clients and JVB server logs as
they are captured at different day/time;

During investigation, one of the jabberd client is a jitsi-android
(self enhanced version) which provides full xmpp logs and debug messages.

The observed problem also happen on jitsi-windows and jitsi-ubuntu,
hence ruling out jitsi-android as the cause.)

When connected to jit.si server, the jabberd client did received
“ReceiveStreamPushBufferDataSource” from JVB and proceed to setup the
full audio graph

to receiving incoming downstream audio data as seen from the log.

====== log on android-jitsi (log on jit.si server) ============

02-10 14:37:30.355: I/Jitsi(12450): [96] net.sf.fmj.media.Log.info()
Resetting queue, last seq added: 9223372036854775806, current seq:
40105

02-10 14:37:30.355: I/Jitsi(12450): [97] fmj.createProcessor() ###
Creating Processor for DataSource ###:
org.jitsi.impl.neomedia.device.ReceiveStreamPushBufferDataSource@cab9a
ef
for type: raw

==============================================

However, when connected to my own XMPP/JVB server, there is no
downstream audio received i.e. missing
“ReceiveStreamPushBufferDataSource” from JVB.

This leads to the observed problem as it does not procced to setup the
incoming decoding audio graph, and hence there is no audio received on
jitsi-android.

#### Following depict the actual own system setup for investigation:

NAT Port map 10000~20000 for 115.66.221.38 <==> 192.168.1.8 as
required when JVB is behind NAT.

# NAT setting if run behind firewall for sip-communicator.properties

org.jitsi.videobridge.NAT_HARVESTER_LOCAL_ADDRESS=192.168.1.8

org.jitsi.videobridge.NAT_HARVESTER_PUBLIC_ADDRESS=115.66.221.38

192.168.1.8 (XMPP/JVB) < == 115.66.221.38 (Public) <=====>
118.201.189.22 (public) ==> 192.168.1.108 (jitsi-windows)

                                                               >>

192.168.1.100 (jitsi-android) <==

I then proceed to execute JVB under live mode and captured the log
message as below

Following are my conclusion/understanding from the logged messages:

1. Gather candidates for component audio.RTP/RTPC has harvested 2
proposed candidates for connection (see log).

2. CheckList.handleNominationConfirmed() confirm the selected
candidate pair for RTP/RTCP stream connection

#### Selected pair for stream audio.RTP: 192.168.1.8:10001/udp/host ->
118.201.189.22:5000/udp/srflx (audio.RTP)

#### Selected pair for stream audio.RTCP: 192.168.1.8:10002/udp/host
-> 118.201.189.22:5001/udp/srflx (audio.RTCP

3. JVB decides to use remote client local address “192.168.1.108:5000”
which is behind firewall for audio streaming. This leads to timeout
(unreachable).

What makes you think that it uses this address? Do you see it sending media to it?

Initially, checks are sent for all pairs. The fact that some of them timeout does not mean that they are used. On the contrary, because checks sent from them timed out, they are considered failed and will
*not* be used.

Regards,
Boris

------------------------------

Message: 2
Date: Fri, 12 Feb 2016 11:49:42 +0800
From: "cmeng.gm" <cmeng.gm@gmail.com>
To: <dev@jitsi.org>
Subject: [jitsi-dev] [jitsi-android] What are future plans for this
  project?
Message-ID: <000001d16548$694eaef0$3bec0cd0$@gmail.com>
Content-Type: text/plain; charset="utf-8"

Actually I have spent more than one year of personal effort in enhancing jitsi-andorid, with the following change history below.

I have found out that Jitsi has actually now own by a commercial company Atlassian. This has pulled me back to release the source. I have great concern it will become another example of genymotion; and the released source become copyrighted by Atlassian.

Any good advice.

Regards,

CM Eng

==========================================

Project Jitsi-Android Release Notes

=========================================================================

Version: 0.4.0

Author: cmeng

Upload Date: 01/25/2016

Improvements:

- Fixed coin(conference-info) IQ stanza handler by implementing IQRequestHandler

=========================================================================

Version: 0.3.13

Author: cmeng

Upload Date: 12/25/2015

Improvements:

- Removed g722 codec support; as libjng722.so has incomplete implementation

- Fixed handler problem when multiple jingle transport-info are received as stand-alone stanza;

  all except one are lost in sessionInitiateSyncRoot.wait() during video call - Empathy caller

=========================================================================

Version: 0.3.12

Author: cmeng

Upload Date: 12/05/2015

Improvements:

- Fixed to limit rotated previewSize not to exceed physical device supported dimensions

- Fixed to ensure local stream media player scaled dimension meets OpenGL requirements

- Fixed to ensure the local video preview display is in correct aspect ratio

- Removed unnecessary initRemoteVideo(callPeer) on resume - let remote video event triggers to improve VC reliability

- Removed fireVideoEvent in playerRealizeComplete(); let remote video event handles to improve VC reliability

- Added support for direct VideoCall in chat window

=========================================================================

Version: 0.3.11

Author: cmeng

Upload Date: 11/23/2015

Improvements:

- Fixed OpenGL Invalid Operation when incorrect video dimension given

=========================================================================

Version: 0.3.10

Author: cmeng

Upload Date: 11/20/2015

Improvements:

- Fixed audio graph creation errors (raw type handlers)

- Fixed local preview display incorrect proportional ratio in portrait mode

- Fixed resource access errors

- Implemented auto video rotation for remote video streaming

- Added support for jni lib auto ndk-builder/compilation

- Updated Opus jni source to latest 1.1.1rc release

=========================================================================

Version: 0.3.9

Author: cmeng

Upload Date: 10/14/2015

Improvements:

- Fixed avatar icon update problem

- Added participant join information

- Close chatPanel and leave chatRoom when chatRoom closed

=========================================================================

Version: 0.3.8

Author: cmeng

Upload Date: 10/13/2015

Improvements:

- Added sender avatar in message view holder

- Added Status for conference participants etc

- Fixed otr status and background color (was disabled while adding muc
feature)

- Fixed multiple panel created for the same chatroom

- Fixed notification pop and pending intend

Tag 0.3.7

- Added in muc room creation

- Display sender avatar and status

- Implemented notification sound and muc chatpanel open

Tag 0.3.6

- Added muc room creation

- Fixed system exceptions

Tag 0.3.5

- Implemented conference chat (major changes to Jitsi-android chat
structure)

- muc group chat basic function working

- muc room setup, invitation send and acceptInvitation

- multi-sessions support

=========================================================================

Version: 0.3.3

Author: cmeng

Upload Date: 09/18/2015

Improvements:

- Fixed ByteStream file transfer with multiple <streamhost/>

- Fixed smack Sock5ByteStream File Transfer Exception (smack routine)

=========================================================================

Version: 0.3.2

Author: cmeng

Upload Date: 09/09/2015

Improvements:

- Upgraded to Gradle 2.6

- Fixed all fmj initialization error (remove android incompatible codec)

- Fixed to use only fmj codec

- Added V8 video codec

- Fixed contactList exception on screen rotation

=========================================================================

Version: 0.3.0

Author: cmeng

Upload Date: 09/07/2015

Improvements:

- Ported to use Smack 4.1.3 library (first release - major changes)

- Add Stream Management support v2/v3

- Add Scram Authentication

- Add Smack debugger

- Add ping manager

Tag 0.2.3

- Major import of source from jitsi - fixed imported source errors

- update asmack dnsjava.jar source to 2.1.7

Tag 0.2.2

- Fix re-triggered file send request on android view redraw (view position
changed)

- Update filename for open on redraw (enable file button trigger)

- Clean up unused png files

- Implemented file transfer with BOB

- Fix file transfer with Jingle and multi-<streamhost/> sockets exception (support pidgin file transfer)

- fix call function

=========================================================================

Version: 0.2.0

Author: cmeng

Upload Date: 08/11/2015

Improvements:

- File transfer with IBB and Si fully implemented

- File / Folder open function working

- Change ContactListFragment to single instance (avoid multiple listener and file send)

Tag 0.1.10

- Full concurrent file transfer implemented

- File transfer structure revamp to support multi-file transfer

Tag 0.1.7 release

- cleanup FileTransferActivator

- Cleanup FileTransfer handlers (UI and events)

- Fixed all attachment options

=========================================================================

Version: 0.1.5

Author: cmeng

Upload Date: 07/24/2015

Improvements:

- Added file size check and thumbnail sending support - asmack si support need to change to support thumbnail rx

- Fixed camera photo / video attachment sending (need more work on file history implementation - incomplete jitsi implementation)

- Clean up UI

=========================================================================

Version: 0.1.3

Author: cmeng

Upload Date: 07/22/2015

Improvements:

- Added full implementation for file transfer support including notifications

- Added contact's history erase function

=========================================================================

Version: 0.1.2

Author: cmeng

Upload Date: 06/08/2015

Improvements:

- Added chatFragment background color based on encryption type and verification states

- Synchronize text entry with the actual remote JID (with multiple chatFragment on Adapter)

- Fixed otrContact resource comparing error when inject message

- Add missing resources

- Fixed speex path reference error

- Added auto update function

- Major import latest Jitsi source from windows and updated for android

- Modified build.xml to do proper setup-lib to avoid android conflict class

- Ported to use spongycastle instead of bouncycastle.jar

- Updated libjitsi.jar etc to split fmj.jar and neomedia libs-asset

- Deleted all jar classes in libs which have corresponding source imported.

- Fix build.gradle to add in native .so files for linkage

- Ported to use asmack 4.0 library - modified source to work with it.

=========================================================================

Version: 0.1.1

Author: cmeng

Upload Date: 05/25/2015

Improvements:

- Implemented mult-dex support to overcome android 64K limit

- Implemented encryption OTR V3

- Fixed Known Finger Settings option

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.jitsi.org/pipermail/dev/attachments/20160212/76b1e94d/attachment.html>

------------------------------

Subject: Digest Footer

_______________________________________________
dev mailing list
dev@jitsi.org
http://lists.jitsi.org/mailman/listinfo/dev

------------------------------

End of dev Digest, Vol 35, Issue 53
***********************************

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


#2

Hi Boris Grozev,

After I terminated the videobridge call, I observed the following debug messages log from JVB.

JVB 2016-02-12 13:16:36.561 INFO: [65] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10004/udp/host -> 92.168.1.100:5001/udp/host (audio.RTCP), failing.
JVB 2016-02-12 13:16:36.562 INFO: [63] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10003/udp/host -> 192.168.1.100:5000/udp/host (audio.RTP), failing.
JVB 2016-02-12 13:16:49.041 INFO: [125] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10002/udp/host -> 118.201.189.22:5001/udp/srflx (audio.RTCP), failing.
JVB 2016-02-12 13:16:49.041 INFO: [124] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10001/udp/host -> 118.201.189.22:5000/udp/srflx (audio.RTP), failing.
JVB 2016-02-12 13:16:51.557 INFO: [127] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10004/udp/host -> 192.168.1.100:5001/udp/host (audio.RTCP), failing.
JVB 2016-02-12 13:16:51.558 INFO: [126] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10003/udp/host -> 192.168.1.100:5000/udp/host (audio.RTP), failing.
JVB 2016-02-12 13:17:04.040 INFO: [63] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10001/udp/host -> 118.201.189.22:5000/udp/srflx (audio.RTP), failing.
JVB 2016-02-12 13:17:04.041 INFO: [65] org.ice4j.ice.ConnectivityCheckClient.processTimeout() timeout for pair: 192.168.1.8:10002/udp/host -> 118.201.189.22:5001/udp/srflx (audio.RTCP), failing.

Look like JVB was actually sending UDP data out during the active call session???

It might have been just sending ICE consent checks and no media.

But why none is being observed on wireshark or received by both jitsi-android and jitsi-windows clients???

My best guess is that the problem stems from the way the jitsi client configures jitsi-videobridge (over COLIBRI). A good amount of changes have been introduced since it was last tested or used actively (by our team).

Regards,
Boris

···

On 11/02/16 23:30, cmeng.gm wrote: