[jitsi-dev] Can't listen on tcp socket


#1

Wanted to check out doing a video call over tcp so I blocked the udp ports,
but noticed this on the bridge:

2015-07-05 19:57:49.764 INFO: [35]
org.ice4j.ice.harvest.MultiplexingTcpHostHarvester.readFromChannel() Failed
to handle TCP socket Socket[addr=/xxxxxxxxx,port=64242,localport=4443]:
java.io.IOException: End of stream!

I have single port mode turned on and thought I saw something about there
being a potential issue there (is there?) but even after disabling it I
still see the same issue.

Anyone seen this? Known configuration problem on my side, perhaps? This
is running on an AWS instance and I've got 4443/tcp forwarded.

-brian


#2

Hi Brian,

Wanted to check out doing a video call over tcp so I blocked the udp
ports, but noticed this on the bridge:

2015-07-05 19:57:49.764 INFO: [35]
org.ice4j.ice.harvest.MultiplexingTcpHostHarvester.readFromChannel()
Failed to handle TCP socket
Socket[addr=/xxxxxxxxx,port=64242,localport=4443]: java.io.IOException:
End of stream!

This indicates that something (presumably a client) opened a TCP connection to the bridge, but didn't send data in the format we expect (maybe it sent no data at all). You can inspect a network dump for details on what was sent. What we expect is:

1. If "SSLTCP" is enabled, then the exact bytes defined in GoogleTurnSSLCandidateHarvester#SSL_CLIENT_HANDSHAKE.

2. A STUN Binding Request preceded by a two-byte length field (from RFC4571).

I have single port mode turned on and thought I saw something about
there being a potential issue there (is there?) but even after disabling
it I still see the same issue.

Anyone seen this? Known configuration problem on my side, perhaps?
This is running on an AWS instance and I've got 4443/tcp forwarded.

I can't think of any issues like this, and single-port shouldn't affect it.

Regards,
Brian

···

On 06/08/15 16:40, Brian Baldino wrote:


#3

Thanks Boris,
I did a quick network dump and I do see the SSL_CLIENT_HANDSHAKE packet
being sent. This is just using chrome, so shouldn't be anything special on
the client side here. Any other ideas? I'll try and add some logging to
ice4j to see if I can get some more information.

···

On Fri, Aug 7, 2015 at 8:32 AM, Boris Grozev <boris@jitsi.org> wrote:

Hi Brian,

On 06/08/15 16:40, Brian Baldino wrote:

Wanted to check out doing a video call over tcp so I blocked the udp
ports, but noticed this on the bridge:

2015-07-05 19:57:49.764 INFO: [35]
org.ice4j.ice.harvest.MultiplexingTcpHostHarvester.readFromChannel()
Failed to handle TCP socket
Socket[addr=/xxxxxxxxx,port=64242,localport=4443]: java.io.IOException:
End of stream!

This indicates that something (presumably a client) opened a TCP
connection to the bridge, but didn't send data in the format we expect
(maybe it sent no data at all). You can inspect a network dump for details
on what was sent. What we expect is:

1. If "SSLTCP" is enabled, then the exact bytes defined in
GoogleTurnSSLCandidateHarvester#SSL_CLIENT_HANDSHAKE.

2. A STUN Binding Request preceded by a two-byte length field (from
RFC4571).

I have single port mode turned on and thought I saw something about
there being a potential issue there (is there?) but even after disabling
it I still see the same issue.

Anyone seen this? Known configuration problem on my side, perhaps?
This is running on an AWS instance and I've got 4443/tcp forwarded.

I can't think of any issues like this, and single-port shouldn't affect it.

Regards,
Brian

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#4

So from what I can see from the network trace and logs now, the chrome
client appears to send the client hello from 3 different ports
back-to-back. Jitsi receives the client hello and responds with a server
hello to each one, but the client never seems to see anything after (after
the readFromChannel that reads the client hello, the next read returns -1).
The client never seems to transition to ice: connected state. I've tried
disabling ssltcp on the bridge, but that seems to fail in an even more
fundamental way. Any suggestions on debugging?

Thanks!
-brian

···

On Fri, Aug 7, 2015 at 10:52 AM, Brian Baldino <brian@highfive.com> wrote:

Thanks Boris,
I did a quick network dump and I do see the SSL_CLIENT_HANDSHAKE packet
being sent. This is just using chrome, so shouldn't be anything special on
the client side here. Any other ideas? I'll try and add some logging to
ice4j to see if I can get some more information.

On Fri, Aug 7, 2015 at 8:32 AM, Boris Grozev <boris@jitsi.org> wrote:

Hi Brian,

On 06/08/15 16:40, Brian Baldino wrote:

Wanted to check out doing a video call over tcp so I blocked the udp
ports, but noticed this on the bridge:

2015-07-05 19:57:49.764 INFO: [35]
org.ice4j.ice.harvest.MultiplexingTcpHostHarvester.readFromChannel()
Failed to handle TCP socket
Socket[addr=/xxxxxxxxx,port=64242,localport=4443]: java.io.IOException:
End of stream!

This indicates that something (presumably a client) opened a TCP
connection to the bridge, but didn't send data in the format we expect
(maybe it sent no data at all). You can inspect a network dump for details
on what was sent. What we expect is:

1. If "SSLTCP" is enabled, then the exact bytes defined in
GoogleTurnSSLCandidateHarvester#SSL_CLIENT_HANDSHAKE.

2. A STUN Binding Request preceded by a two-byte length field (from
RFC4571).

I have single port mode turned on and thought I saw something about
there being a potential issue there (is there?) but even after disabling
it I still see the same issue.

Anyone seen this? Known configuration problem on my side, perhaps?
This is running on an AWS instance and I've got 4443/tcp forwarded.

I can't think of any issues like this, and single-port shouldn't affect
it.

Regards,
Brian

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#5

Does the bridge send a server hello?

Regards,
Boris

···

On 07/08/15 13:36, Brian Baldino wrote:

So from what I can see from the network trace and logs now, the chrome
client appears to send the client hello from 3 different ports
back-to-back. Jitsi receives the client hello and responds with a
server hello to each one, but the client never seems to see anything
after (after the readFromChannel that reads the client hello, the next
read returns -1). The client never seems to transition to ice: connected
state. I've tried disabling ssltcp on the bridge, but that seems to
fail in an even more fundamental way. Any suggestions on debugging?


#6

Yeah, bridge does send server hello back.

···

On Fri, Aug 7, 2015 at 2:56 PM, Boris Grozev <boris@jitsi.org> wrote:

On 07/08/15 13:36, Brian Baldino wrote:

So from what I can see from the network trace and logs now, the chrome
client appears to send the client hello from 3 different ports
back-to-back. Jitsi receives the client hello and responds with a
server hello to each one, but the client never seems to see anything
after (after the readFromChannel that reads the client hello, the next
read returns -1). The client never seems to transition to ice: connected
state. I've tried disabling ssltcp on the bridge, but that seems to
fail in an even more fundamental way. Any suggestions on debugging?

Does the bridge send a server hello?

Regards,
Boris

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#7

I got around to enabling debug logs on the chrome side, and it appears to
loop over and over on the same log messages:

[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D362B0:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520683394433548287|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763, last
ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36518:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520401920329252863|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763, last
ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36780:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520401919456837631|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763, last
ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D35B78:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520964870282674175|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812, last
ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D35DE0:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520964869410258943|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812, last
ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36048:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520683395305963519|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812, last
ping received: 0, last data_received: 0
...etc

Seem familiar at all? It does this over and over and over.

···

On Thu, Aug 13, 2015 at 11:40 AM, Brian Baldino <brian@highfive.com> wrote:

I got around to enabling debug logs on the chrome side, and it appears to
loop over and over on the same log messages:

[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D362B0:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520683394433548287|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763, last
ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36518:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520401920329252863|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763, last
ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36780:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520401919456837631|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763, last
ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D35B78:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520964870282674175|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812, last
ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D35DE0:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520964869410258943|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812, last
ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36048:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520683395305963519|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812, last
ping received: 0, last data_received: 0
...etc

Seem familiar at all? It does this over and over and over. Attached the
entire log.

On Fri, Aug 7, 2015 at 3:03 PM, Brian Baldino <brian@highfive.com> wrote:

Yeah, bridge does send server hello back.

On Fri, Aug 7, 2015 at 2:56 PM, Boris Grozev <boris@jitsi.org> wrote:

On 07/08/15 13:36, Brian Baldino wrote:

So from what I can see from the network trace and logs now, the chrome
client appears to send the client hello from 3 different ports
back-to-back. Jitsi receives the client hello and responds with a
server hello to each one, but the client never seems to see anything
after (after the readFromChannel that reads the client hello, the next
read returns -1). The client never seems to transition to ice: connected
state. I've tried disabling ssltcp on the bridge, but that seems to
fail in an even more fundamental way. Any suggestions on debugging?

Does the bridge send a server hello?

Regards,
Boris

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#8

Hey Brian,

I got around to enabling debug logs on the chrome side, and it appears
to loop over and over on the same log messages:

[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D362B0:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520683394433548287|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36518:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520401920329252863|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36780:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520401919456837631|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D35B78:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520964870282674175|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D35DE0:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520964869410258943|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36048:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520683395305963519|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812,
last ping received: 0, last data_received: 0
...etc

Seem familiar at all? It does this over and over and over.

Doesn't ring any bells.

Some suggestions for narrowing down the possible causes:

Try to reproduce on beta.meet.jit.si (I just ran a test and it works for me with both chrome stable and canary).

Try to disable multiplexing HTTP and media if you are using it (I don't think this has been tested on an amazon instance yet).

See if any TCP segments leave the browser machines but don't reach the server (or reach the server but don't reach the method that you mentioned in MultiplexingTCPHostHarvester).

Regards,
Boris

···

On 13/08/15 14:11, Brian Baldino wrote:

On Thu, Aug 13, 2015 at 11:40 AM, Brian Baldino <brian@highfive.com > <mailto:brian@highfive.com>> wrote:

    I got around to enabling debug logs on the chrome side, and it
    appears to loop over and over on the same log messages:

    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D362B0:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520683394433548287|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006763, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D36518:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520401920329252863|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006763, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D36780:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520401919456837631|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006763, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D35B78:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520964870282674175|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006812, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D35DE0:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520964869410258943|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006812, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D36048:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520683395305963519|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006812, last ping received: 0, last data_received: 0
    ...etc

    Seem familiar at all? It does this over and over and over. Attached
    the entire log.

    On Fri, Aug 7, 2015 at 3:03 PM, Brian Baldino <brian@highfive.com > <mailto:brian@highfive.com>> wrote:

        Yeah, bridge does send server hello back.

        On Fri, Aug 7, 2015 at 2:56 PM, Boris Grozev <boris@jitsi.org > <mailto:boris@jitsi.org>> wrote:

            On 07/08/15 13:36, Brian Baldino wrote:

                So from what I can see from the network trace and logs
                now, the chrome
                client appears to send the client hello from 3 different
                ports
                back-to-back. Jitsi receives the client hello and
                responds with a
                server hello to each one, but the client never seems to
                see anything
                after (after the readFromChannel that reads the client
                hello, the next
                read returns -1). The client never seems to transition
                to ice: connected
                state. I've tried disabling ssltcp on the bridge, but
                that seems to
                fail in an even more fundamental way. Any suggestions
                on debugging?

            Does the bridge send a server hello?

            Regards,
            Boris

            _______________________________________________
            dev mailing list
            dev@jitsi.org <mailto:dev@jitsi.org>
            Unsubscribe instructions and other list options:
            http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#9

Thanks for the help, Boris! Looks like I actually see the same behavior on
beta.meet.jit.si: I blocked port 7777 for udp and ICEConnectionState never
gets passed "Checking". Wonder what the deal is there. I'll have to try
this from home, too, and see if there's any difference compared to our work
network.

···

On Thu, Aug 13, 2015 at 3:22 PM, Boris Grozev <boris@jitsi.org> wrote:

Hey Brian,

On 13/08/15 14:11, Brian Baldino wrote:

I got around to enabling debug logs on the chrome side, and it appears
to loop over and over on the same log messages:

[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D362B0:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520683394433548287|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36518:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520401920329252863|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36780:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520401919456837631|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D35B78:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520964870282674175|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D35DE0:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520964869410258943|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36048:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520683395305963519|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812,
last ping received: 0, last data_received: 0
...etc

Seem familiar at all? It does this over and over and over.

Doesn't ring any bells.

Some suggestions for narrowing down the possible causes:

Try to reproduce on beta.meet.jit.si (I just ran a test and it works for
me with both chrome stable and canary).

Try to disable multiplexing HTTP and media if you are using it (I don't
think this has been tested on an amazon instance yet).

See if any TCP segments leave the browser machines but don't reach the
server (or reach the server but don't reach the method that you mentioned
in MultiplexingTCPHostHarvester).

Regards,
Boris

On Thu, Aug 13, 2015 at 11:40 AM, Brian Baldino <brian@highfive.com >> <mailto:brian@highfive.com>> wrote:

    I got around to enabling debug logs on the chrome side, and it
    appears to loop over and over on the same log messages:

    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D362B0:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520683394433548287|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006763, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D36518:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520401920329252863|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006763, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D36780:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520401919456837631|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006763, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D35B78:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520964870282674175|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006812, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D35DE0:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520964869410258943|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006812, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D36048:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520683395305963519|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006812, last ping received: 0, last data_received: 0
    ...etc

    Seem familiar at all? It does this over and over and over. Attached
    the entire log.

    On Fri, Aug 7, 2015 at 3:03 PM, Brian Baldino <brian@highfive.com >> <mailto:brian@highfive.com>> wrote:

        Yeah, bridge does send server hello back.

        On Fri, Aug 7, 2015 at 2:56 PM, Boris Grozev <boris@jitsi.org >> <mailto:boris@jitsi.org>> wrote:

            On 07/08/15 13:36, Brian Baldino wrote:

                So from what I can see from the network trace and logs
                now, the chrome
                client appears to send the client hello from 3 different
                ports
                back-to-back. Jitsi receives the client hello and
                responds with a
                server hello to each one, but the client never seems to
                see anything
                after (after the readFromChannel that reads the client
                hello, the next
                read returns -1). The client never seems to transition
                to ice: connected
                state. I've tried disabling ssltcp on the bridge, but
                that seems to
                fail in an even more fundamental way. Any suggestions
                on debugging?

            Does the bridge send a server hello?

            Regards,
            Boris

            _______________________________________________
            dev mailing list
            dev@jitsi.org <mailto:dev@jitsi.org>
            Unsubscribe instructions and other list options:
            http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#10

So I tried this at home and from my windows machine I'm able to
successfully connect via tcp to both beta.meet.jit.si and my own
deployment. However on mac I'm still not able to connect to either one,
which seems odd. Interestingly enough, if I try a telnet connection to
beta.meet.jit.si 443 from my mac, it gets just as far as it does on my pc:

11:57:36 brian@Brians-MacBook-Pro-1036:~ $ telnet beta.meet.jit.si 443
Trying 46.105.57.130...
Connected to beta.meet.jit.si.
Escape character is '^]'.

···

On Fri, Aug 14, 2015 at 9:38 AM, Brian Baldino <brian@highfive.com> wrote:

Thanks for the help, Boris! Looks like I actually see the same behavior
on beta.meet.jit.si: I blocked port 7777 for udp and ICEConnectionState
never gets passed "Checking". Wonder what the deal is there. I'll have to
try this from home, too, and see if there's any difference compared to our
work network.

On Thu, Aug 13, 2015 at 3:22 PM, Boris Grozev <boris@jitsi.org> wrote:

Hey Brian,

On 13/08/15 14:11, Brian Baldino wrote:

I got around to enabling debug logs on the chrome side, and it appears
to loop over and over on the same log messages:

[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D362B0:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520683394433548287|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36518:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520401920329252863|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36780:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520401919456837631|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006763,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D35B78:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520964870282674175|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D35DE0:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520964869410258943|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812,
last ping received: 0, last data_received: 0
[8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
Jingle:Conn[05D36048:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520683395305963519|-]:
UpdateState(): pings_since_last_response_=, rtt=3000, now=1386006812,
last ping received: 0, last data_received: 0
...etc

Seem familiar at all? It does this over and over and over.

Doesn't ring any bells.

Some suggestions for narrowing down the possible causes:

Try to reproduce on beta.meet.jit.si (I just ran a test and it works for
me with both chrome stable and canary).

Try to disable multiplexing HTTP and media if you are using it (I don't
think this has been tested on an amazon instance yet).

See if any TCP segments leave the browser machines but don't reach the
server (or reach the server but don't reach the method that you mentioned
in MultiplexingTCPHostHarvester).

Regards,
Boris

On Thu, Aug 13, 2015 at 11:40 AM, Brian Baldino <brian@highfive.com >>> <mailto:brian@highfive.com>> wrote:

    I got around to enabling debug logs on the chrome side, and it
    appears to loop over and over on the same log messages:

    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D362B0:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520683394433548287|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006763, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D36518:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520401920329252863|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006763, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D36780:audio:+X2LObQm:1:0:local:tcp:192.168.159.240:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520401919456837631|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006763, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D35B78:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520964870282674175|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006812, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D35DE0:audio:vCXUXptJ:1:0:local:tcp:192.168.59.3:0
->A68rf+Y5:1:1694498815:stun:ssltcp:server_ip_2:4443|---W|6520964869410258943|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006812, last ping received: 0, last data_received: 0
    [8592:12844:0813/112811:VERBOSE1:port.cc(1106)]
    Jingle:Conn[05D36048:audio:u/O7fRFq:1:0:local:tcp:192.168.56.1:0
->oD85iIM4:1:2130706431:local:ssltcp:server_ip_1:4443|---W|6520683395305963519|-]:
    UpdateState(): pings_since_last_response_=, rtt=3000,
    now=1386006812, last ping received: 0, last data_received: 0
    ...etc

    Seem familiar at all? It does this over and over and over. Attached
    the entire log.

    On Fri, Aug 7, 2015 at 3:03 PM, Brian Baldino <brian@highfive.com >>> <mailto:brian@highfive.com>> wrote:

        Yeah, bridge does send server hello back.

        On Fri, Aug 7, 2015 at 2:56 PM, Boris Grozev <boris@jitsi.org >>> <mailto:boris@jitsi.org>> wrote:

            On 07/08/15 13:36, Brian Baldino wrote:

                So from what I can see from the network trace and logs
                now, the chrome
                client appears to send the client hello from 3 different
                ports
                back-to-back. Jitsi receives the client hello and
                responds with a
                server hello to each one, but the client never seems to
                see anything
                after (after the readFromChannel that reads the client
                hello, the next
                read returns -1). The client never seems to transition
                to ice: connected
                state. I've tried disabling ssltcp on the bridge, but
                that seems to
                fail in an even more fundamental way. Any suggestions
                on debugging?

            Does the bridge send a server hello?

            Regards,
            Boris

            _______________________________________________
            dev mailing list
            dev@jitsi.org <mailto:dev@jitsi.org>
            Unsubscribe instructions and other list options:
            http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#11

Note that on beta the bridge uses port 4443 for TCP.

Boris

···

On 16/08/15 14:28, Brian Baldino wrote:

So I tried this at home and from my windows machine I'm able to
successfully connect via tcp to both beta.meet.jit.si
<http://beta.meet.jit.si> and my own deployment. However on mac I'm
still not able to connect to either one, which seems odd. Interestingly
enough, if I try a telnet connection to beta.meet.jit.si
<http://beta.meet.jit.si> 443 from my mac, it gets just as far as it
does on my pc:

11:57:36 brian@Brians-MacBook-Pro-1036:~ $ telnet beta.meet.jit.si
<http://beta.meet.jit.si> 443
Trying 46.105.57.130...
Connected to beta.meet.jit.si <http://beta.meet.jit.si>.
Escape character is '^]'.


#12

Oops, yes. Just double checked on 4443 and see the same result from telnet.

···

On Mon, Aug 17, 2015 at 8:05 AM, Boris Grozev <boris@jitsi.org> wrote:

On 16/08/15 14:28, Brian Baldino wrote:

So I tried this at home and from my windows machine I'm able to
successfully connect via tcp to both beta.meet.jit.si
<http://beta.meet.jit.si> and my own deployment. However on mac I'm
still not able to connect to either one, which seems odd. Interestingly
enough, if I try a telnet connection to beta.meet.jit.si
<http://beta.meet.jit.si> 443 from my mac, it gets just as far as it
does on my pc:

11:57:36 brian@Brians-MacBook-Pro-1036:~ $ telnet beta.meet.jit.si
<http://beta.meet.jit.si> 443
Trying 46.105.57.130...
Connected to beta.meet.jit.si <http://beta.meet.jit.si>.
Escape character is '^]'.

Note that on beta the bridge uses port 4443 for TCP.

Boris

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#13

Hey Boris,
Have you committed the fix for this for aws? I just updated the bridge and
notice I can connect over tcp where I couldn't before.

···

On Mon, Aug 17, 2015 at 9:28 AM, Brian Baldino <brian@highfive.com> wrote:

Oops, yes. Just double checked on 4443 and see the same result from
telnet.

On Mon, Aug 17, 2015 at 8:05 AM, Boris Grozev <boris@jitsi.org> wrote:

On 16/08/15 14:28, Brian Baldino wrote:

So I tried this at home and from my windows machine I'm able to
successfully connect via tcp to both beta.meet.jit.si
<http://beta.meet.jit.si> and my own deployment. However on mac I'm
still not able to connect to either one, which seems odd. Interestingly
enough, if I try a telnet connection to beta.meet.jit.si
<http://beta.meet.jit.si> 443 from my mac, it gets just as far as it
does on my pc:

11:57:36 brian@Brians-MacBook-Pro-1036:~ $ telnet beta.meet.jit.si
<http://beta.meet.jit.si> 443
Trying 46.105.57.130...
Connected to beta.meet.jit.si <http://beta.meet.jit.si>.
Escape character is '^]'.

Note that on beta the bridge uses port 4443 for TCP.

Boris

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev


#14

Hey Brian,

Hey Boris,
Have you committed the fix for this for aws? I just updated the bridge
and notice I can connect over tcp where I couldn't before.

I thought I'd left a comment here, but apparently I didn't. Apologies.

We found and fixed a problem which would cause ICE failures over TCP if the bridge doesn't get a response to its STUN Binding Request within 200ms (which would happen once every couple of attempts on our office wifi for example). As far as I can tell, this is distinct from your issue, because you were seeing some DTLS exchange, while in this case we don't even see ICE complete.

Regards,
Boris

···

On 09/10/15 13:01, Brian Baldino wrote:

On Mon, Aug 17, 2015 at 9:28 AM, Brian Baldino <brian@highfive.com > <mailto:brian@highfive.com>> wrote:

    Oops, yes. Just double checked on 4443 and see the same result from
    telnet.

    On Mon, Aug 17, 2015 at 8:05 AM, Boris Grozev <boris@jitsi.org > <mailto:boris@jitsi.org>> wrote:

        On 16/08/15 14:28, Brian Baldino wrote:

            So I tried this at home and from my windows machine I'm able to
            successfully connect via tcp to both beta.meet.jit.si
            <http://beta.meet.jit.si>
            <http://beta.meet.jit.si> and my own deployment. However on
            mac I'm
            still not able to connect to either one, which seems odd.
            Interestingly
            enough, if I try a telnet connection to beta.meet.jit.si
            <http://beta.meet.jit.si>
            <http://beta.meet.jit.si> 443 from my mac, it gets just as
            far as it
            does on my pc:

            11:57:36 brian@Brians-MacBook-Pro-1036:~ $ telnet
            beta.meet.jit.si <http://beta.meet.jit.si>
            <http://beta.meet.jit.si> 443
            Trying 46.105.57.130...
            Connected to beta.meet.jit.si <http://beta.meet.jit.si>
            <http://beta.meet.jit.si>.
            Escape character is '^]'.

        Note that on beta the bridge uses port 4443 for TCP.

        Boris

        _______________________________________________
        dev mailing list
        dev@jitsi.org <mailto:dev@jitsi.org>
        Unsubscribe instructions and other list options:
        http://lists.jitsi.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
dev@jitsi.org
Unsubscribe instructions and other list options:
http://lists.jitsi.org/mailman/listinfo/dev