I ran into issues hitting the max filedescriptors setting and found that it was because the server never terminates idle connections. I have an idle timeout setting of 1200 seconds (20min). If I make an LDAP request from a client to the directory server, the tcp connection stays in ESTABLISHED state on the server side forever. I ran tcpdump on the client side and not a single packet of traffic was sent to the server during for hours.
Any idea why this connection would not be terminated after 1200 sec?
Idle timeout enforcement is passive, it occurs when we've polled some activity and we're walking over the connection table. If you had made an additional connection or sent an operation on another established connection the idle one should have been disconnected.
Brian Fender wrote:
I ran into issues hitting the max filedescriptors setting and found that it was because the server never terminates idle connections. I have an idle timeout setting of 1200 seconds (20min). If I make an LDAP request from a client to the directory server, the tcp connection stays in ESTABLISHED state on the server side forever. I ran tcpdump on the client side and not a single packet of traffic was sent to the server during for hours.
Any idea why this connection would not be terminated after 1200 sec?
-- Fedora-directory-users mailing list Fedora-directory-users@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-users
This is actually an apache webserver making the connections to directory server. What I see through tcpdump and netstat is that apache creates an LDAP connection to perform a search, and the connection sits idle for hours in established state. The webserver eventually re-uses the random port it made the initial request on to talk to a client, so the LDAP connection no longer shoes up as established on the client side. On the server side, however, it still shows the connection as established forever. There are many other apache children talking to the same LDAP server in parallel, and the number of open filehandles constantly increases.
I realize that it is possible that the webserver is not properly tearing the connection down or a firewall may be blocking it, but shouldn't the server application notice that that connection was idle for more than 20min and time it out anyway?
________________________________
From: fedora-directory-users-bounces@redhat.com [mailto:fedora-directory-users-bounces@redhat.com] On Behalf Of Ulf Weltman Sent: Friday, August 10, 2007 12:49 PM To: General discussion list for the Fedora Directory server project. Subject: Re: [Fedora-directory-users] directory server setting fail toterminate idle connections
Idle timeout enforcement is passive, it occurs when we've polled some activity and we're walking over the connection table. If you had made an additional connection or sent an operation on another established connection the idle one should have been disconnected.
Brian Fender wrote:
I ran into issues hitting the max filedescriptors setting and found that it was because the server never terminates idle connections. I have an idle timeout setting of 1200 seconds (20min). If I make an LDAP request from a client to the directory server, the tcp connection stays in ESTABLISHED state on the server side forever. I ran tcpdump on the client side and not a single packet of traffic was sent to the server during for hours.
Any idea why this connection would not be terminated after 1200 sec?
________________________________
-- Fedora-directory-users mailing list Fedora-directory-users@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-users
Brian Fender wrote:
This is actually an apache webserver making the connections to directory server. What I see through tcpdump and netstat is that apache creates an LDAP connection to perform a search, and the connection sits idle for hours in established state. The webserver eventually re-uses the random port it made the initial request on to talk to a client, so the LDAP connection no longer shoes up as established on the client side. On the server side, however, it still shows the connection as established forever. There are many other apache children talking to the same LDAP server in parallel, and the number of open filehandles constantly increases.
I realize that it is possible that the webserver is not properly tearing the connection down or a firewall may be blocking it, but shouldn’t the server application notice that that connection was idle for more than 20min and time it out anyway?
You want Apache to keep the connections open. It creates a pool of LDAP connections to use for all authentication. TCP/IP connections are expensive so it keeps them open to issue search and bind requests when doing authentication.
There may be a bug in the pooling code but how many connections are we talking about?
rob
With a lightly used, it takes about two weeks to hit 8000 connections. The number of connections does constantly drop, however not nearly as fast as it increases. I would expect it to level off at some point, but this never happens.
I believe that the reason it is creating new connections is because the apache processes are hitting max-requests-per-child and respawning before they ever have a need to re-use their LDAP connection. I think the connection is being closed on the client side at this point, and when another LDAP request actually comes in it makes a new connection.
The apache server is extremely busy, it just rarely talks to the LDAP server.
-----Original Message----- From: fedora-directory-users-bounces@redhat.com [mailto:fedora-directory-users-bounces@redhat.com] On Behalf Of Rob Crittenden Sent: Tuesday, August 14, 2007 8:31 AM To: General discussion list for the Fedora Directory server project. Subject: Re: [Fedora-directory-users] directory server setting fail toterminateidle connections
Brian Fender wrote:
This is actually an apache webserver making the connections to
directory
server. What I see through tcpdump and netstat is that apache creates
an LDAP connection to perform a search, and the connection sits idle
for
hours in established state. The webserver eventually re-uses the
random
port it made the initial request on to talk to a client, so the LDAP connection no longer shoes up as established on the client side. On
the
server side, however, it still shows the connection as established forever. There are many other apache children talking to the same
LDAP
server in parallel, and the number of open filehandles constantly increases.
I realize that it is possible that the webserver is not properly
tearing
the connection down or a firewall may be blocking it, but shouldn't
the
server application notice that that connection was idle for more than 20min and time it out anyway?
You want Apache to keep the connections open. It creates a pool of LDAP connections to use for all authentication. TCP/IP connections are expensive so it keeps them open to issue search and bind requests when doing authentication.
There may be a bug in the pooling code but how many connections are we talking about?
rob
389-users@lists.fedoraproject.org