Dear SSSD developers,

I have started to create this patch in order to address the reclamation filed on Fedora 12:

The problem shows up as continuous flood of messages to event log (2 every 10 minutes) about expired Kerberos context.
Mar 28 06:39:17 node-2 sssd_be: GSSAPI Error: The referenced context has expired (Unknown error)
Mar 28 06:39:17 node-2 sssd_be: GSSAPI Error: The referenced context has expired (Unknown error)

The messages show were caused by expired Kerberos context. The assumption made in ldap_child.c that stated that Kerberos ticket is only needed during connection setup, so it forcibly set the Kerberos ticket lifetime to 5 minutes. The assumption is unfortunately wrong as with MIT Kerberos implementation Kerberos encryption and integrity is always enforced when supported by both server and client. So after 5 minutes the connection Kerberos ticket expired and connection was broken polluting event log and putting additional load on DS and KDC due to constant reconnects.

Please, note that Kerberos authentication with LDAP server is the default for IPA domain.

The natural solution for the problem was to increase Kerberos ticket lifetime. Farther investigations show that it is necessary also to close connection in timely fashion, as even closing connection with expired ticket caused message in event log to appear.

During implementation of the above fixes I have also identified several more problems:
1. In case of 2 or more requests executed in parallel (all started before connection was established) it was possible to begin 2 connection operations in parallel. One of the established connections was finally leaked leaving connection to hang around.
2. The were no failover retry login LDAP ID backend itself. More over after the first server failure (even on stale cached connection) the whole backend was put to OFFLINE.

The patch attached to this message addresses all the above issues and as well as:
1. Reduces amount of duplicate code related to LDAP connection and retry logic.
2. Puts the connection and retry logic on the same architectural level in ldap_id.c (now all this logic is handled by sdap_account_ function family), that should benefit readability of code.

The patch design is as follows:

1. New entity sdap_id_op represents high level LDAP query operation. It is responsible of:
   a) Keeping track of LDAP connection
   b) Keeping track of number of reconnect retries
2. New entity sdap_id_connection represents connection attempt made by LDAP ID backend and later tracks connection usage.
3. sdap_id_ctx keeps track of all currently open connections and ensures:
   a) That there is only one connection attempt in progress at a time. Further operations are queued until connection is completed.
   b) That cached connection is released in timely fashion
   c) That all connections are closed when all operations using them are complete and they are released from cache.
4. The retry logic of all LDAP operations is updated to use new facility.

I have attached to versions of this patch:
- one against sssd-1.1.0. This version was thoroughly tested and currently is in preproduction testing on several systems in my IPA domain.
- the other against git repository. I have only made sure that the patch compiled.

Looking forward for your reply, Eugene Indenbom