[SSSD] RFC - disable the cleanup task by default

Simo Sorce simo at redhat.com
Mon Apr 13 13:16:18 UTC 2015


On Mon, 2015-04-13 at 11:05 +0200, Jakub Hrozek wrote:
> On Wed, Apr 08, 2015 at 12:52:33PM +0200, Jakub Hrozek wrote:
> > On Wed, Apr 08, 2015 at 12:11:06PM +0200, Lukas Slebodnik wrote:
> > > On (08/04/15 11:59), Jakub Hrozek wrote:
> > > >On Wed, Apr 08, 2015 at 11:56:59AM +0200, Pavel Březina wrote:
> > > >> On 04/07/2015 05:08 PM, Jakub Hrozek wrote:
> > > >> >On Tue, Apr 07, 2015 at 05:02:33PM +0200, Jakub Hrozek wrote:
> > > >> >>On Tue, Apr 07, 2015 at 04:39:30PM +0200, Jakub Hrozek wrote:
> > > >> >>>So unless account_cache_expiration is set to something sensible, only
> > > >> >>>empty groups are removed and that's quite a corner case.
> > > >> >>
> > > >> >>Correction - also groups who have never logged in might be removed. This
> > > >> >                     ~~~~~~
> > > >> >                     users, sorry
> > > >> >>could be useful for pruning info after "ls -l" on directory like /tmp
> > > >> >>for example.
> > > >> >>
> > > >> >>But my opinion on the whole cleanup task still stands.
> > > >> 
> > > >> Sure we can disable it, it don't oppose.
> > > >> 
> > > >> Or we can also increase the default value for ldap_purge_cache_timeout to
> > > >> days instead of hours.
> > > >
> > > >This wouldn't help the biggest problem though which is when the cache cleanup
> > > >starts for a huge database on a busy server, the single transaction blocks
> > > >any other transaction..
> > > 
> > > Disabling clean-up task might help busy servers. There is hight probability
> > > all users/groups in sssd cache are used and refreshed very often
> > > 
> > > But could it cause problems in other use cases?
> > 
> > So far I can think of one use case:
> >     The users never log in and are only used for identity lookups. At
> >     the same time, the users on the server tend to change (be
> >     added/deleted). And at the same time, the removed entries are never
> >     requested (ie files are removed before someone does getpwuid() of
> >     the owner).
> > 
> > I can imagine ls -l on /tmp on a university server where students come
> > and go might hit this problem.
> > 
> > > 
> > > BTW we already have documented how to disable clean-up task.
> > > man sssd-ldap -> ldap_purge_cache_timeout
> > 
> > Noone reads documentation :-) We had two RHEL customer cases this week
> > where sssd_be was spinning out of control during a cleanup task and users
> > couldn't log in. For one customer (jb on #sssd), the sssd_be process was
> > even killed and restated.
> > 
> > > 
> > > maybe we should write wiki page with performance tuning.
> > 
> > In my opinion it would be better the other way around, add a wiki page
> > for cache size tuning. Disk space is cheap, but slow logins look bad.
> 
> Simo, what do you think?

Hmmm if the cleanup task is causing such issues then it is better to
disable it by default, in the short term, then think about how we can
make it smarter.

In the end, what we really want is to be able to prune users that
disappeared from the main DB. But it is not really important for use to
actually go and remove them from the database, only to not return them
to the userspace. So perhaps we should just care about expiration times,
and if nothing (enumeration or direct lookup) refreshes the user then we
just do not return it on enumeration. That would probably be sufficient.

Space is not always cheap, but if that is a problem then I think the
only way to do proper pruning is to spin a separate process, and have it
avoid long transactions. It should build a list of users to prune w/o
keeping a transaction open and then have small transactions for each
modifications it thinks needs to be done where it checks if the user
entry has been modified between the time it looked it up and the time it
decided to store any change (in case the main backend process updated it
in the meanwhile). I think it is key that this is a separate process
where we can also change the priority and make it very low prio.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York




More information about the sssd-devel mailing list