[SSSD] RFC - disable the cleanup task by default

Jakub Hrozek jhrozek at redhat.com
Mon Apr 13 13:21:44 UTC 2015


On Mon, Apr 13, 2015 at 09:16:18AM -0400, Simo Sorce wrote:
> On Mon, 2015-04-13 at 11:05 +0200, Jakub Hrozek wrote:
> > On Wed, Apr 08, 2015 at 12:52:33PM +0200, Jakub Hrozek wrote:
> > > On Wed, Apr 08, 2015 at 12:11:06PM +0200, Lukas Slebodnik wrote:
> > > > On (08/04/15 11:59), Jakub Hrozek wrote:
> > > > >On Wed, Apr 08, 2015 at 11:56:59AM +0200, Pavel Březina wrote:
> > > > >> On 04/07/2015 05:08 PM, Jakub Hrozek wrote:
> > > > >> >On Tue, Apr 07, 2015 at 05:02:33PM +0200, Jakub Hrozek wrote:
> > > > >> >>On Tue, Apr 07, 2015 at 04:39:30PM +0200, Jakub Hrozek wrote:
> > > > >> >>>So unless account_cache_expiration is set to something sensible, only
> > > > >> >>>empty groups are removed and that's quite a corner case.
> > > > >> >>
> > > > >> >>Correction - also groups who have never logged in might be removed. This
> > > > >> >                     ~~~~~~
> > > > >> >                     users, sorry
> > > > >> >>could be useful for pruning info after "ls -l" on directory like /tmp
> > > > >> >>for example.
> > > > >> >>
> > > > >> >>But my opinion on the whole cleanup task still stands.
> > > > >> 
> > > > >> Sure we can disable it, it don't oppose.
> > > > >> 
> > > > >> Or we can also increase the default value for ldap_purge_cache_timeout to
> > > > >> days instead of hours.
> > > > >
> > > > >This wouldn't help the biggest problem though which is when the cache cleanup
> > > > >starts for a huge database on a busy server, the single transaction blocks
> > > > >any other transaction..
> > > > 
> > > > Disabling clean-up task might help busy servers. There is hight probability
> > > > all users/groups in sssd cache are used and refreshed very often
> > > > 
> > > > But could it cause problems in other use cases?
> > > 
> > > So far I can think of one use case:
> > >     The users never log in and are only used for identity lookups. At
> > >     the same time, the users on the server tend to change (be
> > >     added/deleted). And at the same time, the removed entries are never
> > >     requested (ie files are removed before someone does getpwuid() of
> > >     the owner).
> > > 
> > > I can imagine ls -l on /tmp on a university server where students come
> > > and go might hit this problem.
> > > 
> > > > 
> > > > BTW we already have documented how to disable clean-up task.
> > > > man sssd-ldap -> ldap_purge_cache_timeout
> > > 
> > > Noone reads documentation :-) We had two RHEL customer cases this week
> > > where sssd_be was spinning out of control during a cleanup task and users
> > > couldn't log in. For one customer (jb on #sssd), the sssd_be process was
> > > even killed and restated.
> > > 
> > > > 
> > > > maybe we should write wiki page with performance tuning.
> > > 
> > > In my opinion it would be better the other way around, add a wiki page
> > > for cache size tuning. Disk space is cheap, but slow logins look bad.
> > 
> > Simo, what do you think?
> 
> Hmmm if the cleanup task is causing such issues then it is better to
> disable it by default, in the short term, then think about how we can
> make it smarter.
> 
> In the end, what we really want is to be able to prune users that
> disappeared from the main DB. But it is not really important for use to
> actually go and remove them from the database, only to not return them
> to the userspace. So perhaps we should just care about expiration times,
> and if nothing (enumeration or direct lookup) refreshes the user then we
> just do not return it on enumeration. That would probably be sufficient.

Some users are scared about disabling the cache cleanup because the
cache might grow out of control. I kinda suspect they don't realize the
next request for a nonexistant user that would hit the DP would also
remove the stray entry, though.

> 
> Space is not always cheap, but if that is a problem then I think the
> only way to do proper pruning is to spin a separate process, and have it
> avoid long transactions. It should build a list of users to prune w/o
> keeping a transaction open and then have small transactions for each
> modifications it thinks needs to be done where it checks if the user
> entry has been modified between the time it looked it up and the time it
> decided to store any change (in case the main backend process updated it
> in the meanwhile). I think it is key that this is a separate process
> where we can also change the priority and make it very low prio.
> 

That's pretty much what Sumit proposed on IRC as well.

Thanks!



More information about the sssd-devel mailing list