On Wed, Feb 13, 2013 at 07:34:54PM -0500, Dmitri Pal wrote:
Hello,
I was thinking a bit about the ticket
https://fedorahosted.org/sssd/ticket/1713
AFAIU the problem is that in the case when there are many netgroups they
are expiring at the same time and every so often the system has to
refetch all of them.
The ticket calls for a job that would run asynchronously and refresh
things that already expired. While this might be useful it seems like a
big task. I was wondering about a different "complementary" approach
that might help with making entries not expire at the same time. Would
it be helpful it we as an option parameter something like:
<map>_extended_expiration_range = %of_the_cache_lifetime.
If not defined it will work as right now. If defined (per map) it will
set the window of expiration. Here is how it would work.
Say the expiration time is 10h and midpoint cache is at 50%. Without
activity after 10 hours all entries would expire and the user would
experience a delay. If the new parameter is defined say 100% the
expiration timestamp will be set in between 10h and 10h + 10h * 100% =
20h randomly. It does not solve the problem of the refresh after 20h if
there is no activity happening, but if there is some minimal activity
happening within 10h and 20h window not all netgroups would expire at
the same time and not all of them need to be refreshed at the same time.
Does this make sense? Would it make things better?
I was experimenting with the problem this morning as I know this is a
big pain point for a customer.
I think the solution can be actually much simpler:
Right now, the midpoint refresh works if the expiration is N percent
into the cached entry timeout BUT not past the cache validity.
I think the simplest solution would be to either add a new option or
extend the current one to always return from cache even if the entry is
past its cache validity and schedule a refresh in the background. The
downside would obviously be that if the entry was removed from the LDAP
server then the first lookup would always hit it (unless it was scooped
by the cleanup task).