[SSSD] SSSD/Memberof profiling results
Jan Zelený
jzeleny at redhat.com
Fri Oct 21 11:22:42 UTC 2011
Hello everybody,
during last couple days, I've been profiling sssd storing large groups (i.e.
groups with many members and only flat membership structure) and I want to
share my findings. In my particular case these members were users, since that
is potentially the most complex memberof operation due to memberuid attribute.
First I would like to stress that numbers given here are not exact, but they
give a good impression about what consumes *CPU time* the most. I was
originally focused on the memberof plugin because we suspected that this will
be the greatest performance killer. But in the end the scope was much more
wide. For testing I used callgrind connected to the backend, debug level has
been set to 0. As remote server I used FreeIPA. The test case was
1. start sssd (in following inspection I ignored BE initialization)
2. getent group large-group (no member groups, ~2k member users)
3. shut down sssd
First some of my observations regarding memberof plugin:
=====================================
- the operation of the entire plugin took 8.73% of total CPU time
- this number can be reduced only a little bit:
-- removal of dn validation where not entirely necessary will gain ~0.55%
-- replacing comparison of ldb_dn by strcmp will gain ~0.2%
(I suspect it's not safe though)
-- using a hash table to store memberuid operations could gain up to 2.99%
(note that this is how much mbof_muop_append takes at the moment,
so in reality the gain will be likely lower)
Now something out of memberof scope:
=========================
- ldb_transaction_commit is called for each added user
(in the end it takes ~ 14.5% of CPU time)
-- we also have to consider disk write time which isn't measured by callgrind
-- here I suggest that recent Jakub's patch is extended to cover all
schema types and all potentially large transactions
- ltdb_search is called 10513 times and it takes 11.54% of CPU time
-- ldb_search (only prepares the search query) takes 6.27%
-- the total sysdb lookup consumption are previous two numbers combined
-- there should be also a little bit more time counted in from the memberof
plugin, but I consider that to be insignificant right now
That is all for the moment. If you have any ideas for test cases, feel free to
let me know.
Personally I believe that the solution to performance boost is to optimize
sysdb operations on high level since that is where we know what will be stored
and can adjust the process a bit (like using one transaction instead of couple
thousands). Right now I will focus on the number of searches performed when
storing group, I'd like to reduce this number at least by half.
If you have any other suggestions or comments, I'll welcome them.
Thanks
Jan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part.
URL: <https://lists.fedorahosted.org/pipermail/sssd-devel/attachments/20111021/46b14c47/attachment.sig>
More information about the sssd-devel
mailing list