Fruits and Nuts (or, need help with nested LDAP groups)
by John Gorkos
I've been handed a requirement to created nested groups in our LDAP directory
for our Posix logins to ease our admin load. We are using RHEL6.1 and SSSD
1.5.x from Redhat, along with OpenLDAP 2.4.x. I have loaded up a test VM with
all the appropriate software, but for the life of me I can't make it work.
For starters, I'm using the rfc2307bis schema, meaning I want a groupOfNames
with member attributes that are complete DNs.
I've included all the appropriate files/config at the end of this email, but the
base premis is this:
I have an ou=People organizational group with four users (Alpha, Bravo,
Charlie, and Delta). Each is an inetOrgPerson and posixAccount with a
gidNumber of 1000 and a uidNumber of 10001-10004.
I have an ou=Groups organizational group with two posixGroup groups (cn=Fruits
and cn=Nuts) with gidNumbers of 1001 and 1002. At the same level, I have 8
more groups (Apple, Banana, Cherry, Date and Almond, Brazil, Cashew and
Doughnut). These are just groupOfNames object, not posixGroups. As you would
expect, Fruits' members list will contain cn=[Apple|Banana|Cherry|
Date],ou=Groups,dc=example,dc=com and Nuts' members list will contain
cn=[Almond|Brazil|Cashew|Doughnut],ou=Groups,dc=example,dc=com.
We'll start off my example without them, and add them as needed.
Also, just for completeness, there is a cn=Users,ou=Groups entry with
gidNumber=1000 (for the default group of each user).
Now, the fun part. I want to have user Alpha be a Fruit. So, I add his dn to
cn=Fruits:
dn: cn=Fruits,ou=Groups,dc=example,dc=com
objectClass: groupOfNames
objectClass: posixGroup
gidNumber: 1001
cn: fruits
member: cn=User Alpha,ou=People,dc=example,dc=com
(here's user alpha's definition)
dn: cn=User Alpha,ou=People,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
cn: User Alpha
sn: Alpha
gidNumber: 1000
homeDirectory: /home/alpha
uidNumber: 10001
uid: alpha
I ask NSS about him, and I get this:
# id alpha
uid=10001(alpha) gid=1000(users) groups=1000(users),1001(fruits)
Well, that's great. Now lets add him to nuts, too:
# ldapsearch -xLLL cn=nuts
dn: cn=Nuts,ou=Groups,dc=example,dc=com
objectClass: groupOfNames
objectClass: posixGroup
gidNumber: 1002
cn: nuts
member: cn=User Alpha,ou=People,dc=example,dc=com
And now he's a fruit and a nut. Perfect.
# id alpha
uid=10001(alpha) gid=1000(users) groups=1000(users),1002(nuts),1001(fruits)
In my organization, there are lots of nuts. We'd like to categorize our nuts
a little better, and User Alpha is definitely a Doughnut. So, we'll take him
our of the cn=Nuts group and put him in cn=Doughnut.
After just adding him to cn=Doughnut, we don't expect that list to show up in
the nss list of groups, because Doughnut is not a posixGroup. Here's the list
structure so far:
dn: cn=Nuts,ou=Groups,dc=example,dc=com
objectClass: groupOfNames
objectClass: posixGroup
gidNumber: 1002
cn: nuts
dn: cn=Doughnut,ou=Groups,dc=example,dc=com
objectClass: groupOfNames
cn: doughnut
member: cn=User Alpha,ou=People,dc=example,dc=com
# id alpha
uid=10001(alpha) gid=1000(users) groups=1000(users),1001(fruits)
Perfect. He's still a fruit, but not yet a nut.
Now, the kicker. According to my understanding, I should be able to put
cn=Doughnut,ou=Groups,dc=example,dc=com into the member list of
cn=Nuts,ou=Groups,dc=example,dc=com and our buddy User Alpha should become
part of posix group nuts:
dn: cn=Nuts,ou=Groups,dc=example,dc=com
objectClass: groupOfNames
objectClass: posixGroup
gidNumber: 1002
cn: nuts
member: cn=Doughnut,ou=Groups,dc=example,dc=com
dn: cn=Doughnut,ou=Groups,dc=example,dc=com
objectClass: groupOfNames
cn: doughnut
member: cn=User Alpha,ou=People,dc=example,dc=com
But alas:
# id alpha
uid=10001(alpha) gid=1000(users) groups=1000(users),1001(fruits)
And this is where the gnashing of teeth, pulling of hair, and wailing begin.
# getent group
root:x:0:root
...
users:*:1000:
fruits:*:1001:alpha
nuts:*:1002:
# getent group nuts
nuts:*:1002:
This one really puzzles me:
# sss_groupshow fruits
Error initializing the tools - no local domain
So, that's where I stand, on the brink between success and failure. On the
precipitice of unemployment, and I beseech the wisdom of the crowd to rescue
me. Somewhere in the following sssd.conf file, or in my LDAP schema, or in
SOMETHING, I'm missing a line, or have something misconfigured. I'm at your
collective mercy.
sssd.conf:
[sssd]
config_file_version = 2
reconnection_retries = 3
sbus_timeout = 30
services = nss, pam
domains = LDAP
[nss]
filter_groups = root
filter_users = root
reconnection_retries = 3
entry_cache_timeout = 60
entry_cache_nowait_timeout = 30
enum_cache_timeout = 30
[pam]
reconnection_retries = 3
[domain/LDAP]
ldap_uri = ldap://smbtest2
auth_provider = ldap
id_provider = ldap
chpass_provider = ldap
te = true
entry_cache_timeout = 10
cache_credentials = false
ldap_id_use_start_tls = True
ldap_schema = rfc2307bis
ldap_search_base = dc=example,dc=com
ldap_tls_cacert = /etc/pki/tls/certs/slapdcert.pem
ldap_enumeration_refresh_timeout = 10
ldap_purge_cache_timeout = 10
ldap_group_nesting_level = 5
debug_level = 5
/etc/nsswitch.conf:
#
passwd: files sss
shadow: files sss
group: files sss
hosts: files dns
bootparams: nisplus [NOTFOUND=return] files
ethers: files
netmasks: files
networks: files
protocols: files
rpc: files
services: files
netgroup: files ldap
publickey: nisplus
automount: files ldap
aliases: files nisplus
/etc/ldap.conf (even though it should not be used, because SSS is handling
NSS)
base dc=example,dc=com
uri ldap://smbtest2
ldap_version 3
scope sub
restart yes
pam_template_login_attribute uid
pam_password md5
bind_policy soft
network_timeout 5
timelimit 10
bind_timelimit 10
tls_cacertfile /etc/pki/tls/certs/slapdcert.pem
ssl start_tls
nss_schema rfc2307bis
nss_base_netgroup ou=Netgroups,dc=example,dc=com
nss_connect_policy persist
nss_map_attribute uniqueMember member
I have slightly modified the LDAP schemas to allow for empty groupOfNames (i.e.
no members), and to make RFC2307bis work.
Any other files, debug output, logs, etc are available on request. If I don't
get this working, we'll wind up having to use netgroups and nis-like
construts, which I feel is a giant step backward in time. Finally, I realize
this is a "devel" list and not an "end user help" list, but I'm unsure of
where else to turn.
Thanks in advance.
John Gorkos
12 years, 5 months
Re: [SSSD] [PATCH] Plug memory leaks in sysdb_ops (Jakub Hrozek)
by Mark London
Jakub - I figured out why my debug log file was empty. I only had debugging enabled for nss and pam!
In any event, yes, I now see the memory leaks. My valgrind summary for sssd_be, after running it for a while, is:
==31457== LEAK SUMMARY:
==31457== definitely lost: 128,911 bytes in 1,312 blocks
==31457== indirectly lost: 105,416,263 bytes in 899,812 blocks
==31457== possibly lost: 3,303,640 bytes in 27,120 blocks
==31457== still reachable: 758,871 bytes in 4,324 blocks
==31457== suppressed: 0 bytes in 0 blocks
I tried using your patch file, but it couldn't properly patch my version of sysdb_ops.c (I was using a compiled version of 1.6.1).
I think what I'll do is to reinstall the REDHAT version of SSSD, disable caching, and wait until a properly patched version of SSSD is either released from Redhat or the SSSD site. Thanks. - Mark
12 years, 5 months
Re: [SSSD] SSSD + CYRUS/IMAP + LDAP + NSS caching = constantly growing sssd_be memory usage?
by Mark London
Dmitri - I added valgrind to the sssd_be command, and didn't get any
error messages except for a bunch of these messages:
==30135== Warning: invalid file descriptor -1 in syscall close()
I believe these are minor, according to what I've read on the web.
Here's another question. When I turn on debugging, I get the following
log files:
-rw------- 1 root root 208 Oct 22 21:36 sssd.log
-rw------- 1 root root 30970503 Oct 22 17:27 sssd_nss.log
-rw------- 1 root root 1913856 Oct 22 17:27 sssd_pam.log
-rw------- 1 root root 0 Oct 22 16:58 sssd_PSFC.log
My domain is PSFC, yet the sssd_PSFC.log file is empty. Is that normal
to be empty, even with debug set to 10? I believe that's the log file
that the sssd_be process logs to.
- Mark
dpal(a)redhat.com wrote:
> On 10/21/2011 03:09 PM, Mark London wrote:
>> Hi - I've compiled and installed the latest version of SSSD (1.6.1),
>> with caching enabled, for a Redhat 6 computer running CYRUS IMAP mail
>> server software (with SASL). Users are authenticated via LDAP, and
>> the LDAP server is running as part of a Windows domain server. Mail
>> is sent using sendmail, with SMTP authentication. We have
>> appropriately 200 users (who read their mail using various
>> applications, including the server's Squirrelmail webmail interface).
>>
>> Using both the version of SSSD that came with Redhat 6 and the newly
>> compiled, the sssd_be process will constantly increase it's memory
>> usage over time. Within a day, the process is now using up 2G:
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
>> COMMAND
>> 22900 root 20 0 2002m 1.7g 5628 S 0.0 14.8 9:41.42 sssd_be
>>
>> Why is there a constant need for extra memory, and is the amount of
>> memory being used, "normal" for a mail server, that only 200 users?
>> Here is my sssd.conf file. Thanks for any suggestions or help that
>> anyone can provide. - Mark
>
> Seems like a time to run Valgrind. Looks like a leak to me ...
>
>> ----------------------------- sssd.conf --------------------------------
>> [sssd]
>> config_file_version = 2
>> # Number of times services should attempt to reconnect in the
>> # event of a crash or restart before they give up
>> reconnection_retries = 3
>>
>> # If a back end is particularly slow you can raise this timeout here
>> sbus_timeout = 30
>> services = nss, pam
>>
>> # SSSD will not start if you do not configure any domains.
>> # Add new domain configurations as [domain/<NAME>] sections, and
>> # then add the list of domains (in the order you want them to be
>> # queried) to the "domains" attribute below and uncomment it.
>> ; domains = LOCAL,LDAP
>>
>> domains = PSFC
>>
>> [nss]
>> # The following prevents SSSD from searching for the root user/group in
>> # all domains (you can add here a comma-separated list of system accounts that
>> # are always going to be /etc/passwd users, or that you want to filter out).
>> filter_groups = root
>> filter_users = root
>> reconnection_retries = 3
>> debug_level = 0
>>
>> # The entry_cache_timeout indicates the number of seconds to retain an
>> # entry in cache before it is considered stale and must block to refresh.
>> # The entry_cache_nowait_timeout indicates the number of seconds to
>> # wait before updating the cache out-of-band. (NSS requests will still
>> # be returned from cache until the full entry_cache_timeout). Setting this
>> # value to 0 turns this feature off (default).
>> #entry_cache_timeout = 600
>> #entry_cache_nowait_timeout = 300
>>
>> [pam]
>> reconnection_retries = 3
>> debug_level = 0
>>
>> # LDAP domain where the LDAP server is an Active Directory server.
>>
>> [domain/PSFC]
>> description = LDAP domain with AD server
>> enumerate = true
>> min_id = 501
>> cache_credentials = TRUE
>>
>> id_provider = ldap
>> chpass_provider = none
>> auth_provider = ldap
>> ldap_uri = ldaps://xxxxxxxx
>> ldap_tls_cacertdir = /etc/openldap/cacerts
>> ldap_schema = rfc2307bis
>> ldap_search_base = dc=psfc,dc=mit,dc=edu
>> ldap_user_search_base = dc=psfc,dc=mit,dc=edu
>> ldap_group_search_base = dc=psfc,dc=mit,dc=edu
>> ldap_default_bind_dn = CN=ADldapreadonly,OU=Computer Group,OU=PSFC Users,DC=psfc,DC=mit,DC=edu
>> ldap_default_authtok_type = password
>> ldap_default_authtok = ldapread
>> ldap_user_object_class = person
>> ldap_user_name = sAMAccountName
>> ldap_user_uid_number = msSFU30UidNumber
>> ldap_user_gid_number = msSFU30GidNumber
>> ldap_user_home_directory = msSFU30HomeDirectory
>> ldap_user_shell = msSFU30LoginShell
>> ldap_user_principal = userPrincipalName
>> ldap_group_object_class = group
>> ldap_group_member = msSFU30PosixMember
>> ldap_user_member_of = msSFU30PosixMemberOf
>> ldap_group_name = name
>> ldap_group_gid_number = msSFU30GidNumber
>> ldap_force_upper_case_realm = True
>>
>>
>>
>> _______________________________________________
>> sssd-devel mailing list
>> sssd-devel(a)lists.fedorahosted.org
>> https://fedorahosted.org/mailman/listinfo/sssd-devel
>
>
12 years, 5 months
SSSD/Memberof profiling results
by Jan Zelený
Hello everybody,
during last couple days, I've been profiling sssd storing large groups (i.e.
groups with many members and only flat membership structure) and I want to
share my findings. In my particular case these members were users, since that
is potentially the most complex memberof operation due to memberuid attribute.
First I would like to stress that numbers given here are not exact, but they
give a good impression about what consumes *CPU time* the most. I was
originally focused on the memberof plugin because we suspected that this will
be the greatest performance killer. But in the end the scope was much more
wide. For testing I used callgrind connected to the backend, debug level has
been set to 0. As remote server I used FreeIPA. The test case was
1. start sssd (in following inspection I ignored BE initialization)
2. getent group large-group (no member groups, ~2k member users)
3. shut down sssd
First some of my observations regarding memberof plugin:
=====================================
- the operation of the entire plugin took 8.73% of total CPU time
- this number can be reduced only a little bit:
-- removal of dn validation where not entirely necessary will gain ~0.55%
-- replacing comparison of ldb_dn by strcmp will gain ~0.2%
(I suspect it's not safe though)
-- using a hash table to store memberuid operations could gain up to 2.99%
(note that this is how much mbof_muop_append takes at the moment,
so in reality the gain will be likely lower)
Now something out of memberof scope:
=========================
- ldb_transaction_commit is called for each added user
(in the end it takes ~ 14.5% of CPU time)
-- we also have to consider disk write time which isn't measured by callgrind
-- here I suggest that recent Jakub's patch is extended to cover all
schema types and all potentially large transactions
- ltdb_search is called 10513 times and it takes 11.54% of CPU time
-- ldb_search (only prepares the search query) takes 6.27%
-- the total sysdb lookup consumption are previous two numbers combined
-- there should be also a little bit more time counted in from the memberof
plugin, but I consider that to be insignificant right now
That is all for the moment. If you have any ideas for test cases, feel free to
let me know.
Personally I believe that the solution to performance boost is to optimize
sysdb operations on high level since that is where we know what will be stored
and can adjust the process a bit (like using one transaction instead of couple
thousands). Right now I will focus on the number of searches performed when
storing group, I'd like to reduce this number at least by half.
If you have any other suggestions or comments, I'll welcome them.
Thanks
Jan
12 years, 5 months
SSSD + CYRUS/IMAP + LDAP + NSS caching = constantly growing sssd_be memory usage?
by Mark London
Hi - I've compiled and installed the latest version of SSSD (1.6.1),
with caching enabled, for a Redhat 6 computer running CYRUS IMAP mail
server software (with SASL). Users are authenticated via LDAP, and the
LDAP server is running as part of a Windows domain server. Mail is sent
using sendmail, with SMTP authentication. We have appropriately 200
users (who read their mail using various applications, including the
server's Squirrelmail webmail interface).
Using both the version of SSSD that came with Redhat 6 and the newly
compiled, the sssd_be process will constantly increase it's memory usage
over time. Within a day, the process is now using up 2G:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
COMMAND
22900 root 20 0 2002m 1.7g 5628 S 0.0 14.8 9:41.42 sssd_be
Why is there a constant need for extra memory, and is the amount of
memory being used, "normal" for a mail server, that only 200 users?
Here is my sssd.conf file. Thanks for any suggestions or help that
anyone can provide. - Mark
----------------------------- sssd.conf --------------------------------
[sssd]
config_file_version = 2
# Number of times services should attempt to reconnect in the
# event of a crash or restart before they give up
reconnection_retries = 3
# If a back end is particularly slow you can raise this timeout here
sbus_timeout = 30
services = nss, pam
# SSSD will not start if you do not configure any domains.
# Add new domain configurations as [domain/<NAME>] sections, and
# then add the list of domains (in the order you want them to be
# queried) to the "domains" attribute below and uncomment it.
; domains = LOCAL,LDAP
domains = PSFC
[nss]
# The following prevents SSSD from searching for the root user/group in
# all domains (you can add here a comma-separated list of system accounts that
# are always going to be /etc/passwd users, or that you want to filter out).
filter_groups = root
filter_users = root
reconnection_retries = 3
debug_level = 0
# The entry_cache_timeout indicates the number of seconds to retain an
# entry in cache before it is considered stale and must block to refresh.
# The entry_cache_nowait_timeout indicates the number of seconds to
# wait before updating the cache out-of-band. (NSS requests will still
# be returned from cache until the full entry_cache_timeout). Setting this
# value to 0 turns this feature off (default).
#entry_cache_timeout = 600
#entry_cache_nowait_timeout = 300
[pam]
reconnection_retries = 3
debug_level = 0
# LDAP domain where the LDAP server is an Active Directory server.
[domain/PSFC]
description = LDAP domain with AD server
enumerate = true
min_id = 501
cache_credentials = TRUE
id_provider = ldap
chpass_provider = none
auth_provider = ldap
ldap_uri = ldaps://xxxxxxxx
ldap_tls_cacertdir = /etc/openldap/cacerts
ldap_schema = rfc2307bis
ldap_search_base = dc=psfc,dc=mit,dc=edu
ldap_user_search_base = dc=psfc,dc=mit,dc=edu
ldap_group_search_base = dc=psfc,dc=mit,dc=edu
ldap_default_bind_dn = CN=ADldapreadonly,OU=Computer Group,OU=PSFC Users,DC=psfc,DC=mit,DC=edu
ldap_default_authtok_type = password
ldap_default_authtok = ldapread
ldap_user_object_class = person
ldap_user_name = sAMAccountName
ldap_user_uid_number = msSFU30UidNumber
ldap_user_gid_number = msSFU30GidNumber
ldap_user_home_directory = msSFU30HomeDirectory
ldap_user_shell = msSFU30LoginShell
ldap_user_principal = userPrincipalName
ldap_group_object_class = group
ldap_group_member = msSFU30PosixMember
ldap_user_member_of = msSFU30PosixMemberOf
ldap_group_name = name
ldap_group_gid_number = msSFU30GidNumber
ldap_force_upper_case_realm = True
12 years, 5 months
Why is SUA needed for SSSD to work with a Microsoft AD LDAP server?
by Greg Lehmann
I have been reading up on SUA and can't see any mention of LDAP changes being made when SUA is enabled. As far as I can see all the posix bits are already there and SSSD should work fine without SUA. I'd really like to know why I need SUA as it involves a whole lot of other changes in AD that I don't particularly want. We have Windows Server 2008 R2.
TIA
Greg
12 years, 5 months
[PATCH] Use fake users for RFC2307bis nested group processing
by Jakub Hrozek
https://bugzilla.redhat.com/show_bug.cgi?id=742052
Attached are two patches that should be commited to 1.5. They are
already present in 1.6 and later branches.
Instead of downloading all user attributes and saving complete user data
during nested RFC2307bis group processing, download only the necessary
attributes (objectclass and name) and save the users as "fake".
This lowers the network traffic needed to download all the attributes, reduces
the cache size and provides a minor speedup.
The speedup really is minor compared to what dereference brings because
the RFC2307bis processing code still needs to perform at least N LDAP
lookups for N group members that are not cached.
12 years, 5 months
Announcing SSSD 1.5.14
by Stephen Gallagher
The SSSD team is proud to announce the version 1.5.14 enhancement and
bugfix release of the System Security Services Daemon.
As always, it can be downloaded from https://fedorahosted.org/sssd/
== Highlights ==
* Improved handling of users and groups with multi-valued name
attributes (aliases)
* Performance enhancements
* Initgroups on RFC2307bis/FreeIPA
* HBAC rule processing
* Improved process-hang detection and restarting
* Enabled the midpoint cache refresh by default (fewer cache misses on
commonly-used entries)
* Cleaned up the example configuration
== Detailed Changelog ==
Jakub Hrozek (22):
* Improve error message for LDAP password constraint violation
* Fix uninitialized pointer read in sdap_gssapi_get_default_realm()
* IPA access: hostname comparison should be case-insensitive
* Add sysdb interface to get name aliases
* Add a sysdb_get_direct_parents function
* Store name aliases for users, groups
* Return users and groups based on alias
* Use explicit base 10 for converting strings to integers
* Fix typo in sysdb_get_direct_parents
* Add option to follow symlinks to check_file()
* Append PID to sbus server socket name, let clients use a symlink
* Streamline the example config
* Do not delete requests inside hash_iterate loop
* Check if dp_requests hash table exists before using it
* Fix off-by-one error in remove_socket_symlink()
* Report on errno, not return code in create_socket_symlink
* Add a missing break
* Sanitize DN in sysdb_get_direct_parents
* gitignore additions
* Utility functions for LDAP nested schema initgroups
* Use fewer transactions during RFC2307bis initgroups
* Use fewer transactions during IPA initgroups
Jan Zeleny (2):
* man page fix (lists are comma-separated)
* Fixed timeout handling in responders
Marko Myllynen (3):
* Add missing options to sssd.api.conf
* Unbreak ./configure
* Update sssd-example.conf
Pavel Březina (3):
* sss_ldap_err2string() - function created
* sss_ldap_err2string() - ldap_err2string() to sss_ldap_err2string()
* Added quiet option to pam_sss
Stephen Gallagher (17):
* Bumping version to 1.5.14
* Add option to specify the kerberos replay cache dir
* Fix typo in %configure
* Remove all libtool .la files from RPM
* Improve documentation of libipa_hbac
* Add libipa_hbac documentation to the -devel package
* MONITOR: Correctly detect lack of response from services
* Do not build documentation on RHEL 5
* Fix typo in specfile
* MAN: Add more information about internal credential storage
* HBAC: fix typos preventing proper hostgroup evaluation
* HBAC: Do not save member/memberOf links
* HBAC: Use originalMember for identifying servicegroups
* HBAC: Use originalMember for identifying hostgroups
* BUILDSYS: Fix --without-manpages
* MONITOR: fix timeout conversion
* Updating translation files
Sumit Bose (1):
* Do not access memory out of bounds
12 years, 5 months