On 5/7/20 4:38 AM, Mary Georgiou via FreeIPA-users wrote:
Hello,
In our set-up, we have a DB with all the users and groups, which we use as ground truth for provisioning the forementioned objects in FreeIPA (2 master servers + replicas). We are continuously synchronizing entries (~60000 users and 60000 groups, where groups might have 0 to 20000 members) from the DB to FreeIPA. In each cycle of synch, we are figuring out the differences and add, delete, or change existing entries.
The first sync (through which we had to import all 120000 objects) clogged the server totally, and after tweaking the 389DS we ended up disabling the memberOf plugin where it finally worked (we followed the FreeIPA documentation[1]). One of the advice to follow is to do the sync and then run the fixup task in the server where the provisioning happened. The fixup still clogs the server after some point and stops. The errors we get in the log are the following:
[06/May/2020:18:16:59.862308719 +0200] - INFO - memberof-plugin - memberof_fixup_task_thread - Memberof task starts (filter: "(|(objectclass=inetuser)(objectclass=inetadmin)(objectclass=nsmemberof))") ... [06/May/2020:20:07:49.545606214 +0200] - ERR - libdb - BDB2055 Lock table is out of available lock entries [06/May/2020:20:07:49.547921580 +0200] - ERR - idl_new_delete_key - idl_new.c BAD 22, err=12 Cannot allocate memory [06/May/2020:20:07:49.548930035 +0200] - ERR - addordel_values_sv - database index operation failed BAD 1130, err=12 Cannot allocate memory [06/May/2020:20:07:49.549779631 +0200] - ERR - addordel_values_sv - database index operation failed BAD 1140, err=12 Cannot allocate memory [06/May/2020:20:07:49.550612745 +0200] - ERR - index_addordel_values_ext_sv - database index operation failed BAD 1230, err=12 Cannot allocate memory [06/May/2020:20:07:49.551444741 +0200] - ERR - index_add_mods - database index operation failed BAD 1041, err=12 Cannot allocate memory [06/May/2020:20:07:49.552457769 +0200] - ERR - index_add_mods - database index operation failed BAD 1040, err=12 Cannot allocate memory [06/May/2020:20:07:49.553305019 +0200] - ERR - ldbm_back_modify - index_add_mods failed, err=12 Cannot allocate memory
We just saw this in a different case.
First do a ldapsearch as follows:
# ldapsearch -D "cn=directory manager" -W -b "YOUR_DB SUFFIX" (|(objectclass=inetuser)(objectclass=inetadmin)(objectclass=nsmemberof))
How many entries are turned, save this value add 10000 to it and use it below:
For now try setting this attribute under cn=config
ldapmodify -D "cn=directory manager" -W dn: cn=config,cn=ldbm database,cn=plugins,cn=config changetype: modify replace: nsslapd-idlistscanlimit nsslapd-idlistscanlimit: YOUR_NUMBER_FROM_THE SEARCH
Then try the fixup task again.
HTH, Mark
We increased the number of DB locks and set the `nsslapd-cache-autosize` to 50% (server has currently 13G of memory). The only thing we saw was that one thread was using 100% of one of the CPUs. Any advice on how to deal with this? We would really need to have memberOf attribute. Thank you in advance! Best Regards Mary Georgiou [1] https://www.freeipa.org/page/V4/Performance_Improvements _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org