I make some synchronization features with freeipa. For this i need to fetch users from LDAP db. By default search limit is 2000 users both in GUI and API. So i made modofications in freeipa configs
ldapmodify -x -D "cn=Directory Manager" -W -H ldap://localhost
dn: cn=config changetype: modify replace: nsslapd-sizelimit nsslapd-sizelimit: 20000000
so all started to work fine, for starter there were just around 3000 users in db so it worked fine, returned all users. then i started to generate new users to see perfomance of DB, so i used "pkey_only: true" in "user_find" command so when quantity of users reached 90k users it spend around 25sec to response, so ok its tolerable.... so all 90k+ users still returned by API
So issue happened when quantity of users reached around 100к+ users, so search limit shrinked to 49629 users shrikage info started to add to responses
"count": 49629, "truncated": true, "messages": [ { "type": "warning", "name": "SearchResultTruncated", "message": "Search result has been truncated: Configured administrative server limit exceeded", "code": 13017, "data": { "reason": "Configured administrative server limit exceeded" } } ]
i do not do anything with freeipa server, just worked python script which generated users, script worked delicate and slow , he always waited for response from API that user created before creating another one user, so there were no high parallel overload on DB
so i searched for nsslapd-sizelimit again with command: ldapsearch -x -D "cn=Directory Manager" -W -H ldap://localhost -b "cn=config" nsslapd-sizelimit but is showed normally nsslapd-sizelimit: 20000000 so with settings all fine, it shows that number that i set
so what happened? Where freeipa get this strange limit in 49629 users and why? How to handle this? so projected size of LDAP DB is around 15 millons users I tried to workaround this with python ldap-3 library in hope it make some more low-level requests without any limits, but via python script i have this limit too, so its totally global for all, UI, API and python API... So how to remove this strange limit?
Anton PalkoBrosov via FreeIPA-users wrote:
I make some synchronization features with freeipa. For this i need to fetch users from LDAP db. By default search limit is 2000 users both in GUI and API. So i made modofications in freeipa configs
ldapmodify -x -D "cn=Directory Manager" -W -H ldap://localhost
dn: cn=config changetype: modify replace: nsslapd-sizelimit nsslapd-sizelimit: 20000000
so all started to work fine, for starter there were just around 3000 users in db so it worked fine, returned all users. then i started to generate new users to see perfomance of DB, so i used "pkey_only: true" in "user_find" command so when quantity of users reached 90k users it spend around 25sec to response, so ok its tolerable.... so all 90k+ users still returned by API
So issue happened when quantity of users reached around 100к+ users, so search limit shrinked to 49629 users shrikage info started to add to responses
"count": 49629, "truncated": true, "messages": [ { "type": "warning", "name": "SearchResultTruncated", "message": "Search result has been truncated: Configured administrative server limit exceeded", "code": 13017, "data": { "reason": "Configured administrative server limit exceeded" } } ]
i do not do anything with freeipa server, just worked python script which generated users, script worked delicate and slow , he always waited for response from API that user created before creating another one user, so there were no high parallel overload on DB
so i searched for nsslapd-sizelimit again with command: ldapsearch -x -D "cn=Directory Manager" -W -H ldap://localhost -b "cn=config" nsslapd-sizelimit but is showed normally nsslapd-sizelimit: 20000000 so with settings all fine, it shows that number that i set
so what happened? Where freeipa get this strange limit in 49629 users and why? How to handle this? so projected size of LDAP DB is around 15 millons users I tried to workaround this with python ldap-3 library in hope it make some more low-level requests without any limits, but via python script i have this limit too, so its totally global for all, UI, API and python API... So how to remove this strange limit?
There is also a search time limit. It's possible that could be related. If you look in the dirsrv access log it may tell you, or you can deduce it from the etime.
rob
freeipa-users@lists.fedorahosted.org