Hi,

On Wed, Jun 12, 2024 at 3:35 PM Anton PalkoBrosov via FreeIPA-users <freeipa-users@lists.fedorahosted.org> wrote:
I make some synchronization features with freeipa. For this i need to fetch users from LDAP db.
By default search limit is 2000 users both in GUI and API.
So i made modofications in freeipa configs

ldapmodify -x -D "cn=Directory Manager" -W -H ldap://localhost

dn: cn=config
changetype: modify
replace: nsslapd-sizelimit
nsslapd-sizelimit: 20000000

so all started to work fine, for starter there were just around 3000 users in db so it worked fine, returned all users.
then i started to generate new users to see perfomance of DB, so i used "pkey_only: true" in "user_find" command so when quantity of users reached 90k users  it spend around 25sec to response, so ok its tolerable....
so all 90k+ users still returned by API

So issue happened when quantity of users reached around 100к+ users, so search limit shrinked to 49629 users
shrikage info started to add to responses

"count": 49629,
        "truncated": true,
        "messages": [
            {
                "type": "warning",
                "name": "SearchResultTruncated",
                "message": "Search result has been truncated: Configured administrative server limit exceeded",
                "code": 13017,
                "data": {
                    "reason": "Configured administrative server limit exceeded"
                }
            }
        ]

The search results can be truncated because of size limit but also because of time limits. You can find more information in this chapter:
Adjusting the search size and time limit

Note that adjusting the size or time limits using the ipa config-mod command with the --searchrecordslimit or the --searchtimelimit options affects the number of entries returned by ipa commands, such as ipa user-find.
In addition to these limits, the settings configured at the Directory Server level are also taken into account and may impose stricter limits. For more information on Directory Server limits, see the Red Hat Directory Server Administration Guide.

flo

i do not do anything with freeipa server, just worked python script which generated users, script worked delicate and slow , he always waited for response from API that user created before creating another one user, so there were no high parallel overload on DB

so i searched for nsslapd-sizelimit again with command:
ldapsearch -x -D "cn=Directory Manager" -W -H ldap://localhost -b "cn=config" nsslapd-sizelimit
but is showed normally nsslapd-sizelimit: 20000000
so with settings all fine, it shows that number that i set

so what happened? Where  freeipa get this strange limit in 49629 users and why? How to handle this? so projected size of LDAP DB is around 15 millons users
I tried to workaround this with python ldap-3 library in hope it make some more low-level requests without any limits, but via python script i have this limit too, so its totally global for all, UI, API and python API...
So how to remove this strange limit?
--
_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue