Hey Florence
Your suggestion seems to be fixed the limit problem for my case. That is interesting
because when checking the logs I see the search using my user cn but the anonymous-limit
change was effective somehow.
Thank you all for your help.
—
Givaldo Lins
> On Feb 16, 2018, at 8:54 AM, Florence Blanc-Renaud via FreeIPA-users
<freeipa-users(a)lists.fedorahosted.org> wrote:
>
>> On 02/15/2018 06:42 PM, Bret Wortman via FreeIPA-users wrote:
>>> On 02/15/2018 12:27 PM, Florence Blanc-Renaud wrote:
>>>> On 02/15/2018 05:01 PM, Bret Wortman via FreeIPA-users wrote:
>>>>> On 02/15/2018 09:29 AM, Florence Blanc-Renaud wrote:
>>>>>> On 02/15/2018 02:40 PM, Bret Wortman via FreeIPA-users wrote:
>>>>>>> On 02/15/2018 07:09 AM, Florence Blanc-Renaud via
FreeIPA-users wrote:
>>>>>>>> On 02/15/2018 11:47 AM, Bret Wortman via FreeIPA-users
wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 02/15/2018 04:50 AM, Florence Blanc-Renaud wrote:
>>>>>>>>>> On 02/15/2018 10:08 AM, Florence Blanc-Renaud via
FreeIPA-users wrote:
>>>>>>>>>>> On 02/14/2018 05:58 PM, Bret Wortman wrote:
>>>>>>>>>>>> On 02/14/2018 10:22 AM, Florence
Blanc-Renaud wrote:
>>>>>>>>>>>>> On 02/14/2018 12:52 PM, Bret Wortman
via FreeIPA-users wrote:
>>>>>>>>>>>>> I did figure out that I can use
>>>>>>>>>>>>>
>>>>>>>>>>>>> # ldapsearch -D 'directory
manager' -W -E pr=20000 -b idnsname=damascusgrp.com,cn=dns,dc=damascusgrp,dc=com
>>>>>>>>>>>>>
>>>>>>>>>>>>> to list out all the entries, but the
format isn't what I'm expecting.
>>>>>>>>>>>>>
>>>>>>>>>>>>> What I'm actually trying to do is
move our whole infrastructure from one set of old & busted servers to some shiny new
VMs. We'd like to extract the data and start fresh, as our replication agreements just
don't seem to be working as expected. Changes to one don't always make it to the
other and vice versa. While I'd love to dig in and solve that, it's easier right
now to try to extract the data and reload it into a new server, build new replicas, then
unbind & re-bind every client to the new server using ansible since we also lost our
internal CA in the process.
>>>>>>>>>>>>>
>>>>>>>>>>>>> So while our current configuration is
a mess, we can't afford to lose all the host/user/dns/hbac data in our servers. Thus,
I've been capturing the output to text using various ipa *-find commands and have
parsers to turn those back into new entries on the fresh hosts. DNS is the only thing
that's holding me up.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Bret
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 02/14/2018 06:33 AM, Bret
Wortman wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also, this doesn't solve the
fact that the Web UI always produces an error dialog whenever accessing our primary zone.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 02/13/2018 02:19 PM, Natxo
Asenjo via FreeIPA-users wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Feb 13, 2018 at 8:13
PM, Natxo Asenjo <natxo.asenjo(a)gmail.com <mailto:natxo.asenjo@gmail.com>>
wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> the canonical way to do
this is using ldap paging, with
>>>>>>>>>>>>>>> ldapsearch you could try
using the -E pr=xxxx parameter, where
>>>>>>>>>>>>>>> xxxx could be 1000 for
instance. That way you know you are always
>>>>>>>>>>>>>>> under the limit imposed
by the server.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> if you use -E
pr=1000/noprompt, it will not prompt to continue, nicer for scripts obviously.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Groeten,
>>>>>>>>>>>>>>> natxo
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>>>>> FreeIPA-users mailing list
--freeipa-users(a)lists.fedorahosted.org
>>>>>>>>>>>>>>> To unsubscribe send an email
tofreeipa-users-leave(a)lists.fedorahosted.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>>> FreeIPA-users mailing list --
freeipa-users(a)lists.fedorahosted.org
>>>>>>>>>>>>> To unsubscribe send an email to
freeipa-users-leave(a)lists.fedorahosted.org
>>>>>>>>>>>>>
>>>>>>>>>>>> Hi Bret,
>>>>>>>>>>>>
>>>>>>>>>>>> the search limits can be set at multiple
levels:
>>>>>>>>>>>> - for the whole 389-ds server
>>>>>>>>>>>> nsslapd-sizelimit (in cn=config)
>>>>>>>>>>>> nsslapd-lookthroughlimit (in
cn=config,cn=ldbm database,cn=plugins,cn=config)
>>>>>>>>>>>>
>>>>>>>>>>>> - for operations performed through ipa *
commands (or the webGUI):
>>>>>>>>>>>> ipaSearchRecordsLimit (in
cn=ipaConfig,cn=etc,$BASEDN)
>>>>>>>>>>>>
>>>>>>>>>>>> - for each user:
>>>>>>>>>>>> nssizelimit and nsLookThroughLimit
attributes (in uid=$USER,cn=users,cn=accounts,$BASEDN)
>>>>>>>>>>>>
>>>>>>>>>>>> You are probably hitting one of these
limits in your ipa *-find command.
>>>>>>>>>>>>
>>>>>>>>>>>> HTH,
>>>>>>>>>>>> Flo
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> So I found almost all of these:
>>>>>>>>>>>
>>>>>>>>>>> # ldapsearch -D 'cn=directory
manager' -W -b 'cn=config' cn=config | grep nsslapd-sizelimit
>>>>>>>>>>> nsslapd-sizelimit: 2000
>>>>>>>>>>>
>>>>>>>>>>> # ldapsearch -D 'cn=directory
manager' -W -b 'cn=config,cn=ldbm database,cn=plugins,cn=config' | grep
lookthroughlimit
>>>>>>>>>>> nsslapd-lookthroughlimit: 100000
>>>>>>>>>>>
>>>>>>>>>>> # ldapsearch -D 'cn=directory
manager' -W -b 'cn=ipaConfig,cn=etc,dc=damascusgrp,dc=com' | grep
ipaSearchRecordsLimit
>>>>>>>>>>> ipaSearchRecordsLimit: 99999
>>>>>>>>>>>
>>>>>>>>>>> # ldapsearch -D 'cn=directory
manager' -W -b 'uid=admin,cn=users,cn=accounts,dc=damascusgrp,dc=com' | grep
-i limit
>>>>>>>>>>> (returns data but nothing matches)
>>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> nsSizeLimit and nsLookThroughLimit are
operational attributes, meaning that a standard ldapsearch will not return them. You need
either to specifically request them by providing them in the attributes list:
>>>>>>>>>>
>>>>>>>>>> $ ldapsearch -D 'cn=directory manager' -W
-b $BASE nssizelimit nslookthroughlimit
>>>>>>>>>>
>>>>>>>>>> or you can also specify + instead of the
attributes in order to get all operational attributes:
>>>>>>>>>> $ ldapsearch -D 'cn=directory manager' -W
-b $BASE +
>>>>>>>>>>
>>>>>>>>>> HTH,
>>>>>>>>>> Flo
>>>>>>>>>>
>>>>>>>>>>> The first doesn't seem to be something I
can change. It's stuck at 2000, but since my issue occurs at 5000, I'm not worried
about it. I believe that I'm missing something in the fourth search that might point
me toward the attributes you mentioned but I'm not sure where.
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The 5000 limit rings a bell to me. It is the
anonymous size limit. Can you check:
>>>>>>>>> $ ldapsearch -D 'cn=directory manager' -W -b
cn=config -s base nsslapd-anonlimitsdn
>>>>>>>>>
>>>>>>>>> it will provide you with a DN of the entry defining
the anonymous limits (usually cn=anonymous-limits,cn=etc,$BASEDN), then:
>>>>>>>>>
>>>>>>>>> $ ldapsearch -D 'cn=directory manager' -W -b
cn=anonymous-limits,cn=etc,$BASEDN nsSizeLimit nsLookThroughLimit
>>>>>>>>>
>>>>>>>>> Now we should check the access log
(/var/log/dirsrv/slapd-xxx/access) corresponding to your command to retrieve the DNS
entries, and ensure which user identity is actually performing the search.
>>>>>>>>>
>>>>>>>>> Flo
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> FreeIPA-users mailing list --
freeipa-users(a)lists.fedorahosted.org
>>>>>>>>>> To unsubscribe send an email to
freeipa-users-leave(a)lists.fedorahosted.org
>>>>>>>>>
>>>>>>>>
>>>>>>>> Both those limits are 5000.
>>>>>>>>
>>>>>>>> What should I be looking for in the access log? It gets a
lot of traffic and narrowing down exactly which entry represents the original search is
challenging. I'd rather not post the whole thing because it's pretty large &
noisy.
>>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> you can perform the ipa dnsrecord-find --all command, while
tail'ing the access log. I will be interested in the lines containing the SEARCH with
a err=4 (size limit exceeded). For this SEARCH, please note the connection number
(conn=xx) and find the BIND for the same connection number (it will be located before the
SEARCH). For each line (BIND and SEARCH) both the operation and the RESULT are
interesting, for instance when doing ipa user-find:
>>>>>>>
>>>>>>> [date] conn=514 op=0 BIND dn="" method=sasl
version=3 mech=GSS-SPNEGO
>>>>>>> [date] conn=514 op=0 RESULT err=0 tag=97 nentries=0
etime=0.0004754873 dn="uid=admin,cn=users,cn=accounts,$BASEDN"
>>>>>>> ...
>>>>>>> [date] conn=514 op=2 SRCH
base="cn=users,cn=accounts,$BASEDN" scope=1
filter="(objectClass=posixaccount)" attrs="krbCanonicalName loginShell
sshpubkeyfp gidNumber homeDirectory uidNumber uid givenName title sn krbPrincipalName mail
nsAccountLock telephoneNumber ipaSshPubKey"
>>>>>>> [date] conn=514 op=2 RESULT err=0 tag=101 nentries=6
etime=0.0006313857
>>>>>>>
>>>>>>> Flo
>>>>>>>
>>>>>>>> Bret
>>>>>>>> _______________________________________________
>>>>>>>> FreeIPA-users mailing list --
freeipa-users(a)lists.fedorahosted.org
>>>>>>>> To unsubscribe send an email to
freeipa-users-leave(a)lists.fedorahosted.org
>>>>>>> _______________________________________________
>>>>>>> FreeIPA-users mailing list --
freeipa-users(a)lists.fedorahosted.org
>>>>>>> To unsubscribe send an email to
freeipa-users-leave(a)lists.fedorahos
>>>>>> Is this what you're after?
>>>>>>
>>>>>> conn=121474 op=0 SRCH base="" scope=0
filter="(objectClass=*)" attrs="* altServer naming Contexts
supportedControl supportedExtension supportedFeatures supportedLDAPVersion
supportedSASLMechanisms domaincontrollerfunctionality defaultnamingcontext lastusn
highestcommittedusn aci"
>>>>>> conn=121474 op=0 RESULT err=0 tag=101 nentries=1 etime=0
>>>>>> conn=121474 op=1 BIND dn="" method=sasl version=3
mech=GSSAPI
>>>>>> conn=121474 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL
bind in progress
>>>>>> conn=121474 op=2 BIND dn="" method=sasl version=3
mech=GSSAPI
>>>>>> conn=121474 op=2 RESULT err=14 tag=97 nentries=0 etime=0, SASL
bind in progress
>>>>>> conn=121474 op=3 BIND dn="" method=sasl version=3
mech=GSSAPI
>>>>>> conn=121474 op=3 RESULT err=0 tag=101 nentries=0 etime=0
dn="fqdn=loader4.my.net,cn=computers,cn=accoutns,dc=my,dc=net"
>>>>>> :
>>>>>> conn=121474 op=3033 SRCH
base="cn=accounts,dc=my,dc=net" scope=2
filter="(&(objectClass=ipaHost)(fqdn=sfile6.my.net))"
attrs="objectClass cn fqdn serverHostName memberOf ipaSshPubKey ipaUniqueID"
>>>>>> conn=121474 op=3033 RESULT err=0 tag=101 nentries=1 etime=0
notes=P pr_idx=0 pr_cookie=-1
>>>>>> _______________________________________________
>>>>>> FreeIPA-users mailing list --
freeipa-users(a)lists.fedorahosted.org
>>>>>> To unsubscribe send an email to
freeipa-users-leave(a)lists.fedorahosted.org
>>>>> Hi,
>>>>>
>>>>> the command ipa dnsrecord-find should create a trace similar to the
following:
>>>>>
>>>>> [date] conn=538 op=0 BIND dn="" method=sasl version=3
mech=GSS-SPNEGO
>>>>> [date] conn=538 op=0 RESULT err=0 tag=97 nentries=0
etime=0.0004778829 dn="uid=admin,cn=users,cn=accounts,$BASEDN"
>>>>> ...
>>>>> [date] conn=538 op=7 SRCH
base="idnsname=$ZONE,cn=dns,$BASEDN" scope=2
filter="(&(objectClass=top)(objectClass=idnsrecord))" attrs="pTRRecord
APLRecord SigRecord nAPTRRecord HIPRecord kXRecord aRecord aFSDBRecord hInfoRecord
sRVRecord mDRecord certRecord nSECRecord idnsName tXTRecord DHCIDRecord dSRecord
dNameRecord TLSARecord mXRecord sSHFPRecord cNAMERecord IPSECKEYRecord aAAARecord
rRSIGRecord RPRecord DLVRecord URIRecord mInfoRecord KeyRecord a6Record nXTRecord
LocRecord nSRecord SPFRecord"
>>>>> [date] conn=538 op=7 RESULT err=0 tag=101 nentries=11
etime=0.0005895601
>>>>>
>>>>> In your case, since the search results are truncated, there will
probably be err=4 (size limit exceeded) or err=11 (admin limit exceeded), and nentries=
the limit you are hitting.
>>>>>
>>>>> Please keep in mind that 389-ds access log is buffered, it may get
some time between the event and its logging in the file.
>>>>>
>>>>> Flo
>>>> Okay, found it.
>>>>
>>>> conn=128104 op=0 BIND dn="" method=sasl version=3
mech=GSS-SPNEGO
>>>> conn=128104 op=0 RESULT err=0 tag=97 nentries=0 etime=0
dn="uid-admin,cn=users,cn=accoutns,dc=my,dc=net"
>>>> ...
>>>> conn=128104 op=7 SRCH
base="idnsname=my.net,cn=dns,dc=my,dc=net" scope=2
filter="(&(objectClass=top)(objectClass=idnsrecord))"
attrs="sSHFPRecord HIPRecord SPFRecord kXRecord nXTRecord mXRecord aAAARecord
mDRecord aRecord DLVReocrd TLSARecord * pTRRecord SigRecord idnsname aFSDBRecord APLRecord
URIRecord nAPTRRecord nSRecord LocRecord dNameRecord RPRecrod DHCIDRecord IPSECKEYRecord
rSIGRecord hInfoRecord cNAMERecord certRecord sRVRecord dSRecord tXTRecord nSECRecord
a6Record KeyRecord mInfoRecord aci""
>>>> conn=128104 op=7 RESULT err=11 tag=101 nentries=5000 etime=2
>>>>
>>>> It was going to our other IPA server in the pair and I didn't notice
that I was doing my "ipa dnsrecord-find" from my workstation, not one of the
servers.
>>>>
>>>> So it looks like I'm getting an admin limit exceeded. What's the
best way to proceed?
>>>
>>> Hi,
>>>
>>> just to be sure, can you perform equivalent ldapsearch with GSS-SPNEGO /
GSSAPI and plain bind:
>>> kinit admin
>>> ldapsearch -Y GSS-SPNEGO -b idnsname=my.net,cn=dns,dc=my,dc=net
"(&(objectClass=top)(objectClass=idnsrecord))"
>>>
>>> kinit admin
>>> ldapsearch -Y GSSAPI -b idnsname=my.net,cn=dns,dc=my,dc=net
"(&(objectClass=top)(objectClass=idnsrecord))"
>>>
>>> ldapsearch -D uid=admin,cn=users,cn=accounts,dc=my,dc=net -W -b
idnsname=my.net,cn=dns,dc=my,dc=net
"(&(objectClass=top)(objectClass=idnsrecord))"
>>>
>>> I would like to see if the auth mechanism has any impact on the limits and
number of returned entries.
>>> Flo
>>>
>>>> _______________________________________________
>>>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>>>> To unsubscribe send an email to
freeipa-users-leave(a)lists.fedorahosted.org
>>>
>> GSS-SPNEGO returns 2000.
>> GSSAPI returns 2000.
>> The third command returns 2000.
> Hi Bret,
>
> this 2000 limit is your cn=config limit (nsslapd-sizelimit), right? If uid=admin does
not define any nssizelimit then it's an expected behavior. But I don't understand
how you sometimes reach the 5000 limit with uid=admin.
>
> I must admit that this issue teased me, and I reproduced a strange behavior. The
limit is not always the same, it looks like sometimes the anonymous lookthrough limit is
picked even though the search is authenticated. A bug on 389-ds has already been reported
but was closed as not reproducible, and it is now re-opened with new information:
>
https://pagure.io/389-ds-base/issue/49296
>
> As a workaround, if you want to be able to get all your DNS records, you will need to
modify the cn=config limit to a value higher than 5000, and also increase the anonymous
lookthrough limit:
>
> ldapmodify -D "cn=directory manager" -W
> dn: cn=config
> changetype: modify
> replace: nsslapd-sizelimit
> nsslapd-sizelimit: <your value here>
>
> dn: cn=anonymous-limits,cn=etc,$BASEDN
> changetype: modify
> replace: nsLookThroughLimit
> nsLookThroughLimit: <your value here>
>
> HTH,
> Flo
>
>> This returns 5732:
>> ldapsearch -D 'cn=directory manager' -W -b
idnsname=my.net,cn=dns,dc=my,dc=net
"(&(objectClass=top)(objectClass=idnsrecord))"
>> _______________________________________________
>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org