On 02/15/2018 06:42 PM, Bret Wortman via FreeIPA-users wrote:
> On 02/15/2018 12:27 PM, Florence Blanc-Renaud wrote:
>> On 02/15/2018 05:01 PM, Bret Wortman via FreeIPA-users wrote:
>>> On 02/15/2018 09:29 AM, Florence Blanc-Renaud wrote:
>>>> On 02/15/2018 02:40 PM, Bret Wortman via FreeIPA-users wrote:
>>>>> On 02/15/2018 07:09 AM, Florence Blanc-Renaud via FreeIPA-users
>>>>> wrote:
>>>>>> On 02/15/2018 11:47 AM, Bret Wortman via FreeIPA-users wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 02/15/2018 04:50 AM, Florence Blanc-Renaud wrote:
>>>>>>>> On 02/15/2018 10:08 AM, Florence Blanc-Renaud via
>>>>>>>> FreeIPA-users wrote:
>>>>>>>>> On 02/14/2018 05:58 PM, Bret Wortman wrote:
>>>>>>>>>> On 02/14/2018 10:22 AM, Florence Blanc-Renaud
wrote:
>>>>>>>>>>> On 02/14/2018 12:52 PM, Bret Wortman via
FreeIPA-users wrote:
>>>>>>>>>>>> I did figure out that I can use
>>>>>>>>>>>>
>>>>>>>>>>>> # ldapsearch -D 'directory
manager' -W -E pr=20000 -b
>>>>>>>>>>>>
idnsname=damascusgrp.com,cn=dns,dc=damascusgrp,dc=com
>>>>>>>>>>>>
>>>>>>>>>>>> to list out all the entries, but the
format isn't what I'm
>>>>>>>>>>>> expecting.
>>>>>>>>>>>>
>>>>>>>>>>>> What I'm actually trying to do is
move our whole
>>>>>>>>>>>> infrastructure from one set of old &
busted servers to
>>>>>>>>>>>> some shiny new VMs. We'd like to
extract the data and
>>>>>>>>>>>> start fresh, as our replication
agreements just don't seem
>>>>>>>>>>>> to be working as expected. Changes to one
don't always
>>>>>>>>>>>> make it to the other and vice versa.
While I'd love to dig
>>>>>>>>>>>> in and solve that, it's easier right
now to try to extract
>>>>>>>>>>>> the data and reload it into a new server,
build new
>>>>>>>>>>>> replicas, then unbind & re-bind every
client to the new
>>>>>>>>>>>> server using ansible since we also lost
our internal CA in
>>>>>>>>>>>> the process.
>>>>>>>>>>>>
>>>>>>>>>>>> So while our current configuration is a
mess, we can't
>>>>>>>>>>>> afford to lose all the host/user/dns/hbac
data in our
>>>>>>>>>>>> servers. Thus, I've been capturing
the output to text
>>>>>>>>>>>> using various ipa *-find commands and
have parsers to turn
>>>>>>>>>>>> those back into new entries on the fresh
hosts. DNS is the
>>>>>>>>>>>> only thing that's holding me up.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Bret
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 02/14/2018 06:33 AM, Bret Wortman
wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, this doesn't solve the fact
that the Web UI always
>>>>>>>>>>>>> produces an error dialog whenever
accessing our primary
>>>>>>>>>>>>> zone.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 02/13/2018 02:19 PM, Natxo Asenjo
via FreeIPA-users
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Feb 13, 2018 at 8:13 PM,
Natxo Asenjo
>>>>>>>>>>>>>> <natxo.asenjo(a)gmail.com
<mailto:natxo.asenjo@gmail.com>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> the canonical way to do this
is using ldap paging, with
>>>>>>>>>>>>>> ldapsearch you could try
using the -E pr=xxxx
>>>>>>>>>>>>>> parameter, where
>>>>>>>>>>>>>> xxxx could be 1000 for
instance. That way you know
>>>>>>>>>>>>>> you are always
>>>>>>>>>>>>>> under the limit imposed by
the server.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> if you use -E pr=1000/noprompt,
it will not prompt to
>>>>>>>>>>>>>> continue, nicer for scripts
obviously.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Groeten,
>>>>>>>>>>>>>> natxo
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>>>> FreeIPA-users mailing list
>>>>>>>>>>>>>>
--freeipa-users(a)lists.fedorahosted.org
>>>>>>>>>>>>>> To unsubscribe send an email
>>>>>>>>>>>>>>
tofreeipa-users-leave(a)lists.fedorahosted.org
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>> FreeIPA-users mailing list --
>>>>>>>>>>>> freeipa-users(a)lists.fedorahosted.org
>>>>>>>>>>>> To unsubscribe send an email to
>>>>>>>>>>>>
freeipa-users-leave(a)lists.fedorahosted.org
>>>>>>>>>>>>
>>>>>>>>>>> Hi Bret,
>>>>>>>>>>>
>>>>>>>>>>> the search limits can be set at multiple
levels:
>>>>>>>>>>> - for the whole 389-ds server
>>>>>>>>>>> nsslapd-sizelimit (in cn=config)
>>>>>>>>>>> nsslapd-lookthroughlimit (in
cn=config,cn=ldbm
>>>>>>>>>>> database,cn=plugins,cn=config)
>>>>>>>>>>>
>>>>>>>>>>> - for operations performed through ipa *
commands (or the
>>>>>>>>>>> webGUI):
>>>>>>>>>>> ipaSearchRecordsLimit (in
cn=ipaConfig,cn=etc,$BASEDN)
>>>>>>>>>>>
>>>>>>>>>>> - for each user:
>>>>>>>>>>> nssizelimit and nsLookThroughLimit attributes
(in
>>>>>>>>>>> uid=$USER,cn=users,cn=accounts,$BASEDN)
>>>>>>>>>>>
>>>>>>>>>>> You are probably hitting one of these limits
in your ipa
>>>>>>>>>>> *-find command.
>>>>>>>>>>>
>>>>>>>>>>> HTH,
>>>>>>>>>>> Flo
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> So I found almost all of these:
>>>>>>>>>>
>>>>>>>>>> # ldapsearch -D 'cn=directory manager' -W
-b 'cn=config'
>>>>>>>>>> cn=config | grep nsslapd-sizelimit
>>>>>>>>>> nsslapd-sizelimit: 2000
>>>>>>>>>>
>>>>>>>>>> # ldapsearch -D 'cn=directory manager' -W
-b
>>>>>>>>>> 'cn=config,cn=ldbm
database,cn=plugins,cn=config' | grep
>>>>>>>>>> lookthroughlimit
>>>>>>>>>> nsslapd-lookthroughlimit: 100000
>>>>>>>>>>
>>>>>>>>>> # ldapsearch -D 'cn=directory manager' -W
-b
>>>>>>>>>>
'cn=ipaConfig,cn=etc,dc=damascusgrp,dc=com' | grep
>>>>>>>>>> ipaSearchRecordsLimit
>>>>>>>>>> ipaSearchRecordsLimit: 99999
>>>>>>>>>>
>>>>>>>>>> # ldapsearch -D 'cn=directory manager' -W
-b
>>>>>>>>>>
'uid=admin,cn=users,cn=accounts,dc=damascusgrp,dc=com' |
>>>>>>>>>> grep -i limit
>>>>>>>>>> (returns data but nothing matches)
>>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> nsSizeLimit and nsLookThroughLimit are operational
>>>>>>>>> attributes, meaning that a standard ldapsearch will
not
>>>>>>>>> return them. You need either to specifically request
them by
>>>>>>>>> providing them in the attributes list:
>>>>>>>>>
>>>>>>>>> $ ldapsearch -D 'cn=directory manager' -W -b
$BASE
>>>>>>>>> nssizelimit nslookthroughlimit
>>>>>>>>>
>>>>>>>>> or you can also specify + instead of the attributes
in order
>>>>>>>>> to get all operational attributes:
>>>>>>>>> $ ldapsearch -D 'cn=directory manager' -W -b
$BASE +
>>>>>>>>>
>>>>>>>>> HTH,
>>>>>>>>> Flo
>>>>>>>>>
>>>>>>>>>> The first doesn't seem to be something I can
change. It's
>>>>>>>>>> stuck at 2000, but since my issue occurs at 5000,
I'm not
>>>>>>>>>> worried about it. I believe that I'm missing
something in
>>>>>>>>>> the fourth search that might point me toward the
attributes
>>>>>>>>>> you mentioned but I'm not sure where.
>>>>>>>>>>
>>>>>>>>
>>>>>>>> The 5000 limit rings a bell to me. It is the anonymous
size
>>>>>>>> limit. Can you check:
>>>>>>>> $ ldapsearch -D 'cn=directory manager' -W -b
cn=config -s base
>>>>>>>> nsslapd-anonlimitsdn
>>>>>>>>
>>>>>>>> it will provide you with a DN of the entry defining the
>>>>>>>> anonymous limits (usually
cn=anonymous-limits,cn=etc,$BASEDN),
>>>>>>>> then:
>>>>>>>>
>>>>>>>> $ ldapsearch -D 'cn=directory manager' -W -b
>>>>>>>> cn=anonymous-limits,cn=etc,$BASEDN nsSizeLimit
nsLookThroughLimit
>>>>>>>>
>>>>>>>> Now we should check the access log
>>>>>>>> (/var/log/dirsrv/slapd-xxx/access) corresponding to your
>>>>>>>> command to retrieve the DNS entries, and ensure which
user
>>>>>>>> identity is actually performing the search.
>>>>>>>>
>>>>>>>> Flo
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> FreeIPA-users mailing list --
>>>>>>>>> freeipa-users(a)lists.fedorahosted.org
>>>>>>>>> To unsubscribe send an email to
>>>>>>>>> freeipa-users-leave(a)lists.fedorahosted.org
>>>>>>>>
>>>>>>>
>>>>>>> Both those limits are 5000.
>>>>>>>
>>>>>>> What should I be looking for in the access log? It gets a lot
>>>>>>> of traffic and narrowing down exactly which entry represents
>>>>>>> the original search is challenging. I'd rather not post
the
>>>>>>> whole thing because it's pretty large & noisy.
>>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> you can perform the ipa dnsrecord-find --all command, while
>>>>>> tail'ing the access log. I will be interested in the lines
>>>>>> containing the SEARCH with a err=4 (size limit exceeded). For
>>>>>> this SEARCH, please note the connection number (conn=xx) and
>>>>>> find the BIND for the same connection number (it will be located
>>>>>> before the SEARCH). For each line (BIND and SEARCH) both the
>>>>>> operation and the RESULT are interesting, for instance when
>>>>>> doing ipa user-find:
>>>>>>
>>>>>> [date] conn=514 op=0 BIND dn="" method=sasl version=3
>>>>>> mech=GSS-SPNEGO
>>>>>> [date] conn=514 op=0 RESULT err=0 tag=97 nentries=0
>>>>>> etime=0.0004754873
dn="uid=admin,cn=users,cn=accounts,$BASEDN"
>>>>>> ...
>>>>>> [date] conn=514 op=2 SRCH
base="cn=users,cn=accounts,$BASEDN"
>>>>>> scope=1 filter="(objectClass=posixaccount)"
>>>>>> attrs="krbCanonicalName loginShell sshpubkeyfp gidNumber
>>>>>> homeDirectory uidNumber uid givenName title sn krbPrincipalName
>>>>>> mail nsAccountLock telephoneNumber ipaSshPubKey"
>>>>>> [date] conn=514 op=2 RESULT err=0 tag=101 nentries=6
>>>>>> etime=0.0006313857
>>>>>>
>>>>>> Flo
>>>>>>
>>>>>>> Bret
>>>>>>> _______________________________________________
>>>>>>> FreeIPA-users mailing list --
freeipa-users(a)lists.fedorahosted.org
>>>>>>> To unsubscribe send an email to
>>>>>>> freeipa-users-leave(a)lists.fedorahosted.org
>>>>>> _______________________________________________
>>>>>> FreeIPA-users mailing list --
freeipa-users(a)lists.fedorahosted.org
>>>>>> To unsubscribe send an email to
freeipa-users-leave(a)lists.fedorahos
>>>>> Is this what you're after?
>>>>>
>>>>> conn=121474 op=0 SRCH base="" scope=0
filter="(objectClass=*)"
>>>>> attrs="* altServer naming Contexts supportedControl
>>>>> supportedExtension supportedFeatures supportedLDAPVersion
>>>>> supportedSASLMechanisms domaincontrollerfunctionality
>>>>> defaultnamingcontext lastusn highestcommittedusn aci"
>>>>> conn=121474 op=0 RESULT err=0 tag=101 nentries=1 etime=0
>>>>> conn=121474 op=1 BIND dn="" method=sasl version=3
mech=GSSAPI
>>>>> conn=121474 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL
>>>>> bind in progress
>>>>> conn=121474 op=2 BIND dn="" method=sasl version=3
mech=GSSAPI
>>>>> conn=121474 op=2 RESULT err=14 tag=97 nentries=0 etime=0, SASL
>>>>> bind in progress
>>>>> conn=121474 op=3 BIND dn="" method=sasl version=3
mech=GSSAPI
>>>>> conn=121474 op=3 RESULT err=0 tag=101 nentries=0 etime=0
>>>>>
dn="fqdn=loader4.my.net,cn=computers,cn=accoutns,dc=my,dc=net"
>>>>> :
>>>>> conn=121474 op=3033 SRCH base="cn=accounts,dc=my,dc=net"
scope=2
>>>>> filter="(&(objectClass=ipaHost)(fqdn=sfile6.my.net))"
>>>>> attrs="objectClass cn fqdn serverHostName memberOf ipaSshPubKey
>>>>> ipaUniqueID"
>>>>> conn=121474 op=3033 RESULT err=0 tag=101 nentries=1 etime=0
>>>>> notes=P pr_idx=0 pr_cookie=-1
>>>>> _______________________________________________
>>>>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>>>>> To unsubscribe send an email to
>>>>> freeipa-users-leave(a)lists.fedorahosted.org
>>>> Hi,
>>>>
>>>> the command ipa dnsrecord-find should create a trace similar to
>>>> the following:
>>>>
>>>> [date] conn=538 op=0 BIND dn="" method=sasl version=3
mech=GSS-SPNEGO
>>>> [date] conn=538 op=0 RESULT err=0 tag=97 nentries=0
>>>> etime=0.0004778829 dn="uid=admin,cn=users,cn=accounts,$BASEDN"
>>>> ...
>>>> [date] conn=538 op=7 SRCH base="idnsname=$ZONE,cn=dns,$BASEDN"
>>>> scope=2
filter="(&(objectClass=top)(objectClass=idnsrecord))"
>>>> attrs="pTRRecord APLRecord SigRecord nAPTRRecord HIPRecord
>>>> kXRecord aRecord aFSDBRecord hInfoRecord sRVRecord mDRecord
>>>> certRecord nSECRecord idnsName tXTRecord DHCIDRecord dSRecord
>>>> dNameRecord TLSARecord mXRecord sSHFPRecord cNAMERecord
>>>> IPSECKEYRecord aAAARecord rRSIGRecord RPRecord DLVRecord URIRecord
>>>> mInfoRecord KeyRecord a6Record nXTRecord LocRecord nSRecord
>>>> SPFRecord"
>>>> [date] conn=538 op=7 RESULT err=0 tag=101 nentries=11
>>>> etime=0.0005895601
>>>>
>>>> In your case, since the search results are truncated, there will
>>>> probably be err=4 (size limit exceeded) or err=11 (admin limit
>>>> exceeded), and nentries= the limit you are hitting.
>>>>
>>>> Please keep in mind that 389-ds access log is buffered, it may get
>>>> some time between the event and its logging in the file.
>>>>
>>>> Flo
>>> Okay, found it.
>>>
>>> conn=128104 op=0 BIND dn="" method=sasl version=3 mech=GSS-SPNEGO
>>> conn=128104 op=0 RESULT err=0 tag=97 nentries=0 etime=0
>>> dn="uid-admin,cn=users,cn=accoutns,dc=my,dc=net"
>>> ...
>>> conn=128104 op=7 SRCH base="idnsname=my.net,cn=dns,dc=my,dc=net"
>>> scope=2 filter="(&(objectClass=top)(objectClass=idnsrecord))"
>>> attrs="sSHFPRecord HIPRecord SPFRecord kXRecord nXTRecord mXRecord
>>> aAAARecord mDRecord aRecord DLVReocrd TLSARecord * pTRRecord
>>> SigRecord idnsname aFSDBRecord APLRecord URIRecord nAPTRRecord
>>> nSRecord LocRecord dNameRecord RPRecrod DHCIDRecord IPSECKEYRecord
>>> rSIGRecord hInfoRecord cNAMERecord certRecord sRVRecord dSRecord
>>> tXTRecord nSECRecord a6Record KeyRecord mInfoRecord aci""
>>> conn=128104 op=7 RESULT err=11 tag=101 nentries=5000 etime=2
>>>
>>> It was going to our other IPA server in the pair and I didn't
>>> notice that I was doing my "ipa dnsrecord-find" from my
>>> workstation, not one of the servers.
>>>
>>> So it looks like I'm getting an admin limit exceeded. What's the
>>> best way to proceed?
>>
>> Hi,
>>
>> just to be sure, can you perform equivalent ldapsearch with
>> GSS-SPNEGO / GSSAPI and plain bind:
>> kinit admin
>> ldapsearch -Y GSS-SPNEGO -b idnsname=my.net,cn=dns,dc=my,dc=net
>> "(&(objectClass=top)(objectClass=idnsrecord))"
>>
>> kinit admin
>> ldapsearch -Y GSSAPI -b idnsname=my.net,cn=dns,dc=my,dc=net
>> "(&(objectClass=top)(objectClass=idnsrecord))"
>>
>> ldapsearch -D uid=admin,cn=users,cn=accounts,dc=my,dc=net -W -b
>> idnsname=my.net,cn=dns,dc=my,dc=net
>> "(&(objectClass=top)(objectClass=idnsrecord))"
>>
>> I would like to see if the auth mechanism has any impact on the
>> limits and number of returned entries.
>> Flo
>>
>>> _______________________________________________
>>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>>> To unsubscribe send an email to
>>> freeipa-users-leave(a)lists.fedorahosted.org
>>
>
> GSS-SPNEGO returns 2000.
>
> GSSAPI returns 2000.
>
> The third command returns 2000.
>
Hi Bret,
this 2000 limit is your cn=config limit (nsslapd-sizelimit), right? If
uid=admin does not define any nssizelimit then it's an expected
behavior. But I don't understand how you sometimes reach the 5000
limit with uid=admin.
I must admit that this issue teased me, and I reproduced a strange
behavior. The limit is not always the same, it looks like sometimes
the anonymous lookthrough limit is picked even though the search is
authenticated. A bug on 389-ds has already been reported but was
closed as not reproducible, and it is now re-opened with new information:
https://pagure.io/389-ds-base/issue/49296
As a workaround, if you want to be able to get all your DNS records,
you will need to modify the cn=config limit to a value higher than
5000, and also increase the anonymous lookthrough limit:
ldapmodify -D "cn=directory manager" -W
dn: cn=config
changetype: modify
replace: nsslapd-sizelimit
nsslapd-sizelimit: <your value here>
dn: cn=anonymous-limits,cn=etc,$BASEDN
changetype: modify
replace: nsLookThroughLimit
nsLookThroughLimit: <your value here>
HTH,
Flo
> This returns 5732:
>
> ldapsearch -D 'cn=directory manager' -W -b
> idnsname=my.net,cn=dns,dc=my,dc=net
> "(&(objectClass=top)(objectClass=idnsrecord))"
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to
> freeipa-users-leave(a)lists.fedorahosted.org
Both these commands fail to return. I've executed each and they simply
don't return; I have to terminate them with Ctrl-C.
Do they need to be executed together, in one invocation?