Hi,
On Thu, May 25, 2023 at 9:53 AM Nicholas Cross <nicholas.cross(a)dice.fm>
wrote:
Hi, thanks for the reply.
This was the process.
1. prep
2. test connectivity - works ok
3. install replica (successfully) - no DNS no CA at this point
4. test connectivity - fails.
If i replace step 4 for the following:
4. install DNS (successfully)
5. install CA (fails - with the "invalid 'cn': must be" error)
If we can ignore the step 4 where the comms check fails and concentrate on
step 5 where the CA fails. I am happy to do so.
But in step 5. it is the comms check that the `ipa-ca-install` command
runs that also fails as far as i can see (no expert here myself, though my
company thinks so LOL)
Successful DNS install logs
http://sprunge.us/7WCtpW
Failed CA install logs
http://sprunge.us/hTVLzL
I believe this error needs to be investigated:
failed with Insufficient
access: SASL(-1): generic failure: GSSAPI Error: No credentials were
supplied, or the credentials were unavailable or inaccessible
(Credential cache is empty)
We can see that the process is able to connect to the master and get a
cookie but later on GSSAPI auth fails.
Are there any corresponding logs in the master in /var/log/httpd/error_log?
You can enable debug logs on the master with the following procedure:
- create /etc/ipa/server.conf:
[global]
debug=True
then restart apache with systemctl restart httpd.
You can also run ipa-healthcheck on the master/the replica to find any
config issue.
flo
[root@ipa011 ~]# ipa-ca-install
Directory Manager (existing master) password:
Run connection check to master
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.
Connection check failed!
See /var/log/ipareplica-conncheck.log for more information.
If the check results are not valid it can be skipped with --skip-conncheck
parameter.
thanks,
Nick
On Thu, 25 May 2023 at 08:18, Florence Blanc-Renaud <flo(a)redhat.com>
wrote:
> Hi,
>
> On Wed, May 24, 2023 at 3:29 PM Nicholas Cross <nicholas.cross(a)dice.fm>
> wrote:
>
>> Hi Flo (and other helpful people on this list),
>>
>> After fixing the SID/PAC issue, i am back to looking as to why the
>> ipa-replica-conncheck fails when installing the CA to a (working) replica.
>>
>> I ran your suggested commands and they look fine now.
>>
>> [root@ipa011 ~]# kdestroy
>> kdestroy: No credentials cache found while destroying cache
>>
>> [root@ipa011 ~]# kinit -V -kt /etc/krb5.keytab host/
>> ipa011.ad.companyx.fm(a)AD.companyx.FM
>> Using default cache: 0
>> Using principal: host/ipa011.ad.companyx.fm(a)AD.companyx.FM
>> Using keytab: /etc/krb5.keytab
>> Authenticated to Kerberos v5
>>
>> [root@ipa011 ~]# kvno HTTP/ipa011.ad.companyx.fm(a)AD.companyx.FM
>> HTTP/ipa011.ad.companyx.fm(a)AD.companyx.FM: kvno = 1
>>
>> I can run the following BEFORE i install the replica and it works 100%,
>> it is part of our pre-reqs in our docs to make sure comms work.
>>
>> ipa-replica-conncheck --master=ipa008.ad.companyx.fm
>> --auto-master-check --realm=AD.companyx.FM --principal=nicholas.cross
>>
>> After i install the replica with ipa-replica-install. (no DNS, no CA)
>> then running the above ipa-replica-conncheck command fails and we see the
>> errors as described.
>>
>
> ipa-replica-conncheck is intended to be run *before* replica
> installation, as stated in the man page:
> ipa-replica-conncheck - Check a replica-master network connection
> before installation
>
> The reason is that it opens server sockets listening on the tested ports,
> for the master to try to reach the replica. If the replica is already
> installed, the ports are already in use by the ldap server, the apache web
> server etc...
>
> I understand that you tried to run ipa-replica-conncheck to ensure that
> ipa-dns-install or ipa-ca-install won't hit any issue. Did you actually try
> to install the DNS service and the CA service on the replica? If you see
> errors in those steps we can help troubleshoot, based on the produced logs,
> but ipa-replica-conncheck won't be of any use here.
>
> flo
>
>>
>> Some logs:
>> 1. before ipa-replica-install - conn check works.
>>
http://sprunge.us/4yrInD
>> 2. after successful ipa-replica-install - but conn check fails
>>
http://sprunge.us/gvqEle
>>
>> new replica versions
>> [root@ipa011 ~]# rpm -qa | grep ipa | sort
>> almalinux-logos-ipa-90.5.1-1.1.el9.noarch
>> ipa-client-4.10.1-6.el9.x86_64
>> ipa-client-common-4.10.1-6.el9.noarch
>> ipa-common-4.10.1-6.el9.noarch
>> ipa-healthcheck-0.12-1.el9.noarch
>> ipa-healthcheck-core-0.12-1.el9.noarch
>> ipa-selinux-4.9.8-7.el9_0.noarch
>> ipa-server-4.10.1-6.el9.x86_64
>> ipa-server-common-4.10.1-6.el9.noarch
>> ipa-server-dns-4.10.1-6.el9.noarch
>> libipa_hbac-2.8.2-2.el9.x86_64
>> python3-ipaclient-4.10.1-6.el9.noarch
>> python3-ipalib-4.10.1-6.el9.noarch
>> python3-ipaserver-4.10.1-6.el9.noarch
>> python3-libipa_hbac-2.8.2-2.el9.x86_64
>> sssd-ipa-2.8.2-2.el9.x86_64
>>
>> current master versions
>> [nicholas.cross@ipa008 ~]$ rpm -qa | grep ipa | sort
>> almalinux-logos-ipa-90.5.1-1.1.el9.noarch
>> ipa-client-4.10.0-8.el9_1.x86_64
>> ipa-client-common-4.10.0-8.el9_1.noarch
>> ipa-common-4.10.0-8.el9_1.noarch
>> ipa-healthcheck-0.9-9.el9.noarch
>> ipa-healthcheck-core-0.9-9.el9.noarch
>> ipa-selinux-4.9.8-7.el9_0.noarch
>> ipa-server-4.10.0-8.el9_1.x86_64
>> ipa-server-common-4.10.0-8.el9_1.noarch
>> ipa-server-dns-4.10.0-8.el9_1.noarch
>> libipa_hbac-2.7.3-4.el9_1.3.x86_64
>> python3-ipaclient-4.10.0-8.el9_1.noarch
>> python3-ipalib-4.10.0-8.el9_1.noarch
>> python3-ipaserver-4.10.0-8.el9_1.noarch
>> python3-libipa_hbac-2.7.3-4.el9_1.3.x86_64
>> sssd-ipa-2.7.3-4.el9_1.3.x86_64
>>
>> thanks,
>> Nick
>>
>> On Tue, 23 May 2023 at 10:01, Florence Blanc-Renaud <flo(a)redhat.com>
>> wrote:
>>
>>> Hi,
>>>
>>> the replica-conncheck error means that a call to server_conncheck
>>> reached the wrong server. ipa-replica-conncheck performs multiple checks:
>>> - first from the replica to the existing master (here we seem to be
>>> good)
>>> - then from the existing master to the replica, by doing a call to the
>>> XMLRPC api server_conncheck on the master. If the connection from replica
>>> to master fails, another server is tried (in this case, the replica
>>> launches server_conncheck on itself), but there is a security that ensures
>>> the right server is handling the call.
>>> The logs shows that the connection fails because of SASL auth failure:
>>> 2023-05-22T18:14:03Z INFO Connection to
>>>
https://ipa010.ad.companyx.fm/ipa/json failed with Insufficient
>>> access: SASL(-1): generic failure: GSSAPI Error: No credentials were
>>> supplied, or the credentials were unavailable or inaccessible (Credential
>>> cache is empty)
>>>
>>> Are you able to do kinit -kt /etc/krb5.keytab
>>> host/<replicafqdn>@<REALM> on the replica? And then kvno
>>> HTTP/<serverfqdn>@<REALM> ?
>>> flo
>>>
>>> On Tue, May 23, 2023 at 9:55 AM Nicholas Cross via FreeIPA-users <
>>> freeipa-users(a)lists.fedorahosted.org> wrote:
>>>
>>>> That was the /var/log/ipareplica-conncheck.log log file
>>>>
>>>> it does looks like a DNs issue, but im not sure where.
>>>>
>>>> dns resolves the host fine on the host
>>>>
>>>> [root@ipa011 ~]# host ipa011
>>>> ipa011.ad.companyx.fm has address 10.32.225.7
>>>>
>>>> [root@ipa011 ~]# grep ipa /etc/ipa/default.conf
>>>> host = ipa011.ad.companyx.fm
>>>> xmlrpc_uri =
https://ipa011.ad.companyx.fm/ipa/xml
>>>> ca_host = ipa010.ad.companyx.fm
>>>>
>>>> it's odd as i run the connection check before the start of the
>>>> install, to check ports and routes. it works fine.
>>>> replica install works.
>>>> dns install works.
>>>> just the ca installer comes back with this error.
>>>>
>>>> As an additional test i added the dns record for this host into IPA
>>>> before the install. Normally we don't need to, but just as a test,
but it
>>>> made no difference.
>>>>
>>>>
>>>> We do have new DNS forwarders on the network - these are in front of
>>>> the IPA servers. They are there just take the load from the k8s
clusters
>>>> away from IPA DNS.
>>>> Would the CA install break if the DNS lookups are "proxied" by
the DNS
>>>> forwarders?
>>>> All DNS tests i can think of work via the forwarders. The IPA clients
>>>> (100s) are all fine with them.
>>>>
>>>> I will update the client to ignore the forwarders, but if you can
>>>> think of anything else to try?
>>>>
>>>> thanks, Nick
>>>> _______________________________________________
>>>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>>>> To unsubscribe send an email to
>>>> freeipa-users-leave(a)lists.fedorahosted.org
>>>> Fedora Code of Conduct:
>>>>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>>>> List Guidelines:
>>>>
https://fedoraproject.org/wiki/Mailing_list_guidelines
>>>> List Archives:
>>>>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>>>> Do not reply to spam, report it:
>>>>
https://pagure.io/fedora-infrastructure/new_issue
>>>>
>>>