Hi All,

I've created an additional new freeipa replica.
The main difficulty was that I rebuilt an existing system and there were remnants of the previous build in the exist ipa replica and this was reported as insufficient acccess rights even through the keys could be manually created using the same commands. After initially assuming that it was a file permissions error and blowing out the permissions using acls I eventually found the link below.
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org/thread/R3ZVGECW2MC4T6F7J3RO2PPHMMKPUJF6/

Manually deleting the entity after a failed install appears to rectify this issue.

I will now promote the original replica to be the master CA server. If anyone is aware of any deficiencies in the process documented https://www.freeipa.org/page/Howto/Promote_CA_to_Renewal_and_CRL_Master it would be appreciated.

Cheers

-----Original Message-----
From: Ian Willis <fedora@checksum.net.au>
To: FreeIPA users list <freeipa-users@lists.fedorahosted.org>
Subject: Re: [Freeipa-users] Re: FreeIPA centos8 update Failed to authenticate to CA REST API
Date: Sat, 16 Jan 2021 14:41:42 +1100

Hi All,

Given the fact that there haven't been any responses to this issue it would appear that the options are limited to the following approach.

Given the current state and the fact that the CA master is the one with the issues. Would the best approach be to
1 Build a new replica with the current patchset
2 Promote the existing replica to be the CA master
3 Rebuild the original problematic server.

Should steps 1 or 2 above be performed in a particular sequence or doesn't it matter.

Based upon the current documentation

  1. Clean deployment from the lost server by removing all replication agreements with it.
  2. Choose another FreeIPA Server with CA installed to become the first master
  3. Nominate this master to be the one in charge or renewing certs and publishing CRLS. This is a manual procedure at the moment (I believe this is documented here https://www.freeipa.org/page/Howto/Promote_CA_to_Renewal_and_CRL_Master
  4. Follow standard installation procedure to deploy a new master on a hardware/VM of your choice



Kind Regards

-----Original Message-----
From: Ian Willis via FreeIPA-users <freeipa-users@lists.fedorahosted.org>
Reply-To: FreeIPA users list <freeipa-users@lists.fedorahosted.org>
To: freeipa-users@lists.fedorahosted.org
Cc: Ian Willis <fedora@checksum.net.au>
Subject: [Freeipa-users] Re: FreeIPA centos8 update Failed to authenticate to CA REST API
Date: Thu, 14 Jan 2021 21:21:36 +1100

Hi All,

Any next steps in fixing the following issue.

The upgrade has failed as the tomcat CA server appears to be unable to connect to the ldap server as the connection is refused. Is there any way to collect more information from from ldap server to ascertain why the connection has failed.

Is it possible to run the upgrade process manually rather than the current automated process.

2021-01-14 09:21:28 [main] FINEST: Getting pidDir=/var/run/pki/tomcat
2021-01-14 09:21:28 [main] FINEST: Getting pidDir=/var/run/pki/tomcat
2021-01-14 09:21:28 [main] SEVERE: Unable to create socket: java.net.ConnectException: Connection refused (Connection refused)
java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)

Going through the information in https://floblanc.wordpress.com/2017/09/11/troubleshooting-freeipa-pki-tomcatd-fails-to-start/

The certificates are and configuration are correct and valid however the failure still occurs. Are there any suggestions which might assist in isolating the issue.


Kind Regards

Ian


-----Original Message-----
From: Ian Willis via FreeIPA-users <freeipa-users@lists.fedorahosted.org>
Reply-To: FreeIPA users list <freeipa-users@lists.fedorahosted.org>
To: freeipa-users@lists.fedorahosted.org
Cc: Ian Willis <fedora@checksum.net.au>
Subject: [Freeipa-users] FreeIPA centos8 update Failed to authenticate to CA REST API
Date: Tue, 12 Jan 2021 22:14:11 +1100

Hi All,

I've been using freeipa configured as a HA pair on Centos for about 12 months and I've been really impressed, however this morning it has started pumping mud. Any suggestions appreciated.

I did a dnf update of the server which appears to have broken the FreeIPA server and I see the following errors from the ipa start

ipactl start 
IPA version error: data needs to be upgraded (expected version '4.8.7-13.module_el8.3.0+606+1e8766d7', current version '4.8.7-12.module_el8.3.0+511+8a502f20')
Automatically running upgrade, for details see /var/log/ipaupgrade.log
...
[Disabling cert publishing]
[Ensuring CA is using LDAPProfileSubsystem]
[Migrating certificate profiles to LDAP]
IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
Unexpected error - see /var/log/ipaupgrade.log for details:
RemoteRetrieveError: Failed to authenticate to CA REST API
The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information


Some information
The broken system.
CentOS Linux release 8.3.2011
ipa-server-4.8.7-13 (the updated server)

The still operational system
CentOS Linux release 8.3.2011
ipa-server-4.8.7-12

The certificate information based upon the following commands appear to be good.

getcert list -f /var/lib/ipa/ra-agent.pem | grep expires
expires: 2021-12-17 14:43:54 AEDT

ldapsearch -D "cn=directory manager" -W -b o=ipaca "(uid=ipara)"

openssl x509 -text -in /var/lib/ipa/ra-agent.pem

From the /var/log/ipaupgrade.log

2021-01-12T09:51:07Z DEBUG request GET https://groats.ipa.bogus.com.au:8443/ca/rest/account/login
2021-01-12T09:51:07Z DEBUG request body ''
2021-01-12T09:51:07Z DEBUG response status 500
2021-01-12T09:51:07Z DEBUG response headers Content-Type: text/html;charset=utf-8

From the ca debug logs /var/log/pki/pki-tomcat/ca/debug.2021-01-12.log

I'm not sure if the following are relevant

2021-01-12 20:50:49 [main] FINEST: Getting log.instance.SignedAudit.events=ACCESS_SESSION_ESTABLISH,ACCESS_SESSION_TERMINATED,AUTH,AUTHORITY_CONFIG,AUTHZ,CERT_PROFILE_APPROVAL,CERT_REQUEST_PROCESSED,CERT_SIGNING_INFO,CERT_STATUS_CHANGE_REQUEST_PROCESSED,CLIENT_ACCESS_SESSION_ESTABLISH,CLIENT_ACCESS_SESSION_TERMINATED,CMC_REQUEST_RECEIVED,CMC_RESPONSE_SENT,CMC_SIGNED_REQUEST_SIG_VERIFY,CMC_USER_SIGNED_REQUEST_SIG_VERIFY,CONFIG_ACL,CONFIG_AUTH,CONFIG_CERT_PROFILE,CONFIG_CRL_PROFILE,CONFIG_DRM,CONFIG_ENCRYPTION,CONFIG_OCSP_PROFILE,CONFIG_ROLE,CONFIG_SERIAL_NUMBER,CONFIG_SIGNED_AUDIT,CONFIG_TRUSTED_PUBLIC_KEY,CRL_SIGNING_INFO,DELTA_CRL_GENERATION,FULL_CRL_GENERATION,LOG_PATH_CHANGE,OCSP_GENERATION,OCSP_SIGNING_INFO,PROFILE_CERT_REQUEST,PROOF_OF_POSSESSION,RANDOM_GENERATION,ROLE_ASSUME,SECURITY_DOMAIN_UPDATE,SELFTESTS_EXECUTION
2021-01-12 20:50:49 [main] FINEST: Getting log.instance.SignedAudit.events=ACCESS_SESSION_ESTABLISH,ACCESS_SESSION_TERMINATED,AUTH,AUTHORITY_CONFIG,AUTHZ,CERT_PROFILE_APPROVAL,CERT_REQUEST_PROCESSED,CERT_SIGNING_INFO,CERT_STATUS_CHANGE_REQUEST_PROCESSED,CLIENT_ACCESS_SESSION_ESTABLISH,CLIENT_ACCESS_SESSION_TERMINATED,CMC_REQUEST_RECEIVED,CMC_RESPONSE_SENT,CMC_SIGNED_REQUEST_SIG_VERIFY,CMC_USER_SIGNED_REQUEST_SIG_VERIFY,CONFIG_ACL,CONFIG_AUTH,CONFIG_CERT_PROFILE,CONFIG_CRL_PROFILE,CONFIG_DRM,CONFIG_ENCRYPTION,CONFIG_OCSP_PROFILE,CONFIG_ROLE,CONFIG_SERIAL_NUMBER,CONFIG_SIGNED_AUDIT,CONFIG_TRUSTED_PUBLIC_KEY,CRL_SIGNING_INFO,DELTA_CRL_GENERATION,FULL_CRL_GENERATION,LOG_PATH_CHANGE,OCSP_GENERATION,OCSP_SIGNING_INFO,PROFILE_CERT_REQUEST,PROOF_OF_POSSESSION,RANDOM_GENERATION,ROLE_ASSUME,SECURITY_DOMAIN_UPDATE,SELFTESTS_EXECUTION
2021-01-12 20:50:49 [main] FINE: Event filters:
2021-01-12 20:50:49 [main] FINE: - CMC_SIGNED_REQUEST_SIG_VERIFY: (Outcome=Failure)
2021-01-12 20:50:49 [main] FINE: - CMC_USER_SIGNED_REQUEST_SIG_VERIFY: (Outcome=Failure)
2021-01-12 20:50:49 [main] FINE: - DELTA_CRL_GENERATION: (Outcome=Failure)
2021-01-12 20:50:49 [main] FINE: - FULL_CRL_GENERATION: (Outcome=Failure)
2021-01-12 20:50:49 [main] FINE: - OCSP_GENERATION: (Outcome=Failure)
2021-01-12 20:50:49 [main] FINE: - RANDOM_GENERATION: (Outcome=Failure)
2021-01-12 20:50:49 [main] FINE: - SELFTESTS_EXECUTION: (Outcome=Failure)
2021-01-12 20:50:49 [main] FINEST: Property log.instance.SignedAudit.trace not found

However where it dies is
2021-01-12 20:50:50 [main] FINEST: Property internaldb.doCloning not found
2021-01-12 20:50:50 [main] FINEST: Getting internaldb.doCloning=true
2021-01-12 20:50:50 [main] FINE: LdapBoundConnFactory: doCloning: true
2021-01-12 20:50:50 [main] FINE: LdapBoundConnFactory: mininum: 3
2021-01-12 20:50:50 [main] FINE: LdapBoundConnFactory: maximum: 15
2021-01-12 20:50:50 [main] FINE: LdapBoundConnFactory: host: oats.ipa.amnesium.com.au
2021-01-12 20:50:50 [main] FINE: LdapBoundConnFactory: port: 636
2021-01-12 20:50:50 [main] FINE: LdapBoundConnFactory: secure: true
2021-01-12 20:50:50 [main] FINE: LdapBoundConnFactory: authentication: 2
2021-01-12 20:50:50 [main] FINE: LdapBoundConnFactory: makeConnection(true)
2021-01-12 20:50:50 [main] FINEST: Getting internaldb.ldapauth.clientCertNickname=subsystemCert cert-pki-ca
2021-01-12 20:50:50 [main] FINEST: Property tcp.keepAlive not found
2021-01-12 20:50:50 [main] FINEST: Getting tcp.keepAlive=true
2021-01-12 20:50:50 [main] FINE: TCP Keep-Alive: true
2021-01-12 20:50:50 [main] FINE: LdapBoundConnection: Connecting to oats.ipa.amnesium.com.au:636 with client cert auth
2021-01-12 20:50:50 [main] FINE: ldapconn/PKISocketFactory.makeSSLSocket: begins
2021-01-12 20:50:50 [main] FINE: SignedAuditLogger: event CLIENT_ACCESS_SESSION_ESTABLISH
2021-01-12 20:50:50 [main] FINEST: Getting pidDir=/var/run/pki/tomcat
2021-01-12 20:50:50 [main] FINEST: Getting pidDir=/var/run/pki/tomcat
2021-01-12 20:50:50 [main] SEVERE: Unable to create socket: java.net.ConnectException: Connection refused (Connection refused)
java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
.....




_______________________________________________
FreeIPA-users mailing list -- 
freeipa-users@lists.fedorahosted.org

To unsubscribe send an email to 
freeipa-users-leave@lists.fedorahosted.org

Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: 
https://fedoraproject.org/wiki/Mailing_list_guidelines

List Archives: 
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org

_______________________________________________
FreeIPA-users mailing list -- 
freeipa-users@lists.fedorahosted.org

To unsubscribe send an email to 
freeipa-users-leave@lists.fedorahosted.org

Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: 
https://fedoraproject.org/wiki/Mailing_list_guidelines

List Archives: 
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org