Well that did it. Thanks!

So for people following this thread, what would have caused this? Were they not set up correctly on installation, or were they somehow removed? If the former, how long could the system run with the incorrect configuration? If the latter, how likely is this to recur?

And yes, I’m glad the upstream error message is being improved :)

On Mar 5, 2020, at 2:51 PM, Rob Crittenden <rcritten@redhat.com> wrote:

You are missing two certmonger CA helpers, dogtag-ipa-ca-renew-agent and
dogtag-ipa-ca-renew-agent-reuse.

Re-add them as root with:

# getcert add-ca -c dogtag-ipa-ca-renew-agent -e
/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit
# getcert add-ca -c dogtag-ipa-ca-renew-agent-reuse -e
/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit --reuse-existing

Confirm with getcert list that every tracked request has a CA value.

That should do the trick, you can now run:

# ipa-server-upgrade

rob

David Carter via FreeIPA-users wrote:
With some redactions:

# getcert list
Number of certificates and requests being tracked: 9.
Request ID '20190815160425':
status: MONITORING
stuck: no
key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
issuer: CN=Certificate Authority,O=XXXXXXXXXXXXX
subject: CN=IPA RA,O=XXXXXXXXXXXXX
expires: 2021-08-04 16:04:27 UTC
key usage: digitalSignature,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
Request ID '20190815160528':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB'
issuer: CN=Certificate Authority,O=XXXXXXXXXXXXX
subject: CN=CA Audit,O=XXXXXXXXXXXXX
expires: 2021-08-04 16:03:26 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20190815160529':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS
Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS
Certificate DB'
issuer: CN=Certificate Authority,O=XXXXXXXXXXXXX
subject: CN=OCSP Subsystem,O=XXXXXXXXXXXXX
expires: 2021-08-04 16:03:24 UTC
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20190815160530':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB'
issuer: CN=Certificate Authority,O=XXXXXXXXXXXXX
subject: CN=CA Subsystem,O=XXXXXXXXXXXXX
expires: 2021-08-04 16:03:25 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20190815160531':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB'
issuer: CN=Certificate Authority,O=XXXXXXXXXXXXX
subject: CN=Certificate Authority,O=XXXXXXXXXXXXX
expires: 2039-08-15 16:03:22 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20190815160532':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB'
issuer: CN=Certificate Authority,O=XXXXXXXXXXXXX
subject: CN=XXXXXXXXXXXXX,O=XXXXXXXXXXXXX
expires: 2021-08-04 16:03:24 UTC
dns: XXXXXXXXXXXXX
key usage: digitalSignature,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
"Server-Cert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20190815160610':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/dirsrv/slapd-XXXXXXXXXXXXX',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/dirsrv/slapd-XXXXXXXXXXXXX/pwdfile.txt'
certificate:
type=NSSDB,location='/etc/dirsrv/slapd-XXXXXXXXXXXXX',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=XXXXXXXXXXXXX
subject: CN=XXXXXXXXXXXXX,O=XXXXXXXXXXXXX
expires: 2021-08-15 16:06:11 UTC
dns: XXXXXXXXXXXXX
principal name: ldap/XXXXXXXXXXXXX@XXXXXXXXXXXXX
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: 
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv XXXXXXXXXXXXX
track: yes
auto-renew: yes
Request ID '20190815160720':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate:
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=XXXXXXXXXXXXX
subject: CN=XXXXXXXXXXXXX,O=XXXXXXXXXXXXX
expires: 2021-08-15 16:07:21 UTC
dns: XXXXXXXXXXXXX
principal name: HTTP/XXXXXXXXXXXXX@XXXXXXXXXXXXX
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: 
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
Request ID '20190815160739':
status: MONITORING
stuck: no
key pair storage: type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
CA: IPA
issuer: CN=Certificate Authority,O=XXXXXXXXXXXXX
subject: CN=XXXXXXXXXXXXX,O=XXXXXXXXXXXXX
expires: 2021-08-15 16:07:40 UTC
principal name: krbtgt/XXXXXXXXXXXXX@XXXXXXXXXXXXX
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-pkinit-KPKdc
pre-save command: 
post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
track: yes
auto-renew: yes


On Mar 4, 2020, at 12:16 PM, Rob Crittenden <rcritten@redhat.com
<mailto:rcritten@redhat.com>> wrote:

David Carter via FreeIPA-users wrote:
Freeipa was running fine, then I guess it tried to update and now I
have an unusable system and it's impacting other systems. I've seen
this error in a few online postings but no fix that I can apply.

When I run it manually:

[root@kdc1 log]# ipa-server-upgrade
Upgrading IPA:. Estimated time: 1 minute 30 seconds
 [1/11]: stopping directory server
 [2/11]: saving configuration
 [3/11]: disabling listeners
 [4/11]: enabling DS global lock
 [5/11]: disabling Schema Compat
 [6/11]: starting directory server
 [7/11]: updating schema
 [8/11]: upgrading server
 [9/11]: stopping directory server
 [10/11]: restoring configuration
 [11/11]: starting directory server
Done.
Update complete
Upgrading IPA services
Upgrading the configuration of the IPA services
[Verifying that root certificate is published]
[Migrate CRL publish directory]
CRL tree already moved
[Verifying that CA proxy configuration is correct]
[Verifying that KDC configuration is using ipa-kdb backend]
[Fix DS schema file syntax]
Syntax already fixed
[Removing RA cert from DS NSS database]
RA cert already removed
[Enable sidgen and extdom plugins by default]
[Updating HTTPD service IPA configuration]
[Updating HTTPD service IPA WSGI configuration]
Nothing to do for configure_httpd_wsgi_conf
[Updating mod_nss protocol versions]
Protocol versions already updated
[Updating mod_nss cipher suite]
[Updating mod_nss enabling OCSP]
[Fixing trust flags in /etc/httpd/alias]
Trust flags already processed
[Moving HTTPD service keytab to gssproxy]
[Removing self-signed CA]
[Removing Dogtag 9 CA]
[Checking for deprecated KDC configuration files]
[Checking for deprecated backups of Samba configuration files]
[Add missing CA DNS records]
IPA CA DNS records already processed
[Removing deprecated DNS configuration options]
DNS is not configured
[Ensuring minimal number of connections]
DNS is not configured
[Updating GSSAPI configuration in DNS]
DNS is not configured
[Updating pid-file configuration in DNS]
DNS is not configured
DNS is not configured
DNS is not configured
DNS is not configured
DNS is not configured
DNS is not configured
DNS is not configured
DNS is not configured
[Upgrading CA schema]
CA schema update complete (no changes)
[Verifying that CA audit signing cert has 2 year validity]
[Update certmonger certificate renewal configuration]
Failed to get request: bus, object_path and dbus_interface must not
be None.
IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run
command ipa-server-upgrade manually.
bus, object_path and dbus_interface must not be None.
The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log
for more information
The tail of the log file (it is very long):

2020-03-03T03:51:44Z DEBUG stderr=
2020-03-03T03:51:44Z DEBUG Loading Index file from
'/var/lib/ipa/sysrestore/sysrestore.index'
2020-03-03T03:51:44Z DEBUG Starting external process
2020-03-03T03:51:44Z DEBUG args=/usr/bin/certutil -d
dbm:/etc/pki/pki-tomcat/alias -L -f /etc/pki/pki-tomcat/alias/pwdfile.txt
2020-03-03T03:51:44Z DEBUG Process finished, return code=0
2020-03-03T03:51:44Z DEBUG stdout=
Certificate Nickname                                         Trust
Attributes
                                                            SSL,S/MIME,JAR/XPI

caSigningCert cert-pki-ca                                    CTu,Cu,Cu
auditSigningCert cert-pki-ca                                 u,u,Pu
Server-Cert cert-pki-ca                                      u,u,u
ocspSigningCert cert-pki-ca                                  u,u,u
subsystemCert cert-pki-ca                                    u,u,u

2020-03-03T03:51:44Z DEBUG stderr=
2020-03-03T03:51:44Z ERROR Failed to get request: bus, object_path
and dbus_interface must not be None.
2020-03-03T03:51:44Z ERROR IPA server upgrade failed: Inspect
/var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2020-03-03T03:51:44Z DEBUG   File
"/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 178,
in execute
   return_value = self.run()
 File
"/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py",
line 56, in run
   raise admintool.ScriptError(str(e))

2020-03-03T03:51:44Z DEBUG The ipa-server-upgrade command failed,
exception: ScriptError: bus, object_path and dbus_interface must not
be None.
2020-03-03T03:51:44Z ERROR bus, object_path and dbus_interface must
not be None.
2020-03-03T03:51:44Z ERROR The ipa-server-upgrade command failed. See
/var/log/ipaupgrade.log for more information
I can run the certutil command separately, so it doesn't seem to be
that. I have no idea what the issue is, where to look, or how I can
fix this. Suggestions?

OS: CentOS 7
IPA version: VERSION: 4.6.5, API_VERSION: 2.231

What is the output of getcert list-cas

One or more is missing.

BTW this terrible error message is fixed upstream and will appear in
future versions of RHEL 8 with a much better description.

rob




_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org