I've been able to trace this failure back to ipa-custodia on ipa01. the
pki-tomcat/ca/debug log on ipa02 shows:
[17/Sep/2019:18:28:35][KeyRetrieverRunner-b06485e9-c2bb-4ccf-8023-0bf93c32b94b]: About to
execute command: [/usr/libexec/ipa/ipa-pki-retrieve-key, caSigningCert cert-pki-ca
b06485e9-c2bb-4ccf-8023-0bf93c32b94b,
ipa01.yyy.com]
[17/Sep/2019:18:28:35][KeyRetrieverRunner-b06485e9-c2bb-4ccf-8023-0bf93c32b94b]: Failed to
retrieve key from any host.
[17/Sep/2019:18:28:35][KeyRetrieverRunner-b06485e9-c2bb-4ccf-8023-0bf93c32b94b]:
KeyRetriever did not return a result.
[17/Sep/2019:18:28:35][KeyRetrieverRunner-b06485e9-c2bb-4ccf-8023-0bf93c32b94b]: Retrying
in 168338 seconds
The apache access log on ipa01 shows a 404:
"GET
/ipa/keys/ca_wrapped/caSigningCert%20cert-pki-ca%20b06485e9-c2bb-4ccf-8023-0bf93c32b?.............
HTTP/1.1" 404 190
And then I see a denial in /var/log/ipa-custodia.audit.log:
2019-09-17 17:58:54 - SimpleCredsAuth-[auth:simple] - PASS: '4289'
authenticated as '48, 48'
2019-09-17 17:58:54 - SimpleHeaderAuth-[auth:header] - PASS: '4289'
authenticated as '(null)'
2019-09-17 17:58:54 - IPAKEMKeys-[authz:kemkeys] - PASS: '4289' authorized
for '/keys'
2019-09-17 17:58:55 - Secrets-[/keys] - DENIED: '(null)'
requested key 'ca_wrapped/caSigningCert cert-pki-ca
b06485e9-c2bb-4ccf-8023-0bf93c32b'
The other keys are being allowed by custodia:
2018-01-24 18:18:31 - SimpleCredsAuth-[auth:simple] - PASS: '15417'
authenticated as '48, 48'
2018-01-24 18:18:31 - SimpleHeaderAuth-[auth:header] - PASS: '15417'
authenticated as '(null)'
2018-01-24 18:18:31 - IPAKEMKeys-[authz:kemkeys] - PASS: '15417' authorized
for '/keys'
2018-01-24 18:18:31 - Secrets-[/keys] - ALLOWED: '(null)'
requested key 'ca/caSigningCert cert-pki-ca'
Any thoughts on why the ca_wrapped request for the intermediate cert is being denied?
I did restart ipa-custodia on ipa01 without any effect.