On 8/25/21 8:32 AM, Spike White wrote:
Because we are researching an ongoing problem reported by L1 server
ops. About 70 – 80 sssd-enabled Linux servers / month drop off the
domain. Out of our current sssd-enabled population of ~20K server,
that’s not horrible. But still it should be better. (Our former
commercial product did better.)
It’s not limited to one particular OS, OS version, build location or
region. We have surveyed; it seems to occur randomly among all OS
versions, regions and locations.
...
We are taking sssd’s default settings for
ad_maximum_machine_account_password_age and
ad_machine_account_password_renewal_opts. ... So when we find servers
that have dropped off the domain, it’s because they have not renewed
their AD machine accounts in 40 days.
We had similar symptoms on CentOS systems at my previous employer,
however, I'm mostly sure that they were resolved by an sssd update
sometime in the last year or two. Are all of your systems fully patched?
If you're seeing the same issue that we saw, one indication would be
that running "klist -kt /etc/krb5.keytab" would print a list including
two KVNOs on systems after they'd dropped off the domain.
I'd also look up machines in AD to find systems that haven't changed
their password in > 40 days, and compare the PasswordLastSet date for a
system you're examining to the dates in the klist output:
https://pipe2text.com/?page_id=121
Import-Module ActiveDirectory
$date = [DateTime]::Today.AddDays(-40)
Get-ADComputer -Filter ‘PasswordLastSet -le $date’ -SearchBase
“OU=WhereIStoreComputers,DC=pipe2,DC=Text,DC=com” -properties
PasswordLastSet
But packet captures proved that adcli update is using TCP on
RHEL7/8.
I *think* that our problems went away after an update changed machine
password renewal to TCP only. You might have a different problem, but
I'd still start with the keytab, and I'd increase the logging level for
sssd because the default logging level didn't give us a lot of
information to go on when we were troubleshooting password renewal. It
doesn't need to be 9, but should be increased to a level that won't
cause you operational difficulty.