Hi all -
So this is something I found and wanted to post it to the team - this is
for RHEL and/or CentOS 7.3 thru 5 so far. It has to do with
selinux_provider and having to explicitly disable it in sssd or things
will randomly fail.
On heavily loaded clients, (and a fair load on IPA cluster) you find
that even if a client has selinux disabled (sometimes because of
application requirements) that ssh access is still randomly denied
because of selinux failures. You need to explicitly add
selinux_provider=none to sssd.conf to avoid seeing these:
sshd[58319]: fatal: Access denied for user xxxxxxxx by PAM account
configuration [preauth]
sshd[58319]: pam_sss(sshd:account): Access denied for user xxxxxxxx: 4
(System error)
If you look in detail you find that the authentication actually works
but when it is sent back to the client, there are random failures for
the same username from time to time. It all seems to be load related, as
I have been unable to find a root cause. An example is that I have a
looping ssh job to just login, create a folder and exit - all via ssh
keys. If you run that for a few hours with a few seconds interval, you
find that out of 1000+ successes, you might see 20-30 random "Access
Denied".
This was confusing at first because sshd only returns that the
authentication failed without any details (return code is 255) but
looking in detailed logs finds the random errors as show above. This all
connects back with the errors I reported last week regarding the same
thing and that I felt it was related to DNS and other settings - it was
not.
Hope this helps someone else..
-K