All,

This is a strange one.  When we exec this command under puppet control:

/usr/sbin/realm permit -R AMER.COMPANY.COM processehcprofiler@AMER.COMPANY.COM

Then sssd_be core dumps (segfault). 

When we run that ‘realm permit’ command natively on the command line, it executes flawlessly.  No core dump.

Naturally, our first thought was that it’s something different in the environment.  So we dumped the environment under which puppet exec resource runs. 

Yes, quite minimal. 

LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.m4a=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.oga=01;36:*.opus=01;36:*.spx=01;36:*.xspf=01;36:

LD_LIBRARY_PATH=

LANG=en_US.UTF-8

HISTCONTROL=ignoredups

HOSTNAME=austgcore25.us.company.com

S_COLORS=auto

PWD=/root

APT_LISTCHANGES_FRONTEND=none

DEBIAN_FRONTEND=noninteractive

APT_LISTBUGS_FRONTEND=none

MAIL=/var/spool/mail/root

SHELL=/bin/bash

TERM=xterm

SHLVL=2

MANPATH=:/opt/puppetlabs/puppet/share/man

PATH=/bin:/usr/bin:/sbin:/usr/sbin

HISTSIZE=1000

LESSOPEN=||/usr/bin/lesspipe.sh %s

_=/bin/env

 

But when we create a bash session with no environment, then add this puppet-supplied environment and run the above realm permit– all is still well.

We can reproduce this easily in puppet.  Just delete our breadcrumb file (so that puppet re-executes this ‘realm permit’ command). And execute another puppet agent run.

Doing this, we obtained a core dump from sssd_be.  And it points to some code in ad_id.c:

[root@austgcore25 tmp]# gdb -c core-sssd_be.15405.austgcore25.us.company.com.1563210863

GNU gdb (GDB) Red Hat Enterprise Linux 8.2-5.el8

Copyright (C) 2018 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.

Type "show copying" and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu".

Type "show configuration" for configuration details.

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>.

Find the GDB manual and other documentation resources online at:

    <http://www.gnu.org/software/gdb/documentation/>.

 

For help, type "help".

Type "apropos word" to search for commands related to "word".

[New LWP 15405]

Reading symbols from /usr/libexec/sssd/sssd_be...Reading symbols from /usr/lib/debug/usr/libexec/sssd/sssd_be-2.0.0-43.el8.x86_64.debug...done.

done.

 

warning: Ignoring non-absolute filename: <linux-vdso.so.1>

Missing separate debuginfo for linux-vdso.so.1

Try: dnf --enablerepo='*debug*' install /usr/lib/debug/.build-id/06/44254f9cbaa826db070a796046026adba58266

 

warning: .dynamic section for "/usr/lib64/libndr-nbt.so.0.0.1" is not at the expected address (wrong library or version mismatch?)

[Thread debugging using libthread_db enabled]

Using host libthread_db library "/lib64/libthread_db.so.1".

Core was generated by `/usr/libexec/sssd/sssd_be --domain AMER.COMPANY.COM --uid 0 --gid 0 --logger=files'.

Program terminated with signal SIGSEGV, Segmentation fault.

#0  0x00007f79444f53c0 in ad_get_account_domain_search (req=req@entry=0x5557b6fd45b0) at src/providers/ad/ad_id.c:1276

1276        state->filter = sdap_combine_filters(state, state->base_filter,

(gdb)

 

This is in RHEL8. 

 

What we suspect is that the shell in which puppet executes this ‘realm permit’ is not supplying something that this executable needs.  If we know what it is, we can preface our puppet exec resource code snippet with the missing piece.

 

Spike

PS This ‘realm permit’ does seem to perform the correct actions eventually.  It adds the expected user to the simple_allow_users line of the appropriate AD domain in /etc/sssd/sssd.conf file.  But because it segfaults and then has to start up again, it takes a very long time to complete the puppet run.  (There’s about 20 – 30 users + groups allowed; it has to segfault on each of them).