[sssd PR#5766][comment] Sources cleanup - part 3.
by alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5766
Title: #5766: Sources cleanup - part 3.
alexey-tikhonov commented:
"""
> After upgrade from json-ed ccache I got:
>
> ```
> [pbrezina /dev/shm/sssd]$ klist
> klist: Internal credentials cache error
> ```
This happens one time:
```
(2021-09-25 11:40:48): [kcm] [ccdb_secdb_list_send] (0x2000): Listing all ccaches
(2021-09-25 11:40:48): [kcm] [sss_sec_list] (0x0400): Listing keys at [persistent/0/ccache/]
(2021-09-25 11:40:48): [kcm] [local_dn_to_path] (0x2000): Secrets path for [cn=77aa5f1e-dd99-4479-9d5a-a1641f346505-0,cn=ccache,cn=0,cn=persistent,cn=kcm] is [77aa5f1e-dd99-4479-9d5a-a1641f346505-0]
(2021-09-25 11:40:48): [kcm] [sss_sec_list] (0x1000): Returning 1 secrets
(2021-09-25 11:40:48): [kcm] [ccdb_secdb_list_send] (0x2000): Found 1 ccaches
...
(2021-09-25 11:40:48): [kcm] [kcm_cmd_send] (0x0400): KCM operation GET_PRINCIPAL
(2021-09-25 11:40:48): [kcm] [sss_sec_get] (0x0400): Retrieving a secret from [persistent/0/ccache/77aa5f1e-dd99-4479-9d5a-a1641f346505-0]
(2021-09-25 11:40:48): [kcm] [secdb_get_cc] (0x0040): Cannot convert data to ccache [12]: Cannot allocate memory, deleting this entry
(2021-09-25 11:40:48): [kcm] [sss_sec_delete] (0x0400): Removing a secret from [persistent/0/ccache/77aa5f1e-dd99-4479-9d5a-a1641f346505-0]
(2021-09-25 11:40:48): [kcm] [kcm_reply_error] (0x0040): KCM operation returns failure [2]: No such file or directory
(2021-09-25 11:40:48): [kcm] [kcm_failbuf_construct] (0x1000): Sent reply with error -1765328188
```
-- `sss_sec_list()` finds ccache in old format but later `sss_sec_get()` fails for this key and deletes this entry, so second `klist` doesn't fail.
Options are:
- try to convert ccache in `sss_sec_list()` and delete malformed entries there: unconditional perf overhead to remove outdated entries once (if any)
- patch `sss2krb5_error()` to translate `ENOENT` to `KRB5_CC_NOTFOUND`. Result will be different output:
```
# klist
klist: Matching credential not found
```
-- looks a little bit better, but I'm not sure this conversion is a right thing in other possible scenarios
- leave it as is. IIRC, ccaches in old format aren't created since F32 and I think we aren't going to rebase Fedora older than 35 on new 2.6 release (is this correct?) So I think chances to meet ccache in old format on F35+ are low and perhaps we can tolerate this one-time error output of klist in this case...
"""
See the full comment at https://github.com/SSSD/sssd/pull/5766#issuecomment-927148719
2 years, 7 months
[sssd PR#5766][comment] Sources cleanup - part 3.
by alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5766
Title: #5766: Sources cleanup - part 3.
alexey-tikhonov commented:
"""
> After upgrade from json-ed ccache I got:
>
> ```
> [pbrezina /dev/shm/sssd]$ klist
> klist: Internal credentials cache error
> ```
This happens one time:
```
(2021-09-25 11:40:48): [kcm] [ccdb_secdb_list_send] (0x2000): Listing all ccaches
(2021-09-25 11:40:48): [kcm] [sss_sec_list] (0x0400): Listing keys at [persistent/0/ccache/]
(2021-09-25 11:40:48): [kcm] [local_dn_to_path] (0x2000): Secrets path for [cn=77aa5f1e-dd99-4479-9d5a-a1641f346505-0,cn=ccache,cn=0,cn=persistent,cn=kcm] is [77aa5f1e-dd99-4479-9d5a-a1641f346505-0]
(2021-09-25 11:40:48): [kcm] [sss_sec_list] (0x1000): Returning 1 secrets
(2021-09-25 11:40:48): [kcm] [ccdb_secdb_list_send] (0x2000): Found 1 ccaches
...
(2021-09-25 11:40:48): [kcm] [kcm_cmd_send] (0x0400): KCM operation GET_PRINCIPAL
(2021-09-25 11:40:48): [kcm] [sss_sec_get] (0x0400): Retrieving a secret from [persistent/0/ccache/77aa5f1e-dd99-4479-9d5a-a1641f346505-0]
(2021-09-25 11:40:48): [kcm] [secdb_get_cc] (0x0040): Cannot convert data to ccache [12]: Cannot allocate memory, deleting this entry
(2021-09-25 11:40:48): [kcm] [sss_sec_delete] (0x0400): Removing a secret from [persistent/0/ccache/77aa5f1e-dd99-4479-9d5a-a1641f346505-0]
(2021-09-25 11:40:48): [kcm] [kcm_reply_error] (0x0040): KCM operation returns failure [2]: No such file or directory
(2021-09-25 11:40:48): [kcm] [kcm_failbuf_construct] (0x1000): Sent reply with error -1765328188
```
-- `sss_sec_list()` finds ccache in old format but later `sss_sec_get()` fails for this key and deletes this entry, so second `klist` doesn't fail.
Options are:
- try to convert ccache in `sss_sec_list()` and delete malformed entries there: unconditional perf overhead to remove outdated entries once (if any)
- patch `sss2krb5_error()` to translate `ENOENT` to `KRB5_CC_NOTFOUND`. Result will be different output:
```
# klist
klist: Matching credential not found
```
-- looks a little bit better, but I'm not sure this conversion is a right thing in other possible scenarios
-- leave it as is. IIRC, ccaches in old format aren't created since F32 and I think we aren't going to rebase Fedora older than 35 on new 2.6 release (is this correct?) So I think chances to meet ccache in old format on F35+ are low and perhaps we can tolerate this one-time error output of klist in this case...
"""
See the full comment at https://github.com/SSSD/sssd/pull/5766#issuecomment-927148719
2 years, 7 months
[sssd PR#5766][comment] Sources cleanup - part 3.
by alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5766
Title: #5766: Sources cleanup - part 3.
alexey-tikhonov commented:
"""
With `['KRB5_TRACE'] = "/dev/stdout"`:
```
____________________________________________________________ test_krb5_auth ____________________________________________________________
Traceback (most recent call last):
File "/home/talex/sssd/src/tests/intg/test_pam_responder.py", line 768, in test_krb5_auth
assert err.find(r"pam_authenticate for user [user1]: Success") != -1
AssertionError: assert -1 != -1
+ where -1 = <built-in method find of str object at 0x7f8c77e010f8>('pam_authenticate for user [user1]: Success')
+ where <built-in method find of str object at 0x7f8c77e010f8> = 'Unable to connect to system bus!\nInfoPipe User lookup with [user1] failed.\nPassword: pam_authenticate for user [user1]: System error\n\nPAM Environment:\n - no env -\n'.find
-------------------------------------------------------- Captured stdout setup ---------------------------------------------------------
Loading random data
Initializing database '/tmp/sssd-intg.78miBVX1/var/krb5kdc/principal' for realm 'PAMKRB5TEST',
master key name 'K/M@PAMKRB5TEST'
0/Success
[23232] 1632575636.946045: Retrieving K/M@PAMKRB5TEST from FILE:/tmp/sssd-intg.78miBVX1/var/krb5kdc/stash.PAMKRB5TEST (vno 0, enctype 0) with result: 0/Success
-------------------------------------------------------- Captured stderr setup ---------------------------------------------------------
kadmin.local: No KCM server found while opening default credentials cache
kadmin.local: No KCM server found while opening default credentials cache
```
`Unable to connect to system bus!` is also suspicious.
`krb5kdc.log`:
```
Sep 25 10:40:09 debian10 krb5kdc[1190](Error): preauth spake failed to initialize: No SPAKE preauth groups configured
Sep 25 10:40:09 debian10 krb5kdc[1190](info): setting up network...
Sep 25 10:40:09 debian10 krb5kdc[1190](info): setsockopt(18,IPV6_V6ONLY,1) worked
Sep 25 10:40:09 debian10 krb5kdc[1190](info): setsockopt(20,IPV6_V6ONLY,1) worked
Sep 25 10:40:09 debian10 krb5kdc[1190](info): set up 4 sockets
Sep 25 10:40:09 debian10 krb5kdc[1191](info): commencing operation
Sep 25 10:40:10 debian10 krb5kdc[1191](info): AS_REQ (8 etypes {18 17 20 19 16 23 25 26}) ::1: CLIENT_NOT_FOUND: user1@PAMKRB5TEST for krbtgt/PAMKRB5TEST@PAMKRB5TEST, Client not found in Kerberos database
```
```
$ strings principal
kiprop/debian10@PAMKRB5TEST
kiprop/debian10@PAMKRB5TEST
4Oakdb5_util@PAMKRB5TEST
kadmin/changepw@PAMKRB5TEST
kadmin/changepw@PAMKRB5TEST
4Oakdb5_util@PAMKRB5TEST
kadmin/admin@PAMKRB5TEST
kadmin/admin@PAMKRB5TEST
4Oakdb5_util@PAMKRB5TEST
kadmin/debian10@PAMKRB5TEST
kadmin/debian10@PAMKRB5TEST
4Oakdb5_util@PAMKRB5TEST
krbtgt/PAMKRB5TEST@PAMKRB5TEST
krbtgt/PAMKRB5TEST@PAMKRB5TEST
4Oadb_creation@PAMKRB5TEST
K/M@PAMKRB5TEST
K/M@PAMKRB5TEST
4Oadb_creation@PAMKRB5TEST
```
`krb5_child.log`:
```
(2021-09-25 10:40:10): [krb5_child[1200]] [get_and_save_tgt] (0x0400): Attempting kinit for realm [PAMKRB5TEST]
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640671: Getting initial credentials for user1@PAMKRB5TEST
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640673: Sending unauthenticated request
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640674: Sending request (183 bytes) to PAMKRB5TEST
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640675: Resolving hostname localhost
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640676: Sending initial UDP request to dgram ::1:10088
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640677: Received answer (151 bytes) from dgram ::1:10088
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640678: Response was not from master KDC
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640679: Received error from KDC: -1765328378/Client not found in Kerberos database
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640680: Retrying AS request with master KDC
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640681: Getting initial credentials for user1@PAMKRB5TEST
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640683: Sending unauthenticated request
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640684: Sending request (183 bytes) to PAMKRB5TEST (master)
(2021-09-25 10:40:10): [krb5_child[1200]] [get_and_save_tgt] (0x0020): 1724: [-1765328378][Client 'user1@PAMKRB5TEST' not found in Kerberos database]
(2021-09-25 10:40:10): [krb5_child[1200]] [map_krb5_error] (0x0020): 1853: [-1765328378][Client 'user1@PAMKRB5TEST' not found in Kerberos database]
```
"""
See the full comment at https://github.com/SSSD/sssd/pull/5766#issuecomment-927120637
2 years, 7 months
[sssd PR#5766][comment] Sources cleanup - part 3.
by alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5766
Title: #5766: Sources cleanup - part 3.
alexey-tikhonov commented:
"""
With `['KRB5_TRACE'] = "/dev/stdout"`:
```
____________________________________________________________ test_krb5_auth ____________________________________________________________
Traceback (most recent call last):
File "/home/talex/sssd/src/tests/intg/test_pam_responder.py", line 768, in test_krb5_auth
assert err.find(r"pam_authenticate for user [user1]: Success") != -1
AssertionError: assert -1 != -1
+ where -1 = <built-in method find of str object at 0x7f8c77e010f8>('pam_authenticate for user [user1]: Success')
+ where <built-in method find of str object at 0x7f8c77e010f8> = 'Unable to connect to system bus!\nInfoPipe User lookup with [user1] failed.\nPassword: pam_authenticate for user [user1]: System error\n\nPAM Environment:\n - no env -\n'.find
-------------------------------------------------------- Captured stdout setup ---------------------------------------------------------
Loading random data
Initializing database '/tmp/sssd-intg.78miBVX1/var/krb5kdc/principal' for realm 'PAMKRB5TEST',
master key name 'K/M@PAMKRB5TEST'
0/Success
[23232] 1632575636.946045: Retrieving K/M@PAMKRB5TEST from FILE:/tmp/sssd-intg.78miBVX1/var/krb5kdc/stash.PAMKRB5TEST (vno 0, enctype 0) with result: 0/Success
-------------------------------------------------------- Captured stderr setup ---------------------------------------------------------
kadmin.local: No KCM server found while opening default credentials cache
kadmin.local: No KCM server found while opening default credentials cache
```
`Unable to connect to system bus!` is also suspicious.
`krb5kdc.log`:
```
Sep 25 10:40:09 debian10 krb5kdc[1190](Error): preauth spake failed to initialize: No SPAKE preauth groups configured
Sep 25 10:40:09 debian10 krb5kdc[1190](info): setting up network...
Sep 25 10:40:09 debian10 krb5kdc[1190](info): setsockopt(18,IPV6_V6ONLY,1) worked
Sep 25 10:40:09 debian10 krb5kdc[1190](info): setsockopt(20,IPV6_V6ONLY,1) worked
Sep 25 10:40:09 debian10 krb5kdc[1190](info): set up 4 sockets
Sep 25 10:40:09 debian10 krb5kdc[1191](info): commencing operation
Sep 25 10:40:10 debian10 krb5kdc[1191](info): AS_REQ (8 etypes {18 17 20 19 16 23 25 26}) ::1: CLIENT_NOT_FOUND: user1@PAMKRB5TEST for krbtgt/PAMKRB5TEST@PAMKRB5TEST, Client not found in Kerberos database
```
`krb5_child.log`:
```
(2021-09-25 10:40:10): [krb5_child[1200]] [get_and_save_tgt] (0x0400): Attempting kinit for realm [PAMKRB5TEST]
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640671: Getting initial credentials for user1@PAMKRB5TEST
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640673: Sending unauthenticated request
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640674: Sending request (183 bytes) to PAMKRB5TEST
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640675: Resolving hostname localhost
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640676: Sending initial UDP request to dgram ::1:10088
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640677: Received answer (151 bytes) from dgram ::1:10088
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640678: Response was not from master KDC
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640679: Received error from KDC: -1765328378/Client not found in Kerberos database
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640680: Retrying AS request with master KDC
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640681: Getting initial credentials for user1@PAMKRB5TEST
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640683: Sending unauthenticated request
(2021-09-25 10:40:10): [krb5_child[1200]] [sss_child_krb5_trace_cb] (0x4000): [1200] 1632580810.640684: Sending request (183 bytes) to PAMKRB5TEST (master)
(2021-09-25 10:40:10): [krb5_child[1200]] [get_and_save_tgt] (0x0020): 1724: [-1765328378][Client 'user1@PAMKRB5TEST' not found in Kerberos database]
(2021-09-25 10:40:10): [krb5_child[1200]] [map_krb5_error] (0x0020): 1853: [-1765328378][Client 'user1@PAMKRB5TEST' not found in Kerberos database]
```
"""
See the full comment at https://github.com/SSSD/sssd/pull/5766#issuecomment-927120637
2 years, 7 months
[sssd PR#5766][comment] Sources cleanup - part 3.
by alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5766
Title: #5766: Sources cleanup - part 3.
alexey-tikhonov commented:
"""
With `['KRB5_TRACE'] = "/dev/stdout"`:
```
____________________________________________________________ test_krb5_auth ____________________________________________________________
Traceback (most recent call last):
File "/home/talex/sssd/src/tests/intg/test_pam_responder.py", line 768, in test_krb5_auth
assert err.find(r"pam_authenticate for user [user1]: Success") != -1
AssertionError: assert -1 != -1
+ where -1 = <built-in method find of str object at 0x7f8c77e010f8>('pam_authenticate for user [user1]: Success')
+ where <built-in method find of str object at 0x7f8c77e010f8> = 'Unable to connect to system bus!\nInfoPipe User lookup with [user1] failed.\nPassword: pam_authenticate for user [user1]: System error\n\nPAM Environment:\n - no env -\n'.find
-------------------------------------------------------- Captured stdout setup ---------------------------------------------------------
Loading random data
Initializing database '/tmp/sssd-intg.78miBVX1/var/krb5kdc/principal' for realm 'PAMKRB5TEST',
master key name 'K/M@PAMKRB5TEST'
0/Success
[23232] 1632575636.946045: Retrieving K/M@PAMKRB5TEST from FILE:/tmp/sssd-intg.78miBVX1/var/krb5kdc/stash.PAMKRB5TEST (vno 0, enctype 0) with result: 0/Success
-------------------------------------------------------- Captured stderr setup ---------------------------------------------------------
kadmin.local: No KCM server found while opening default credentials cache
kadmin.local: No KCM server found while opening default credentials cache
```
`Unable to connect to system bus!` is also suspicious.
"""
See the full comment at https://github.com/SSSD/sssd/pull/5766#issuecomment-927120637
2 years, 7 months
[sssd PR#5766][comment] Sources cleanup - part 3.
by alexey-tikhonov
URL: https://github.com/SSSD/sssd/pull/5766
Title: #5766: Sources cleanup - part 3.
alexey-tikhonov commented:
"""
> I didn't see this debian failure on other pull requests so I would like to wait until we understand what is happening there.
I still don't know what is happening exactly, but I discovered that the same fail is observed on master branch if SSSD is configured `--without-secrets`.
With patch https://github.com/alexey-tikhonov/sssd/commit/1018378bbeae2e6869bdb49e1a... on the top of master branch on demand run result: https://ci-jenkins-csb-sssd.apps.ocp4.prod.psi.redhat.com/job/sssd-ci-on-...
So:
- master branch with default config - **success**
- master branch `--without-secrets`**entire** suite - **fail** on Debian (both 10 and 11, but logs are a bit different)
- master branch `--without-secrets` "-k test_krb5_auth" - **success**
This makes me thinking that some of tests (maybe kcm) leave a leftovers in the working dir that trigger a fail, but `secrets` tests clean this up, so `test_krb5_auth` succeeds. And when we disable `secrets` (either via config option or via code removal) clean up doesn't happen and `test_krb5_auth` fails.
This is the only idea I have at the moment and I must admit it doesn't explain why fail is only seen on Debian... IIRC, Debian was somewhat different with regards to KCM, but I don't remember details.
As to what exactly is failing: `krb5_child` fails to get a ticket - KDC replies 'user not found` - probably `kadmin.local addprinc` fails.
IIUC, test is expected to work with 'FILE:` ccaches, but on Debian11 I saw something like 'Failed to connect to KCM server' on stderr (I'm not sure but looks like this is from `kadmin.local`).
@sumit-bose , @pbrezina , do you have any insights based on the above summary?
"""
See the full comment at https://github.com/SSSD/sssd/pull/5766#issuecomment-927107834
2 years, 7 months
[sssd PR#5791][opened] sysdb: more specific mpg search filter
by sumit-bose
URL: https://github.com/SSSD/sssd/pull/5791
Author: sumit-bose
Title: #5791: sysdb: more specific mpg search filter
Action: opened
PR body:
"""
Originally all user of an mpg domain had an automatically created
user-private group and as a result the ID space was unified in the sense
that a given ID either belongs to a group to to a user with a
user-private group.
With the introduction of id-overrides and the auto_private_groups option
this assumption is not true anymore and as a result the search filter
for GIDs must be more specific with respect to the user objects.
Resolves: https://github.com/SSSD/sssd/issues/5790
"""
To pull the PR as Git branch:
git remote add ghsssd https://github.com/SSSD/sssd
git fetch ghsssd pull/5791/head:pr5791
git checkout pr5791
2 years, 7 months