Hi, 
Just following up with this issue again. I did more testing and found that if I add ldap_use_tokengroups=False, this long pause is significantly reduced. From the last email, it seems it is taking awfully long during sysdb_set_entry_attr during large group update. So I have following questions 

1. Why is adding this flag causing it to speed up that much ?
2. Is there any index, which could be used to speed up sysdb_set_entry_attr call ?
3. If I use ldap_use_tokengroups=False, I see during initgroups it is not fetching a few groups (e.g. domain_users/users), which seems to have a tokenGroups attribute. What can we do to work around this?

(2020-12-28 14:06:20:075623): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 14:06:20:121118): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 14:06:20:121317): [be[mydomain]] [sdap_save_groups] (0x4000): Group 4 members processed!
--
(2020-12-28 14:21:27:074044): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 14:21:27:132745): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 14:21:27:132947): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 14:42:38:786851): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 14:42:38:846446): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 14:42:38:846693): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 15:03:15:421853): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 15:03:15:483935): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 15:03:15:484154): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 15:11:14:331469): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 15:11:14:382334): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 15:11:14:382496): [be[mydomain]] [sdap_save_groups] (0x4000): Group 4 members processed!
--
(2020-12-28 15:33:48:861686): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 15:33:48:921684): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 15:33:48:921889): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 15:53:47:268799): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 15:53:47:325466): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 15:53:47:325696): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 16:13:14:107362): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 16:13:14:166806): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 16:13:14:167047): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 16:16:14:032217): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 16:16:14:080719): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 16:16:14:080890): [be[mydomain]] [sdap_save_groups] (0x4000): Group 4 members processed!
--
(2020-12-28 16:32:42:368442): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 16:32:42:428655): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 16:32:42:428890): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 16:49:12:657265): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 16:49:12:715830): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 16:49:12:716071): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 17:05:17:976739): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 17:05:18:035818): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 17:05:18:036043): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 17:21:14:090058): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 17:21:14:147189): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 17:21:14:147382): [be[mydomain]] [sdap_save_groups] (0x4000): Group 4 members processed!
--
(2020-12-28 17:43:12:423857): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 17:43:12:483306): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 17:43:12:483531): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 18:03:49:458768): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 18:03:49:517741): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 18:03:49:517932): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 18:19:40:572151): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 18:19:40:630836): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 18:19:40:631037): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-28 18:26:14:880640): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-28 18:26:14:933184): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1188@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-28 18:26:14:933429): [be[mydomain]] [sdap_save_groups] (0x4000): Group 4 members processed!


Thanks,
Sanjay Agrawal


On Tuesday, December 22, 2020, 03:45:59 PM EST, Sanjay Agrawal <sanjayagrawal@yahoo.com> wrote:


We are noticing that sometimes sssd take very long to resolve some queries. They are happens when sssd_be is 100% during that interval. SO with debug_level 9 I tried to capture but I am see that no log update in sssd_be. Following is sample output from sssd_be log during this time interval. SO it is stuck in save_group ? The group in question here is very large group. 

(2020-12-21 13:33:37:929203): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-21 13:33:38:018075): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-21 13:33:38:018253): [be[mydomain]] [sdap_save_groups] (0x4000): Group 4 members processed!
--
(2020-12-21 14:18:13:015516): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [orig_member] of entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-21 14:18:13:040536): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-21 14:18:13:040796): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 processed!
--
(2020-12-21 14:18:13:102591): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-21 14:18:18:782683): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-21 14:18:18:783006): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-21 14:23:13:013189): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [orig_member] of entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-21 14:23:13:037017): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-21 14:23:13:037266): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 processed!
--
(2020-12-21 14:23:13:098968): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-21 14:23:13:765062): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-21 14:23:13:765277): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-21 15:16:01:151526): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-21 15:16:07:549879): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-21 15:16:07:550229): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-21 16:06:04:098209): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-21 16:06:09:528503): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-21 16:06:09:528813): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!
--
(2020-12-21 16:38:13:017447): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [orig_member] of entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-21 16:38:13:043604): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-21 16:38:13:043817): [be[mydomain]] [sdap_save_groups] (0x4000): Group 2 processed!
--
(2020-12-21 16:38:13:133200): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-21 16:38:16:725696): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-21 16:38:16:726140): [be[mydomain]] [sdap_save_groups] (0x4000): Group 2 members processed!
--
(2020-12-21 17:26:22:880163): [be[mydomain]] [sysdb_ldb_msg_difference] (0x2000): Replaced/extended attr [member] of entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb]
(2020-12-21 17:26:29:093140): [be[mydomain]] [sysdb_set_entry_attr] (0x0200): Entry [name=group1187@mydomain,cn=groups,cn=mydomain,cn=sysdb] has set [cache, ts_cache] attrs.
(2020-12-21 17:26:29:093619): [be[mydomain]] [sdap_save_groups] (0x4000): Group 0 members processed!

Any help is really appreciated. 
Thanks,
Sanjay Agrawal