URL:
https://github.com/SSSD/sssd/pull/5552
Title: #5552: files: split update into batches
elkoniu commented:
"""
Just run another test run to include `batch_size = 1000`.
@sumit-bose I have only one domain in `sssd.conf`:
```
[sssd]
config_file_version = 2
services = nss, pam, ifp, sudo
debug_level = 0x3ff0
domains = files
user = root
[nss]
timeout = 30000
debug_level = 0x3ff0
debug_microseconds = true
[pam]
timeout = 30000
debug_level = 0x3ff0
[sudo]
timeout = 30000
debug_level = 0x3ff0
[ifp]
timeout = 30000
debug_level = 0x3ff0
[domain/files]
timeout = 30000
debug_level = 0x3ff0
debug_microseconds = true
id_provider = files
```
Time needed to mark domain as inconsistent:
|Version|30 + 100 users|30 + 250 users|30 + 500 users|
|:---:|:---:|:---:|:---:|
|SSSD master|204ms|558ms|1168ms|
|SSSD batch_size = 10|0,5ms|0,5ms|70ms|
|SSSD batch_size = 1000|0,4ms|0,5ms|0,6ms|
Result:
* Each variant of batching speeds up marking domain as inconsistent and solves
https://github.com/SSSD/sssd/issues/5091
Time needed to add / remove 100, 250 and 500 users in bash for loop:
|Version|Add 100|Del 100|Add 250|Del 250|Add 500|Del 500|
|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|Raw files|0m 7.463s|0m 7.618s|0m 19.567s|0m 18.589s|0m 40.271s|0m 39.313s|
|SSSD master|0m 58.315s|1m 2.276s|4m 6.709s|4m 2.857s|14m 2.478s|14m 54.913s|
|SSSD batch_size = 10|2m37.513s|2m38.450s|8m19.219s|8m15.790s|23m49.023s|21m52.085s|
|SSSD batch_size = 1000|2m33.065s|2m33.831s|7m43.810s|7m39.830s|20m13.238s|19m9.398s|
![test-result](https://user-images.githubusercontent.com/2312108/123186997-9502b700-d499-11eb-82f4-c4fed789acaf.png)
"""
See the full comment at
https://github.com/SSSD/sssd/pull/5552#issuecomment-867257247