Hi guys,
while working of my thesis I've notice two features of talloc that we
might use to make sssd faster with only a little effort.
The first one is about decreasing malloc() calls. We should certainly
do some measures here but I don't doubt that malloc() is a big issue in
sssd. We use it almost everywhere + we use it indirectly with talloc
which is about 10% slower (manpage).
Samba team run to this issue as well and therefore they created a new
context type - talloc pool. The documentation says:
"A talloc pool is a pure optimization for specific situations. In the
release process for Samba 3.2 we found out that we had become
considerably slower than Samba 3.0 was. Profiling showed that malloc(3)
was a large CPU consumer in benchmarks. For Samba 3.2 we have
internally converted many static buffers to dynamically allocated ones,
so malloc(3) being beaten more was no surprise. But it made us slower."
The great thing about this is that it is completely transparent. The
context can be created with talloc_pool(ctx, size) and it works this
way:
- if we are allocating data on a pool context, it takes the desirable
amount of space from the pool,
- if the context is a descendant of the pool context, it takes the space
from the pool as well,
- if the pool is out of memory, it creates a new, normal context,
- if we change the parent of a child of talloc pool to a parent that is
outside of this pool, the whole pool memory will not be freed until
the child is freed.
Example:
/* allocate 1KiB in a pool */
TALLOC_CTX *pool_ctx = talloc_pool(NULL, 1024);
/* take 512B from the pool, 512B left */
void *ptr = talloc_size(pool_ctx, 512);
/* 1024B > 512B, this will create new talloc chunk */
void *ptr2 = talloc_size(ptr, 1024);
/* the pool still contains 512 free bytes
* this will take 200B from them */
void *ptr3 = talloc_size(ptr, 200);
/* this will destroy context 'ptr3' but the memory
* is not freed, the available space in the pool
* will increase to 512B */
talloc_free(ptr3);
/* this works as usual :) */
talloc_free(pool_ctx);
Basically we can create large pool context for each responder/dp request
and create temporary context on the pool context or its descendants.
That's all that would need to change.
I will send a description of the other feature in another mail.