-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 09/10/2010 09:42 AM, Dmitri Pal wrote:
Stephen Gallagher wrote:
> On 09/08/2010 09:04 AM, Stephen Gallagher wrote:
>> I've also been thinking about how we're going to handle processing the
>> nested groups, and I think what I'm going to do is take advantage of
>> some of the nicer features of libcollection.
>
>> Internal processing of setnetgrent_send() will recursively call out a
>> subrequest to setnetgrent_send() again for each of the named nested
>> groups. The setnetgrent_recv() function will return a libcollection
>> object containing all of the results from that request (as well as any
>> additional subrequests called). When the results come back up, they can
>> be added together trivially using the col_add_collection_to_collection()
>> interface with the col_add_mode_clone mode.
>
>
> I've added some additional details about how I would like to do the
> nesting and loop detection to the wiki page. Comments welcome.
>
Sorry I am having trouble understanding this algorithm.
But may be it is because i do not understand the tevent_req interface to
the level needed here.
AFAIU the
struct tevent_req setnetgrent_send(char *netgroupname, hash_table_t
*nesting)
call will ask for the netgroups, while another call
errno_t setnetgrent_recv(tevent_req *req, struct collection **entries)
is the call that will be executed when the response from the server is
received.
No, this is where you are mistaken. The call that will be executed when
the request is finished is specified by the caller right after invoking
setnetgrent_send() by using the tevent_set_callback() function.
This function that is called must then invoke setnetgrent_recv() in
order to read out the final result data that is available.
The problems I have are with the item 4).
If it removes the netgroup from the hash how the hash ever grows?
You're confusing the hash with the result set (which will be a
libcollection object). I'm thinking about changing the way I do the
nested invocation so that the toplevel hides the need for the hash.
The idea behind the hash is actually to have it double as a reference
count and a loop-detection mechanism. It doesn't need to be a hash (it
could just as easily be a b-tree), but since we already have an
efficient hash available, I was just going to use that.
The idea is that for every time we recurse down a level, we will add the
name of that netgroup to the hash. Before recursing down again, we'll
make sure that the new name is not a key in the hash. If it is, we know
we've hit a loop and should break processing.
When we recurse up a level, we need to remove this entry from the
tracking hash so that it's possible to recurse down into it again in a
different branch of the tree. There is a pro and con to this approach.
Pro: We can store the complete result sets of all of the member
netgroups individually, so if they're requested directly or indirectly
again later, we don't have to go back to the server. This is a very
real, tangible advantage.
Con: It does mean that if the same member netgroup appears twice in the
nest that we will return additional copies. This is allowable by the
standard and is more of a configuration bug than anything else, so I
don't think it necessarily makes a lot of sense to try optimizing it
away at this point.
Trying to graph it:
(netgroupA has nested members netgroupB and netgroupC)
netgroupA
|
- ----netgroupB
|
- ----netgroupC
(netgroupB has nested member netgroupD)
netgroupB
|
- ----netgroupD
(netgroupC has nested member netgroupD)
netgroupC
|
- ----netgroupD
In this situation, my result set will ACTUALLY be:
netgroupA
|
- ----netgroupB
| |
| ----netgroupD
|
- ----netgroupC
|
----netgroupD
The result set for netgroupA WILL have two copies of netgroupD
It should grow when the requests and responses (!) are processed in
a
nested way.
But I do not think that this is possible with the interface we have (at
least how it is described).
If the request for a netgroup sent and then the response is received and
we are processing a response and find that the netgroup has nested
netgroups what do we do?
You misunderstood. We're doing the nesting internally and only
responding once all the recursive calls are complete. So when the
callback invokes setnetgrent_recv(), it's going to receive a
libcollection object that is 100% complete.
Am I missing something?
Yes, see above :)
Also there is not design for the
int innetgr(const char *netgroup, const char *host,
const char *user, const char *domain);
Is this intentional or just an omission?
It's intentional. It would be really nice if there was actually an
interface for this, but unfortunately libc internally wraps this by
calling setnetgrent(), looping through getnetgrent() then endgrent() and
then manually searching the result list for the 3-tuple specified.
It would be much more efficient if we could handle it internally, but
it's a terrible interface.
Also I think we should have the following optimization:
Each fetched netgroup goes to the cache with a timestamp. If the
expiration is say 30 sec and there is a netgroup C nested into two
independent netgroups A and B and A is fetched and then B is fetched
before the expiration timeout of the C, then the netgroup C should be
taken from the cache rather than refetched.
We're not talking about the LDB cache here. We're talking about a memory
cache that is automatically reaped by a tevent timer. The on-disk cache
already behaves exactly the way you described above.
The idea behind the memory cache is that we guarantee that for one full
timeout lifetime, the complete results of this top-level netgroup will
remain constant, even if during that time one of the nested groups changed.
Another thing that I just realized is that you create a flat result set
collection by appending nested groups rather than creating a collection
with tree structure and iterating it as a flat collection. While your
approach is probably the right one I wanted to draw attention to the
fact that the option of having a tree style collection with nested
referenced (or copied) subcollections and then traversing the tree as if
it is a flat collection is also available. I do not know if you looked
at such structure and whether it would help better if we need to do some
optimization (now or later). Just something to consider.
I may use the tree style approach. I haven't designed that level of
detail yet. As long as it can be traversed as if it is flat, either
approach is acceptable.
- --
Stephen Gallagher
RHCE 804006346421761
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Fedora -
http://enigmail.mozdev.org/
iEYEARECAAYFAkyOFWUACgkQeiVVYja6o6OU1wCcDxK03pX7ZuS8vL8fA5yz17yo
h78AmwSRpqL+IeimnbekbYduNsDAAOfH
=yDIk
-----END PGP SIGNATURE-----