[PATCH] dynamic providers

Jim Meyering jim at meyering.net
Mon Dec 20 18:23:12 UTC 2010


Jeff Darcy wrote:
> On 12/18/2010 05:26 AM, Jim Meyering wrote:
>>> I don't think the code uses any of the provider
>>> fields after a back-end request is initiated, so we might not need to
>>> wait for those to complete, but
>>>
>>> 	(a) trying to guarantee that for every library to which we pass those
>>> 	fields, forever, is questionable from a maintenance POV;
>>>
>>> 	(b) that still leaves us vulnerable to initiation-phase conflicts with
>>> 	other threads;
>>>
>>> 	(c) queued replication jobs will make that even more complicated.
>>
>> Are you arguing that ref-counting cannot work at all, or merely
>> that some hypothetical partial implementation would not be sustainable?
>
> I think ref-counting can work.  The above points are my attempt to
> highlight some of the concerns that might affect choice of one
> alternative over another.  The idea of dropping refcounts immediately
> after request initation is pretty tempting, as it allows objects to be
> freed earlier and some cleanup might be more convenient from the
> original context than when it completes, but the issues above might
> make such an implementation more trouble than it's worth.
>
> It also occurs to me that there are other issues to be considered wrt
> provider changes and pending replication requests.  When a replication
> request gets to the front of the queue, it's very nice if its provider
> object is still there, but what if that object has been modified in a
> way relevant to the original replication decision?  Should we
> re-evaluate the replication request in the context of the current
> policy and provider fields?  Should we re-evaluate all replication
> decisions for that object, in the context of the entire current
> provider table? If credentials changed, which should we use?  Can we
> assume that deletion of a provider followed by addition of another
> with the same name is equivalent to modification in place?

I had been assuming that with ref-counting, when a provider
is "in use" (i.e., in queue), iwhd would prohibit any attempt
to modify or delete it.

In fact, with dynamic creation and deletion,
I see no need to allow editing an existing provider.
Rather, a client desiring that behavior would remove
the provider (if possible), and recreate it with the
modified parameters.  There are few enough parameters
that the convenience of allowing editing doesn't seem
to be great enough to outweigh the disadvantage of
maintaining the additional (not strictly necessary) interface.

Does anyone think it's worthwhile to retain
the "edit provider parameters" interface?

>> I have been considering alternative approaches that would be
>> easier to code and maintain:
>>
>>    - allow only addition of providers, not deletion, but with some
>>        reasonably large static limit.  This is probably a non-starter,
>>        since even ignoring malicious abuse[*], an erroneous "client"
>>        script could easily send requests that would fill the table
>>        to the maximum.
>
> This would also be a problem with dynamic allocation, as long as
> there's an enforced maximum number of providers . . . and without such
> a limit a malicious or stupid script could cause us to run out of
> memory instead of provider slots.
>
>>    - use garbage collection rather than explicit "free" calls.
>>        Is there even a good multi-thread aware garbage collector for C?
>
> The Boehm-GC page (http://www.hpl.hp.com/personal/Hans_Boehm/gc/)
> refers to a parallel version developed at UTokyo, but I have no idea
> how good it is.  Is it worth trying out bdwgc itself?

It looks like Boehm's GC code works fine in a multi-threaded
application, so I will try it.

Not having to worry about explicitly freeing memory seems like
it will be quite liberating:

  1) no need to ref-count; gc takes care of freeing as needed.

  2) assuming write-once (no editing), there is no need to worry about
     which provider is "in use" or "deleted."  When deleting one,
     simply remove its name from the table.  Any existing users still
     have a pointer to the actual data.


More information about the iwhd-devel mailing list