IWHD API Suggestions

Jim Meyering jim at meyering.net
Fri Sep 9 17:59:49 UTC 2011


Matt Wagner wrote:
> On Fri, Sep 09, 2011 at 06:56:34PM +0200, Jim Meyering wrote:
>> Maybe you already do this?
>>
>> At least with the current architecture, I would suggest that you read as
>> much as you need initially to construct the graph of relationships.  While
>> doing that, you can add back-links, as needed, so that when you iterate
>> through UUIDs, each query merely examines your in-memory representation
>> and you get results in O(1) time per UUID.
>
> I've discussed doing this as a work-around in the interim. But this
> leaves us mirroring the entirety of images, builds, target images, and
> provider images in our app. (Well, the metadata for each, not the actual
> image files, of course.)
>
> But this pretty quickly grows into 50+ API calls to populate the above.
> Doing that once is tolerable, but the problem is that this information
> can't really be cached, so we need to do this for every request, which
> basically negates the benefit of trying to read it all on our end in the
> first place. Is there any sort of optimization to this that you see?

Before diving into this rabbit hole, I have to ask "Why?"
Is performance somehow so poor that parsing the result of 50 "GET"s is
causing trouble?

The alternatives really are not trivial.

>> Then, if the read-all phase is still found to be too slow, say, due to the
>> number of queries required, we can think about adding an API to provide
>> larger batches of information, rather than imposing application-specific
>> structure on iwhd.
>
> I don't understand how this is application-specific. A build can show me
> what image it is for. There's no way to provide that functionality in
> reverse and have an image show me what builds it has?

Not without compromising a fundamental design attribute:

  iwhd is an object store

It does not know which objects correspond to images, builds, target
images, etc.  All it has is semantic-free "objects", each in some
bucket, and possibly with its own set of attributes.

> It seems like this could be broadly useful.

We talked this to death a few months ago.
The consensus was that the easiest way to provide the functionality
would be to ensure that all changes to iwhd go through a layer like
Michal's graph-augmenting code.  Then, each addition automatically
adds the back-links that you'd like.  That layer (and only it) knows
about the semantics of the objects, so it can provide the required
back-links.


More information about the iwhd-devel mailing list