Bundles V2

Lukas Krejci lkrejci at redhat.com
Tue Feb 19 15:23:13 UTC 2013


On Monday, February 18, 2013 09:36:07 Jay Shaughnessy wrote:
> Lukas,
> 
> (note: I wrote most of this before the presentation last Friday, so some
> of it is answered or dated.  Just posting it to get the thread moving...
> -Jay)
> 
> Looking at the latest revision of the wiki. The design is progressing
> well.  I'm definitely liking the emphasis on migrating to a
> Bundle-centric approach and the drive for simplifying the deployment and
> storage models while also making things more robust. A couple of
> questions/comments:
> 
> * The move to Repo-based content authz seems like a good idea that will
> map well to the current Group-based resource authz.  And therefore
> should be easily adapted/understood by users (and us).
> 
> * I'm a little confused by the unique constraint on (Package.name,Repo)
> where Package-Repo is ManyToMany.  That would mean I could have
> different distinct Packages with the same name, in different repos.  But
> I could also have the same Package in different Repos.  How will I be
> able to identify a test.war in different repos as being the same app, or
> different apps with the same name?   This does prevent two different
> test.war apps in the same repo, which may make sense, but actually maybe
> it's too restrictive.
> 
> Keeping this ManyToMany is probably a good idea as it is analogous to
> the Resource-Group relationship wrt Repo-based authz.  But then you may
> need to keep and re-purpose Package.classification as something that can
> further identify a Package.  Name may not be enough.  The same is true
> of resources, to identify a resource you can't go by name, the
> "classifier" is ancestry.
> 

I share your concerns about the possible confusion about the difference of a 
single package being in two different repos vs. 2 packages with the same name 
in two repos.

At the same time I felt it necessary to keep the possibility of 1 package in 
more repos for the precise reason you mention - be flexible enough for allowing 
different levels of authz on the same package (through access from different 
repos), I think that is an important feature that we should keep.

On the other hand, the restriction to disallow 2 packages with the same name 
in the same repo is to keep the similarity to an ordinary filesystem.

I think your suggestion to include some kind of possibility for the user to 
distinguish between the packages by some other means than a name is good. A 
description field of sorts (classification on package and a description on a 
bundle, merged into one field perhaps) might be a good candidate for the user 
to put down notes further identifying the particular package. 

I am not convinced about the need for allowing 2 packages with the same name 
in the same repo, but I don't have any strong opinion on it. If you think that 
allowing it would justify the increased potential for confusion then we can 
easily incorporate it. After all, the name is not designed as a primary key, 
but rather only has a unique constraint.

> * +1 to removal of ContentSource, we'll need to work out the details of
> the CSP feature

Definitely. And I think providing some infrastructure for periodically executed 
scripts in the server is a far more versatile and useful feature than trying 
to keep ContentSources around with all the complexity in the model, UI and API 
that they require.

> * +1 to removal of PackageCategory
> * +1 to deprecation/removal of PackageType
> 
> * With respect to the PackageVersion changes:
> **  +1 to removing architecture.  Does this mean we can also kill off
> the rhq_architecture table?
> ** I get why you don't want displayName.  But you may want to make this
> a redundant, immutable field, set to the package name. Not sure, but it
> may be helpful to not have to join with Package to get the name.

Agreed, I think we will see some more candidates for denormalization as we 
will come up with the queries to support the remote API/local SLSB calls.

> ** +1 to removing displayVersion, not sure I see a reason for this, either.
> ** +1 to removing MD5 if we can.   Maybe it was the only digest we could
> get for certain remote content?  I don't know.

We had the ability to download the package bits lazily and the content source 
might have been able to only provide certain kinds of hash for a package (it 
could also provide none). Still I am not sure why we needed 2 kinds of hashes 
on a single package.

With possibility of lazily downloaded bits of a package being stripped, I 
don't see a reason for 2 kinds of hashes on a package at all.

> ** license_name, license_version?  These aren't mentioned, can we get
> rid of them?
> 

I've actually overlooked these :) Looking at their usages, I think these fields 
fall into the same category as many other fields on the PackageVersion. They're 
not used anywhere - the only place that ever set the license name or version 
was the content source synchronization code that we're thankfully getting rid 
of ;)

> * Mime types:
> ** +1 to the approach, I think this will be very good as the underlying
> mechanism for moving the model forward.
> ** I'm not sure the mime-type determination approach can work like it
> does for bundle types. It can only work if only one server plugin can
> determine a mime-type, right?  Is this going to be true?  Maybe it is ,
> I'm just asking.
> 

My initial thoughts were that there could be any number of "mimetype plugins" 
and the first one to identify a package would "win", the same approach as with 
the bundle plugins. If none of the plugins would be able to identify a 
package, the RHQ server itself would provide a fallback mechanism - we'd be 
able to identify the basic types like ZIP, jar (even though jars don't have 
their separate standardized mimetype so we'd probably have to have the 
supported mimetypes as well as the behavior of the default mimetype detector 
heavily documented so that agent plugin authors can rely on the results).

> * With respect to Content discovery, you say: "...The latter point is
> useful for content discovered in the plugins using the Content API (that
> don't end up in any repo), because it will allow for two
> "mywebapp.war:1.0" to coexist in the database, each coming from
> different resource". This content without a Repo, what is it "attached"
> to?  How is it tracked/removed/accessed?
> 

It's attached directly to the resources via the InstalledPackage. You can't 
remove it, nor can you currently (=in the current implementation) "move" to a 
repository.

Actually if we were to keep the agent-side content discovery functionality, 
we'd have to come up with a scenario for this.

My current thinking, not reflected in anything on the wiki yet, is that based 
on the input from the content facet, we'd create bundle deployments on the 
server side that would not be attached to any repo and would only be 
accessible from the resource in the UI. But otherwise these guys would be full 
featured bundles. I haven't worked out yet the implications in the authz.

> * Bundle Consequences:
> * We need better terminology for "normal bundle" and "content bundle".
> I don't know what it should be but it should likely reflect that some
> bundles are
> file-system/generic/bundle-distribution-file-zip/bundle-plugin-deployed
> and others are [mime-]typed/resource-type-plugin-deployed.
> 

Yes, we do, and I struggle to make the distinction clear :) On one hand we try 
to make them look the same, yet try to keep them as separate concepts :)

Maybe "bundle" and "recipe-less bundle"? But then there might be bundle 
plugins that wouldn't require a recipe.

Or maybe "free-form bundle" and "managed bundle"? Free-form to emphasize the 
ability of the user to modify the behavior via recipes while "managed" would 
emphasize that RHQ somehow takes care of everything about them.

> ** I think I prefer just adding to the Configuration wrapped by the
> existing ResourceTypeBundleConfiguration to absorb the <content> stuff.
> As opposed to introducing BundleDestinationDefinition.  It already
> exists and Configuration is super-flexible for making additions.  Or,
> perhaps introduce a BundleDestinationDefinition pojo that can be
> retrieved via ResourceTypeBundleConfiguration (like
> getDestinationDefinitions()).
> 

I think that will be possible. Having the BundleDestinationDefinition as a 
separate entity helped me while I was iterating over the design myself. At one 
point I needed to store a configuration definition in it, for which a simple 
configuration object seemed like a really bad choice ;) But in the end the 
design doesn't need that anymore.

> ** The diagram indicated that the Bundle name is derived from package,
> is that right?
> 

Yes, but again that's not something I have a strong opinion about. In fact, it 
may only be my oversight - I think the bundle name is read from the recipe, 
isn't it? In that case, we might just default the bundle name to the package 
name during the creation if no other name is provided from the bundle plugin 
but keep them separate.

> ** Overall I'm still absorbing the proposed changes here. The loss of
> BundleType for a Bundle is something that makes sense given that the
> Bundle will already have a mime-type.
> 

Not only that. If we want to keep the BundleType around to carry the 
information it had - i.e. what resource type is responsible for deploying and 
(somewhat newly) the configuration definition needed for deployment (this was 
previously implied but we're making it explicit to be able to handle 
<content>) we'd not be able to have the BundleType related to the Bundle 
anymore, as the decision what bundle type to use to deploy certain bundle 
would only be made at the time of the creation of bundle destination.

> * Can we simplify Repo stuff?  Do we really need a Repo-Resource
> relationship?
> 

We don't need it.


> * Can we also eliminate all of the Advisory stuff?

Oh yes! :) Consider anything missing from the design as being ripped (unless 
we find I overlooked it or that we need to bring it back somehow) :)



More information about the rhq-devel mailing list