On Monday, February 18, 2013 09:36:07 Jay Shaughnessy wrote:
Lukas,
(note: I wrote most of this before the presentation last Friday, so some
of it is answered or dated. Just posting it to get the thread moving...
-Jay)
Looking at the latest revision of the wiki. The design is progressing
well. I'm definitely liking the emphasis on migrating to a
Bundle-centric approach and the drive for simplifying the deployment and
storage models while also making things more robust. A couple of
questions/comments:
* The move to Repo-based content authz seems like a good idea that will
map well to the current Group-based resource authz. And therefore
should be easily adapted/understood by users (and us).
* I'm a little confused by the unique constraint on (Package.name,Repo)
where Package-Repo is ManyToMany. That would mean I could have
different distinct Packages with the same name, in different repos. But
I could also have the same Package in different Repos. How will I be
able to identify a test.war in different repos as being the same app, or
different apps with the same name? This does prevent two different
test.war apps in the same repo, which may make sense, but actually maybe
it's too restrictive.
Keeping this ManyToMany is probably a good idea as it is analogous to
the Resource-Group relationship wrt Repo-based authz. But then you may
need to keep and re-purpose Package.classification as something that can
further identify a Package. Name may not be enough. The same is true
of resources, to identify a resource you can't go by name, the
"classifier" is ancestry.
I share your concerns about the possible confusion about the difference of a
single package being in two different repos vs. 2 packages with the same name
in two repos.
At the same time I felt it necessary to keep the possibility of 1 package in
more repos for the precise reason you mention - be flexible enough for allowing
different levels of authz on the same package (through access from different
repos), I think that is an important feature that we should keep.
On the other hand, the restriction to disallow 2 packages with the same name
in the same repo is to keep the similarity to an ordinary filesystem.
I think your suggestion to include some kind of possibility for the user to
distinguish between the packages by some other means than a name is good. A
description field of sorts (classification on package and a description on a
bundle, merged into one field perhaps) might be a good candidate for the user
to put down notes further identifying the particular package.
I am not convinced about the need for allowing 2 packages with the same name
in the same repo, but I don't have any strong opinion on it. If you think that
allowing it would justify the increased potential for confusion then we can
easily incorporate it. After all, the name is not designed as a primary key,
but rather only has a unique constraint.
* +1 to removal of ContentSource, we'll need to work out the
details of
the CSP feature
Definitely. And I think providing some infrastructure for periodically executed
scripts in the server is a far more versatile and useful feature than trying
to keep ContentSources around with all the complexity in the model, UI and API
that they require.
* +1 to removal of PackageCategory
* +1 to deprecation/removal of PackageType
* With respect to the PackageVersion changes:
** +1 to removing architecture. Does this mean we can also kill off
the rhq_architecture table?
** I get why you don't want displayName. But you may want to make this
a redundant, immutable field, set to the package name. Not sure, but it
may be helpful to not have to join with Package to get the name.
Agreed, I think we will see some more candidates for denormalization as we
will come up with the queries to support the remote API/local SLSB calls.
** +1 to removing displayVersion, not sure I see a reason for this,
either.
** +1 to removing MD5 if we can. Maybe it was the only digest we could
get for certain remote content? I don't know.
We had the ability to download the package bits lazily and the content source
might have been able to only provide certain kinds of hash for a package (it
could also provide none). Still I am not sure why we needed 2 kinds of hashes
on a single package.
With possibility of lazily downloaded bits of a package being stripped, I
don't see a reason for 2 kinds of hashes on a package at all.
** license_name, license_version? These aren't mentioned, can we
get
rid of them?
I've actually overlooked these :) Looking at their usages, I think these fields
fall into the same category as many other fields on the PackageVersion. They're
not used anywhere - the only place that ever set the license name or version
was the content source synchronization code that we're thankfully getting rid
of ;)
* Mime types:
** +1 to the approach, I think this will be very good as the underlying
mechanism for moving the model forward.
** I'm not sure the mime-type determination approach can work like it
does for bundle types. It can only work if only one server plugin can
determine a mime-type, right? Is this going to be true? Maybe it is ,
I'm just asking.
My initial thoughts were that there could be any number of "mimetype plugins"
and the first one to identify a package would "win", the same approach as with
the bundle plugins. If none of the plugins would be able to identify a
package, the RHQ server itself would provide a fallback mechanism - we'd be
able to identify the basic types like ZIP, jar (even though jars don't have
their separate standardized mimetype so we'd probably have to have the
supported mimetypes as well as the behavior of the default mimetype detector
heavily documented so that agent plugin authors can rely on the results).
* With respect to Content discovery, you say: "...The latter
point is
useful for content discovered in the plugins using the Content API (that
don't end up in any repo), because it will allow for two
"mywebapp.war:1.0" to coexist in the database, each coming from
different resource". This content without a Repo, what is it "attached"
to? How is it tracked/removed/accessed?
It's attached directly to the resources via the InstalledPackage. You can't
remove it, nor can you currently (=in the current implementation) "move" to a
repository.
Actually if we were to keep the agent-side content discovery functionality,
we'd have to come up with a scenario for this.
My current thinking, not reflected in anything on the wiki yet, is that based
on the input from the content facet, we'd create bundle deployments on the
server side that would not be attached to any repo and would only be
accessible from the resource in the UI. But otherwise these guys would be full
featured bundles. I haven't worked out yet the implications in the authz.
* Bundle Consequences:
* We need better terminology for "normal bundle" and "content
bundle".
I don't know what it should be but it should likely reflect that some
bundles are
file-system/generic/bundle-distribution-file-zip/bundle-plugin-deployed
and others are [mime-]typed/resource-type-plugin-deployed.
Yes, we do, and I struggle to make the distinction clear :) On one hand we try
to make them look the same, yet try to keep them as separate concepts :)
Maybe "bundle" and "recipe-less bundle"? But then there might be
bundle
plugins that wouldn't require a recipe.
Or maybe "free-form bundle" and "managed bundle"? Free-form to
emphasize the
ability of the user to modify the behavior via recipes while "managed" would
emphasize that RHQ somehow takes care of everything about them.
** I think I prefer just adding to the Configuration wrapped by the
existing ResourceTypeBundleConfiguration to absorb the <content> stuff.
As opposed to introducing BundleDestinationDefinition. It already
exists and Configuration is super-flexible for making additions. Or,
perhaps introduce a BundleDestinationDefinition pojo that can be
retrieved via ResourceTypeBundleConfiguration (like
getDestinationDefinitions()).
I think that will be possible. Having the BundleDestinationDefinition as a
separate entity helped me while I was iterating over the design myself. At one
point I needed to store a configuration definition in it, for which a simple
configuration object seemed like a really bad choice ;) But in the end the
design doesn't need that anymore.
** The diagram indicated that the Bundle name is derived from
package,
is that right?
Yes, but again that's not something I have a strong opinion about. In fact, it
may only be my oversight - I think the bundle name is read from the recipe,
isn't it? In that case, we might just default the bundle name to the package
name during the creation if no other name is provided from the bundle plugin
but keep them separate.
** Overall I'm still absorbing the proposed changes here. The
loss of
BundleType for a Bundle is something that makes sense given that the
Bundle will already have a mime-type.
Not only that. If we want to keep the BundleType around to carry the
information it had - i.e. what resource type is responsible for deploying and
(somewhat newly) the configuration definition needed for deployment (this was
previously implied but we're making it explicit to be able to handle
<content>) we'd not be able to have the BundleType related to the Bundle
anymore, as the decision what bundle type to use to deploy certain bundle
would only be made at the time of the creation of bundle destination.
* Can we simplify Repo stuff? Do we really need a Repo-Resource
relationship?
We don't need it.
* Can we also eliminate all of the Advisory stuff?
Oh yes! :) Consider anything missing from the design as being ripped (unless
we find I overlooked it or that we need to bring it back somehow) :)