On 9/8/2011 8:15 PM, John Sanda wrote:
Let me try to provide some clarification around terminology based on
what exists in the code base today.
* change set
A collection of changes where each change in the collection corresponds
to a particular file. This is somewhat analogous to a commit in a
version control system. Consider an initial commit for a file. With
respect to the VCS, the only change represented by that commit is the
addition of the file to version control. Each change set has an
incrementing version associated with it, similar to say revision numbers
in SVN.
* coverage change set
The initial change set that the agent generates is called a coverage
change set. This is to let the server know that we are simply adding
files for drift monitoring. A coverage change set includes all files
that are being monitored for drift. It is in effect a snapshot.
[jshaughn] Perhaps
we should just drop the term "coverage" and formally
call this the "Initial Changeset". Originally a configuration could
have more than one coverage changeset but that's no longer true. So
perhaps initial changeset is all we need.
* drift change set
A drift change set contains only a set of deltas. Unlike a coverage
change set, it includes only those files that have changed.
* snapshot
A snapshot is the set of all files being monitored for drift at a
particular point in time. Discrete points in time are determined by
change set versions that are created and added to the system.
Some additional thoughts...
While a coverage change set is really a snapshot, it might be somewhat
confusing from an implementation point of view to call it that because
both the coverage change set and drift change set share the same file
format. And I guess to be precise a coverage change set is a specific
type of snapshot in that it includes only version whereas a snapshot can
include any number of change sets.
Given what I have previously stated, I think it is accurate to called a
pinned change set a snapshot; however, I think pinned change set is more
intuitive at this stage since we have been using the term coverage
change set.
[jshaughn] I think changesets are more tangible and more in line with
what a user deals with in the UI. Each changeset represents, or is
backed by, a snapshot, but snapshots are an aggregation, and calculated
on the fly. As long as it's easy for a user to see the snapshot for a
changeset (the aggregation of the initial changeset and following
deltas, I think working with, and pinning changesets is the way to go.
I hope this helps more than it confuses.
- John
On 9/8/11 6:21 PM, Charles Crouch wrote:
> Moving the conversation Jay and John were having to mail...
>
> Thanks
> Charles
>
>
>
> jshaughn writes:
> We've been talking a lot about the "Golden Image" but we haven't
really been formal about its definition or about what we'll actually be able to do.
So this is to get things flowing...
>
>> From what we've done so far it seems to me that a "golden image"
is a specific changeset for a specific drift configuration. Since a drift configuration
monitors immutable directories we don't have to worry about it changing and
invalidating an associated golden image.
> I'd like to propose that we change the terminology at this point to be in line
with our current drift vocabulary. Instead of golden image I propose "Pinned
Changeset" (other options: "Trusted", "Fixed",
"Blessed", "Golden"). I like "Pinned" because pinning is a
well-known term, pinned, pin, pinning, pinnable are all available when describing the
feature, and it has easy visualization for icons, etc. Also, were going to need one of
those other adjectives later...
> [ccrouch: So just to be clear, a "Pinned Changeset" contains knowledge of
the total set of files being monitored at the time the changeset was pinned? i.e. using
terminology from
here:http://rhq-project.org/display/RHQ/Drift+Management#DriftManagement-... its
really a Snapshot?]
[jshaughn] Hopefully the prepended terminology atthe top makes
this more
clear. A user would pin the changeset because that's what he's working
with/looking at in the GUI. But yes, each changeset represents a
snapshot and that is the set of files to compare against.
So, this begs the question, do we need the term 'changeset' or is it
confusing things? Should we just use the term 'Snapshot' and maybe
provide two snapshot views: Full and Delta? So in the UI we'd deal with
a Snapshot subtab, and the tree would show Drift Configuration nodes
with Snapshot children. By default you'd see the delta fileset for the
Snapshot (today's changeset) but could also easily get the full fileset
view. This would mainly be a presentation level change as the backing
mechanisms would probablty reman mainly the same.
The fallout of this would in fact be that we would pin Snapshots, which
may be more intuitive, I'm not sure.
>
> [ccrouch: Sidebar: I think we need to
updatehttp://rhq-project.org/display/RHQ/Drift+Management#DriftManagement... and
remove the term "Change Set" or indicate its an abstract concept. IIUC the only
two related things we can have in the system right now are "Coverage Change Set"
and "Drift Change Set"?
>
> [jsanda: I'm down with pinned. In fact, IntelliJ uses the same thing with some
views. You can pin or unpin certain windows.]
>
> Today our drift configurations are all "Resource Drift Configurations
(RDC)". Meaning they are only specific to a single resource. And at the moment we
don't support pinned changesets. By default we apply a "rolling
changeset", meaning we look for drift compared with the most recent snapshot, the
snapshot being the initial changeset plus all changes since that time. If we were to
allow pinning it would mean the user could select a changeset and pin it for the
configuration. And after that point all drift detection for that configuration would be
against that changeset. With that in mind, we could allow a new option on a
configuration: "pin initial changeset", which would mean that the initial
changeset (version 0) is immediately pinned and variations from that changeset would
trigger drift entries. In this way reverting back to a pinned version of a file (or
removal of an unwanted file) would not generate any more drift entries.
> [ccrouch:what's so important about pinning the initial changeset? why
wouldn't you want to pin an arbitrary changeset?]
[jshaughn] Maybe that was just
an unneeded convenience. I suggest we
forget about that potential option for now, strike it from the record.
>
> Before we talk about group-level or type-level pinning we may want to make a
fundamental decision. Do pinned changesets always originate from an existing resource,
or can they be defined external to any existing resource?
> [ccrouch:If this is a simplifying assumption then it seems reasonable for now]
>
> Although a drift configuration could be treated much like an alert def or metric
schedule, a pinned changeset is not like those things. Because it actually involves the
results of processing the configuration. To continue the analogy, we can build templates
for an alert def but not for resulting alerts. Here we need to find a way to easily
templatize the actual file-set resulting from a configuration.
>
> To a degree we have this already if we do make the assumption that a pinned
changeset can be leveraged only from a trusted resource that has processed a desired
configuration. Since the mechanism we would have to build to load a globally accessible
trusted changeset would provide what you get from a resource already, I suggest we do
require that it come from a trusted resource. I can't think of a reason why we would
need a standalone tool to capture a trusted changeset as opposed to using an inventoried
resource to do it. Having said that, we could feasibly provide such a standalone tool.
But it may be tricky, the configuration would still need to be defined in order to
generate the initial changeset.
>
> [jsanda: One reason I can think of for providing a tool to capture a trusted
change set is to better integrate with other management tools. A lot of folks for
example use puppet and chef for configuration management. A trusted change set might
already exist in some form as part of a puppet configuration. An external tool, which
could be some rich CLI apis, could better facilitate extracting pulling that
information out of the other system and getting it into RHQ. The tool could also
facilitate and require the creation of the configuration. ]
> [cccrouch: agreed, but I think this could be a useful extension, not an initial
requirement]
>
> So, for now, I assume that a pinned changeset will come from a trusted, inventoried,
resource. And this is where things get interesting :0)
>
> After thinking on this for a bit I think we need to introduce the "Golden
Configuration" (other options: "Trusted", "Blessed",
"[Drift] Configuration Template"). Not completely dissimilar to say, an
alert-def template because it would be type based and could apply across inventory. But
not exactly the same because it would optionally not be applied by default. Whether
we'd want to work this into the template stuff or handle it independently I don't
know. I like the term "Golden" here because people have already been using the
term, seem to like it, and understand what it represents. The key thing to understand is
that a golden configuration comes with a pinned changeset. So when adding a golden
configuration to a resource, the initial changeset (version 0) is provided and pinned.
>
> The idea is this, that a configuration for an RDC (i.e. a changeset for some
resource) could be promoted to a golden configuration. At which point it would be copied
(the drift config, changeset (pinned or most recent changeset), and drift entries) and
flagged for use globally.
>
> [ccrouch: I'm confused by the terminology here. WRT "changeset (pinned or
most recent changeset), and drift entries)" do you mean "Coverage Change
Set", "Drift Change Set", "Snapshot" or something else? Can we
make sure we are only using terms we are defining in this mail or are already
defined:http://rhq-project.org/display/RHQ/Drift+Management#DriftManageme... ]
[jshaughn] A golden configuration is really the combination of a
configuration and the file set that we want to see for that config. The
file set is represented by a changeset. When promoting a resource's
configuration to "golden", we need to know which changeset is going to
define the file set. "pinned or most recent" means that if the user has
a pinned changeset for that configuration then that will be the
changeset used, otherwise, the most recent changeset will be used.
If we were to remove the changeset terminology as proposed above, the
previous would read the following way, is it more clear? :
A golden configuration is really the combination of a configuration and
the file set that we want to see for that config. The file set is a
snapshot. When promoting a resource's configuration to "golden", we
need to know which snapshot is going to define the file set. "pinned or
most recent" means that if the user has a pinned snapshot for that
configuration then that will be the snapshot used, otherwise, the most
recent snapshot will be used.
Continuing in this vein, does "Golden Snapshot" sound better to anyone?
That emphasizes the file set over the configuration (the configuration
would still be associated, of course).
>
> Then, at the resource (or eventually, the group) level, when adding a drift
configuration, one could optionally select from the available golden configurations for
the type. Whether the golden changeset is copied for each resource, or referenced,
I'm not sure. A copy would maybe add flexibility and maybe make coding easier, but
copying all of the drift entries may be cumbersome for lots of resources. Maybe just
copying the changeset would work, leaving only one copy of all of the drift entries.
Also, there are plugin considerations to work out. The concept of trusted changesets
would be RHQ specific, but plugins actually store changesets and drift entries, so there
is that to finagle.
>
> [jsanda: Instinctively, I like storing references instead of copies for the same
reasons I like that we store references to content. I think it will better enable us
to do drift management on a larger scale. If I purge resource from inventory, I do
have to remove all of the drift entries for the trusted configuration. I only have to
remove a single reference. Likewise with assigning a trusted config to a resource. I
am just adding a single reference as opposed to copying lots of drift entries. ]
>
> If we wanted to align golden configurations like templates, we could add an option
to apply it to every member of a type on import, like today.
>
> In the GUI I think this would require either a top menu Drift item or be worked into
Administration. The former seems the most flexible, but the latter may be less
cumbersome and may play better with the idea of templates.
>
> As for [compatible] resource groups, I think perhaps the idea of golden
configurations may work well. Associating a golden config across the group members
makes, I think, more sense than having a lot of resource specific configurations.
>
> [ccrouch: I think have a single read-only golden configuration across multiple
resources, is a good simplifying assumption]
[jshaughn] I agree that as much sharing
as possible will be optimal, the
question is only what we need to be able to provide the necessary
features and gui views. I think we absolutely have to find a way not to
duplicate the actual file set.
>
> We will, I think in any pinned changeset impl, need to be able to supply to the
agent the pinned changeset file. This may need to be reverse-engineered from the drift
entries. We can probably make this such that the agent performs all drift detection and
the server is relieved of that need. But we also need to consider the implications of
sending perhaps large changeset files to the agent (along with the new configuration).
My initial impression is that it's not a big deal because it will be roughly
equivalent to the initial change set that *would have* been generated on the agent. With
this seeded the agent will need send only drift changesets. So, I'm optimistic about
the server not having to perform detection. I think that will simplify the mechanism and
keep the UI representation of changesets more clear (i.e. no "adding" drift
entries to a changeset, or adding server-generated changesets).
>
> [jsanda: I think it makes sense to store the pinned change set on the agent for a
couple reasons. Suppose I have 10 EAP servers on a box and am doing drift management
on all of them. The agent can store a single copy of the pinned change set for all of
those EAP servers, assuming that they are using the same trusted configuration. This
will also inventory sync easier/faster. The agent won't need to download 10
separate snapshots from the server, just the one.]
> _______________________________________________
> rhq-devel mailing list
> rhq-devel(a)lists.fedorahosted.org
>
https://fedorahosted.org/mailman/listinfo/rhq-devel
_______________________________________________
rhq-devel mailing list
rhq-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/rhq-devel