drift: Golden Images
John Sanda
jsanda at redhat.com
Fri Sep 9 00:15:59 UTC 2011
Let me try to provide some clarification around terminology based on
what exists in the code base today.
* change set
A collection of changes where each change in the collection corresponds
to a particular file. This is somewhat analogous to a commit in a
version control system. Consider an initial commit for a file. With
respect to the VCS, the only change represented by that commit is the
addition of the file to version control. Each change set has an
incrementing version associated with it, similar to say revision numbers
in SVN.
* coverage change set
The initial change set that the agent generates is called a coverage
change set. This is to let the server know that we are simply adding
files for drift monitoring. A coverage change set includes all files
that are being monitored for drift. It is in effect a snapshot.
* drift change set
A drift change set contains only a set of deltas. Unlike a coverage
change set, it includes only those files that have changed.
* snapshot
A snapshot is the set of all files being monitored for drift at a
particular point in time. Discrete points in time are determined by
change set versions that are created and added to the system.
Some additional thoughts...
While a coverage change set is really a snapshot, it might be somewhat
confusing from an implementation point of view to call it that because
both the coverage change set and drift change set share the same file
format. And I guess to be precise a coverage change set is a specific
type of snapshot in that it includes only version whereas a snapshot can
include any number of change sets.
Given what I have previously stated, I think it is accurate to called a
pinned change set a snapshot; however, I think pinned change set is more
intuitive at this stage since we have been using the term coverage
change set.
I hope this helps more than it confuses.
- John
On 9/8/11 6:21 PM, Charles Crouch wrote:
> Moving the conversation Jay and John were having to mail...
>
> Thanks
> Charles
>
>
>
> jshaughn writes:
> We've been talking a lot about the "Golden Image" but we haven't really been formal about its definition or about what we'll actually be able to do. So this is to get things flowing...
>
> > From what we've done so far it seems to me that a "golden image" is a specific changeset for a specific drift configuration. Since a drift configuration monitors immutable directories we don't have to worry about it changing and invalidating an associated golden image.
>
> I'd like to propose that we change the terminology at this point to be in line with our current drift vocabulary. Instead of golden image I propose "Pinned Changeset" (other options: "Trusted", "Fixed", "Blessed", "Golden"). I like "Pinned" because pinning is a well-known term, pinned, pin, pinning, pinnable are all available when describing the feature, and it has easy visualization for icons, etc. Also, were going to need one of those other adjectives later...
> [ccrouch: So just to be clear, a "Pinned Changeset" contains knowledge of the total set of files being monitored at the time the changeset was pinned? i.e. using terminology from here: http://rhq-project.org/display/RHQ/Drift+Management#DriftManagement-Terminology its really a Snapshot?]
>
> [ccrouch: Sidebar: I think we need to update http://rhq-project.org/display/RHQ/Drift+Management#DriftManagement-Terminology and remove the term "Change Set" or indicate its an abstract concept. IIUC the only two related things we can have in the system right now are "Coverage Change Set" and "Drift Change Set"?
>
> [jsanda: I'm down with pinned. In fact, IntelliJ uses the same thing with some views. You can pin or unpin certain windows.]
>
> Today our drift configurations are all "Resource Drift Configurations (RDC)". Meaning they are only specific to a single resource. And at the moment we don't support pinned changesets. By default we apply a "rolling changeset", meaning we look for drift compared with the most recent snapshot, the snapshot being the initial changeset plus all changes since that time. If we were to allow pinning it would mean the user could select a changeset and pin it for the configuration. And after that point all drift detection for that configuration would be against that changeset. With that in mind, we could allow a new option on a configuration: "pin initial changeset", which would mean that the initial changeset (version 0) is immediately pinned and variations from that changeset would trigger drift entries. In this way reverting back to a pinned version of a file (or removal of an unwanted file) would not generate any more drift entries.
> [ccrouch:what's so important about pinning the initial changeset? why wouldn't you want to pin an arbitrary changeset?]
>
> Before we talk about group-level or type-level pinning we may want to make a fundamental decision. Do pinned changesets always originate from an existing resource, or can they be defined external to any existing resource?
> [ccrouch:If this is a simplifying assumption then it seems reasonable for now]
>
> Although a drift configuration could be treated much like an alert def or metric schedule, a pinned changeset is not like those things. Because it actually involves the results of processing the configuration. To continue the analogy, we can build templates for an alert def but not for resulting alerts. Here we need to find a way to easily templatize the actual file-set resulting from a configuration.
>
> To a degree we have this already if we do make the assumption that a pinned changeset can be leveraged only from a trusted resource that has processed a desired configuration. Since the mechanism we would have to build to load a globally accessible trusted changeset would provide what you get from a resource already, I suggest we do require that it come from a trusted resource. I can't think of a reason why we would need a standalone tool to capture a trusted changeset as opposed to using an inventoried resource to do it. Having said that, we could feasibly provide such a standalone tool. But it may be tricky, the configuration would still need to be defined in order to generate the initial changeset.
>
> [jsanda: One reason I can think of for providing a tool to capture a trusted change set is to better integrate with other management tools. A lot of folks for example use puppet and chef for configuration management. A trusted change set might already exist in some form as part of a puppet configuration. An external tool, which could be some rich CLI apis, could better facilitate extracting pulling that information out of the other system and getting it into RHQ. The tool could also facilitate and require the creation of the configuration. ]
> [cccrouch: agreed, but I think this could be a useful extension, not an initial requirement]
>
> So, for now, I assume that a pinned changeset will come from a trusted, inventoried, resource. And this is where things get interesting :0)
>
> After thinking on this for a bit I think we need to introduce the "Golden Configuration" (other options: "Trusted", "Blessed", "[Drift] Configuration Template"). Not completely dissimilar to say, an alert-def template because it would be type based and could apply across inventory. But not exactly the same because it would optionally not be applied by default. Whether we'd want to work this into the template stuff or handle it independently I don't know. I like the term "Golden" here because people have already been using the term, seem to like it, and understand what it represents. The key thing to understand is that a golden changeset comes with a pinned changeset. So when adding a golden configuration to a resource, the initial changeset (version 0) is provided and pinned.
>
> The idea is this, that a configuration for an RDC (i.e. a changeset for some resource) could be promoted to a golden configuration. At which point it would be copied (the drift config, changeset (pinned or most recent changeset), and drift entries) and flagged for use globally.
>
> [ccrouch: I'm confused by the terminology here. WRT "changeset (pinned or most recent changeset), and drift entries)" do you mean "Coverage Change Set", "Drift Change Set", "Snapshot" or something else? Can we make sure we are only using terms we are defining in this mail or are already defined: http://rhq-project.org/display/RHQ/Drift+Management#DriftManagement-Terminology ]
>
> Then, at the resource (or eventually, the group) level, when adding a drift configuration, one could optionally select from the available golden configurations for the type. Whether the golden changeset is copied for each resource, or referenced, I'm not sure. A copy would maybe add flexibility and maybe make coding easier, but copying all of the drift entries may be cumbersome for lots of resources. Maybe just copying the changeset would work, leaving only one copy of all of the drift entries. Also, there are plugin considerations to work out. The concept of trusted changesets would be RHQ specific, but plugins actually store changesets and drift entries, so there is that to finagle.
>
> [jsanda: Instinctively, I like storing references instead of copies for the same reasons I like that we store references to content. I think it will better enable us to do drift management on a larger scale. If I purge resource from inventory, I do have to remove all of the drift entries for the trusted configuration. I only have to remove a single reference. Likewise with assigning a trusted config to a resource. I am just adding a single reference as opposed to copying lots of drift entries. ]
>
> If we wanted to align golden configurations like templates, we could add an option to apply it to every member of a type on import, like today.
>
> In the GUI I think this would require either a top menu Drift item or be worked into Administration. The former seems the most flexible, but the latter may be less cumbersome and may play better with the idea of templates.
>
> As for [compatible] resource groups, I think perhaps the idea of golden configurations may work well. Associating a golden config across the group members makes, I think, more sense than having a lot of resource specific configurations.
>
> [ccrouch: I think have a single read-only golden configuration across multiple resources, is a good simplifying assumption]
>
> We will, I think in any pinned changeset impl, need to be able to supply to the agent the pinned changeset file. This may need to be reverse-engineered from the drift entries. We can probably make this such that the agent performs all drift detection and the server is relieved of that need. But we also need to consider the implications of sending perhaps large changeset files to the agent (along with the new configuration). My initial impression is that it's not a big deal because it will be roughly equivalent to the initial change set that *would have* been generated on the agent. With this seeded the agent will need send only drift changesets. So, I'm optimistic about the server not having to perform detection. I think that will simplify the mechanism and keep the UI representation of changesets more clear (i.e. no "adding" drift entries to a changeset, or adding server-generated changesets).
>
> [jsanda: I think it makes sense to store the pinned change set on the agent for a couple reasons. Suppose I have 10 EAP servers on a box and am doing drift management on all of them. The agent can store a single copy of the pinned change set for all of those EAP servers, assuming that they are using the same trusted configuration. This will also inventory sync easier/faster. The agent won't need to download 10 separate snapshots from the server, just the one.]
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/rhq-devel
More information about the rhq-devel
mailing list