drift: Golden Images

John Sanda jsanda at redhat.com
Fri Sep 9 00:15:59 UTC 2011


Let me try to provide some clarification around terminology based on 
what exists in the code base today.

* change set
A collection of changes where each change in the collection corresponds 
to a particular file. This is somewhat analogous to a commit in a 
version control system. Consider an initial commit for a file. With 
respect to the VCS, the only change represented by that commit is the 
addition of the file to version control. Each change set has an 
incrementing version associated with it, similar to say revision numbers 
in SVN.

* coverage change set
The initial change set that the agent generates is called a coverage 
change set. This is to let the server know that we are simply adding 
files for drift monitoring. A coverage change set includes all files 
that are being monitored for drift. It is in effect a snapshot.

* drift change set
A drift change set contains only a set of deltas. Unlike a coverage 
change set, it includes only those files that have changed.

* snapshot
A snapshot is the set of all files being monitored for drift at a 
particular point in time. Discrete points in time are determined by 
change set versions that are created and added to the system.

Some additional thoughts...
While a coverage change set is really a snapshot, it might be somewhat 
confusing from an implementation point of view to call it that because 
both the coverage change set and drift change set share the same file 
format. And I guess to be precise a coverage change set is a specific 
type of snapshot in that it includes only version whereas a snapshot can 
include any number of change sets.

Given what I have previously stated, I think it is accurate to called a 
pinned change set a snapshot; however, I think pinned change set is more 
intuitive at this stage since we have been using the term coverage 
change set.

I hope this helps more than it confuses.

- John


On 9/8/11 6:21 PM, Charles Crouch wrote:
> Moving the conversation Jay and John were having to mail...
>
> Thanks
> Charles
>
>
>
> jshaughn writes:
> We've been talking a lot about the "Golden Image" but we haven't really  been formal about its definition or about what we'll actually be able to  do.  So this is to get things flowing...
>
> > From what we've done so far it seems to me that a "golden image" is a  specific changeset for a specific drift configuration.  Since a drift  configuration monitors immutable directories we don't have to worry  about it changing and invalidating an associated golden image.
>
> I'd like to propose that we change the terminology at this point to be  in line with  our current drift vocabulary.  Instead of golden image I  propose "Pinned Changeset" (other options: "Trusted", "Fixed",  "Blessed", "Golden").  I like "Pinned" because pinning is a well-known  term, pinned, pin, pinning, pinnable are all available when describing  the feature, and it has easy visualization for icons, etc.  Also, were  going to need one of those other adjectives later...
> [ccrouch: So just to be clear, a "Pinned Changeset" contains knowledge of the total set of files being monitored at the time the changeset was pinned? i.e. using terminology from here: http://rhq-project.org/display/RHQ/Drift+Management#DriftManagement-Terminology its really a Snapshot?]
>
> [ccrouch: Sidebar: I think we need to update  http://rhq-project.org/display/RHQ/Drift+Management#DriftManagement-Terminology and remove the term "Change Set" or indicate its an abstract concept. IIUC the only two related things we can have in the system right now are "Coverage Change Set" and "Drift Change Set"?
>
> [jsanda: I'm down with pinned. In fact, IntelliJ uses the same thing with some  views. You can pin or unpin certain windows.]
>
> Today our drift configurations are all "Resource Drift Configurations  (RDC)". Meaning they are only specific to a single resource.   And at  the moment we don't support pinned changesets.  By default we apply a  "rolling changeset", meaning we look for drift compared with the most  recent snapshot, the snapshot being the initial changeset plus all  changes since that time.  If we were to allow pinning it would mean the  user could select a changeset and pin it for the configuration.  And  after that point all drift detection for that configuration would be  against that changeset.  With that in mind, we could allow a new option on a configuration: "pin initial changeset", which would mean  that the initial changeset (version 0) is immediately pinned and  variations from that changeset would trigger drift entries. In this way  reverting back to a pinned version of a file (or removal of an unwanted  file)  would not generate any more drift entries.
> [ccrouch:what's so important about pinning the initial changeset? why wouldn't you want to pin an arbitrary changeset?]
>
> Before we talk about group-level or type-level pinning we may want to  make a fundamental decision.  Do pinned changesets always originate from  an existing resource, or can they be defined external to any existing  resource?
> [ccrouch:If this is a simplifying assumption then it seems reasonable for now]
>
> Although a drift configuration could be treated much like an alert def  or metric schedule, a pinned changeset is not like those things.   Because it actually involves the results of processing the configuration.  To continue the analogy, we can build templates for an  alert def but not for resulting alerts.  Here we need to find a way to  easily templatize the actual file-set  resulting from a configuration.
>
> To a degree we have this already if we do make the assumption that a  pinned changeset can be leveraged only from a trusted resource that has  processed a desired configuration.  Since the mechanism we would have to  build to load a globally accessible trusted changeset would provide what  you get from a resource already, I suggest we do require that it come  from a trusted resource.  I can't think of a reason why we would need a  standalone tool to capture a trusted changeset as opposed to using an  inventoried resource to do it.  Having said that, we could feasibly  provide such a standalone tool.  But it may be tricky, the configuration  would still need to be defined in order to generate the initial changeset.
>
> [jsanda: One  reason I can think of for providing a  tool  to capture a trusted  change  set is to better integrate with  other  management tools. A lot of  folks  for example use puppet and  chef for  configuration management. A   trusted change set might already  exist in  some form as part of a puppet   configuration. An external  tool, which  could be some rich CLI apis,   could better facilitate  extracting  pulling that information out of the   other system and  getting it into  RHQ. The tool could also facilitate  and  require the  creation of the  configuration. ]
> [cccrouch: agreed, but I think this could be a useful extension, not an initial requirement]
>
> So, for now, I assume that a pinned changeset will come from a trusted,  inventoried, resource.  And this is where things get interesting :0)
>
> After thinking on this for a bit I think we need to introduce the  "Golden Configuration" (other options: "Trusted", "Blessed", "[Drift]  Configuration Template").   Not completely dissimilar to say, an  alert-def template because it would be type based and could apply across  inventory.  But  not exactly the same because it would optionally not be  applied by default.  Whether we'd want to work this into the template  stuff or handle it independently I don't know.  I like the term "Golden"  here because people have already been using the term, seem to like it,  and understand what it represents.  The key thing to understand is that  a golden changeset comes with a pinned changeset.  So when adding a  golden configuration to a resource, the initial changeset (version 0) is  provided and pinned.
>
> The idea is this, that a configuration for an RDC (i.e. a changeset for  some resource) could be promoted to a golden configuration.  At which  point it would be copied (the drift config, changeset (pinned or most  recent changeset), and drift entries) and flagged for use globally.
>
> [ccrouch: I'm confused by the terminology here. WRT "changeset (pinned or most  recent changeset), and drift entries)" do you mean "Coverage Change Set", "Drift Change Set", "Snapshot" or something else? Can we make sure we are only using terms we are defining in this mail or are already defined: http://rhq-project.org/display/RHQ/Drift+Management#DriftManagement-Terminology ]
>
> Then, at the resource (or eventually, the group) level, when adding a  drift configuration, one could optionally select from the available  golden configurations for the type.   Whether the golden changeset is  copied for each resource, or referenced, I'm not sure.  A copy would  maybe add flexibility and maybe make coding easier, but copying all of  the drift entries may be cumbersome for lots of resources. Maybe just  copying the changeset would work, leaving only one copy of all of the  drift entries. Also, there are plugin considerations to work out.  The  concept of trusted changesets would be RHQ specific, but plugins  actually store changesets and drift entries, so there is that to finagle.
>
> [jsanda: Instinctively,  I like storing references  instead of copies for the  same  reasons I  like that we store  references to content. I think it  will  better enable  us to do drift  management on a larger scale. If I  purge  resource from  inventory, I  do have to remove all of the drift  entries  for the trusted   configuration. I only have to remove a single  reference.  Likewise  with  assigning a trusted config to a resource. I  am just adding  a  single  reference as opposed to copying lots of drift  entries. ]
>
> If we wanted to align golden configurations like templates, we could add  an option to apply it to every member of a type on import, like today.
>
> In the GUI I think this would require either a top menu Drift item or be  worked into Administration.  The former seems the most flexible, but the  latter may be less cumbersome and may play better with the idea of  templates.
>
> As for [compatible] resource groups, I think perhaps the idea of golden  configurations may work well.  Associating a golden config across the  group members makes, I think, more sense than having a lot of resource  specific configurations.
>
> [ccrouch: I think have a single read-only golden configuration across multiple resources, is a good simplifying assumption]
>
> We will, I think in any pinned changeset impl, need to be able to supply  to the agent the pinned changeset file.  This may need to be  reverse-engineered  from the drift entries.  We can probably make this  such that the agent performs all drift detection and the server is  relieved of that need.  But we also need to consider the implications of  sending perhaps large changeset files to the agent (along with the new  configuration).  My initial impression is that it's not a big deal  because it will be roughly equivalent to the initial change set that  *would have* been generated on the agent. With this seeded the agent  will need send only drift changesets.  So, I'm optimistic about the  server not having to perform detection. I think that will simplify the  mechanism and keep the UI representation of changesets more clear (i.e.  no "adding" drift entries to a changeset, or adding server-generated  changesets).
>
> [jsanda: I  think it makes sense to store the pinned  change set on the agent for  a  couple reasons. Suppose I have 10 EAP  servers on a box and am  doing   drift management on all of them. The  agent can store a single  copy of   the pinned change set for all of  those EAP servers, assuming  that they   are using the same trusted  configuration. This will also  inventory sync   easier/faster. The agent  won't need to download 10  separate snapshots   from the server, just  the one.]
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/rhq-devel




More information about the rhq-devel mailing list