inventory connection changes and/or different use cases with auto-discovery / adding resources manually / plugin updates / resource key purpose

Lukas Krejci lkrejci at redhat.com
Mon Jan 3 13:47:41 UTC 2011


Hi Andreas,

If I understand your usecase correctly, you basically want to have different 
views on the log files.

The difficulties you're describing are real but I don't think we would solve 
them by adding some more logic and asking plugins questions like: "you just 
discovered this resource. Given the current inventory, is it really valid?"

This would potentially complicate matters very much and I think it would 
conceptually break our approach of the resource key being a unique identifier 
of a resource.

Your log file views don't exist as standalone things outside of the RHQ 
inventory. I think this is the fundamental cause of your difficulties with RHQ 
way of doing things but at the same time, I think the resource upgrade 
functionality can still help you. Basically, if your RK calculation changes, 
the only person that knows about how to handle the changes is you as the 
plugin dev.

I also think that by rearranging your resource type hierarchy you could make 
your life easier.

What about this:

LogFile (RK = path)
-- View (RK = a random string, only manual add supported, with a "default" 
view maybe autodiscovered, the config attributes would be set in the plugin 
config)

With this approach, you get rid of the problem of LP vs. LP+. Because View's 
RK is just a random string, you also get rid of the ARM problem. It basically 
leaves the responsibility of "duplicate detection" on the user (which I think 
is fair given the fact that views are only manually added).

Even if you want the duplicate detection to work, you could base the View's RK 
on the concatenation of your config attributes and have the resource upgrade 
facet to "move" the resource from one key to another. 
Because RK is the agent-side ID of a resource there are checks in place that 
prevent two resources with the same RK (under the same parent) to exist in the 
inventory.

Other answers are inlined below.

Thanks for the input, this is turning into a very interesting debate :)

Lukas

On Monday, January 03, 2011 13:36:28 Andreas Dietrich wrote:
>   Hi,
> 
> there seems to be another related and important problem to the whole
> issue with resource keys, auto discovery, adding resources manually and
> updating the plugin (configuration):
> 
> ===============================================================
> To explain it I'd like to explain it with help of our own case:
> 
> - L) we have a Logreader plugin in place with various config attributes
> (logfilePath, errorPattern, warnPattern, excludePattern,
> skipInitialLines, ...)
> 
> - what we want from each side regarding the resource keys:
> 
> -- AD) auto discovery: if we discovered a log file we do not want to
> discover it again with another resource key
>      => LP (log file path only approach): logfilePath should be
> sufficient as resource key
> 
> -- ARM) adding resources manually/Logreaders: we would like to be able
> to add different Logreaders for the same log file
>      => LP+ (log file path + other attributes approach): logfilePath +
> all or part of the other config attributes (see L) above) should make up
> the (initial) resource key
> 
> - U) if we update our Logreader plugin and have added some new attribute
> that has to go into the resource key calculation, we would basically
> auto-discover every existing Logreader when using LP+ because the
> resource key calculation has changed
>      => IR (ignore auto discovered resources on plugin attribute update
> approach): we would have to ignore all the newly discovered resources
> (which is not always simple to do because one has to check if there is
> no Logreader that is really newly discovered and a duplicate of an
> existing one)
>      => SR (import the newly discovered resources and uninventory the
> old ones approach): this is something we don't like as explained in
> detail here:
> 
> http://metlos.wordpress.com/2010/10/06/upgrading-resources-in-rhq/
> 
> => since LP and LP+ are valid requirements, but go against each other
> with the current RHQ implementation, I think they should not be based on
> the same "resource key" definition
> 

As I tried to explain above, I don't think this is completely true. In fact, 
the usecase you're describing is fairly a non-standard one, because you're 
trying to create "logical" resources that you let your users define as opposed 
to the conventional approach of RHQ where each resource corresponds to 
"physical" thing installed on the machine on its own. 

> ===============================================================
> S) (solution to the LP <-> LP+ / AD <-> ARM <-> U conflict)
>  From my point of view we basically only need the "resource key" for
> auto-discovery-identity-management (LP approach above).
>
> (=> We could think of it as "auto discovery resource key" or "if auto
> discovery finds a new resource, have a look if we should treat it as new
> or existing => thus compare the resource keys")
>

If autodiscovery generates the same resource key, the resource is not 
considered new, but rather discarded, because resource with such RK already 
exists.

> If we still need it for ARM: If we wan't to add resources manually there
> could be a hint "Another resource with the same logfilePath already
> exists. Are you sure to continue adding this resource (configuration)?".
>

I think this already is in place since RK is the agent-side ID of a resource. 
Two resources with the same RK (under given parent) can't therefore be allowed 
anywhere in the inventory. This is btw. also checked for during the resource 
upgrade. If you upgrade a resource and the new resource key is equal to the RK 
of one of the siblings, the resource upgrade fails for that resource and the 
RK is not changed.
 
> S.1) The resource key should now be the same as the existing one, e.g.
> "/mypath/logfile.log" and thus it would not be a unique sibling anymore.
> Does this matter so much?

Yes, it does, because RK is the agent-side ID of a resource.

> S.1.U) Additionally this would solve the problems related to U since the
> logfilePath most likely stays the same.
> 
> 
> ===============================================================
> Some comments to other thoughts and their implications I had
> implementing something else than S.1:
> 
> S.2) The resource gets a new id and a resource key appended by e.g. some
> smallest positive integer >= 1 so the resource key does not exist yet,
> e.g. "/mypath/logfile.log (3)" if "/mypath/logfile.log (2)" already
> exists. (One should know this concept from MS Windows/Office behaviour ;-))

If this was the approach for all the resource types, you'd duplicate your 
inventory on each discovery run, e.g. each 15 minutes ;)

> => S.2.ADC) auto discovery conflict: if we would uninventory the first
> discovered resource "/mypath/logfile.log" and have "/mypath/logfile.log
> (3)" left we would again discover a new resource with key
> "/mypath/logfile.log" without further "investigation" ala "resource_key
> like '/mypath/logfile.log%'" (S.2.ADE - auto discovery duplicate
> resource key exists algorithm).
> => S.2.U) U would be solved with S.2.ADE
> => why so much effort if S.1 is sufficient
> 
> S.3) Leaving the resource key null would lead to a similar problem like
> S.2.ADC, but there would potentially be a lot more resource keys of
> different logfilePaths we can't find a simple solution like the S.2.ADE
> for. ===============================================================
> 
> 
> What do you think? Did I miss something suggesting to implement S + S.1?
> 
> Kind regards
> Andreas :-)
> 
> On 01/03/2011 11:39 AM, Lukas Krejci wrote:
> > Hi,
> > 
> > The name of the resource cannot be changed programatically in RHQ<= 3 in
> > the plugin code. The only option you have is either to change the name
> > manually using the UI or use the RHQ CLI and have a script that would
> > periodically scan for changes in the plugin configs and rename the
> > resources accordingly.
> > 
> > In RHQ 4 however, this is going to be possible from within your plugin
> > code with the resource upgrade functionality, but you're still going to
> > have to do a bit of manual setup to get it working.
> > 
> > You can read about the resource upgrade on our wiki:
> > http://www.rhq-project.org/display/RHQ/Design+-+Resource+Upgrade
> > 
> > Using the resource upgrade functionality, you are currently able to
> > upgrade the resource's key, name and description. But the RHQ server is
> > by default configured to ignore the name and description changes (so
> > that we don't override the custom values users might have assigned to
> > their resources). Currently there is no way of changing that
> > configuration from the UI.
> > 
> > To enable the upgrade of resource name and description, you have to
> > change the RHQ server system configuration using this SQL (for
> > Postgres):
> > 
> > INSERT INTO RHQ_SYSTEM_CONFIG (ID, PROPERTY_KEY, PROPERTY_VALUE) VALUES
> > (nextval('rhq_system_config_id_seq'),
> > 'RESOURCE_GENERIC_PROPERTIES_UPGRADE', 'true');
> > 
> > and restart the server.
> > 
> > In your plugin code, you'll have to make your discovery class implement
> > the ResourceUpgradeFacet interface and implement the upgrade() method
> > along the lines of:
> > 
> > public ResourceUpgradeReport
> > upgrade(ResourceUpgradeContext<HttpHostResourceComponent>  context) {
> > 
> > Configuration pluginConfig = context.getPluginConfiguration();
> > 
> > //TODO extract the values from plugin config
> > String hostname = null;
> > String port = null;
> > 
> > //TODO implement
> > String newResourceName = generateResourceName(hostname, port);
> > 
> > ResourceUpgradeReport report = new ResourceUpgradeReport();
> > report.setNewResourceName(newResourceName);
> > 
> > return report;
> > }
> > 
> > Lukas
> > 
> > On Tuesday, December 28, 2010 07:04:51 Lin Gao wrote:
> >> Hi:
> >>     If the value of one inventory connection is part of the resource
> >> 
> >> component name, how to make the resource component name reflecting the
> >> changes of the inventory connection value.
> >> 
> >>    below is an example:
> >> 
> >>    rhq-plugin.xml:
> >> <server name="HttpServer-Host" description="Http server host and port"
> >> 
> >>               discovery="HttpHostDiscoveryComponent"
> >> 
> >> class="HttpHostResourceComponent"
> >> 
> >>               supportsManualAdd="true">
> >> 
> >> <plugin-configuration>
> >> <c:simple-property name="host" description="host of target http server"
> >> /> <c:simple-property name="port"  description="port of target http
> >> server" /> </plugin-configuration>
> >> </server>
> >> 
> >> HttpHostDiscoveryComponent.java:
> >> 
> >> public Set<DiscoveredResourceDetails>  discoverResources(
> >> 
> >>               ResourceDiscoveryContext<HttpHostResourceComponent> 
> >>               context) throws InvalidPluginConfigurationException,
> >>               Exception {
> >>           
> >>           Set<DiscoveredResourceDetails>  result = new
> >> 
> >> HashSet<DiscoveredResourceDetails>();
> >> 
> >>           for(Configuration config: context.getPluginConfigurations()){
> >>           
> >>               String host =
> >>               config.getSimpleValue(PLUGIN_CONFIG_HOST_KEY,
> >> 
> >> null);
> >> 
> >>               Integer port =
> >> 
> >> Integer.valueOf(config.getSimpleValue(PLUGIN_CONFIG_PORT_KEY, "80"));
> >> 
> >>               String resourceKey = "HTTP_CHECK_" + host + ":" + port;
> >>               String resourceName = "Http Checker on: " + host + ":" +
> >>               port; String resourceVersion = "1.0.0";
> >>               String resourceDescription = "Http Checker on server: " +
> >> 
> >> host + ":" + port + ".";
> >> 
> >>               DiscoveredResourceDetails resource = new
> >> 
> >> DiscoveredResourceDetails(context.getResourceType(), resourceKey,
> >> resourceName, resourceVersion, resourceDescription, config, null);
> >> 
> >>               result.add(resource);
> >>           
> >>           }
> >>           return result;
> >>       
> >>       }
> >> 
> >> If the host is defined to: 127.0.0.1 and port to: 80 when the server is
> >> added manually, the name of this resource component in the navigate tree
> >> is: "Http Checker on: 127.0.0.1:80", after that, if the host is
> >> changed(inventory->connection->edit...), the name of the resource
> >> component in the navigate tree is not changed, even re load the tree, so
> >> how to make it changed also?
> > 
> > _______________________________________________
> > rhq-devel mailing list
> > rhq-devel at lists.fedorahosted.org
> > https://fedorahosted.org/mailman/listinfo/rhq-devel
> 
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/rhq-devel


More information about the rhq-devel mailing list