Need advice about agent plugin design
by Steven North
I am trying to design an RHQ/JON agent plugin to manage a software
resource with the following characteristics:
- there is the software itself (the installation);
- there are a variable number of "bundles" of configuration information
about 250KB in size each which need to be read from and written to the
agent; and
- there are "log" files which can 10-50MB in size each which need to be
read from the agent.
I think I am pretty clear on how to handle the software itself--just
like any number of other agents.
I am not sure how to handle the configuration bundles and the large log
files.
We might want to have the RHQ/JON server manage different versions of
these configuration files and distribute them to multiple remote agents.
Is there some existing domain object that would handle the read/write
aspect of the configuration bundles (zip files)? Could the "package"
concept be used for these? Would we need to create a new domain object
on the server side for these bundles? If so, is there an example of
this kind of thing?
For the log files, I see some mention of the SupportFacet. Would this
be appropriate for retrieving large log files? Is there an example of this?
We expect to access the configuration bundles and the log files using
remote client operations because we have a separate GUI tool to
build/edit the configuration bundles and to correlate and analyze the
log files. Is there an example of using a remote client to pull files
from and push files to remote agents?
Thanks in advance for any advice you can give or examples you can point to.
Steve
10 years, 1 month
MeasurementUnits (EPOCH_MILLISECONDS, EPOCH_SECONDS)
by Jiri Kremser
Hi,
in plugin descriptor, there can be a metric definition with unit type "epoch_milliseconds" or "epoch_seconds" (rhq-configuration.xsd allows it). What kind of metric should could be represented by epoch_milliseconds? Shouldn't the exact time moments (what values epoch_milliseconds represent) be addressed rather by traits?
I am asking because the values of this type are not formatted (https://bugzilla.redhat.com/show_bug.cgi?id=857144).
I think these two unit types should be removed from MeasurementUnits and xml schema, however there might be some plugins out there using it, so what about deprecation? Are there any edge cases, when these unit types do make sense?
JK
10 years, 2 months
Some proposed changes to Manual Import functionality
by spinder
Hi All,
After discussions with Jay, I wanted to describe some changes that I think we should make in support of 'Manual Import' functionality in the plugin container and potentially on the server side.
The issue:
With BZ 883825 we stumbled into odd behavior while trying to test out 'Manual Import' of Tomcat instances. What wasn't immediately apparent was that those same Tomcat instances were already silently indexed by RHQ in the 'Discovery Queue'. So that while attempting to 'Manual Import' these new instances with a new plugin configuration, the plugin container would take the information from the 'new' import and detect that we'd already had that in inventory(the auto-discovered version) and use the invalid configuration from that version. The result was that a user submitting a Manual Import with valid credentials, was getting rejected because RHQ was silently swapping in the wrong config with no credentials. While the situation just described is ugly, the wider issue is that it occurred because we were never clear about what should happen when one tries to 'Manual Import' a new resource resulting in a resource key that matches an existing resource. Essentially such a behavior, if allowed, would result in a Resource configuration/status update which was not the original point.
To summarize, RHQ/JON does not do enough to detect and deter when 'Manual Import' is being used to inadvertently update the state of a Resource already indexed by RHQ. Current suggestions are to:
i)Update the RHQ code to better detect and prevent such behavior with appropriate error messages
- modify InventoryManager to error out when 'Manual Import' is called and an existing RHQ resource is already indexed. The new logic should handle Resources in all InventoryStatus.* states better since changes to existing Resources should only be handled via our existing Resource update mechanism. This means better agent side and server side logs messages when this situation is attempted in the future. I'm also looking into throwing better UI errors after "Manual Import" attempts. Right now we only really support Plugin Configuration exceptions(bad creds) or Plugin Container exceptions of which this doesn't fit in either category well.
ii) update released/docs/support information to better describe this situation and workaround
- this just means documenting this behavior and how to avoid getting into this state. An administrator attempting to inventory resources could get into this state and it's confusing to debug/detect if auto discovery is doing it's job well.
iii)clarify the javadoc on the remote api documentation so that plugin developers know to not attempt to handle such unintentional updates.
- Trying to support 'Manual Import', without filtering out existing Resources that match the same identity, is a subtle point that we haven't gotten right in the past. The DiscoveryBossRemote public api javadoc implies that we will handle cases where 'Manual Import' specifies a plugin config that maps to an existing resource. We need to update this api description here to be clearer about which situations should not be handled by 'Manual Import'.
This last point is tricky because there's no simple way to tell a plugin developers how to detect if the plugin configuration will map to an existing Resource already. There is perhaps a CLI query that they can put together but it's not a great solution. Logging error messages to the server and agent logs also isn't a great solution here from a CLI users standpoint.
I think we should actively discourage this usage of 'Manual Import' that can overlap with Resource update. Are there any other opinions here?
-Simeon
10 years, 11 months
upgrading RHQ with rhqctl
by John Sanda
So far the development and testing of rhqctl has been focused on new installations. When no options are specified, the install command installs and starts the storage node, the server, and the agent. All three are started in the background. The agent is started with rhq-agent-wrapper.sh. Jay will need to chime on how things vary on Windows. When upgrading from a previous version of RHQ, does the install command need to do any extra work? I do not think any extra work is needed for the server and the agent, but if the auto-update feature of the agent is disabled, should rhqctl do anything special.
Upgrading the storage node will involve the following tasks:
* Run a drain operation. This turns off writes and then flushes memtables to disk.
* Shut down the node.
* Take a snapshot of data files.
* Install the new version (to a different directory than the existing version).
* Update cassandra.yaml with configuration changes.
- John
11 years
bundle relative paths using ".."
by John Mazzitelli
Someone opened up a pandora's box with this one: https://bugzilla.redhat.com/show_bug.cgi?id=917085
Title: "Relative paths to rhq:file element fail if parent deploy destination has not yet been created"
I need to solicit opinions, because this is now my second night wallowing in this code wondering how I should handle it, if I should at all.
Here's the crux of the problem:
Someone wrote a bundle recipe and wants to write a bundle file to a location specified via a relative path (thus, relative to the bundle destination directory) - the problem is the relative path used "..":
<rhq:file name="app.properties" destinationFile="../../conf/app.properties" />
Now, look at the problem this poses. This file is no longer going to be placed UNDER the bundle destination directory! The author of the recipe is actually asking us to put the file outside. So we have a relative path that is supposed to be under the destination directory, but it really isn't.
Technically, this is possible - today, you can do this via absolute paths:
<rhq:file name="app.properties" destinationFile="/opt/jboss/conf/app.properties" />
Because the earlier path was a relative path, we assume its under the destination directory and we do things accordingly - if it really isn't under the dest dir, things break. I tried converting these "relative" paths that have ".." in them to be absolute, canonical paths, but that ends up breaking other things, specifically, problems occur if the relative path ends up NOT outside of the dest dir (i.e. it is now the opposite problem):
<rhq:file name="app.properties" destinationFile="some-location/subdir/../file.txt" />
So, unless I write special code to handle these different scenarios, things break all over.
My question - should we allow for relative paths in the recipe that use ".."?
Rather than use relative path, the user could either:
1) use an absolute path if its known (rather than say ../../conf/app.properties, say "/opt/jboss/server/default/conf/app.properties" - this assumes you know your app server is in /opt/jboss)
2) deploy the bundle with a destination directory of the top most directory you need, and your relative paths no longer need ".." (rather than say ../../conf/app.properties, your dest directory should be the jboss install dir and your relative path then becomes "server/default/conf/app.properties")
If we do not allow relative paths to have ".." in them, what problems does this cause? I see some myself - it probably isn't user-friendly to not allow them to do this, and it requires the user to know the dest dir IN the recipe (in order to put an absolute path in, they need to know the full path, which includes dest dir), which goes against the design of what we want people to be able to do in recipes.
I could opt for some hybrid approach - I wrote up some code that is working but ONLY if the relative path using ".." takes the file outside of the dest dir - if the relative path has ".." but stays under the dest dir, other odd things happen. But then, why specify a path like that in the first place? Seems odd - when would you ever want to say something like destinationFile="some-location/subdir/../file.txt" ?
Sorry for the rambling... if you understand any of this, chime in with thoughts.
11 years
[as7] AbstractServerComponentTest#validateStartScriptEnv
by Thomas Segismont
Hi,
As part of as7 plugin integration tests, we check on platform other than
Windows that the detected start script environment will always have an
LD_LIBRARY_PATH entry.
See http://pastebin.com/s0vkm28s
The thing is that this will only be true if the shell from which AS7 was
started had an LD_LIBRARY_PATH variable.
So I think we should get rid of this part. Am I missing something?
Thanks,
Thomas
11 years
GWT defer … - possibility for plugins?
by Heiko W.Rupp
Hi Mike, Jirka,
I saw this commit
> commit e4ac4b1b628ae5e577510045ead7967a603d4c1c
> UI Performance - Move non-essential javascripts (like charting) out of CoreGUI.gwt.xml to CoreGUI.html and add the 'defer' script eva
Could that mechanism also allow UI plugins - in the sense that e.g. those plugins all implement some
"java script interface" and we sort of loop over the .html files that implement them and e.g. populate a menu entry
with that?
I guess that they still need to be added to the root gwt.xml file at compile time?
Heiko
11 years
BZ 883825
by jean-frederic clere
Hi,
I am trying to understand why the stuff is broken in release/jon3.1.x.
The tomcat rhq-plugin.xml is like the master one for
principal/credentials but in the plugin it seems null/null is received
even I enter the right user/password.
It seems fixed in master but I can't find the patch.
Any hints?
Cheers
Jean-Frederic
11 years
rhqctl docs udpated
by John Sanda
I have updated the rhqctl docs[1] with info on the new options that have been added to the install command. There are now options to specify directories for the storage node and the agent to install in non-default locations. And there are another set of options for specifying alternative config files for each service. For example, --server-config can be used to specify a properties file to use and replace the default rhq-server.properties.
[1] https://docs.jboss.org/author/display/RHQ/RHQ+Control+Script
- John
11 years