Need advice about agent plugin design
by Steven North
I am trying to design an RHQ/JON agent plugin to manage a software
resource with the following characteristics:
- there is the software itself (the installation);
- there are a variable number of "bundles" of configuration information
about 250KB in size each which need to be read from and written to the
agent; and
- there are "log" files which can 10-50MB in size each which need to be
read from the agent.
I think I am pretty clear on how to handle the software itself--just
like any number of other agents.
I am not sure how to handle the configuration bundles and the large log
files.
We might want to have the RHQ/JON server manage different versions of
these configuration files and distribute them to multiple remote agents.
Is there some existing domain object that would handle the read/write
aspect of the configuration bundles (zip files)? Could the "package"
concept be used for these? Would we need to create a new domain object
on the server side for these bundles? If so, is there an example of
this kind of thing?
For the log files, I see some mention of the SupportFacet. Would this
be appropriate for retrieving large log files? Is there an example of this?
We expect to access the configuration bundles and the log files using
remote client operations because we have a separate GUI tool to
build/edit the configuration bundles and to correlate and analyze the
log files. Is there an example of using a remote client to pull files
from and push files to remote agents?
Thanks in advance for any advice you can give or examples you can point to.
Steve
10 years, 1 month
Re: Help with Drools JON/RHQ plugin
by John Mazzitelli
[I'm forwarding this to the rhq-devel mailing list - this is a good tech
discussion that should be made out in the open in the community]
Edson,
> In the end I was able to test the plugin by nuking the installation and re-installing after each change, as you suggested.
Nice - can you tell me which H2 files you had to restore? Was it just
one of those .db files or all of them?
> My questions are:
>
> 1. I would like to change it to make the plugin a top level server instead of running inside the JMX Server. Is it possible? How can I do it?
You can probably do this. In your plugin descriptor, just mimic what the
JMX descriptor has, tweeking as you need. In this case, you would not
need any "<runs-inside>" metadata.
You can probably reuse the JMX server component as well. Or at least
subclass it. Sounds like you want to override the discovery component so
it only detect things that has your MBean in it. So I would investigate
extending the JMX's plugin component "JMXDiscoveryComponent":
http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=blob;f=modules/plugins/j...
Maybe write your own, or some how reuse the discover method:
org.rhq.plugins.jmx.JMXDiscoveryComponent.discoverResources(ResourceDiscoveryContext)
this method takes the process scan results (see the JMX plugin
<process-scan> for what its looking for in the operating system process
table - essentially it is looking for java executables). Sounds like you
want to do a little extra - you want to find all java VMs that have your
MBean in its platform MBean Server.
> 2. Related to above, I guess, how do I configure auto-discovery? My criteria is: any java process (JVM) that contains a DroolsManagementAgent MBean running should be discovered.
Right. See the JMXDiscoveryComponent class linked above for how you do
the first part. The JMX plugin is configured via <process-scan> metadata
to auto-discover java apps that are running. you'd then have to add your
own discovery code to look for whatever specific info you want to
further refine those results. In your case, you want to attach to that
java VM, peek in its platform MBeanServer and see if your MBean exists.
If it does, you'd return data for your discovered server. Otherwise,
you'd return nothing to indicate you found nothing.
> 3. I was able to configure all the metrics I want to collect, but how to I collect configuration data? I.e., things that were configured in the application and that rarely change, but that I want to keep track of and store as an historical audit trail in JON/RHQ. E.g. "Event Processin Mode = stream", "Assert Behavior = identity", "Max threads = 10".
Sounds like you want what we call a metric "trait". A trait is like a
metric but it can be any string (not just a double) and we track changes
to them over time (under the Measuerment tab, there is a subtab, I think
called "Traits"). It is assumed traits can change, but they change
rarely (less so then dynamic measurement data like "number of DB
connections" or "CPU utilization" or "free memory"). There are plenty of
examples of trait definitions in many of our plugins. See the agent
plugin or the jboss-as plugin or even the JMX plugin itself - look for
<metric> definitions that have dataType="Trait" attributes. One example
in the JMX plugin descriptor is:
<metric displayName="Heap Initial Size"
property="{HeapMemoryUsage.init}"
displayType="summary"
dataType="trait"
description="The amount of heap that the Java virtual
machine initially requests from the operating system"
units="bytes"/>
12 years, 8 months
Out-of-band data for configurations
by Heiko W.Rupp
Hi,
currently when writing a configuration, we can only return "success" or throw an Exception on failure.
Is there a way to convey extra information on success?
Background: in as7 when e.g. changing the a port definition for the socket bindings, the result on
success may be "reload required" that should be presented to the user
(actually, that should even be a badge in the resource tree until the reload is done).
Any ideas?
Heiko
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Brendan Lane, Charlie Peters, Michael Cunningham, Charles Cachera
12 years, 8 months
Reduce Database Configuration Complexity - Proposal
by Stefan Negrea
Hello Everybody,
I have been working on BZ 7222298 (https://bugzilla.redhat.com/show_bug.cgi?id=722298) to port dbreset to Oracle. During this work I got confused over data source configuration.
Currently there are two sets of settings (rhq.test.ds and rhq.dev.ds) for data source configuration (primarily used for unit tests). The two sets are almost identical (username, passsword, type-mapping), the main difference is db-name. Additionally, there is a flag used to switch between the two sets. And here is where the confusion starts... Not all the scripts use (or use correctly) the flag to switch between the settings. A lot of the scripts/tests only use rhq.test.ds (and completely ignore rhq.dev.ds or the flag). The original dbreset was not making full use of the settings either. At any single time, it is possible to use only one set.
My initial dbreset port was very confusing and complicated because there are so many settings. I had to take into account fall-back scenarios (when one setting was not defined for one profile but it was a for the other). So, I took a step back and looked at ways to simplify/improve this.
And here is my proposal:
1) Remove both rhq.dev.ds and rhq.test.ds; and remove the flag to switch between the two sets.
2) Consolidate everything into one set of settings - rhq.ds.
3) To use a different database for building/testing/resetting the db then just override rhq.ds (eg. -Drhq.ds.db-name=rhq_dev) at command line. Or better, create separate profiles in .m2/settings.xml and activate each profile as needed for maven builds.
Having only one set will:
1) make it easy to understand data source settings (only rhq.ds to configure)
2) reduce the code complexity (no fall-back, no flags to switch between settings).
3) make it easy to override the correct settings for your particular database configuration.
4) make new unit tests consistent and reduce the possibility to make mistakes.
Here is the branch with the changes I made for dbreset:
http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=shortlog;h=refs/heads/st...
And here is the commit with most of the changes proposed for rhq.ds:
http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=commit;h=187643
Any suggestions or input for my proposal? Is there a reason to keep two sets of tests? Are there any use cases that cannot be accomplished by profiles/settings override?
Thank you,
Stefan Negrea
Software Engineer
12 years, 9 months
code review
by Filip Drabek
Hi,
as you know there is a problem with memory leak in augeas based plugins.
(https://bugzilla.redhat.com/show_bug.cgi?id=721152)
I would like to ask you for code review of my changes in augeas based
plugins. The code is in branch release-3.0.0
commit 4cfd2eb0b6f12bb9e4cd77862a0aa0e2292580e2 and
b787f3937f7618bfc83aad8bbc4e4ed67eb97b3e
The problem was that after initialization of augeas we were calling
augeas.load() to load the configuration files, but
we were not calling method augeas.close() to release the memory in heap.
The solution is:
- added method finalize where we can close the augeas.
- once augeas is initialized and the method load() is called multiple
times the memory usage is still growing.
So we can not reuse the augeas instance and we need to call method
close() after each augeas use (started by method load())
I have tried more solutions like new augeas components or implementing
inner classes for each use of augeas, but in the end
the code was always much worse than in this implemented solution.
Thanks
Filip
12 years, 9 months
PropertySimple.defaultValue
by Ian Springer
Either Pidgin or #rhq is acting weird for me today, so I'm not sure if
the below made it into the room...
(12:04:06 PM) ip1: mazz, pilhuhn - just read the defaultValue discussion
above
(12:04:57 PM) ip1: iirc, the original intent of defaultValue was a way
to specify what default value the managed resource would use for an
optional prop that was unset/null
(12:05:58 PM) ip1: the thinking was that the config editor gui could
display it somewhere (eg - a new Default column)
(12:07:10 PM) ip1: but that never got implemented. instead, we decided
to just state what the default value for an optional prop within the
prop's description
(12:08:39 PM) ip1: the nice part of instead making it a first class
attribute would be that then the plugin descriptor parser could complain
if no defaultValue was specified for a required="false" prop
(12:08:59 PM) ip1: to encourage (or force) plugin devs to specify it
(12:11:11 PM) ip1: but the complication is that a given prop's default
could be different across managed resource versions (e.g. different
default for ds max connections prop for jboss 4.0 and jboss 4.2)
(12:12:09 PM) ip1: using the description as we currently do, you could
just state that in the description
12 years, 9 months
Problems upgrading JMX plugin
by Edson Tirelli
Hi,
Charles asked me to create this thread, so here we go. :)
Background: we have a request to provide an RHQ/JON plugin for Drools [1].
Drools exposes its monitoring stats through standard JMX MBeans (both static
and dynamic). So I created a JMX derived plugin for Drools [2]. During the
development of the plugin (read: creation of the plugin descriptor) I had
problems every time I would make changes to the plugin structure (other than
adding metrics) and trying to redeploy the plugin. Most of the time, it
would "corrupt the database" and I would have to perform a reinstall of the
whole RHQ server. Simply restarting server and agent would not be enough.
Talking to Mazz, he suggested that I nuked the database before redeploying,
and that was what I did, using a fresh install for each test.
Plugin upgrades assumptions: I understand that certain changes might make it
impossible to keep track of history after redeploying a new version of a
given plugin. For instance, one of the things I did was to move a service
that was directly under the server to inside another service. In my
understanding, in such cases, the ideal solution would be for the server to
provide as much access to historical data as possible, but if some data
become unaccessible, it would not be a big deal. BUT it is mandatory that
the server works with the new plugin and start collecting data under the new
structure, without any problems. The same is valid for
adding/removing/renaming operations/metrics/services.
The problem: assuming my understanding of how a plugin upgrade should work
is correct, then there might be a serious problem with the RHQ server or JMX
base plugin. After every change I did during the development of the plugin
(other than adding metrics) followed by a redeploy, the plugin and the
server would stop working. Apparently, some kind of corruption of the
database would occur, preventing the whole server from continuing working
and the only possible solution that I found was to reinstall the whole
server.
Disclaimer: there is one thing I haven't tested: since I was developing the
plugin, the version number remained the same for all the redeploys:
<plugin name="DroolsRHQPlugin"
...
version="1.0"
...
I don't know if the RHQ server uses version number as a key for the plugin
data model, what might explain the database corruption.
Configuration used: RHQ 4.0.1, embedded H2 database.
To reproduce, install RHQ and deploy the Drools plugin [2], collect some
data from a sample application like [3]. Change the plugin by moving a
service around or renaming an operation, etc. Redeploy.
Thanks,
Edson
[1] http://www.jboss.org/drools
[2]
https://github.com/droolsjbpm/droolsjbpm-integration/tree/master/drools-r...
[3]
https://github.com/droolsjbpm/droolsjbpm-integration/blob/master/droolsjb...
--
Edson Tirelli
JBoss Drools Core Development
JBoss by Red Hat @ www.jboss.com
12 years, 9 months
SNMP plugin patch for handling SNMP notifications; testing for plugins
by Elias Ross
This is my first contribution.
I was working on the JMX side but it was decided we needed to support
the functionality of SNMP more. I was attempting to use the RHQ SNMP
listener for Python but although the plugin does support SNMP traps,
it does not do the two-way "inform" messages as described here:
http://www.cisco.com/en/US/docs/ios/11_3/feature/guide/snmpinfm.html
I wanted this support so I added it. So I verified that it now works
with pySNMP V2 inform messages.
I also spent some time on adding support for test ability. There
doesn't seem to be a good plugin suite to leverage, so I started to
craft something called "ComponentTest". I would like to have this
reviewed and maybe put into a common test library. There doesn't seem
to be any testing for many of the plugins which I think is
unfortunate.
I also ran into issues with how events are queued and polled for over
60 seconds. Necessarily, I had to change the minimum poll value to 1
second so I can run the test in a reasonable timeframe. This interval
is now configurable as well, but defaults to 60 as before.
Some of the whitespace and other changes are inadvertent.
12 years, 9 months
drift merged into master
by John Sanda
I just merged the drift branch into master. Running dbsetup will be
required due to some recent schema changes made in the drift branch.
- John
12 years, 9 months