Need advice about agent plugin design
by Steven North
I am trying to design an RHQ/JON agent plugin to manage a software
resource with the following characteristics:
- there is the software itself (the installation);
- there are a variable number of "bundles" of configuration information
about 250KB in size each which need to be read from and written to the
agent; and
- there are "log" files which can 10-50MB in size each which need to be
read from the agent.
I think I am pretty clear on how to handle the software itself--just
like any number of other agents.
I am not sure how to handle the configuration bundles and the large log
files.
We might want to have the RHQ/JON server manage different versions of
these configuration files and distribute them to multiple remote agents.
Is there some existing domain object that would handle the read/write
aspect of the configuration bundles (zip files)? Could the "package"
concept be used for these? Would we need to create a new domain object
on the server side for these bundles? If so, is there an example of
this kind of thing?
For the log files, I see some mention of the SupportFacet. Would this
be appropriate for retrieving large log files? Is there an example of this?
We expect to access the configuration bundles and the log files using
remote client operations because we have a separate GUI tool to
build/edit the configuration bundles and to correlate and analyze the
log files. Is there an example of using a remote client to pull files
from and push files to remote agents?
Thanks in advance for any advice you can give or examples you can point to.
Steve
10 years, 1 month
Remove Maven SCM Tags
by Stefan Negrea
Hello Everybody,
Just to give some background, I recently replaced the maven release plugin with versions plugin in all the release scripts. Another change, all the scm commands in the scripts are now pure git commands. So the release scripts do not rely on maven for scm operations.
With this in mind, all the pom files have the scm tag. My suspicion is that those were added to support the maven release plugin. I would like to remove all the tags from the all the pom files for a couple of reasons:
1) The scm tags are no longer used by the release process. Maven versions does not need this tag.
2) The scm tags are not correct today because they do not track the correct branch. These tags are supposed to have the full path of the git tag or branch for the respective modules.
3) It is dangerous to have these tags because git is used for day-to-day branching, tagging, and committing. Unaware users that attempt to use maven scm commands would make commits to the wrong branch (most likely master) without knowing. Maven scm plugin relies on this scm tag to track the branch that will receive the commits. However this tag is not correctly updated because of direct git usage.
4) Minimal pom files (without vestiges) are easier to maintain and understand.
Does anybody see a problem with removing the scm tags? If there are no valid use cases, I will go ahead and remove them later next week. This change is very simple and will not impact anything.
Thank you,
Stefan Negrea
Software Engineer
12 years, 5 months
Hadoop plugin questions; JMX
by Elias Ross
I'm seriously looking at the Hadoop plugin but have a couple of
comments and questions.
1. Not sure why the traditional JMX plugin was not used. Is the JMX
plugin not sufficient? I do notice the metrics across multiple MBeans
get grouped together nicely but I don't see much value in doing this.
Can I simply drop this code and start fresh? Or is this an approach
that should maybe be implemented in the JMX plugin itself. For
instance:
<service name="Aggregate Service"
discovery="MBeanResourceDiscoveryComponent"
description="Two MBeans in one service">
<plugin-configuration>
<c:simple-property name="objectName" readOnly="true"
default="captured"/> <!-- captured from the property name -->
</plugin-configuration>
<metric displayName="Foo"
property="hadoop:service=NameNode,name=FSNamesystemState|BlocksTotal"
description="Number of active threads in the system"/>
2. It's incomplete, obviously, and many of the metrics don't show up
with the Cloudera version I need, which is perhaps newer. I do intend
to fix this. Is there any need to keep what's there? (I'd say no.)
3. There are many values that are totals and available, so I'd like to
make ratios out of them. For example, if the total space is 1TB and
the available is 10GB, I'd like to set up an alert on < 10%
availability. What's the recommendation on how to alert on something
like this? (Maybe more of a user question, I guess.)
4. Are there tools for generating rhq-plugin.xml from an MBean server
connection? Basically it would spit out a <service> for each
ObjectName and properties and operations for each MBean attribute and
operation. I would think this could be fairly useful. The generated
plugin would obviously need to be edited by hand but this would save a
lot of work for many users.
12 years, 5 months
RHQ 4.2 released
by Heiko W.Rupp
The RHQ team is pleased to announce the immediate availability of RHQ 4.2
You can read the release notes here:
http://rhq-project.org/display/RHQ/Release+Notes+4.2.0
Downloads are available from SourceForge
http://sourceforge.net/projects/rhq/files/rhq/rhq-4.2.0/
Special thanks go to Ted Won for Korean translation of the installer and Rafael Torres for his
contributions to the Twitter plugins (OAuth support).
Heiko (on behalf of the RHQ team)
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Brendan Lane, Charlie Peters, Michael Cunningham, Charles Cachera
12 years, 6 months
dependency problem with drift tests
by John Sanda
We hit a problem last night with drift tests in the server/jar module
that needs to be addressed after the 4.2 release. I will describe the
problem followed by my proposed solution.
problem:
DriftManagerBean delegates to a server plugin to handle persistence of a
lot of the drift-related entities. Recently I started adding more tests
to exercise functionality in DriftManagerBean. Tests now call through to
the server plugin. Here is the issue. The server/jar module is built
before the server plugins; so, when the drift tests are run (from a
clean work space) they will fail because the server plugin container
won't have a plugin jar file to load since it has not been built yet.
solution:
One possible solution would be to move the tests to the drift-rhq server
plugin module so that they run after both server/jar and the plugin are
built. I think that there is a better solution though. We move the
tests, along with some others, to a separate integration test suite
module. We set up a new jenkins job to run those integration tests after
each run of the master job.
There are two benefits to this approach. First, these tests can be used
with other drift server plugin implementations. This could provide a
really nice test harness for the plugins.
Secondly and more importantly, we are headed down an unsustainable path
with respect to build execution time for server/jar. A couple weeks ago
I had to run all of the server/jar tests in order to reproduce some
failures that were happening on jenkins. On my quad core box with 16 GB
of ram, it took over 17 minutes to run the tests. I want to move some of
these tests over to the integration test suite module. This will help
reduce overall build times for the master job on jenkins, and will also
make it a little more palatable for people to run all of the server/jar
tests before a commit and/or push.
- John
12 years, 6 months
Testing SLSB methods that make agent calls
by John Sanda
We have a lot of tests for SLSB methods that involve making agent calls.
A lot of these tests use stub implementations of the agent service
interfaces. Three times recently I got bit by trying to send objects
that were still attached to the persistence context to the agent. This
resulted in the following (now very familiar) exception,
Caused by: org.hibernate.PersistentObjectException: detached entity
passed to persist: org.rhq.core.domain.configuration.Configuration
Once I saw what was going on the problem was easy to fix. One thing
bothered me though. I have been putting a lot of effort into producing
thorough tests for the drift SLSB code, and I wasn't able to reproduce
this in my automated tests. Jay made the very shrewd suggestion of using
HibernateDetachUtility in my agent service stubs. Doing so allowed me to
reproduce the same failures I hit during live, end-to-end testing. Here
is one example of what I did,
agentServiceContainer.driftService = new TestDefService() {
@Override
public void unscheduleDriftDetection(int resourceId,
DriftDefinition driftDef) {
detach(driftDef);
}
@Override
public void updateDriftDetection(int resourceId, DriftDefinition
driftDef) {
detach(driftDef);
}
@Override
public void updateDriftDetection(int resourceId, DriftDefinition
driftDef, DriftSnapshot driftSnapshot) {
detach(driftDef);
detach(driftSnapshot);
}
private void detach(Object object) {
try {
HibernateDetachUtility.nullOutUninitializedFields(object,
SERIALIZATION);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
};
With the above code in place I was able to reproduce the same
PersistentObjectException. If you are writing or updating any tests that
exercise code that makes calls to the agent, I strongly encourage you to
follow Jay's suggestion as well. It will make the tests a lot more robust.
- John
12 years, 6 months
Release 4.1.0 beta available
by Heiko W.Rupp
The RHQ team is happy to announce the immediate availability of a BETA release of the upcoming
4.1.0 release.
This beta contains a wealth of new features and bug fixes and serves as a preview for 4.1 and
to gather feedback. Please note that some of the new features are still work in progress (and
will be for some time) - this means that *you* can still influence the new features by your
feedback.
You can find the release notes at http://rhq-project.org/display/RHQ/Release+Notes+4.1.0+beta
which also contain a link to the downloads.
We want to especially thank Elias Ross, Fusayuki Minamoto, Jijun Liu, Zhongqianglee
and Rafael Torres Coelho Soares for their contributions
Heiko on behalf of the whole team.
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Brendan Lane, Charlie Peters, Michael Cunningham, Charles Cachera
12 years, 6 months
Dependent resources
by Heiko W.Rupp
Hi,
I was recently in a call with a not to be named customer and I want to share some outcome of the
call, as this opens an interesting perspective on future architectures of RHQ.
We have been thinking about "Relation ship service in the past" (see e.g. http://rhq-project.org/display/RHQ/Design-Relationship+Service),
but not in this extend.
The customer has a strong demand to be able to start and stop apps with their dependencies. The following graphic illustrates this:
Before the Application (s) on AS can start, the compute grid must be started,
as well as the DB and the load balancer. Also in compute grid and the AS
also depend on some MQ broker. For app3 to start, the grid + MQ needs to
be present and for app4, only MQ.
Also for stopping the order needs to be reversed (with checks about what may
still be needed, but this is not the subject of this email).
Veritas cluster manager is able to produce such a graph as displayed above
and also to hande this dependencies.
RHQ actually already provides the biggest part of this puzzle: triggering start +
stop operations on the managed resources.
We got the idea that they could actually implement the triggering logic by implementing
a server plugin, that reads an external file that hosts the dependency graph and then
triggers the start and stop operations in turn.
As server plugins have "controls", there are even "buttons" in the UI to trigger those start
and stop "meta-operations".
But:
- it is not intuitive to start app4 to go to Admin->ServerPlugins->....
- ops people (for the apps) may not even have access to the Admin section of RHQ
So we would need a way to associate such an operation of a server plugin to an
actual resource, so that the operator would go to App4, operations tab and issue a
"start with dependencies" operation, that goes to the server plugin, which then does
the work of scheduling all the needed start operations on plugin level (in turn and with
waiting for results).
There are now several options on how to represent the metadata for this
- server plugin could indicate which operations it would offer for what resource types
- agent plugins would get a new <server-operation> section that contains the name of the
server plugin + metadata about parameters etc.
In any case the name of the resource where the server-operation is triggered from needs to
be passed to the server plugin, so that it can go to the right entry in the table and do its work.
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Brendan Lane, Charlie Peters, Michael Cunningham, Charles Cachera
12 years, 6 months