Aggregate alerting feature - Bug 1019472
by Elias Ross
Hi, I know you've seen a lot of patches from me. I've been doing some major
work on getting our fairly large RHQ 4.9 installation working.
Speaking of fairly large, one feature that RHQ lacks is alerting based on
group average metrics. I added a server plugin feature that can alert on
this data.
One obvious use case: You have a group of 10 hosts serving web traffic. If
the amount of traffic drops during a 10 minute period from the previous
day, say by 30%, you would like to be alerted. Of course during the day
itself there are traffic fluctuations by more than 30%, so comparisons
against an absolute baseline isn't particularly useful. It also isn't
useful to alert per host, as some hosts may be receiving no traffic or
increased traffic, depending on load balance configurations, server
upgrades, or whatever. Or having to manage 10 alerts (or more) isn't easy.
The way the plugin works (and you can see the documentation here:
https://bugzilla.redhat.com/show_bug.cgi?id=1019472 ) is every 5 minutes,
there are calculations made using the most recent data, and yesterday's
data. (There are other rules, such as comparing last week's data, or
absolute value comparisons, or availability checks like the percentage of
up servers.) Then when an alert occurs, an alert definition is created at
the resource group level (since RHQ has no UI support for this), and an
alert is sent.
What I have is fully functional, but required some patches to RHQ core to
support alert notifications for resource groups.
I don't know if this feature is in demand, but it would be nice to see the
following introduced, in order:
0) The feature included as part of the server plugin suite, but perhaps
disabled by default.
1) Patches included to fix alerting for resource groups. (Patches are part
of the bug.)
2) UI support for listing and clearing alerts for resource groups. (Partial
patches in the bug, should be easy to fix.)
3) Alert definitions created through the UI, not through resource tags.
This requires expanding the types of alert definitions RHQ formally
supports.
4) An API for adding alert definitions through the command line interface.
(Because adding a tag to a resource group is supported through the CLI.)
Thoughts?
10 years, 5 months
Announcing RHQ Agent Plugin Plugin
by Thomas Segismont
Hi everyone,
I'm happy to announce the release of RHQ Agent Plugin Plugin!
Plugin Plugin? Yes. Plugin Plugin: a Maven plugin to help you build RHQ
Agent plugins.
It provides a handful of very convenient mojos for:
* Packaging
* Validation
* Deployment to a local RHQ Server (dev-container)
* Uploading to to a remote RHQ Server
* RHQ CLI script or command execution
* Standalone Plugin Container setup for integration tests
Dependent agent plugins are supported (many plugins depend on the parent
JMX plugin, for example).
The RHQ Agent Plugin Plugin documentation [1] is hosted on Github Pages
and the bits are available on JBoss Releases and Maven Central repositories.
Next week I will start to update agent plugins POM files in RHQ master
branch.
If you need help to get started with your own plugin development, feel
free to reply.
Regards,
Thomas
10 years, 5 months
Script execution environment
by Amy Wu
Hi,
I'm working on a WebLogic agent plugin and to support a Start Server operation, I mimicked the behavior in jbossas5's plugin's ApplicationServerOperationsDelegate where I basically allow users to provide a script location to start the server and execute the script via ProcessExecution. I am running into an issue where I am seeing different behavior between executing my script via RHQ vs outside of RHQ.
When I run my startWeblogic script outside of RHQ in my console, when my WebLogic server starts, it will use the Platform MBean Server for its WebLogic Runtime MBean Server. If I run jconsole and connect to my WebLogic process, I am able to see both JVM platform MBeans as well as WebLogic runtime MBeans.
However, when I run the same script inside of RHQ, WebLogic starts up with the following warning:
Warning: The JMX MBean PlatformMBeanServerUsed attribute is true, but the Platform MBeanServer was created without the hooks for the WLS security infrastructure. The Platform MBeanServer will NOT be used and Platform MBeans will NOT be available via the WLS Runtime or Domain Runtime MBeanServers. This can occur if you have defined Platform MBeanServer system properties or JVM options (-Dcom.sun.management.jmxremote or JRockit -XManagement).\n To allow the Platform MBeanServer to be used, you must either remove the system properties/JVM options or start WLS with the following system property:\n -Djavax.management.builder.initial=weblogic.management.jmx.mbeanserver.WLSMBeanServerBuilder \n If you want to eliminate this log error and do not need Platform MBeans to be available via WLS, then set the PlatformMBeanUsed attribute in the JMXMBean to false.
My theory is that when I run the startup script outside of RHQ, Weblogic detects no Platform MBeanServer exists for the JVM and creates one for use. But when the startip script is run inside of RHQ, is it possible a Platform MBean Server already exists? Or are there any additional properties or JVM options that RHQ adds that causes the startup script to behave differently when executing via RHQ?
I did verify that if I pass in "-Djavax.management.builder.initial=weblogic.management.jmx.mbeanserver.WLSMBeanServerBuilder" as an argument, I am able to see all the expected MBeans via jconsole, but I would like to know why this is happening before accepting this workaround.
Thanks
--
Amy Wu
Consultant, Red Hat Consulting
Red Hat
amywu(a)redhat.com
http://www.redhat.com
10 years, 5 months
Tomcat plugin: wrong metadata for "Tomcat Virtual Host" component
by Thomas Segismont
Hi,
While testing the "validator" mojo of the Maven plugin for agent plugins
that I'm working on, I found an issue with the Tomcat plugin.
It never came out on RHQ master builds because the validate-all module
is not configured to validate the Tomcat plugin ...
So here is the validator message:
===
Component class [org.jboss.on.plugins.tomcat.TomcatServerComponent] for
resource type [Tomcat Server] from plugin [Tomcat] does not support the
child creation facet but has metadata saying it can.
===
This is due to the "Tomcat Virtual Host" resource type having the
"createDeletePolicy='both'" attribute.
It was introduced by commit 417fbb5, which is included in RHQ4.7 and up.
It does not harm and if you try to create a "Tomcat Virtual Host" you'll
see an exception like:
===
org.rhq.core.clientapi.agent.PluginContainerException: Component does
not support the
[org.rhq.core.pluginapi.inventory.CreateChildResourceFacet] interface:
ResourceContainer[resource=Resource[id=10004,
uuid=78283a3a-f8d3-4a5b-9b92-e966a4bba150, type={Tomcat}Tomcat Server,
key=/home/tsegismont/Tools/apache-tomcat-7.0.37, name=Tomcat
(127.0.0.26:8080), parent=localhost.localdomain, version=7.0.37.0],
syncState=SYNCHRONIZED, componentState=STARTED, avail=UP]
===
I created a BZ:
Bug 1037736 - Tomcat plugin: wrong metadata for "Tomcat Virtual Host"
component
https://bugzilla.redhat.com/show_bug.cgi?id=1037736
Once it is fixed I will add the Tomcat plugin in the validate-all module.
Regards,
Thomas
10 years, 5 months