tmpwatch can erase the EMS jar
by Elias Ross
I have a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1146167
Basically I have a tmpwatch process in place (EL6) and eventually it
will delete the org-mc4j-ems-impl123xxx.jar file causing all sorts of
trouble.
Is there some reason EMS does this sort of thing? Why not just include
the impl file in the JMX plugin?
I'm going to make the change to not use an embedded impl. Please let
me know if this is not the right idea.
9 years, 6 months
Alerts on REST created resources
by Van Dillon
Hello All,
I'm using an agent plugin that depends on the No-op plugin to define
resources that I push into RHQ (4.10.0) using REST.
I've found that if I create a platform with REST as the parent for these
resources then alerts on metrics defined in the plugin don't trigger.
Alerts work if I use an agent backed platform as the parent for the REST
pushed resources. They work even when the agent is down.
Is this a known limitation or a bug?
Thanks,
Van Dillon
9 years, 6 months
Forced server MAINTENANCE mode seems too drastic
by Larry O'Leary
I have recently come across what I believe is an oversight in how the
storage node can force the server into maintenance mode.
The problem scenario appears to affect multiple users and can lead to
system downtime. It boils down to the fact that if a storage node goes
down for any reason, you can't start it back up or use the alerting
functionality to handle such failures. This is not limited to just the
storage node resource. It can impact all resources in the system as long
as the storage node's state is such that the server gets stuck in
maintenance mode.
What is not clear, why is the entire communication layer being shutdown
just because the storage node is unavailable? I understand that we can
not collect metrics, but the agent's measurement reports could easily be
rejected and told to try again in a minute. In the meantime,
availability and most importantly, operations/management should still be
allowed.
In the current implementation, once the storage node goes offline, the
user must wait for the data center management group to restart the
entire system. This is because the RHQ server and the agent responsible
for managing the storage node, become unavailable for both management
and monitoring. The result is that the entire system goes offline and
can not be recovered from the RHQ server itself.
Are there plans to fix this? This issue can occur:
- during a scheduled restart of the storage node
- when a restart operation of the storage node fails
- when the storage node throws an error that requires a restart
Thanks,
--
Larry O'Leary
https://plus.google.com/+LarryOLeary
9 years, 6 months
Updating dev-container's agent when compiling module
by Michael Burman
Hi,
When working with the agent's modules (such as plugin-container), these files are not currently updated when compiling with -Pdev, unlike many server modules. Does anyone have an objection on why it shouldn't be done? I'm currently targeting ${rhq.defaultDevContainerPath}/rhq-agent (like the rhq-server path), but I'm assuming someone could have changed the agent's path, in which case this wouldn't work. Is anyone actually changing the agent-path on dev-container? Or have any other objections?
I opened PR #126 for comments also.
- Micke
9 years, 6 months
Releasing agent memory to OS
by Lukas Krejci
Hi all,
in the continuing quest to minimize the memory requirements of agent, I've
come across these 2 params in the Java tuning guide [1]:
-XX:MinHeapFreeRatio and
-XX:MaxHeapFreeRatio
These 2 basically define the thresholds of free heap space when the allocation
or freeing of the system memory occurs (more details in the tuning guide [1]).
Setting these thresholds to the same value basically translates to the maximum
free heap (i.e. the maximum amount of allocated memory above the memory
required by the objects) at any given point in time.
Setting these to 5 (meaning maximum 5% free heap) lowered the total memory
usage from 188M to 154M during "quiet" times and to 165M during peak times in
the agent lifecycle (peak probably happening during discovery) in the default
settings with a smallish inventory (9 servers, 1017 services, 98 metrics per
minute).
Of course, having such "tight" heap slows down memory allocation during peak
times but IMHO our agent should not seek the maximum performance but rather
the lowest possible overhead.
What do you think about introducing these 2 parameters to the agent startup
command?
Lukas
[1] http://www.oracle.com/technetwork/java/gc-tuning-5-138395.html#0.0.0.
%20Total%20Heap|outline
9 years, 6 months
Calculation of group average metrics - bug?
by Elias Ross
I have a scenario I'd like input on. For example, you have two servers S1
S2 handing 6 requests per minute, then add a third (at T1) handling 4 per
minute. This is what the data looks like:
T0 T1
------------------
S1 6 4
S2 6 4
S3 x 4
x represents no data at this time (since the server was not alive.)
RHQ calculates the per-server average as 6 for T0 and 4 for T1. Which is
okay, but really there was no additional requests on the network for the
group, so you'd expect the graph to be flat. So the graph actually dips
at/before T1.
I'm wondering if this should either be corrected in some way so that either
you get the count at T0 and T1 for the number of data points found, or the
average is calculated based on the total number of schedules, not number of
data points found.
9 years, 6 months