Heiko,
On Mar 8, 2014, at 12:24 AM, Heiko W.Rupp <hrupp(a)redhat.com> wrote:
Steven,
Am 08.03.2014 um 00:29 schrieb Steven North <swn(a)ocsystems.com>:
> Thanks for the suggestions. It turns out we were only looking at about 10,000
metrics per minute going through a single agent. The agent was running at about an
average CPU load of 50% (which seems high), with periodic spikes to 100-150% for 1 second
(while collection was happening). I did a local setup which logged about 20,000 metrics
per minute and saw a much lower average CPU load for the agent (<5%) and with periodic
1 second spikes to 90% CPU during collection. I had to run with 512MB heap, but only
seemed to be using half that.
what version of RHQ are you looking at?
In 4.10 (hopefully out very soon), we have addressed a good number of issues - especially
one where the agent core itself was using a lot of cpu to select which metrics to take.
Also the heap usage should be considerably lower, which should also reduce (CPU for)
garbage collection and memory pressure.
I think the customer is using version is JON 3.1.2 == RHQ 4.6 and I was using 3.1.0 = RHQ
4.4. I have also run tests on RHQ 4.9. My tests on RHQ 4.4 & 4.9 didn't show any
real issues, other than having to increase the heap size from the default 128MB to 512MB.
In my monitoring, GC was not much of an issue, only about 1-2% CPU during the 1 second
collection interval (after the heap size increase). I may have an opportunity to gather
more information about what was happening at the customer site next week. They are going
to JON 3.2 soon, so events may overtake this investigation.
The other aspect to take into account is the plugin's getValues() method, which also
has sometimes potential for improvement.
Yes, I was very aware of that possibility which lead to my investigations. The messages
in the agent log about metrics collection times and CPU monitoring of my examples helped
convince me that our RTI plugin was not the culprit. There must be another factor in the
customer environment that I have yet to discover.
But let us know when you have findings that we can incorporate
(pull-requests welcome :-)
Will do. I hope it's just some configuration item they have overlooked.
Thanks,
Steve
Heiko
_______________________________________________
rhq-devel mailing list
rhq-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/rhq-devel