Added some comment/link to the input from Steve about some "operator
summary view", generally called "Control Center" (german
"Leitstand")
below...
Kind regards
Andreas :-)
On 02/14/2011 12:13 PM, Heiko W.Rupp wrote:
Very good feedback from Steve Millidge:
Anfang der weitergeleiteten E-Mail:
> Von: "Steve Millidge"<smillidge(a)c2b2.co.uk>
> Datum: 13. Februar 2011 14:50:06 MEZ
> An: "'Heiko W.Rupp'"<hrupp(a)redhat.com>
> Betreff: RE: New RHQ Groups Summary/Activity pages.. thoughts?
>
> Heiko,
>
> Some initial thoughts from experience out in the field. Apologies for the
> length!
>
> The Summary pages are one of the best features of RHQ and the page that
> delivers the most benefit to customers. I was a bit sad to see it disappear
> from the "first tab" in RHQ 4 when you click on a resource. One great
> feature that could be added on the Resource summary screen is a rollup of
> all child resources alerts, OOB metrics, operations and config changes. For
> example it would be great when clicking on a JBoss AS to see that an alert
> had been generated from a JMS queue hosted by the server. If this was
> carried out throughout the whole resource tree you could easily see by
> clicking on a Platform any alerts generated by any servers/services on that
> platform. As it is at the moment you need to have a good idea what the
> problem may be to drill down to the correct resource level and check
> metrics, config changes and alerts etc.
>
> Addition of group summaries would be a great addition and is better than the
> current group page which tends to just show availability. A key thing for a
> group summary view would be to quickly draw an operator's attention to
> anomalies within the group.
>
> One thing to add at a group level would be a traffic light for availability
> which can be drilled down. Green for all members up, Amber for some are down
> some are up, Red for all resources down. You can then drill down for exact
> details of each group member.
>
> Recent metrics at group level as the sparklines. I'm not sure how much value
> this will add as the range bars may be a bit wide. Perhaps a moving average
> value with some indication of the range of values within the group could be
> more useful.
>
> Alerts, Config changes and Operations should be shown when they were
> generated by a group alert or operations on the whole group. I think it
> would also be useful to show whether an operation had been carried out on an
> individual group member but not the whole group. Especially if it failed.
> Perhaps the traffic light concept could be used next to the operation to
> show whether it was executed successful on all members or only some.
> In general good summarisation is a feature lacking in RHQ at the
moment. RHQ
> does a fantastic job of gathering a load of fine grained metrics etc. at a
> resource level and is therefore invaluable as an admin tool for very
> technical operators. What is needed is some "top level" view that can be
> created to give an operator a warm feeling that everything is OK and then if
> anything anomalous occurs, the operator requires immediate visual feedback
> and a signpost to the suffering resource. Followed by rapid drill down to
> the detailed metrics and timeframe where the problem occurred. These
> operators will then typically call in deep support responsible for that
> component who can use the current functionality to get at all the
> information they require to diagnose.
>
> For example for one of our customers we developed a single JSP page
> dashboard which we deployed to the RHQ server which gave a summarised view
> of the state of their system. All servers were listed with traffic lights.
> Key group metrics were also shown with current average and total values for
> the group and traffic lights to indicate anomalous metrics were in the
> group. These traffic lights then could be clicked on to show the server with
> the anomalous value. For example number of messages on a JMS queue was a
> critical metric. This queue is present on 34 servers so the top level
> dashboard showed the summed count and average for all servers with a traffic
> light if the value was out of some bounds. Clicking on the traffic light and
> count gave a view of the queue counts on each server with a traffic light
> next to each. Further drill down was a deep link into the RHQ page for the
> resource (queue).
See also my comments here that refer to such a Control Center
view as we
implemented it for ourselves:
https://fedorahosted.org/pipermail/rhq-devel/2011-February/000585.html
(starting from "Control Center View")
Depending on the amount of servers/customers(groups) it is important for
it to
- spot everything (important) on an aggregated level fitting on an HTML
area of let's say 1000x600 px
(if we assume an operator has at least some screen resolution of
1024x768 and a standard browser and an OS task bar on the screen)
-- e.g. on really big environments with >> 100 servers some kind of
color-coded pixel map can be efficient for the top-level view
-- it could make sense to break this view down in a standard "filtered
view" and an "unfiltered view" (we have this in place)
--- the filtered view more or less only shows the more critical states
and hides the uncritical ones
--- for the daily work the filtered view is much more useful and
generally faster
- drill down options as some mentioned above leading to the most
important RHQ pages
- be fast (should not take more than 30 s to load)
-- since this may heavily depend on the amount of alerts,
unavailability, ... (and of course resources, groups, system changes in
general, ...) there should be some "emergency" view as well
--- this is e.g. necessary if many resources have problems caused by
some "bottleneck issues" (complete networks down, os/db/RHQ server/RHQ
agent/RHQ plugin updates causing problems on many resources at once, ...)
--- we have this for our own and it shows only alerts, unavailability,
... for critical resources (platforms, db instances, ping services) (we
call it the "VIP view" :-) )
- robust (ideally it should be as independent as possible from the
general RHQ frameworks to not suffer from performance or implementation
problems from the rest)
-- e.g. we are only relying on the RHQ db (oracle, views, table
functions, left outer joins that "can ignore" missing data) and running
it on a different machine, JBoss based on Eclipse BIRT reports
--- I guess this is not a good approach for the official RHQ version
which seems more db independent, based on JBoss, GWT, Seam, ..., but it
definitely has some advantages that could eventually be combined somehow
- quick access to (bulk) operations, also based on groups, platforms,
customers, ... (as we implemented it for ourselves saving us a lot of
time where current RHQ 3.0 is not really usable)
-- deletion of alerts
-- acknowledgement of alerts, unavailability, conspicious setups (alert
definitions)
--- this especially helps multiple working operators to quickly assign
the above to somebody or "mark" them with a meaningful acknowledgement
state or comment for further investigation/processing
-- uninventory of resources (e.g. if a whole resource tree had become
unavailable because it does not exist/is not needed anymore)
-- enable/disable alert definitions (e.g. useful on maintenance scenarios)
> On the subject of auto-refresh. I would be careful here as some
of the group
> metric calculations could be expensive to perform and therefore auto-refresh
> may hammer the database server. I think some of the values would need to be
> precalculated in the database if autorefresh at the group level was
> performed. If the heavyweight nature of the database queries can be solved
> autorefresh would be useful.
>
> Steve Millidge
> Director
> C2B2
>
> Providing the foundations for Enterprise Scale Java.
>
> T: 08450 539457
> M: 07920 100626
> W:
www.c2b2.co.uk
> E: smillidge(a)c2b2.co.uk