Something we're focusing on at the moment is improving the ability to
extract metrics data from a Beaker installation without relying on the
main server (either via the web UI or the XML-RPC interface). The
problem with relying on either of those is that some of the more
interesting queries can be quite resource intensive, and end up
interfering with the operation of the job scheduler.
For the more volatile metrics (like current system utilisation and the
state of the recipe queue), Beaker 0.11 will be sending several
additional signals to Graphite. We likely won't have a nice dashboard
for those in this release (instead relying on a few direct links to
appropriately designed graphs in Graphite web UI), but creating a
"Beaker Dashboard" for an installation is definitely on the cards for
the subsequent release.
For the more resource intensive queries though, I'd like to be able to
rely on data aggregation systems like Teiid and business reporting tools
like Jasper Reports. That means:
1. Identifying the Beaker metrics which we think are interesting (and
aren't covered by the Graphite data)
2. Figuring out how to extract those from the database schema
3. Figuring out how to publish them to Beaker users in a way that allows
them to be used in a reporting system like Jasper, but won't be a
nightmare for us to maintain
Amit's patch at
http://gerrit.beaker-project.org/#/c/1546 is a decent
attempt at 1 and 2, but ultimately fails 3.
Amit, Dan and I spent some time discussing this on IRC, so here's what
we're currently thinking:
1. Create a new location in the Beaker source repo for metrics .sql files
2. Have a section in the admin guide on metrics extraction (as Amit's
patch does), but:
- drop the "Job Congestion Measurement" use case (already covered by
the Graphite metrics)
- drop the "Hardware Utilization and Coverage" use case (this part
of the schema is seriously complicated, so it's better to use the Search
UI on the main server. Perhaps mention using that UI to craft a query
and then adding "&tg_format=atom&list_tgp_limit=0" to get the results as
a machine readable list of links as per
http://beaker-project.org/server-api/http.html#system-inventory-information)
- modify the User Accounting section to focus on specific
architectures, rather than distro versions
- adjust the remaining sections to reference the appropriate .sql
file instead of including the SQL in line
- add a caveat noting that the details of these queries may change
between releases, but this should always be mentioned in the release notes
3. For each .sql file, have a test case which runs that SQL and checks
it gives the same answer as the SQL Alchemy model
Cheers,
Nick.
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Python Applications Team Lead
Beaker Development Lead (
http://beaker-project.org/)
GlobalSync Development Lead (
http://pulpdist.readthedocs.org)