RHQ trait storage

Elias Ross genman at noderunner.net
Wed Apr 23 19:26:20 UTC 2014


On Tue, Apr 22, 2014 at 11:33 AM, Elias Ross <genman at noderunner.net> wrote:

> The schema:
>
> CREATE TABLE IF NOT EXISTS measurement_data_trait (
>     schedule_id int,
>     value varchar,
>     timestamp int, -- when it was inserted
>  PRIMARY KEY (schedule_id)
> );

My inclination was to change this to

CREATE TABLE IF NOT EXISTS measurement_data_trait (
     resource_id int,
     schedule_id int,
     value varchar,
     timestamp int, -- when it was inserted
  PRIMARY KEY (resource,id schedule_id)
);

The reason is that all the traits are selectable by resource_id
directly. They're also stored together for easier caching and
selecting.

The downside is you must know the resource_id before you can do any
insert. This is a problem as the measurement report does not contain
this piece of information. Which would require a lot of changes to
'MeasurementData', the agent, and so forth.

>
> Approach is to:
> 1) Select the current value. (*)
> 2) If it is different then insert into the history table. Use TTL
> columns to delete old records.
> 3) In any case, update the measurement_data_trait table with the most
> current value
>
...

I know this approach is slow (and can block). But for alerting we do
need a select at some point, at least for those alerted on traits.
Oracle tells us this, because the update count lets us know the trait
changed or not.

On IRC, the suggestion was to cache the existing trait values to
determine a trait changed. We can still do that effectively with a
Cassandra-side row cache. The alternative is to use a clustered cache,
but RHQ does not support that sort of clustering yet.

I think my initial implementation will use some sort of thread pool or
async EJB call though, for the select+update, and I'll have to
determine if that is effective or not. I do have a 200+ machine
cluster which will prove if the average cache (assuming my
installation is average) performs well enough.


More information about the rhq-devel mailing list