RHQ trait storage

Elias Ross genman at noderunner.net
Tue Apr 22 18:33:04 UTC 2014


I was thinking about how to do trait storage.

https://planet.jboss.org/post/modeling_metric_data_in_cassandra

This document doesn't really explain the major problem with how to
deal with repeated inserts.

I came up with three approaches for consideration:

1.  Select and insert

The schema:

CREATE TABLE IF NOT EXISTS measurement_data_trait (
    schedule_id int,
    value varchar,
    timestamp int, -- when it was inserted
 PRIMARY KEY (schedule_id)
);

CREATE TABLE IF NOT EXISTS measurement_data_trait_history (
    schedule_id int,
    timestamp x,
    value varchar,
 PRIMARY KEY (schedule_id, timestamp)
);

Approach is to:
1) Select the current value. (*)
2) If it is different then insert into the history table. Use TTL
columns to delete old records.
3) In any case, update the measurement_data_trait table with the most
current value

Benefits:
1) Easy to select the most recent value
2) Logic is straightforward

Problems:
1) Select than insert is slow.
2) Hard to do transactionally and with a lot of traits.
3) Race condition exists where insert may happen twice. (Not really likely.)

(*) Probably the most efficient way is to select is to submit the
select asynchronously, then when it completes, do the update and
insert. This does mean, potentially, an insert or update is lost if
the server is shutdown. The mitigating factor, of course, is trait
reports are periodic and effectively retried.


2. Insert then cleanup

Insert in history every time a new value is reported, also update the
most current value.

Then periodically remove any duplicates.

Benefits:
1) Insert only, no round trips

Problems:
1) Basically defers the 'select' as before, so the same amount of work, really
2) Extra disk space/storage. Cassandra data compression may mitigate
disk usage, but still lots more writes. (Worst case would be a trait
reporting the same value every 30 seconds.)
3) Cannot really update the 'when the trait changed' timestamp --
since we are only inserting

Possible alternatives:
1) Defer compression to when data is selected.


3. Map values to timestamps

Similar to above, but using a set of timestamps.

CREATE TABLE IF NOT EXISTS measurement_data_trait_history (
    schedule_id int,
    value varchar,
    timestamps set<x>,
 PRIMARY KEY (schedule_id, value)
);

Benefits:
1) Insert only

Problems:
1) The number of timestamps may be ever increasing and so a process is
needed to clean it up. TTL expiration doesn't work, if the value
flip-flops back and forth between two sets of values.
2) Therefore, there needs to be another cleanup process to deal with
this. This could be done using a counter that increases for every
update, and once it reaches a certain value forces a "garbage
collection" of sorts.


Thoughts?


More information about the rhq-devel mailing list