On Mon, 2013-07-15 at 18:58 -0400, John Sanda wrote:
On Jul 15, 2013, at 5:46 PM, Jiri Kremser <jkremser(a)redhat.com>
wrote:
> Basically I am +1 on having both metrics, both can be useful for setting up custom
alerts.
>
> But I am concerned that the percentage based metric is not proper for alerting
because of its relative nature. While it make sense when calculating the heap size based
alerts, it is different in disk space, because differences in partition sizes could be
enormous. "Calculated.SystemDiskSpaceUsedPercentage > 90%" may mean many
things on different HW. I know that the alerts are adjustable and one could change the
number to whatever she wants, but we should provide smart defaults.
>
> What about having another one called for instance
"Calculated.SystemDiskSpaceAvailable" that would be calculated as the sum of
free spaces across all partitions where data dirs for C* are located? (because of the way
how C* selects the data dir) And trigger the alert if the number is lower than X, where X
could be the number of megabytes necessary for storing the data for, say, 7 days. Using
this approach, we could have alert notifications saying something like "If you
don't add more disks, you will run out of space in approx 1 week.".
A metric that considers the sum of free space across all partitions would not address the
increases in disk usage (albeit temporary) due to compaction.
I am curious to how this is different then looking at a percentage on
individual partitions? For example, when looking at disk usage for a
traditional relational database, a warning is not seen until either disk
space has run out or it has resulted in data being moved from one
partition to another. The hope is that it would be the latter and you
could then increase said partition's space or add new partitions. In the
end, the database doesn't keep track of how much space is there and how
much of the total space it has used up.
Perhaps we are trying to do too much? The expectation is that the user
provides enough disk space for the storage node to function. Hopefully
they will create an alert on their disk resources that will provide them
with this information. If so, perhaps a metric that provides the sum of
the free space across all the partitions used by the storage node is
sufficient?
--
Larry O'Leary
https://plus.google.com/u/0/112645929986009801513