metrics database discussion
John Sanda
jsanda at redhat.com
Mon Jun 11 20:34:50 UTC 2012
You bring up a good point about different access patterns and
consistency requirements. I should have touched on those in my initial
email. I want to clarify my point about considering how flexible a
technology is for modelling different types of data where access
patterns and consistency and availability requirements differ.
I put together this email to start discussions around a well-known issue
with metrics and events. If there are other future problem about which
we are already aware, then absolutely need to factor those into the
discussions. If future problems, requirements, etc. dictate multiple
technologies then so be it; however, as rightly pointed out, that does
increase the administration and maintenance burdens. If we are
considering solutions A and B for metrics where B is wins out overall
for metrics but A is still a really good option and also lends itself
well to other parts of our data model, then I think A *could*
potentially be a better choice. That is the main point I was getting at.
There is a second point though that is harder to appreciate until you
have a good grasp on some of these other technologies. Data modelling in
the relational world can become somewhat ingrained in our minds, making
it difficult at first to understand how to approach it with some of
these other technologies. There is this almost natural tendency to make
me want to normalize data. Once you are comfortable enough with another,
non-relational technology, you might see advantages over the relational
model that might be enough motivation to consider moving it out of the
relational db.
I agree that we are unlikely to find a one size fits all solution, but I
do want to point out the following. Every solution that I think is worth
consideration offers way to define and adjust the consistency and
availability requirements on a very granular level, usually per
operation, i.e., read/write.
On 06/11/2012 12:17 PM, Alan Santos wrote:
> Hi John - that's very helpful, very comprehensive. One point I'd like to discuss in more detail below.
>
>
> On Jun 11, 2012, at 11:46 AM, John Sanda wrote:
>
>> Events immediately come to mind, but also areas like configuration, alerts, etc. Ideally, we'd like to have everything in a single database and not have to support and maintain multiple technologies. This discussion only applies to metrics (and maybe events as well), but at some point down the road we could be having a similar discussion about another part of the data model. I think it is worthwhile and prudent to consider how flexible a technology is for modelling different types of data.
>
> First, I know your focusing on metrics here, but I don't think we should push off future problems quite yet as the decisions today may limit future choices.
>
> Second, I agree that we need to separate the types of data as you mention but also track differences not only by structure but by access patterns, performance, availability and consistency. There are probably other dimensions to track as well.
>
> For example, a user will probably grimace at the loss of performance metrics but still continue to do business. I could also deal, perhaps, with some inconsistency when looking at metrics. I don't know if the same could be said for other information - e.g. bundles, configuration audit trails, events or inventory. In a similar manner, a slow response time for an audit trail is likely to be much less problematic than a slow response time when drawing a graph.
>
> This point, I believe, warrants further discussion before the requirement is introduced to *not* use multiple storage technologies. Here's another: RHQ as it stands today is fairly monolithic. Consider if it were a future goal to be less so; perhaps being used for performance/availability along with other tools that do their own configuration and provisioning?
>
> I understand this has to be weighed against the overhead of user administration and developer complexity but I'd suggest more discovery with respect to separation of persistence before writing it off. There simply may not be a one-size fits all answer.
>
> -alan
>
>
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/rhq-devel
More information about the rhq-devel
mailing list