metrics database discussion

John Sanda jsanda at redhat.com
Mon Jun 11 20:34:50 UTC 2012


You bring up a good point about different access patterns and 
consistency requirements. I should have touched on those in my initial 
email. I want to clarify my point about considering how flexible a 
technology is for modelling different types of data where access 
patterns and consistency and availability requirements differ.

I put together this email to start discussions around a well-known issue 
with metrics and events. If there are other future problem about which 
we are already aware, then absolutely need to factor those into the 
discussions. If future problems, requirements, etc. dictate multiple 
technologies then so be it; however, as rightly pointed out, that does 
increase the administration and maintenance burdens. If we are 
considering solutions A and B for metrics where B is wins out overall 
for metrics but A is still a really good option and also lends itself 
well to other parts of our data model, then I think A *could* 
potentially be a better choice. That is the main point I was getting at.

There is a second point though that is harder to appreciate until you 
have a good grasp on some of these other technologies. Data modelling in 
the relational world can become somewhat ingrained in our minds, making 
it difficult at first to understand how to approach it with some of 
these other technologies. There is this almost natural tendency to make 
me want to normalize data. Once you are comfortable enough with another, 
non-relational technology, you might see advantages over the relational 
model that might be enough motivation to consider moving it out of the 
relational db.

I agree that we are unlikely to find a one size fits all solution, but I 
do want to point out the following. Every solution that I think is worth 
consideration offers way to define and adjust the consistency and 
availability requirements on a very granular level, usually per 
operation, i.e., read/write.

On 06/11/2012 12:17 PM, Alan Santos wrote:
> Hi John - that's very helpful, very comprehensive.  One point I'd like to discuss in more detail below.
>
>
> On Jun 11, 2012, at 11:46 AM, John Sanda wrote:
>
>>   Events immediately come to mind, but also areas like configuration, alerts, etc. Ideally, we'd like to have everything in a single database and not have to support and maintain multiple technologies. This discussion only applies to metrics (and maybe events as well), but at some point down the road we could be having a similar discussion about another part of the data model. I think it is worthwhile and prudent to consider how flexible a technology is for modelling different types of data.
>
> First,  I know your focusing on metrics here, but I don't think we should push off future problems quite yet as the decisions today may limit future choices.
>
> Second, I agree that we need to separate the types of data as you mention but also track differences not only by structure but by access patterns, performance,  availability and consistency. There are probably other dimensions to track as well.
>
> For example, a user will probably grimace at the loss of performance metrics but still continue to do business. I could also deal, perhaps, with some inconsistency when looking at metrics.   I don't know if the same could be said for other information - e.g. bundles, configuration audit trails, events or inventory.  In a similar manner, a slow response time for an audit trail is likely to be much less problematic than a slow response time when drawing a graph.
>
> This point, I believe, warrants further discussion before the requirement is introduced to *not* use multiple storage technologies.  Here's another: RHQ as it stands today is fairly monolithic. Consider if it were a future goal to be less so; perhaps being used for performance/availability along with other tools that do their own configuration and provisioning?
>
> I understand this has to be weighed against the overhead of user administration and developer complexity but I'd suggest more discovery with respect to separation of persistence before writing it off. There simply may not be a one-size fits all answer.
>
> -alan
>
>
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/rhq-devel



More information about the rhq-devel mailing list