metrics database discussion

Jay Shaughnessy jshaughn at redhat.com
Tue Jun 12 13:23:31 UTC 2012


I agree with Ian.  I don't see a full migration to a new database 
solution.  The amount of work would be tremendous. The benefit to the 
end user is only a somewhat simpler deployment model.   I'd go further, 
which is to decide whether we actually do need to continue supporting 
the RDBMS solution, via a server plugin approach, to be able to support 
smaller installations as we do today.  I think that most customers do 
have the ability to support an RDBMS solution.

I believe we do need to support a new solution for the data being 
discussed, for large installations.  The question is whether the 
solution will be easy enough to install and manage such that it doesn't 
turn away the low to medium-large customers.

I don't think it's really going to be an issue but we also need the 
solution to play well with our notion of HA (multi-server) installations.

One other thing to throw out there is security.  I don't know how this 
plays really, but if we plan on supporting customers with high levels of 
security needs we need to make sure the solution meets those needs.


On 6/11/2012 4:34 PM, John Sanda wrote:
> You bring up a good point about different access patterns and 
> consistency requirements. I should have touched on those in my initial 
> email. I want to clarify my point about considering how flexible a 
> technology is for modelling different types of data where access 
> patterns and consistency and availability requirements differ.
>
> I put together this email to start discussions around a well-known 
> issue with metrics and events. If there are other future problem about 
> which we are already aware, then absolutely need to factor those into 
> the discussions. If future problems, requirements, etc. dictate 
> multiple technologies then so be it; however, as rightly pointed out, 
> that does increase the administration and maintenance burdens. If we 
> are considering solutions A and B for metrics where B is wins out 
> overall for metrics but A is still a really good option and also lends 
> itself well to other parts of our data model, then I think A *could* 
> potentially be a better choice. That is the main point I was getting at.
>
> There is a second point though that is harder to appreciate until you 
> have a good grasp on some of these other technologies. Data modelling 
> in the relational world can become somewhat ingrained in our minds, 
> making it difficult at first to understand how to approach it with 
> some of these other technologies. There is this almost natural 
> tendency to make me want to normalize data. Once you are comfortable 
> enough with another, non-relational technology, you might see 
> advantages over the relational model that might be enough motivation 
> to consider moving it out of the relational db.
>
> I agree that we are unlikely to find a one size fits all solution, but 
> I do want to point out the following. Every solution that I think is 
> worth consideration offers way to define and adjust the consistency 
> and availability requirements on a very granular level, usually per 
> operation, i.e., read/write.
>
> On 06/11/2012 12:17 PM, Alan Santos wrote:
>> Hi John - that's very helpful, very comprehensive.  One point I'd 
>> like to discuss in more detail below.
>>
>>
>> On Jun 11, 2012, at 11:46 AM, John Sanda wrote:
>>
>>>   Events immediately come to mind, but also areas like 
>>> configuration, alerts, etc. Ideally, we'd like to have everything in 
>>> a single database and not have to support and maintain multiple 
>>> technologies. This discussion only applies to metrics (and maybe 
>>> events as well), but at some point down the road we could be having 
>>> a similar discussion about another part of the data model. I think 
>>> it is worthwhile and prudent to consider how flexible a technology 
>>> is for modelling different types of data.
>>
>> First,  I know your focusing on metrics here, but I don't think we 
>> should push off future problems quite yet as the decisions today may 
>> limit future choices.
>>
>> Second, I agree that we need to separate the types of data as you 
>> mention but also track differences not only by structure but by 
>> access patterns, performance,  availability and consistency. There 
>> are probably other dimensions to track as well.
>>
>> For example, a user will probably grimace at the loss of performance 
>> metrics but still continue to do business. I could also deal, 
>> perhaps, with some inconsistency when looking at metrics.   I don't 
>> know if the same could be said for other information - e.g. bundles, 
>> configuration audit trails, events or inventory.  In a similar 
>> manner, a slow response time for an audit trail is likely to be much 
>> less problematic than a slow response time when drawing a graph.
>>
>> This point, I believe, warrants further discussion before the 
>> requirement is introduced to *not* use multiple storage 
>> technologies.  Here's another: RHQ as it stands today is fairly 
>> monolithic. Consider if it were a future goal to be less so; perhaps 
>> being used for performance/availability along with other tools that 
>> do their own configuration and provisioning?
>>
>> I understand this has to be weighed against the overhead of user 
>> administration and developer complexity but I'd suggest more 
>> discovery with respect to separation of persistence before writing it 
>> off. There simply may not be a one-size fits all answer.
>>
>> -alan
>>
>>
>>
>> _______________________________________________
>> rhq-devel mailing list
>> rhq-devel at lists.fedorahosted.org
>> https://fedorahosted.org/mailman/listinfo/rhq-devel
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/rhq-devel



More information about the rhq-devel mailing list