long term scalability thoughts

John Sanda jsanda at redhat.com
Wed Aug 10 21:47:49 UTC 2011


minor correction below...

On 8/10/11 5:22 PM, John Sanda wrote:
> Several thoughts....
>
> First, that quote strikes me as misinformed. The "old school" database
> be it oracle or (preferably) postgresql is perfectly capable of handling
> large amounts of data with great performance. With that said, there are
> obviously situations as with metrics where performance *is* an issue.
> And with metrics the database will eventually become the bottleneck.
>
> Secondly, as for other solutions like some of the ones mentioned, you
> can very easily wind up with lousy performance, maybe even more so than
> with a RDBMS. Some of these not so old school technologies are designed
> to address specific problem spaces and are not nearly as general purpose
> as a RDBMS. One of the draws to these other technologies is that they
> are designed to scale out horizontally much more easily than you can do
> with a RDBMS. Much of that ease of scaling out stems from the fact these
> technologies often favor availability over consistency whereas the RDBMS
> favors consistency. Backing off of consistency requirements makes it a
> lot easier to build a scalable, distributed system.
>
> Third, if people want to manage an environment twice their current size,
> I think it is an oversimplification to simply say add more machines. It
> is not an easy problem to solve, and in fact if we already have a good
> solution here, I am aware of it. Then again, this is a really big
if we have a good solution here, I am *not* aware of it
> project and there are plenty of new things I am continually discovering :-)
>
> Lastly, I want to point out the approach we are taking with the drift
> subsystem. Persistence of drift entities and content is not managed
> directly through our SLSB layer but instead through server plugins. The
> default plugin does use the SLSB layer and RHQ database. This might be
> suitable for most users. I have also been working on another server
> plugin that uses MongoDB as its persistence store. This might be an
> attractive alternative if/when scalability becomes an issue. MongoDB
> provides a nice solution in terms of scalability using sharding coupled
> with master-slave replication. The key thing to take away though is that
> a flexible infrastructure is being put in place that allows users to
> plug in different back ends with different performance and scalability
> characteristics.
>
> - John
>
>
> On 8/10/11 4:17 PM, Charles Crouch wrote:
>> A paragraph from an unrelated email this morning helped solidify my thinking around where we could potentially end up going wrt to performance and scalability of RHQ...
>>
>> "MongoDB, Membase, Memcache allow data flexibility and cloud scale. These are not old-sk00l Oracle databases that get slow with lots of data. These scale horizontally and are enabled by the scaling of a PaaS. We do of course have MySQL for the bits of data that aren't going to go through the roof."
>>
>> If users want to manage 1m metrics per minute, or 1m events per minute or 1m alerts per minute they are going to need N machines across which to scale the load. But at the same time there are probably going to be areas of the data model that just will never need to scale like that e.g. plugins, potentially the entire inventory (the requirements for reading/writing 1m resources could be very different to 30m *new* metrics every 12hours) and for that data something like a Postgres instance maybe sufficient. As long as we can also scale down the whole architecture to a single box for those people with smaller environments and still have a reliable and performant system.
>>
>> BTW I'm not talking about making RHQ into a SaaS here or anything PaaS related right now. Just how to build a system that users could install on their own hardware that would scale in a close to linear fashion. e.g. If people want to manage an environment twice their current size, they call up Dell and order N more machines, set them up and install RHQ.
>>
>> One further point, I think even if N can get quite large, I think users would still see that as a reasonable trade-off for near linear scalability (assuming that's achievable). If people have a really big environment they are more likely to invest in a larger monitoring/management infrastructure.
>>
>> More thoughts please...
>>
>> Thanks
>> Charles
>> _______________________________________________
>> rhq-devel mailing list
>> rhq-devel at lists.fedorahosted.org
>> https://fedorahosted.org/mailman/listinfo/rhq-devel
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/rhq-devel



More information about the rhq-devel mailing list