John Sanda has already done work in this area - he has been
experimenting with MongoDB as the OPTIONAL back end for drift content
storage.
He used the server-side plugin architecture which we introduced last
year to make the backend "pluggable".
For people with smaller environments, they don't have to do anything -
out of box, we support everything in the one backend SQL database.
But for those with larger needs for drift content storage, they flip a
system setting to switch over to mongo storage for drift. Today, it
requires them to have mongoDB already installed and running. This is
just a prototype but I have seen it work myself and it looks like a
viable way to do this kind of thing.
If all works well, we could expand this to make, say, measurement
storage pluggable using the same kind of server plugin mechanism. And
events... and calltime data... and...
On 08/10/2011 04:17 PM, Charles Crouch wrote:
A paragraph from an unrelated email this morning helped solidify my
thinking around where we could potentially end up going wrt to performance and scalability
of RHQ...
"MongoDB, Membase, Memcache allow data flexibility and cloud scale. These are not
old-sk00l Oracle databases that get slow with lots of data. These scale horizontally and
are enabled by the scaling of a PaaS. We do of course have MySQL for the bits of data that
aren't going to go through the roof."
If users want to manage 1m metrics per minute, or 1m events per minute or 1m alerts per
minute they are going to need N machines across which to scale the load. But at the same
time there are probably going to be areas of the data model that just will never need to
scale like that e.g. plugins, potentially the entire inventory (the requirements for
reading/writing 1m resources could be very different to 30m *new* metrics every 12hours)
and for that data something like a Postgres instance maybe sufficient. As long as we can
also scale down the whole architecture to a single box for those people with smaller
environments and still have a reliable and performant system.
BTW I'm not talking about making RHQ into a SaaS here or anything PaaS related right
now. Just how to build a system that users could install on their own hardware that would
scale in a close to linear fashion. e.g. If people want to manage an environment twice
their current size, they call up Dell and order N more machines, set them up and install
RHQ.
One further point, I think even if N can get quite large, I think users would still see
that as a reasonable trade-off for near linear scalability (assuming that's
achievable). If people have a really big environment they are more likely to invest in a
larger monitoring/management infrastructure.
More thoughts please...
Thanks
Charles
_______________________________________________
rhq-devel mailing list
rhq-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/rhq-devel