----- Original Message -----
From: "Michael Foley" <mfoley(a)redhat.com>
To: rhq-users(a)lists.fedorahosted.org
Cc: rhq-devel(a)lists.fedorahosted.org
Sent: Thursday, January 10, 2013 11:13:59 AM
Subject: Re: Metrics Migration Tool - Cassandra
Just the voice of QE on this .... I am hearing lots of discussion on
design decisions on this data migration tool ... which is good.
One word of caution (and I hear this in all your emails ... so I know
you are thinking about this...this is just a reminder): Avoid
premature optimization.
Totally agree with you Mike. Premature optimization can be the root of a lot of evil.
However, in this case I think the expectation was/is that the amount of data to be
migrated is large. The only questions not addressed were "How large? How much
data?" Hopefully I shed some light into this with with the updates to the design
wiki:
https://docs.jboss.org/author/display/RHQ/Metrics+Data+Migration+-+Design
The course of action that makes sense to me:
#1. Baseline the current implementation. Use small and large datasets
typical of what a customer would use. Document the baseline. This
will be incredibly useful moving forward to compare alternative
solutions. You need the baseline.
#2. Determine if any design optimizations need to be made.
#3. Compare alternative solutions against the baseline.
I guess what I am saying ... I think we need a baseline before this
conversation can continue in a meaningful way.
I am not sure we can ever have exact figures for the amount of data to be migrated.
Metrics collection is highly configurable in RHQ. Users can increase/decrease collection
frequency of any metric. Another example is exact configuration of the resource monitored;
a deployment with a single JBoss AS server but with 1000 applications inside will have a
lot more metrics than the average JBoss AS install. The number of installed plugins and
discovered resources is yet another example.
I hope that the estimates [1] (please see the estimation process) gives as stable starting
point for creating test scenarios and some sort of timing baseline. However, I caution
against treating these estimates as 99.9999% accurate because of highly configurable
nature of metrics collection.
[1]
https://docs.jboss.org/author/display/RHQ/Metrics+Data+Migration+-+Design
Thank you,
Stefan Negrea
How can QE help establish a meaningful baseline? What do we need?
Resources in the bladecenter? Determining an SLA or acceptance
criteria for performance? Datasets?
From: "Thomas Segismont" <tsegismo(a)redhat.com>
To: rhq-devel(a)lists.fedorahosted.org,
rhq-users(a)lists.fedorahosted.org
Sent: Thursday, January 10, 2013 12:02:07 PM
Subject: Re: Metrics Migration Tool - Cassandra
Le 09/01/2013 20:32, John Sanda a écrit :
> At this point, all we can do is speculate about how long the
> migration will actually take until we do some load testing. If we
> find that the migration is taking longer than we would like,
> another option could be to explore using the bulk import/export
> utilities provided by each of the databases.
I think working on bulk export files would be far more efficient. And
it
shouldn't be too difficult given the measurement tables have very
simple
schema (migrating to Cassandra may not be as simple as migrating
these
tables data though).
So why not having the two mechanisms:
1. batching with Hibernate which would support a larger number of
deployments (Postgres, Oracle, SQLServer)
2. batching with bulk export files for the supported databases
(Postgres, Oracle)
I know it's double code, test and support but I really doubt #1 can
handle large amounts of data in less than a few hours.
And you're right, we cannot speculate on this and I don't believe we
could make a release without actually trying the tool on different
workloads.
Thomas
_______________________________________________
rhq-users mailing list
rhq-users(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/rhq-users
_______________________________________________
rhq-devel mailing list
rhq-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/rhq-devel