Nice document Stefan!

I am of course interested in testing and software quality.   Is it okay if I enter the conversation in this regard?

Primarily, I want to make sure the design process considers testing and verification.   So to begin that thought process I penciled in a few thoughts on the Wiki ... as shown below.  As you continue with the design and implementation ... is it OK if you add your thoughts to the Testing section as well?

Later on ...this will be very useful information as QE begins engaging on this new feature ....

Thanks!

Michael Foley
QE Lead, JBoss Operations Network 


Testing

  1. Use-cases
    1. Oracle --> Cassandra
    2. Postgres --> Cassandra
    3. Edge cases to consider
      1. high-volume of metrics
      2. low-volume of metrics
      3.    
      4.    
    4. Negative tests cases to consider
      1. connection failures
      2.  
      3.    
  2. Verification ... or, "How do we determine that migration was successful?"
    1. Output generated by the migration tool
      1. error messages ... or absence of error messages
      2. # of rows read in Oracle/Postgres and number of rows? created in Cassandra ...on a per resource basis
    2. Visual inspection in RHQ UI
      1. exactly what in RHQ should remain exactly the same after the backend switches from Oracle/Postgres to Cassandra   
        1.   
        2.    
        3.  
      2.  
  3.  Unit tests  on the data migration tool
    1.    
    2.    
    3.    
  4.  
Labels:


From: "John Sanda" <jsanda@redhat.com>
To: rhq-devel@lists.fedorahosted.org
Cc: rhq-users@lists.fedorahosted.org
Sent: Wednesday, January 9, 2013 10:12:25 AM
Subject: Re: Metrics Migration Tool - Cassandra

After some lengthy discussion with Stefan, I think the migration tool needs to be run while the server is offline or possibly in maintenance mode. The best time to run the tool would be prior to running the RHQ installer. A problem with doing the data migration while the server is running is that we could wind up with skews in the aggregate data. The easiest and fastest way to ensure data is consistent for both pre- and post-upgrade is to run the data migration while the server is down.

- John

On Jan 2, 2013, at 6:20 PM, Stefan Negrea <snegrea@redhat.com> wrote:

> Hello Everybody,
>
>
> With new Cassandra based metrics storage system, existing data will need to be migrated out of existing SQL storage to Cassandra. I put together a simple design document for the data migration tool that will be delivered with RHQ when the new metrics storage system is completed.
>
>
> My initial thoughts about the tool are:
> 1) Restartable migration process, this will be especially useful for users with large amounts of metrics.
> 2) Favour robustness over performance because this migration will be done only once.
> 3) Make the tool external to the product (not part of the installation), so user can do the migration ahead of the upgrade.
>
>
> Here is the full design document:
> https://docs.jboss.org/author/display/RHQ/Metrics+Data+Migration+-+Design
>
>
> Feedback is more than welcomed since this tool is still in the early planning stages.
>
>
>
> Thank you,
> Stefan Negrea
>
> Software Engineer
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel@lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel

_______________________________________________
rhq-users mailing list
rhq-users@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/rhq-users