Cassandra - Data Migration

Thomas Segismont tsegismo at redhat.com
Thu Mar 14 15:18:48 UTC 2013


Le 14/03/2013 12:42, Heiko W.Rupp a écrit :
> Stefan,
>
> thanks for sharing that numbers and experience.
>
> Postgres allows for some speedup in inserting data, but obviously that is not the part we
> need to optimize.
>
> - would it be feasible to e.g. dump the content of postgres into a "text file" and feed this
> into cassandra for import?

I had same question and post a test case comparing JDBC batch and bulk 
export (with "copy from"):

https://lists.fedorahosted.org/pipermail/rhq-devel/2013-January/002441.html


That would eliminate the jdbc-roundtripping and could even
> allow to fill cassandra offline (while having the server work against postgres). If that
> offline dump is in cassandra, the tool would only read the data from the time the above
> snapshot was taken and transfer it over (not needed to total offline migration).
>
> In postgres one of the tricks to speed up imports is to use the "copy" command
>
> postgres=# \h copy
> Command:     COPY
> Description: copy data between a file and a table
>
> for this.
>
> Other tricks are to either use no transactions at all or to use very large
> transactions for the inserts and not per row.
> Also for mass-data inserts it helps to have no indexes on the table and only
> add them after all the data is inserted.
>
> I don't know if there is something similar in Cassandra though
>
>
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
>




More information about the rhq-devel mailing list