Le 14/03/2013 12:42, Heiko W.Rupp a écrit :
Stefan,
thanks for sharing that numbers and experience.
Postgres allows for some speedup in inserting data, but obviously that is not the part
we
need to optimize.
- would it be feasible to e.g. dump the content of postgres into a "text file"
and feed this
into cassandra for import?
I had same question and post a test case comparing JDBC batch and bulk
export (with "copy from"):
https://lists.fedorahosted.org/pipermail/rhq-devel/2013-January/002441.html
That would eliminate the jdbc-roundtripping and could even
allow to fill cassandra offline (while having the server work against
postgres). If that
offline dump is in cassandra, the tool would only read the data from the time the above
snapshot was taken and transfer it over (not needed to total offline migration).
In postgres one of the tricks to speed up imports is to use the "copy" command
postgres=# \h copy
Command: COPY
Description: copy data between a file and a table
for this.
Other tricks are to either use no transactions at all or to use very large
transactions for the inserts and not per row.
Also for mass-data inserts it helps to have no indexes on the table and only
add them after all the data is inserted.
I don't know if there is something similar in Cassandra though
_______________________________________________
rhq-devel mailing list
rhq-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/rhq-devel