Cassandra Schema Management Tool

John Sanda jsanda at redhat.com
Fri May 24 18:23:14 UTC 2013


On May 24, 2013, at 12:36 PM, Stefan Negrea <snegrea at redhat.com> wrote:

> 
> 
> ----- Original Message -----
>> From: "John Sanda" <jsanda at redhat.com>
>> To: rhq-devel at lists.fedorahosted.org
>> Sent: Friday, May 24, 2013 11:02:03 AM
>> Subject: Re: Cassandra Schema Management Tool
>> 
>> 
>> On May 24, 2013, at 11:46 AM, Stefan Negrea <snegrea at redhat.com> wrote:
>> 
>>> Hello Everybody,
>>> 
>>> Here are the design plans for the new Cassandra schema management tool:
>>> 
>>> 1) A version table will keep track of the current schema version
>>> 2) The initial setup
>>> a) Will create:
>>>   i)Initial RHQ admin user/password
>>>   ii) RHQ keyspace
>>>   iii) Schema versioning column family (=table)
>>> b) Once the initial setup is done, it will never run again
>>> 3) Everything else will be installed/updated via incremental changes:
>>> a) One change file per version change.
>>> b) Multiple changes can be bundled in a change file.
>>> c) The name of the change file will be used for database versioning
>>> purposes.
>>> d) If a change has already been applied it will not be applied again for
>>> the lifetime of the installation.
>>> e) If an incremental change fails, the upgrade process stops.
>>> f) The upgrade files will be in XML format, with a very simplistic XML
>>> Schema.
>>> 4) There will not be a single schema file for new installs.
>>> a) All installs will follow this incremental upgrade process.
>>> 5) All schema modifications outside of the schema manager will not be
>>> supported and are at risk.
>>> 
>>> 
>>> Starting with a simple and minimalist tool is the best solution (minimal
>>> set features, more robust). I expect the tool to evolve over time but if
>>> we keep things simple, then it will be to easier to maintain over time.
>>> 
>> 
>> Why do we need xml?
> 
> I chose XML because it is a simple way to structure information. The goal is to have multiple updates per file. For example, creating three column families in a single update should be split into three smaller steps inside a big update file. 
> 
> As an example:
> 001_Fancy_Update.xml:
>  - step: create CF A
>  - step: update CF B
> 002_Fancier_Update.xml:
>  - step: remove CF B
>  - step: create CF C
>  - step: create CF D
>  - step: create CF E
> 
> 
> I am open to any other suggestions for file formats. JSON is not a good candidate because there is no standard way to handle new lines. Any suggestions of something else that we could use?
> 
> 

Why not just a text file of CQL commands. Both the cassandra maven plugin and cqlsh support being passed a file of CQL commands. We can do the same. XML/JSON/etc. just means more code to maintain and validate.

We will need some minimal capability for executing statements dynamically. When the installer runs and the schema is created, the replication factor will be set according to the cluster size. And if it is a single node cluster, then we will set gc_grace_seconds on each table to zero. With a single node, tombstones can be GC'd immediately. We can support this kind of stuff with simple tokenization though.


More information about the rhq-devel mailing list