dev-container with multiple storage nodes

John Sanda jsanda at redhat.com
Sat Jun 15 18:36:57 UTC 2013


I just pushed some changes to provide support for running multiple storage nodes with a dev-container build. By default, you should get a dev-container just like you did prior to my changes. Running multiple nodes relies in part on using localhost aliases just like we do with the Cassandra integration tests. If you on Fedora or another Linux distro, you do not have do do anything to set up the aliases. For Mac OS X and Windows you need to create the aliases. On Mac OS X you can do,

$ sudo ifconfig lo0 alias 127.0.0.2 up
$ sudo ifconfig lo0 alias 127.0.0.3 up


There is a script named storage_setup.groovy that gets executed when the dev profile is active. It does not install storage nodes. It sets up the necessary directory structure so that you can easily install nodes. Let's say we are rebuilding our dev-container and want to deploy 3 storage nodes.

$ mvn clean package -Pdev -Drhq.storage.num-nodes=3

This will create dev-container/rhq-server which is the regular, full dev-container server, and it also creates dev-container/rhq-server-2 and dev-container/rhq-server-3. The rhq-server-2 and rhq-server-3 directories are minimal shells that provide the necessary pieces for running storage nodes. Each of these additional server directories contains the necessary scripts along with a configured rhq-storage.properties and a series of symlinks to provide a fast, lightweight solution for deploying multiple storage nodes with the dev-container.

Now let's suppose we already have our dev-container built and want to deploy additional an storage node. From the appserver module run,

$ mvn -o groovy:execute -Pdev -Dsource=src/main/script/storage_setup.groovy -Drhq.storage.num-nodes=2

This will generate dev-container/rhq-server-2. If we later decide that we want two more nodes, run,

$ mvn -o groovy:execute -Pdev -Dsource=src/main/script/storage_setup.groovy -Drhq.storage.num-nodes=4

The script will detect that you already have rhq-server and rhq-server-2 and will then only set up rhq-server-3 and rhq-server-4. Each node has to be (and is) configured with a unique JMX port. If you deploy multiple storage nodes prior to running the server installer, you will need to configure the rhq.cassandra.seeds property in rhq-server.properties by hand. So if you have two nodes, e.g., dev-container/rhq-server/rhq-storage and dev-container/rhq-server-2/rhq-storage, the property should look like,

rhq.cassandra.seeds=127.0.0.1|7299|9142,127.0.0.2|7300|9142

Let me know if there are any questions or if anyone runs into any problems. Given that the implementation relies on symlinks, I do not expect that this will work on Windows unless you are running something like cygwin; however, as I mentioned at the beginning, there should be no changes whatsoever to dev-container/rhq-server regardless of whether you run one or multiple nodes.

- John


More information about the rhq-devel mailing list