On Wed, Sep 11, 2013 at 4:48 PM, Elias Ross <genman@noderunner.net> wrote:

Actually another issue is JDK7u40 on x86_64 EL6, I get this:


...

And another issue is the upgrade fails as well with a similar error. It looks like the cassandra-jvm.properties bundled has the same bad -Xss setting as before.
 
00:14:47,289 INFO  [org.rhq.storage.installer.StorageInstaller] Updating rhq-server.properties...
00:14:47,302 INFO  [org.rhq.storage.installer.StorageInstaller] Starting RHQ Storage Node
00:14:47,350 ERROR [org.rhq.storage.installer.StorageInstaller] The storage node reported the following errors while trying to start:

Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

$ find . -name \*.jar | xargs ls -lS
-rw-r--r-- 1 elias_ross iadops 18026619 Feb 12  2013 ./jbossas/modules/system/layers/base/org/jboss/as/console/eap/jboss-as-console-1.5.0.Final-redhat-2-resources.jar
-rw-r--r-- 1 elias_ross iadops 12545254 Sep 10 16:04 ./modules/org/rhq/rhq-cassandra-installer/main/rhq-cassandra-ccm-core-4.9.0.jar
-rw-r--r-- 1 elias_ross iadops 12545254 Sep 10 17:06 ./modules/org/rhq/rhq-installer-util/main/rhq-cassandra-ccm-core-4.9.0.jar

I was not sure which one to patch (why are there these identical files?) but I patched the first...

unzip  ./modules/org/rhq/rhq-cassandra-installer/main/rhq-cassandra-ccm-core-4.9.0.jar conf/cassandra-jvm.properties
unzip  cassandra.zip conf/cassandra-jvm.properties
vi conf/cassandra-jvm.properties
zip -u cassandra.zip conf/cassandra-jvm.properties
zip -u ./modules/org/rhq/rhq-cassandra-installer/main/rhq-cassandra-ccm-core-4.9.0.jar cassandra.zip

... continued install ...

Then:

00:23:36,418 ERROR [org.rhq.cassandra.schema.VersionManager] com.datastax.driver.core.exceptions.DriverInternalError: An unexpected error occured server side: java.lang.RuntimeException: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
00:23:36,418 INFO  [org.rhq.cassandra.schema.AbstractManager] Shutting down existing cluster connections
00:23:38,068 ERROR [org.rhq.enterprise.server.installer.InstallerServiceImpl] Could not complete Cassandra schema installation: An unexpected error occured server side: java.lang.RuntimeException: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM: java.lang.RuntimeException: com.datastax.driver.core.exceptions.DriverInternalError: An unexpected error occured server side: java.lang.RuntimeException: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at org.rhq.cassandra.schema.VersionManager.create(VersionManager.java:128) [rhq-cassandra-schema-4.9.0.jar:4.9.0]
at org.rhq.cassandra.schema.VersionManager.install(VersionManager.java:92) [rhq-cassandra-schema-4.9.0.jar:4.9.0]
at org.rhq.cassandra.schema.SchemaManager.install(SchemaManager.java:100) [rhq-cassandra-schema-4.9.0.jar:4.9.0]
at org.rhq.enterprise.server.installer.InstallerServiceImpl.prepareDatabase(InstallerServiceImpl.java:494) [rhq-installer-util-4.9.0.jar:4.9.0]
at org.rhq.enterprise.server.installer.InstallerServiceImpl.install(InstallerServiceImpl.java:261) [rhq-installer-util-4.9.0.jar:4.9.0]
at org.rhq.enterprise.server.installer.Installer.doInstall(Installer.java:117) [rhq-installer-util-4.9.0.jar:4.9.0]
at org.rhq.enterprise.server.installer.Installer.main(Installer.java:58) [rhq-installer-util-4.9.0.jar:4.9.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_40]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_40]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40]
at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40]
at org.jboss.modules.Module.run(Module.java:262) [jboss-modules.jar:1.2.0.CR1]
at org.jboss.modules.Main.main(Main.java:329) [jboss-modules.jar:1.2.0.CR1]
Caused by: com.datastax.driver.core.exceptions.DriverInternalError: An unexpected error occured server side: java.lang.RuntimeException: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM

I'm sure this came from me at one time running 3 instances of Cassandra at a time on 4.8--which wasn't really supported--and the other two nodes are down.

Update: Starting the other two nodes didn't help either.

It's also really stupid that RHQ can't figure out JAVA_HOME on EL6 (/etc/alternatives/java or /usr/java/default/bin/java).