New Storage Cluster - Sizing Guidelines
by Stefan Negrea
Hello Everybody,
We've been working on sizing guidelines for the new Storage Cluster. We focused on two aspects: disk and compute power. We condensed all of our findings into a Google Spreadsheet and wiki page (more about this below). While we are still capturing data and refining results we wanted to get this in the hands of users to get feedback.
Data size on disk was our primary focus since it is relative to the number of metrics schedules and collection interval. The underlying data storage is not trivial but thanks to John Sanda's hard work we now have an algorithm that gives a baseline. In terms of computer power, we wanted to see how far we can scale a deployment with the new Storage Cluster. While we have collected some early numbers via simulations, this time we wanted to use a real life deployment. We are still collecting data for the compute part and we are still far from the upper limit.
The Google Spreadsheet:
URL: http://goo.gl/kWI29e
Sheets:
1. Disk Size Algorithm - Interactive algorithm to calculate the baseline for data size on disk. Either update rows 6 through 9, columns A & B or add new rows in that section as per your installation. Once all the data is inserted you will get a total the bottom of the spreadsheet with estimate size per node. The number of storage nodes is also configurable (row 28, column B). If you have more storage nodes the data will get distributed across the cluster so the amount of disk required per node decreases.
2. Discrete Data - data collected from running simulations for collecting metrics and running aggregation. The amount disk used was captured at the end of each run without forcing compaction.
3. Long Run Charts & Long Run Data - During the 60 days simulation, data size on disk was collected on an hourly basis. That gives an indication on the patterns for how data grows (along with occasional spikes due to compaction).
4. Metrics Collection testbed - Data captured during the compute power testing. Data here is very preliminary and we will augment it as we collect more.
5. Changelog - we will keep a change log as we update the content.
How Use it:
1. If you just want to explore our findings, just follow the link above.
2. If you want to get an estimate for your particular installation or future installation, make a copy of the spreedsheet on your Google Drive. Then update the columns/rows marked with bright bleu on the "Disk Size Algorithm" sheet. Make sure to check back periodically the original spreadsheet as we are still working on refinements.
Wiki with additional information:
https://docs.jboss.org/author/display/RHQ49/Storage+Sizing+Analysis
Please feel free to reach to us if you have any questions or comments, we will be glad to answer and help. Any feedback is more than appreciated.
Thank you,
Stefan Negrea
Software Engineer
10 years, 3 months
Allocate exception for servlet ServerInvokerServlet
by Attila Heidrich
Something has happened to several of my RHQ servers. The 4.10 snapshot show
the same for ages, but now I see the same on the test version of the 4.9 as
well.
I found other guys with the same problem, but found no solution untill now.
https://access.redhat.com/site/solutions/625983
Any ide how to solve this?
I have tried to change the listen address from 0.0.0.0 to a more specific
one, but this is in rhq-server.properties, so I guess it is just taken at
install.
I have tried to remove the rhq-server java properties from root's home, and
restart the server (since "rhqctl install" didn't do anything), but the
listener address of the JBoss server is still the same: 0.0.0.0.
Attila
10 years, 3 months
Announcing RHQ Agent Plugin Plugin
by Thomas Segismont
Hi everyone,
I'm happy to announce the release of RHQ Agent Plugin Plugin!
Plugin Plugin? Yes. Plugin Plugin: a Maven plugin to help you build RHQ
Agent plugins.
It provides a handful of very convenient mojos for:
* Packaging
* Validation
* Deployment to a local RHQ Server (dev-container)
* Uploading to to a remote RHQ Server
* RHQ CLI script or command execution
* Standalone Plugin Container setup for integration tests
Dependent agent plugins are supported (many plugins depend on the parent
JMX plugin, for example).
The RHQ Agent Plugin Plugin documentation [1] is hosted on Github Pages
and the bits are available on JBoss Releases and Maven Central repositories.
Next week I will start to update agent plugins POM files in RHQ master
branch.
If you need help to get started with your own plugin development, feel
free to reply.
Regards,
Thomas
10 years, 3 months
4.9 seems to ignore the alert dampening settings
by Attila Heidrich
Hi!
It seems that there's no use to set any dampening for the alert tamplates,
I got all alerts for any of them.
I usually use "one for several hours"... practically I only use templates,
so the servers inherits the settings.
Is there any trick I miss? I have used 4.8 only a few days long, but as far
as I can remember, it worked as I expected.
Attila
10 years, 3 months
RHQ server subsystem elements Availability check failes
by Attila Heidrich
All resources under the RHQ server subsystems are reported unavailable:
Alert subsystem
Communications
Group definitions
Measurement
Remote API
f.e. for the "alert subsystem" the following exception is displayed in
the Component Errors window:
java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy60.isRegistered(Unknown Source)
at org.mc4j.ems.impl.jmx.connection.bean.DMBean.isRegistered(DMBean.java:188)
at org.rhq.plugins.jmx.MBeanResourceComponent.isMBeanAvailable(MBeanResourceComponent.java:242)
at org.rhq.plugins.jmx.MBeanResourceComponent.getAvailability(MBeanResourceComponent.java:229)
at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocation.call(ResourceContainer.java:654)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.jboss.remoting3.NotOpenException: Writes closed
at org.jboss.remoting3.remote.RemoteConnectionChannel.openOutboundMessage(RemoteConnectionChannel.java:108)
at org.jboss.remoting3.remote.RemoteConnectionChannel.writeMessage(RemoteConnectionChannel.java:297)
at org.jboss.remotingjmx.protocol.v2.Common.write(Common.java:180)
at org.jboss.remotingjmx.protocol.v2.ClientConnection$TheConnection.isRegistered(ClientConnection.java:741)
at sun.reflect.GeneratedMethodAccessor1895.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.mc4j.ems.impl.jmx.connection.support.providers.proxy.JMXRemotingMBeanServerProxy.invoke(JMXRemotingMBeanServerProxy.java:59)
... 13 more
Any idea how to fix it?
Attila
10 years, 3 months