deploying metrics db during RHQ installation

John Sanda jsanda at redhat.com
Tue Sep 11 20:00:06 UTC 2012


I do not think auto import is sufficient though for out of the box self-management. This is why I suggested an event notification mechanism whereby an event listener could perform any necessary management functions on the auto imported resources. For those resources that ought to be self-managed, like the RHQ server, RHQ agent, and metrics db, we should consider doing things like automatically setting up event logging and any alert definitions that facilitate that effort.

On Sep 11, 2012, at 3:24 PM, "Larry O'Leary" <loleary at redhat.com> wrote:

> +1 
> 
> And the same goes for the RHQ server.
> 
> The downside of course is additional agent load and larger metric
> reports.
> 
> Perhaps this is a plug-in configuration setting? For example, from the
> administration page you select agent plug-ins and from there you have an
> option to configure a plug-in. In the configuration you can specify per
> resource/child resource settings of Auto Import = Yes|No.
> 
> 
> 
> On Tue, 2012-09-11 at 21:11 +0200, Lukas Krejci wrote:
>> Slightly OT, but I think it is worth adding this to the discussion.
>> 
>> Some aspects of your problem are similar to an extent with a problem I have 
>> been thinking about recently - that of user visibility of agent 
>> performance/configuration/problems, etc.
>> 
>> It is I think unacceptable to continue with the current practice of the RHQ 
>> agent being "just another resource". Consider this imaginary conversation:
>> 
>> CUSTOMER: my agent doesn't collect metrics
>> US: send us agent logs
>> CUSTOMER: I can't even SSH in, the agent seems to clog up the machine.
>> US: Try running the "stop" operation on RHQ agent resource, maybe it's gonna 
>> push through.
>> CUSTOMER: What? I don't have such thing in my inventory.
>> US: Tough.
>> 
>> Apart from the fact that the user should have been notified about a coming 
>> problem on the agent prior to the agent becoming unresponsive, I DO think that 
>> agents are part of RHQ deployment and RHQ as such should be self-manageable. 
>> There should be no user intervention needed to setup the "parts" needed for 
>> management of the whole deployment.
>> And it should be the same with the agents managing Cassandra cluster.
>> 
>> As you say - these "resources", however we decide to implement it, should be 
>> autoimported and I think it should also NOT be possible to uninventory them 
>> (maybe not even disable them).
>> 
>> On Tuesday, September 11, 2012 12:33:33 John Sanda wrote:
>>> There are a couple items I wanted to follow up from my original email. The
>>> first is that the Cassandra node should be committed in inventory by the
>>> end of the installation process. Today we do not have the ability to
>>> auto-import resources, at least not top-level servers. We need more than
>>> the ability to just auto import. We need an event notification mechanism
>>> where an event is fired could when the server receives an inventory report
>>> that includes new resources. An event listener could then import resources
>>> of interest and perform other tasks like scheduling operations, configuring
>>> event logging, etc. While this will be necessary for our metrics db, it
>>> might be generally useful feature as well.
>>> 
>>> The second thing I wanted to follow up on is the schedule operations, alert
>>> definitions, event logging, etc. that should be set up and configured. What
>>> exactly needs to be defined and configured is TBD. It will become more
>>> clear as this work moves further along.
>>> 
>>> - John
>>> 
>>> On Sep 11, 2012, at 10:37 AM, John Sanda <jsanda at redhat.com> wrote:
>>>> Below is a summary of my thoughts thus far on what will be involved with
>>>> deploying the new metrics database during RHQ installation. This
>>>> information is available on the wiki at
>>>> https://docs.jboss.org/author/display/RHQ/Acceptance+Criteria.
>>>> 
>>>> Let's start with the simple case involving a single node. During the RHQ
>>>> installation process, the user will have to provide several pieces of
>>>> information so that we can install and configure the node. For basic
>>>> configuration settings that require values this includes:
>>>> 
>>>> * The host name of the machine on which Cassandra will be installed
>>>> * A writable directory where Cassandra will store the commit log and data
>>>> files * User credentials (more on this in the security section)
>>>> * Location for backups (more on this in the backup section)
>>>> 
>>>> For advanced settings for which we will supply default values, this
>>>> includes:
>>>> 
>>>> * The various ports including rcp_port which is the port Cassandra listens
>>>> on for Thrift clients * key and row cache settings like the cache sizes
>>>> and retention periods for cache entries * Settings for the commit log
>>>> which control when writes are acked and how frequently it is flushed to
>>>> disk * The JMX port which is the port on which Cassandra listens for JMX
>>>> connections
>>>> 
>>>> Nearly all of the user-supplied configuration values will go into
>>>> cassandra.yaml. The JMX port will be written into cassandra-env.sh. We
>>>> will need to generate these files with the values supplied by the user.
>>>> The location of backups is unique in that it pertains to how RHQ will
>>>> manage the Cassandra node. That information will need to be stored in the
>>>> RHQ database.
>>>> 
>>>> Users are not required to install and run an agent on the same machine on
>>>> which the RHQ server runs. The user will however have to install an agent
>>>> onto the machine on which the Cassandra node is installed so that we can
>>>> fully manage it. This brings up the order of installation with respect to
>>>> the Cassandra node, the RHQ server, and the RHQ agent. The key thing is
>>>> that when the RHQ installer finishes, the Cassandra node should be
>>>> running and fully configured for management. By that I mean that the
>>>> Cassandra node should already be imported into the server's inventory,
>>>> maintenance operations should already be scheduled, alert definitions
>>>> should already be created, event logging configured, and so on.
>>>> 
>>>> If deploying multiple nodes, there are two additional properties that must
>>>> be set - seeds and initial_token. seeds is a list of host names or IP
>>>> addresses where nodes will be running. This list is used at start up so
>>>> that a node can learn the topology of the cluster. initial_token
>>>> specifies the range of the token ring for which the node is responsible.
>>>> There is a simple function to use for calculating tokens. It is possible
>>>> to run multiple nodes on a single machine for production use, but each
>>>> node has to have its own IP address.
>>>> 
>>>> - John
> 
> 
> 
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel



More information about the rhq-devel mailing list