Need advice about agent plugin design

Steven North swn at ocsystems.com
Thu Jun 16 23:01:18 UTC 2011


John,

Nice to talk with you and thanks for the response.

Please see below...

On 6/16/2011 2:23 PM, John Mazzitelli wrote:
> OK, Steve, sorry for not answering this earlier. The reason is because I
> don't think I have good answers for them :) Either that or I don't have
> a good grasp of what you need.
>
> On 05/29/2011 09:13 PM, Steven North wrote:
>> I am trying to design an RHQ/JON agent plugin to manage a software
>> resource with the following characteristics:
>>
>> - there is the software itself (the installation);
>> - there are a variable number of "bundles" of configuration information
>> about 250KB in size each which need to be read from and written to the
>> agent; and
>> - there are "log" files which can 10-50MB in size each which need to be
>> read from the agent.
> We have the Event subsystem that can read log files and report them back
> up to the server - one log message equals one RHQ event. However, this
> is normally not used in an on-going basis for constant and continual
> monitoring of log files. People usually just enable events on a resource
> for a short time, to collect logs for a particular resource while doing
> debugging or analysis. The backend event storage in RHQ is strictly in
> the normal RHQ database and as such is not efficient for storing large,
> ongoing log files (do not think of this as a "Splunk" type model). So
> I'm not sure if the events subsystem is what you want, but at least its
> something to look at.
OK.  I looked at that and it basically scrapes error/debug messages from 
log files and turns them into events.  That would not be appropriate for 
the amount/kind of data we have.
>> I think I am pretty clear on how to handle the software itself--just
>> like any number of other agents.
>>
>> I am not sure how to handle the configuration bundles and the large log
>> files.
>> We might want to have the RHQ/JON server manage different versions of
>> these configuration files and distribute them to multiple remote agents.
> We used to have the concept of "Raw Configuration" (which was to support
> raw configuration files) but time constraints among other things cause
> us to stop that work (though some of that work still exists in the code,
> its not fully baked, we plan on returning to that at some point). So,
> this is a non-starter unless and until the raw config support is completed.
>
> We could look at the "ResourceFactoryFacet" which is how we do things
> like deploy individual files to a resource (we use this, for example, to
> deploy a WAR file to JBossAS). Its conceivable this can be used to ship
> any file you want - this is the<content>  metadata tag in the descriptor.
>
> Finally, the new bundle subsystem can also be considered, however, its
> purpose was mainly to support shipping out bulk distributions of entire
> software installs (like a JBossAS distribution zip file or something
> like that). But there is no reason why it couldn't be used to ship out
> smaller bundles of content. I just committed into the master branch the
> ability to push bundles to non-platform resources (see BZ 644328 at
> https://bugzilla.redhat.com/show_bug.cgi?id=644328 ), which would
> probably make this a bit easier and might make it be more acceptable to
> use bundles for something like this.
>
> I'm just throwing things out there - you'd have to look at these
> concepts and see what makes sense and if you want to use those to do
> what you want.
OK, it sounds like the ResourceFactoryFacet is the way to go, or the new 
bundle support.  I'll have to take a closer look.
>> Is there some existing domain object that would handle the read/write
>> aspect of the configuration bundles (zip files)?  Could the "package"
>> concept be used for these?  Would we need to create a new domain object
>> on the server side for these bundles?  If so, is there an example of
>> this kind of thing?
>>
> Right - the "package" concept you are talking about is that
> "ResourceFactoryFacet" /<content>  thing I mentioned above. That might
> be useful here.
>
> Creating a new domain object or adding server-side EJBs cannot be done.
> We aren't pluggable in that manner. That would require a fork/patch to
> RHQ, and obviously mean you'd have to package and build RHQ yourself
> with your changes.
>
> We are pluggable on the agent side using agent plugins, and we are
> pluggable on the server side using server-plugins, but as for adding
> domain objects and server-side business tier objects like EJBs, we
> aren't easily pluggable there.
To be honest, I didn't want to go this route.  ;-)
>> For the log files, I see some mention of the SupportFacet.  Would this
>> be appropriate for retrieving large log files?  Is there an example of this?
>>
> Yes, I believe (from just talking to you) you figured this out :) The
> SupportFacet allows you to zip up any files you want. There is no
> support for capturing snapshots in the GUI today, its only available
> through the CLI. If you think this is a useful feature, we should
> probably consider doing something in the GUI to have a better
> integration here.
Right, I did that through the CLI.  I expect the large log files would 
only be "pulled" from the agent in response to an alert, though it might 
be a regularly scheduled operation for tracking changes.  If I had no 
constraints, I would have that log file be stored by the server with the 
annotation of the alert or operation that caused it to be collected, and 
later someone could extract that from the server via CLI and analyze it 
in a separate tool, or launch the tool directly from the server GUI with 
the log file.  Or even more pie in the sky, an alert might trigger a 
change of configuration via the agent to collect diagnostic information 
for a set interval, and then resume normal settings and pull the 
diagnostic data back for storage.  I don't know if that is feasible (or 
desirable).
>> We expect to access the configuration bundles and the log files using
>> remote client operations because we have a separate GUI tool to
>> build/edit the configuration bundles and to correlate and analyze the
>> log files.  Is there an example of using a remote client to pull files
>> from and push files to remote agents?
> The architecture and design of RHQ is to specifically prohibit remote
> clients from talking directly to agents. Servers talk to agents and if
> remote clients want to talk to agents, they must do so indirectly
> through the server over the remote server API.
>
> You could write a server-side plugin to act as a proxy to the agent and
> do specific things, but the server plugin would still need to go through
> the server-side EJB/SLSB API to talk to the agent.
>
> Your remote clients can push/pull content from the agent indirectly
> using things like the ResourceFactoryFacet (the package stuff you were
> referring to, though that is not really appropriate for log files and
> config files and the like, that is more for things like content that
> doesn't change - like versioned jar libraries and things like that) or
> the SupportFacet or the Bundle subsystem.
>
> Of course, there is nothing stopping your custom agent plugin component
> from exposing its own server-socket (for example) and listen for custom
> remote client comm, but that just circumvents RHQ entirely. Still, that
> is a possibility. Depends on what you want to do.
Right, I didn't mean pushing/pulling to/from an agent directly, always 
through the server.
> John Mazz
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/rhq-devel



More information about the rhq-devel mailing list