Need advice about agent plugin design
by Steven North
I am trying to design an RHQ/JON agent plugin to manage a software
resource with the following characteristics:
- there is the software itself (the installation);
- there are a variable number of "bundles" of configuration information
about 250KB in size each which need to be read from and written to the
agent; and
- there are "log" files which can 10-50MB in size each which need to be
read from the agent.
I think I am pretty clear on how to handle the software itself--just
like any number of other agents.
I am not sure how to handle the configuration bundles and the large log
files.
We might want to have the RHQ/JON server manage different versions of
these configuration files and distribute them to multiple remote agents.
Is there some existing domain object that would handle the read/write
aspect of the configuration bundles (zip files)? Could the "package"
concept be used for these? Would we need to create a new domain object
on the server side for these bundles? If so, is there an example of
this kind of thing?
For the log files, I see some mention of the SupportFacet. Would this
be appropriate for retrieving large log files? Is there an example of this?
We expect to access the configuration bundles and the log files using
remote client operations because we have a separate GUI tool to
build/edit the configuration bundles and to correlate and analyze the
log files. Is there an example of using a remote client to pull files
from and push files to remote agents?
Thanks in advance for any advice you can give or examples you can point to.
Steve
10 years, 1 month
Replace old graphs in RHQ with GWT ones. GSOC 2012
by Denis Krusko
Hello!
My Name is Denis Krusko.
I am graduate student of Siberian Federal University (Russia).
I want to work with on the idea "Replace old graphs in RHQ with GWT ones"
I need to define milestones for 3 month like (proposed Heiko W.Rupp):
- evaluate Graphing options
- implement graphing to replace what is there
- resources
---- half time --
- possibility to e.g. take two metrics and graph them in one diagram
- write documentation
- graphs for the different kinds of groups ( can be in a later phase of
the project)
- remove old code
I would be grateful for comments and ideas.
Thank You,
Denis Krusko
11 years, 11 months
[GSOC-2012] Implement a Forge plugin to create RHQ plugins - idea
by Shameera Rathnayaka
Hi devs,
(I discussed this idea in the irc channel yesterday and would like to
present it here.)
I am an undergraduate student from University of Moratuwa, Sri Lanka. I
would like
to apply for GSOC 2012 with jboss community. Therefore I first went through
the ideas list[1]
published in the site and I am interested in contributing to "Implement a
Forge
plugin to create RHQ plugins" project. I already have chatted through the
irc channel
#rhq, and from that i could get a good introduction to Forge and RHQ
projects. However
as I haven't used them earlier can you please tell me what are the
technologies/frameworks
which i need to know in prior to involving in this project? I will use
this time to getting familiar
with them?.
I am very familiar with java as a programming language and have done many
projects with
it during my academic and internship period. I was able to successfully set
up forge in my
computer and I am currently reading the resources[2][3] provided withing
the above discussion.
I hope to research further about this to write a proposal for this proposed
idea.
It would be very helpful if you can provide me some advice on from where
should i start, what
are the things to be familiar with first etc.
Thank you very much!
[1] https://community.jboss.org/wiki/GSOC12Ideas
[2] http://vimeo.com/16171314
[3] http://www.redhat.com/f/pdf/Write_A_Plugin_WP_web.pdf
--
Shameera Rathnayaka
Undergraduate
Department of Computer Science and Engineering
University of Moratuwa.
Sri Lanka.
Blog : http://shameerarathnayaka.blogspot.com/
12 years
[GSoC 2012] Project: Implement an RHQ agent in Python
by Krzysztof Kwaśniewski
Hello!
I would like to apply this year for a GSoC project entitled
„Implement an RHQ agent with Python” that I have found on your
proposals' list [1].
As this is my first message in this list, firstly I would like to
write a bit about me. My name is Krzysztof Kwaśniewski. I study IT at
Gdańsk University of Technology in Poland, 4th year. Just recently I
graduated, become a Bachelor and started the second (MSc.) stage of
studies. Last year I have taken part in GSoC with the Tcl organization,
however their application was declined this year. My task was to prepare
an extension to GDB that was intended to make debugging of Tcl
interpreter easier. I have implemented that extension in Python - the
repository of my last year's project may be found under the link [2].
Apart from Python, which is one of my favorite programming
languages, I also like Java and C very much. Moreover I know C# and a
couple of other languages that I have - however - little contact with.
For about 7 years I have been using Linux.
Heiko Rupp has defined following initial requirements that need to
be concerned for this year's project:
- installing the needed stuff
- writing the 'agent'
- agent reports some metrics and availability of resources
- agent discovers the resources
- agent startable at system boot
- prepare documentation for both the developer and the end user
Definitely the project needs clarification and we will do our best
to make it. The points above are only initial ones. I'm sure that any
comments from other members of this list would be helpful to make this
project successful and provide useful software. If you have any ideas or
requirements for this project, please let me know about them.
Thanks
Krzysiek
--
[1]
https://community.jboss.org/wiki/GSOC12Ideas?_sscc=t#Implement_an_RHQ_age...
[2] http://chiselapp.com/user/krzykwas/repository/nredebug1/dir?ci=tip
12 years, 1 month
(as7) itests: @Inject resources?
by Heiko W.Rupp
I am just trying to write an itest for the socket binding groups and see myself
spending a lot more time searching for the resource(s) in the plugin container
than for the actual test.
Could we implement some @Inject (a la CDI)
@Inject(key="socket-binding-group=standard-sockets)
Resource testTarget
and/or
@Inject(type="subsystem=foo,type=*")
List<Resource> testTargets
of
@Inject(type="socket-binging-group")
List<Resource> bindingGroups;
If the test code is executed with @Test(groups="standalone"), the standalone server would be used as parent
otherwise with groups="domain" the HC
Of course the above is as7 specific wrt type, but I guess could be handled by the layer doing the injection.
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Mark Hegarty, Charlie Peters, Michael Cunningham, Charles Cachera
12 years, 1 month
AS7 Subsystems, Service vs. Server
by Stefan Negrea
Hello Everybody,
Working on AS7 plugin, I noticed that most (but not all) of the AS7 subsystems are configured as server resources. So what should new subsystems being added to the plugin be mapped to, service or server? It is a hard decision to take with no guidelines in place. Heiko suggested I should to ask for community feedback.
I see one obvious case, if a subsystem does not have any children then it should be a service. And of course there are the two extremes; all subsystems mapped to server resources or all subsystems mapped to service resources.
Should we use a middle ground approach (mix servers and services)? What would be a good differentiator for what gets mapped to server vs. service among the subsystems?
Or just go with one single type of resource for all subsystem? Should it be service or server?
Thank you,
Stefan Negrea
Software Engineer
12 years, 1 month
Re: [itest,as7] two test failures in standalone mode
by Heiko W.Rupp
Am 25.03.2012 um 04:01 schrieb Ian Springer:
> On 03/24/2012 09:16 AM, Heiko W.Rupp wrote:
>> Things seem to have improved (somewhat)
>>
>> When I run the itests a few times in a row, the results are very different.
>>
>> In the most positive case, I get only one failure
>
>
> That's good - we're making progress. However, the operations tests are still failing for me when I try to enable them. See the comments I added to that BZ.
I've seen that and am trying to chase it.
Currently with sources from after the weekend tests constantly fail on RHEL with
Failed tests: arquillianBeforeTest(org.rhq.modules.plugins.jbossas7.itest.standalone.StandaloneServerComponentTest): Failed to execute @AfterDiscovery method [public void org.rhq.modules.plugins.jbossas7.itest.AbstractJBossAS7PluginTest.installManagementUsers() throws org.rhq.core.clientapi.agent.PluginContainerException] - cause: java.lang.reflect.InvocationTargetException
So everything else is skipped.
>> Failed tests: testReleaseVersionTrait(org.rhq.modules.plugins.jbossas7.itest.standalone.StandaloneServerComponentTest): Unexpected value for trait [_skm:release-version]. expected:<7.1.1.Final> but was:<7.1.0.Final-redhat-1>
>>
>> Where I think the assumption may be wrong, it means the tests in total run correctly.
>>
>> (command line is [hrupp@pintsize jboss-as-7]$ mvn verify -DskipTests=false -Pas7.itest -Das7.version=7.1.1.Final )
>
>
> Hmm, the "7.1.0.Final-redhat-1" value makes it looks like you're connected to EAP6, but your command line should download/unzip/start AS 7.1.1...
Actually it may have been that an eap was indeed runinng, which would indicate that the port setup would be wrong, but I retried with a
running eap in standalone mode and the situation did not change - still "arquillianBeforeTest" is failing.
On OS/X I see the same failure.
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Mark Hegarty, Charlie Peters, Michael Cunningham, Charles Cachera
12 years, 1 month
Re: Re: [GSoC 2012] Project: Implement an RHQ agent in Python
by Krzysztof Kwaśniewski
As to my answer written yesterday, I got confused about this one:
(...)
> It would then talk to the rhq server via REST protocol and - add new
> platforms ("boxes" in rhq speak) if they get discovered [1]
Is the code mentioned in [1] used for that kind of subscription mechanism?
(...)
So the code in [1] is for RHQ and has nothing in common with Matahari
and my question doesn't make sense then. ;) Sorry
Thanks
Krzysiek
12 years, 1 month
Re: Re: [GSoC 2012] Project: Implement an RHQ agent in Python
by Krzysztof Kwaśniewski
> Message: 1
> Date: Fri, 23 Mar 2012 09:35:33 +0100
> From: "Heiko W.Rupp"<hrupp(a)redhat.com>
> To: rhq-devel(a)lists.fedorahosted.org
> Subject: Re: [GSoC 2012] Project: Implement an RHQ agent in Python
> Message-ID:<C0A68369-4DC4-4566-A4ED-A46B83EAAB8E(a)redhat.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hello Krzysztof,
>
> cool to hear from you
>
> I have attached a quick drawing tha should illustrate what I have in mind here
>
>
> So the project would involve the box with the blue dashed line.
>
> On Fedora / RHEL (and perhaps other Linuxes), Matahari agents collect data from the OS (e.g. system load, free memory).
> The agent in Python ("PyAgent") would connect to Matahari via qpid (amqp) and find the matahari instances.
Does it mean that Matahari agents subscribe to some kind of „host” which
can be used later on by PyAgent to find them?
>
> It would then talk to the rhq server via REST protocol and
> - add new platforms ("boxes" in rhq speak) if they get discovered [1]
Is the code mentioned in [1] used for that kind of subscription mechanism?
> - add new resources below platform if needed (later phase of the project)
Probably the resources will be „provided” by Matahari in some way,
right? That is - Matahari will have info for PyAgent about what
resources are available to be monitored and what collected data to pass
to RHQ?
> - fetch metrics from matahari and send them to the RHQ server
> - regularly send availabilty of the platforms and resources to the RHQ server
> - if needed [2] send pings to the RHQ server
>
>
> Am 22.03.2012 um 22:29 schrieb Krzysztof Kwaśniewski:
>
>> I would like to apply this year for a GSoC project entitled „Implement an RHQ agent with Python” that I have found on your proposals' list [1].
>> Heiko Rupp has defined following initial requirements that need to be concerned for this year's project:
>>
>> - installing the
>> needed stuff
> Assemble a bill of material and an install guide. Perhaps a shell-script or rpm super-packet for installation
>> -
>> writing the 'agent'
> Core functionality is described above and includes:
>>
>> -
>> agent reports some metrics and availability of resources
>>
>> -
>> agent discovers the resources
>>
>> -
>> agent startable at system boot
> It should be possible that his agent is installed on one box so that it can get started at
> boot time (-> script in /etc/init.d/ )
>
>> - prepare documentation for both the developer and the end user
>> Definitely the project needs clarification and we will do our best to make it. The points above are only initial ones. I'm sure that any comments from other members of this list would be helpful to make this project successful and provide useful software. If you have any ideas or requirements for this project, please let me know about them.
> Does the above make it more clear for you?
Yes, it's quite clear. The description of the project also includes
a piece of info that Java agents are more powerful than PyAgent is
expected to become: „This agent will not implement the full
functionality of the RHQ java agent”. Once I get more familiar with RHQ
and Java agents, I will know the functionality of Java agents and we
could then make some preliminary assumptions about what part of this
functionality is expected to be provided by PyAgent. Or maybe you mean
just the things mentioned above about passing data from Matahari, then
it's also clear.
For now the following tasks have been mentioned:
1. Get to know the platform and install all the necessary software, set
up repository and maybe a blog about the project (?).
2. Connect to Matahari via qpid and find Matahari instances.
3. For each found instance add a corresponding platform to RHQ.
4. Discover the resources provided by each instance of Matahari.
5. Periodically report availability of resources and metrics provided by
Matahari to RHQ.
6. Ensure PyAgent periodically „pings” RHQ with what exactly „ping”
means will be decided later on (whether it is a rest ping, an „UP”
message or something different).
7. Implement PyAgent in a way that will allow it to be started at boot time.
8. Prepare documentation and an installation script/package.
I feel it would help to prepare a set of unit tests for PyAgent,
though it might happen not to be easily testable this way, as huge part
of the logic will be connected with communication over rest and ampq.
> I think you should also start to think about a timeline for the work with some milestones in it.
Right now I'm in a middle of a semester. If I had more time now, I
would probably start the project at once. According to the timeline of
GSoC, the coding phase starts on 21.05 and the mid-term evaluation
starts on 9.07. I will try to outline an approximate schedule below -
what do you think about it?
- I expect it would be possible to realize tasks 1. and 2. until 21.05
if time permits (depending on my duties at the university).
- It's hard to assess the points 3.-5. now - without knowing the api,
however it seems to be well documented. Though I would like not to give
any exact date yet. Maybe I will need 2 weeks for that or maybe 4. In
any case we should still be in June after task 5.
- Pinging RHQ seems to be a rather simple concept and starting PyAgent
at boot time too. I should need at most two weeks for those tasks.
- Another week would be needed for the last task.
This rough timeline is not precise enough, but there will probably
arise more tasks during the implementation. What do you think about the
complexity of each of the tasks 3.-5.?
> Thanks
> Heiko
>
>
> [1] This needs some more code on the RHQ server side, which already exists, but which is not yet in master
> [2] To report agents being alive in the past it was enough to send "UP" availability for the platform. We had a change in availability handling
> and now there is a separate ping for this purpose. It needs to be checked what to implement in the RHQ server REST endpoint for this; I expect
> that the PyAgent would just send a request to a different endpoint for this.
>
Thanks
Krzysiek
12 years, 1 month
[itest,as7] two test failures in standalone mode
by Heiko W.Rupp
java.lang.AssertionError
at org.rhq.modules.plugins.jbossas7.itest.AbstractServerComponentTest.getServerResource(AbstractServerComponentTest.java:59)
at org.rhq.modules.plugins.jbossas7.itest.AbstractServerComponentTest.testMetricsHaveNonNullValues(AbstractServerComponentTest.java:75)
at org.rhq.modules.plugins.jbossas7.itest.standalone.StandaloneServerComponentTest.testMetricsHaveNonNullValues(StandaloneServerComponentTest.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
The assertion error is that getServerResource() was actually not able to obtain the resource and would return null (which then later
would show as random NPEs)
06:20:02,155 WARN [PluginClassLoader] Failed to purge embedded jars directory. Cause: java.io.IOException: Failed to delete file or directory: /tmp/TEST_RHQ_PC_DEPLOYMENTS4718447418621299904.tmp/pcs/pc/tmp/rhq-jboss-as-7-plugin-4.4.0-SNAPSHOT.jar5979248820050881201.classloader
06:20:02,883 WARN [DriftManager] Could not create change sets directory /tmp/TEST_RHQ_PC_DEPLOYMENTS4718447418621299904.tmp/pcs/pc/data/changesets
06:20:03,101 ERROR [InventoryManager] Missing platform plugin(s) - falling back to dummy platform impl; this should only occur in tests!
06:20:03,110 WARN [InventoryManager] Cannot give activated resource its discovery component. Cause: org.rhq.core.clientapi.agent.PluginContainerException: Cannot get discovery component proxy for [null]
06:20:03,111 WARN [InventoryManager] Cannot give activated resource its discovery component. Cause: org.rhq.core.clientapi.agent.PluginContainerException: Cannot get discovery component proxy for [null]
06:20:03,118 WARN [DriftManager] Could not create change sets directory /tmp/TEST_RHQ_PC_DEPLOYMENTS4718447418621299904.tmp/pcs/pc/data/changesets
(6:20:03 AM) Running test [StandaloneServerComponentTest.testMetricsHaveNonNullValues] which is in groups [pc, standalone]...
Locally on RHEL6 I also see
11:00:23,929 WARN [RuntimeDiscoveryExecutor] Exception caught while executing runtime discovery scan rooted at [platform].
java.lang.RuntimeException: java.util.concurrent.RejectedExecutionException
at org.rhq.core.pc.inventory.InventoryManager.synchInventory(InventoryManager.java:1120)
at org.rhq.core.pc.inventory.InventoryManager.handleReport(InventoryManager.java:1054)
and
11:00:23,917 ERROR [RuntimeDiscoveryExecutor] Error in runtime discovery
java.lang.NullPointerException
at org.rhq.core.pc.plugin.PluginComponentFactory.getDiscoveryComponent(PluginComponentFactory.java:84)
at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:243)
at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResourceRecursive(RuntimeDiscoveryExecutor.java:156)
at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.runtimeDiscover(RuntimeDiscoveryExecutor.java:141)
at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.call(RuntimeDiscoveryExecutor.java:100)
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Mark Hegarty, Charlie Peters, Michael Cunningham, Charles Cachera
12 years, 1 month