Need advice about agent plugin design
by Steven North
I am trying to design an RHQ/JON agent plugin to manage a software
resource with the following characteristics:
- there is the software itself (the installation);
- there are a variable number of "bundles" of configuration information
about 250KB in size each which need to be read from and written to the
agent; and
- there are "log" files which can 10-50MB in size each which need to be
read from the agent.
I think I am pretty clear on how to handle the software itself--just
like any number of other agents.
I am not sure how to handle the configuration bundles and the large log
files.
We might want to have the RHQ/JON server manage different versions of
these configuration files and distribute them to multiple remote agents.
Is there some existing domain object that would handle the read/write
aspect of the configuration bundles (zip files)? Could the "package"
concept be used for these? Would we need to create a new domain object
on the server side for these bundles? If so, is there an example of
this kind of thing?
For the log files, I see some mention of the SupportFacet. Would this
be appropriate for retrieving large log files? Is there an example of this?
We expect to access the configuration bundles and the log files using
remote client operations because we have a separate GUI tool to
build/edit the configuration bundles and to correlate and analyze the
log files. Is there an example of using a remote client to pull files
from and push files to remote agents?
Thanks in advance for any advice you can give or examples you can point to.
Steve
10 years, 1 month
POC: Operation scheduling over the REST api
by Heiko W.Rupp
Hi,
feature/rest branch now has the ability to schedule resource operations. This is still very limited as it
only allows for parameter-less operations at the moment and also only reports the operation status,
but show how this can be done in a way that can be done from a variety of languages.
I've tried to use the linking in this approach to start with one public endpoint for a client to know
Here is a log of the actions done with some comments.
For those less familiar with rest, the links below sometimes have some "standard"-rel:
- a rel of "edit" means rud (PUT,GET,DELETE)
- a rel of "self" gives the location of an edit
- a rel of "create" means POST (could also mean PUT when the id of the object to create is known to the client)
Heiko
# Getting the platforms
$ curl --user rhqadmin:rhqadmin http://localhost:7080/rest/1/resource/platforms -HContent-Type:application/json
[{"typeName":"Mac OS X",
"resourceName":"snert",
"parentId":0,
"pluginName":"Platforms",
"resourceId":"10001",
"typeId":10020}]
# Getting one resource
$ curl --user rhqadmin:rhqadmin http://localhost:7080/rest/1/resource/10001 -HContent-Type:application/json
{"typeName":"Mac OS X",
"resourceName":"snert",
"parentId":0,
"pluginName":"Platforms",
"resourceId":"10001",
"links":[
{"rel":"operationDefinitions",
"href":"http://localhost:7080/rest/1/operation/definitions?resourceId=10001"},
{"rel":"self",
"href":"http://localhost:7080/rest/1/resource/10001"},
{"rel":"schedules",
"href":"http://localhost:7080/rest/1/resource/10001/schedules"},
{"rel":"children",
"href":"http://localhost:7080/rest/1/resource/10001/children"}
],
"typeId":10020}
# Get the operation definitions for that resource
$ curl --user rhqadmin:rhqadmin http://localhost:7080/rest/1/operation/definitions\?resourceId=10001 -HContent-Type:application/json
[
{"name":"discovery",
"id":10024,
"links":[
{"rel":"create",
"href":"http://localhost:7080/rest/1/operation/definition/10024?resourceId=10001"}
]
},
{"name":"viewProcessList",
"id":10025,
"links":[
{"rel":"create",
"href":"http://localhost:7080/rest/1/operation/definition/10025?resourceId=10001"}
]
}
]
# Get one operation definition
$ curl --user rhqadmin:rhqadmin http://localhost:7080/rest/1/operation/definition/10025\?resourceId=10001 -HContent-Type:application/json
{"name":"viewProcessList",
"id":10025,
"links":[
{"rel":"create",
"href":"http://localhost:7080/rest/1/operation/definition/10025?resourceId=10001"}
]
}
# Create a new (draft) operation from the definition above
# ( use > /tmp/foo to store the json for further local processing )
$ curl --user rhqadmin:rhqadmin http://localhost:7080/rest/1/operation/definition/10025\?resourceId=10001 -HContent-Typcation/json -X POST
{"name":"viewProcessList",
"id":-970811016,
"state":"creating",
"params":{},
"resourceId":10001,
"definitionId":10025,
"links":[
{"rel":"edit",
"href":"http://localhost:7080/rest/1/operation/-982775384"}
]
}
# View the draft (not necessary for scheduling)
$ curl --user rhqadmin:rhqadmin http://localhost:7080/rest/1/operation/-970811016 -HContent-Type:application/json -X GET
{"name":"viewProcessList",
"id":-970811016,
"state":"creating",
"params":{},
"resourceId":10001,
"definitionId":10025,
"links":[
{"rel":"edit",
"href":"http://localhost:7080/rest/1/operation/-982775384"}
]
}
# Put an update (state field is "ready" now) and thus schedule the operation to run
# input in /tmp/foo is the (modified) return from the POST above
# Note state has to be "ready" for scheduling
$ curl --user rhqadmin:rhqadmin http://localhost:7080/rest/1/operation/-970811016 -HContent-Type:application/json -X PUT -d @/tmp/foo
{
"name":"viewProcessList",
"id":-970811016,
"state":"ready",
"params":{},
"resourceId":10001,
"definitionId":10025,
"links":[
{"rel":"history",
"href":"http://localhost:7080/rest/1/operation/history/rhq-resource-10001-7355903..."}
]
}
# See how the edit link is now gone and a history link points to the operation result
$ curl --user rhqadmin:rhqadmin http://localhost:7080/rest/1/operation/history/rhq-resource-10001-7355903...
Successs
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Brendan Lane, Charlie Peters, Michael Cunningham, Charles Cachera
12 years, 4 months
some git usage guidelines
by Ian Springer
I made a couple gitstakes recently (if interested in the details, see
the bottom of this message). Here are the lessons learned from the
mistakes. Learn them and live them:
a) If at all possible, contain the entire fix for a BZ to a single
commit, but if more than one commit is needed for the fix, make sure to
list all the commit hashes in the BZ. This is critical so that if the
fix needs to be cherry-picked, all of the commits can be cherry-picked
at the same time and pushed atomically. Also, try to limit commits that
fix a BZ to the bare minimum changes required for the fix. Don't include
other fixes (or parts of other fixes) or reformatting (i.e. whitespace
changes). Such additional inclusions make it harder for someone else to
understand or review the fix.Using local branches can help here. Have
one branch per BZ to help isolate the changes you are making for a
specific fix.
b) 99% of the time, when you do a git pull, you should specify --rebase
so that all pending commits are re-applied after the pull. There are two
reasons for this: i) it prevents "merge commits" which clutter the git
history, and ii) when the commits are re-applied, their dates get
updated, which keeps the git history more linear, particularly if a push
is done right after the pull.
You can configure certain branches to always pull with the --rebase flag:
# make 'git pull' on master and release_jon3.x always use rebase
$ git config branch.master.rebase true
$ git config branch.release_jon3.x.rebase true
You can also set up a global option to set the last property for every
new tracked branch:
# setup rebase for every tracking branch
$ git config --global branch.autosetuprebase always
All devs should set the above config options.
I'll incorporate these guidelines into the RHQ wiki in a few days after
I gather any feedback from this email.
------------------------
This page has some other good git tips and tricks:
http://mislav.uniqpath.com/2010/07/git-tips/
-----------------------
Here are the details of my gistakes. The first one caused the Jenkins
job for the JON3 release branch to fail due to a compilation error, and
the second one made it harder to dtermine the root cause of the Jenkins
failure.
1) I pushed two commits to master, each to fix a BZ. However, one of the
BZs required a change from the other BZ's commit. Since I pushed the two
commits at the same time on master, there was no issue there. However, I
cherry-picked the first commit to the release branch and forgot to
cherry-pick the second commit, which caused the above-mentioned
compilation error, due to the fix which actually required both commits.
2) Before pushing the commits, I did a git pull, without specifying the
--rebase option, which resulted in a "merge commit" which unnecessarily
cluttered the git history. The commits had been made two weeks earlier;
but because I did not rebase them, they ended up with the dates from two
weeks ago, which made the git history non-linear and the Jenkins failure
hard to diagnose ("Why did this just fail now when the commit with the
error was made two weeks ago?").
12 years, 4 months
firstResult/maxResults Hibernate warnings
by Ian Springer
I'm running JON 3.0 CR2 and seeing five of these warnings a minute,
pretty much flooding my Server log. In the short term, I think we should
turn the log level down to ERROR for QueryTranslatorImpl. What do others
think?
11:25:10,680 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:25:24,896 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:25:37,147 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:25:43,738 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:26:06,684 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:26:10,681 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:26:24,896 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:26:37,147 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:26:43,738 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:27:06,683 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:27:10,681 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:27:24,897 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:27:37,147 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:27:43,738 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:28:06,684 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:28:10,681 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:28:24,896 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:28:37,148 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:28:43,738 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:29:06,683 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:29:10,689 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:29:24,894 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:29:37,146 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:29:43,739 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:30:06,682 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:30:10,676 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:30:24,896 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:30:37,146 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:30:43,739 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:31:06,684 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:31:10,681 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:31:24,896 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:31:37,147 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:31:43,738 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:32:06,900 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:32:10,680 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:32:24,896 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:32:37,146 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:32:43,734 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:33:06,681 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:33:10,681 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
11:33:24,896 WARN [QueryTranslatorImpl] firstResult/maxResults
specified with collection fetch; applying in memory!
etc.
12 years, 5 months
Q. about operation scheduling
by Heiko W.Rupp
Hi,
(this is wrt REST work)
When I schedule a resource operation with
ResourceOperationSchedule sched = opsManager.scheduleResourceOperation(caller,operation.getResourceId(),operation.getName(),0,0,0,-1,new Configuration(),"TEst");
How do I get the History item from it.
And do I need to care about those "in progress states" at all?
Basically how do I get the required jobId in
OperationHistory history = opsManager.getOperationHistoryByJobId(caller,jobName); ?
Thanks
Heiko
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Brendan Lane, Charlie Peters, Michael Cunningham, Charles Cachera
12 years, 5 months
Seeing parent resource implies seeing children?
by Heiko W.Rupp
Hi,
Question about access rights:
When a user can access a resource X, does that always imply he can also see children of X?
(The other way obviously does not work)
Or can there a case be constructed where the user e.g. only sees a JBAS server, but not its
services?
Thanks
Heiko
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Brendan Lane, Charlie Peters, Michael Cunningham, Charles Cachera
12 years, 5 months
Remove Maven SCM Tags
by Stefan Negrea
Hello Everybody,
Just to give some background, I recently replaced the maven release plugin with versions plugin in all the release scripts. Another change, all the scm commands in the scripts are now pure git commands. So the release scripts do not rely on maven for scm operations.
With this in mind, all the pom files have the scm tag. My suspicion is that those were added to support the maven release plugin. I would like to remove all the tags from the all the pom files for a couple of reasons:
1) The scm tags are no longer used by the release process. Maven versions does not need this tag.
2) The scm tags are not correct today because they do not track the correct branch. These tags are supposed to have the full path of the git tag or branch for the respective modules.
3) It is dangerous to have these tags because git is used for day-to-day branching, tagging, and committing. Unaware users that attempt to use maven scm commands would make commits to the wrong branch (most likely master) without knowing. Maven scm plugin relies on this scm tag to track the branch that will receive the commits. However this tag is not correctly updated because of direct git usage.
4) Minimal pom files (without vestiges) are easier to maintain and understand.
Does anybody see a problem with removing the scm tags? If there are no valid use cases, I will go ahead and remove them later next week. This change is very simple and will not impact anything.
Thank you,
Stefan Negrea
Software Engineer
12 years, 5 months
mod_cluster plugin version
by Ian Springer
I just noticed the mod_cluster plugin version is 1.0.
<plugin name="mod_cluster" displayName="mod_cluster"
description="Monitoring and management of mod_cluster plugin." vers
ion="1.0" package="org.rhq.plugins.modcluster"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:xmlns:rh
q-plugin"
xmlns:c="urn:xmlns:rhq-configuration">
This is inconsistent with all the rest of our plugins, where we don't
define any version in the plugin descriptor, which causes the RHQ Server
to fallback to the version in the plugin jar's manifest, which is same
as the RHQ version (e.g. 4.3.0-SNAPSHOT).
Unless there's some reason mod_cluster should be different, can we
change it to be consistent with the other plugins?
--Ian
12 years, 5 months
commit information in BZ
by Charles Crouch
When people are entering the branch and commit hash for a fix into BZ, can you include a link directly to the git web interface highlighting the commit. I've been noticing Ian has been doing this recently and I think it provides a nice easy way to quickly see the change, e.g. https://bugzilla.redhat.com/show_bug.cgi?id=753264#c2
Thanks
Charles
12 years, 5 months