Need advice about agent plugin design
by Steven North
I am trying to design an RHQ/JON agent plugin to manage a software
resource with the following characteristics:
- there is the software itself (the installation);
- there are a variable number of "bundles" of configuration information
about 250KB in size each which need to be read from and written to the
agent; and
- there are "log" files which can 10-50MB in size each which need to be
read from the agent.
I think I am pretty clear on how to handle the software itself--just
like any number of other agents.
I am not sure how to handle the configuration bundles and the large log
files.
We might want to have the RHQ/JON server manage different versions of
these configuration files and distribute them to multiple remote agents.
Is there some existing domain object that would handle the read/write
aspect of the configuration bundles (zip files)? Could the "package"
concept be used for these? Would we need to create a new domain object
on the server side for these bundles? If so, is there an example of
this kind of thing?
For the log files, I see some mention of the SupportFacet. Would this
be appropriate for retrieving large log files? Is there an example of this?
We expect to access the configuration bundles and the log files using
remote client operations because we have a separate GUI tool to
build/edit the configuration bundles and to correlate and analyze the
log files. Is there an example of using a remote client to pull files
from and push files to remote agents?
Thanks in advance for any advice you can give or examples you can point to.
Steve
10 years, 1 month
RHQ usage of SEAM and AS7 integration
by John Mazzitelli
OK, after looking at this, it appears the first major stumbling block for the AS7 work is our usage of SEAM and how we can run in AS7.
Here's the issue - it appears we are using an older version of SEAM that does not work on AS7. We simply can't use the SEAM we were using. This is not surprising at all (see example threads of similar issues we have or will have: https://community.jboss.org/message/608573#608573 - https://community.jboss.org/blogs/marek-novotny/2011/12/16/dvdstore-migra... ).
What I think we need to consider is one of the following:
a) rip out SEAM entirely and all code in RHQ using it/depending on it (this includes mainly core/gui and enterprise/gui/portal-war modules but also this content_http thing that i dont think we need/should have/is used)
or
b) refactor our GUI code (portal war) to either not use SEAM or to make use of a newer SEAM that can be deployed inside of AS7
Now - if we do b) it means there will be a lot of effort wasted for very little gain. Most of portal-war isn't even used anymore - our GWT UI handles the vast majority of UI features now. So if we do b) its merely work that isn't even going to be used - it will be just so we can successfully deploy the RHQ ear only to later rip out portal war at some point anyway. Now, I can't say how much effort it will take to do b), but clearly, the new SEAM isn't backward compatible with the old - I'm seeing missing classes and things like that at startup. We'd have to integrate with the latest SEAM libraries and all that would entail (e.g. refactoring code perhaps)
But we can't blindly do a) either - all of portal-war is not obsolete - there still are small portions still used (graphing and the content UI are the two main areas that I know still use portal-war - there is also that old content_http servlet thing). Graphing pages, IIRC, are actually still on the old struts stuff - so I'm not even sure if JSF is even used, let alone SEAM. We'd have to investigate that. The content UI I think does use code that relies on SEAM. So ripping out portal-war isn't going to be like ripping off a band-aid - it will still require some delicate maneuvering.
The third option I would like to discuss is porting the rest of portal-war to GWT. Rather than waste time doing b), I think we can take that time and do more useful things that have to get done anyway - that is, port over the final pages in portal war to GWT. Thus, when we do that, I think we can do a) more quickly and without worrying about doing too much damage. The only thing that I think we'd have to look out for are simple things like StartupServlet (which we can easily port into a startup servlet in the GWT app or make a singleton EJB, which is new in EJB 3.1 and is now supported because we'd be on AS7)
Thoughts?
11 years, 4 months
Server side Event bus
by Heiko W.Rupp
Hi,
so the other day we were talking about agents forwarding some information out-of-band to the server
so that the messages can be shown in the message center.
We should perhaps pick that idea up and extend it in a way that the server can push
information into the UI without requiring the user to select a different tab or clicking some
refresh button.
There are many use cases for this:
- out-of band messages from agents to be shown in the message center : mark the message center button accordingly
- updating of the tree when a new child resource has been added
- updating of history pages when an entry has changed
- updating of metrics pages when new data was received
...
The general idea here is to make the UI more responsive in the sense that the
user gets actively informed and does not have to remember to manually check
here and there (and to remember to do it in case X while it is not needed in case Y).
While the above is a nice use case of Errai, I think we don't necessarily use it
for implementation; it could server as example on how to do it though.
Heiko
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Mark Hegarty, Charlie Peters, Michael Cunningham, Charles Cachera
11 years, 6 months
Saving the credentials by browser on login page
by Jiri Kremser
Hi,
I've found this page [1] and I am trying to get this idea working. Lukas suggested the values can be obtained from the hidden form by JS (DOM manipulation). But there will be probably necessary to do the other way i.e. to fill the form and submit it with some kind of no-op somehow on the background to let the browser to store it. Other option could be to use the cookies com.google.gwt.user.client.Cookies.{get|set}Value() but I am not sure about storing the password in plaintext somewhere in the cookie without asking the user.
Browser does basically the same, but at least it asks :]
[1] http://code.google.com/p/google-web-toolkit-incubator/wiki/LoginSecurityF...
11 years, 6 months
Loading data into d3.js graphs
by mike thompson
d3.js is the javascript library that we have decided to use for our charting needs. As a pure javascript library we are integrating into smartGWT via JSNI. As a standalone javacript library, d3 uses either csv or json for populating data in its charts. So now we have the issue of getting the data out of our services api and into json so d3.js can read.
Client Solution:
smartGWT (java) ---> d3.js (javascript)
--- or ---
Server Solution
REST(JAX-RS) ---> d3.js
First, on the client-side we have many options there for transforming java objects to json.
Manually building json strings for each object
GWT Overlay types
GWT Autobeans
GWT JSON serialization libraries
Google Gson
Piriti
RestyGWT
GWT-jsonizer
gwtprojsonserializer
Unfortunately, our server side json serializer (Jackson) is not compatible to run as a client GWT library (read as: something incompatible wont allow it to be compiled to javascript). So this opens the door to client GWT json serializers as a legitimate method of loading services data to d3 without a bunch of coding to get the data into json. One of the factors that separates the client GWT json serializers is whether they can serialize an object graph and if it requires special annotations to serialize the object/field. For instance, Google Gson can just take a pojo object graph and serialize it to json no special manipulation or changing of the domain objects is required.
The advantages of the client side solutions is the ease of implementation. Any GWT service can be easily serialized to json with minimal coding. The disadvantages is that they are double handling the data -- the data gets sent from the server to GWT as custom compressed json which in turn is processed and spit back out as standard json that d3.js can digest.
The server solution means wrapping up whatever services with a JAX-RS wrapper to expose them RESTfully. The downside being it is extra work, and the the upside being that service is now exposed via REST. Performance will also be better due to caching and not having to send the data to the client and then having the client create json from that data. I believe this is the best solution because our customers are given the ability to create/customize charts for themselves.
Anyway, I'm proposing the server/JAX-RS RESTful solution as the way to load data into d3.js for the above reasons. Thoughts?
-- Mike
11 years, 7 months
RHQ plugin development enhancements
by spinder
The act of writing an RHQ plugin is how one integrates a management product into the RHQ management space. While the process is fairly well documented(https://docs.jboss.org/author/display/RHQ/Plugin+Community), there is still quite a lot of initial investment for a user new to RHQ development and even more when integrating management products not based in java.
I think we can significantly improve the plugin development and maintenance process by putting some more effort in the following areas:
Development:
- Add more graphical tooling around plugin development to
a)lower the barrier to entry for new management products
b)shorten the feedback loop during plugin development
Testing:
- Document some best practices and a reference implementation for testing plugins
Deployment/Runtime versions:
- Document some best practices for Deployment/Runtime versioning of plugins
While the enhancements around Testing and Runtime versions of plugins can be considered more advanced topics about how to be more consistent and robust among all available plugins and versions, the enhancements around graphical tooling for plugin development seem fundamental to increasing plugin developer productivity and more pressing.
Specifically around graphical plugin development tooling, the following seem like good enhancement areas to start in:
- graphical plugin descriptor creation. This can be a standalone or integrated part of a Swing|Eclipse plugin|Forge application.
- graphical maven plugin generator. The current command line version works to create an initial setup but a graphical wizard would be vastly more helpful.
- graphical integration with both standalone plugin container AND lightweight configuration editor for faster plugin development/testing. Such a graphical tool would likely have an RHQ/JON installation as a dependency to ease dependency/setup/requirements.
- Additionally you'd want to support the ability of the developer to insert sample/mock values to simulate how the plugin code will respond.
- a few short videos/wink demos showing how to develop i)java plugin ii)java plugins utilizing scripting iii)non-java plugin
* For all of the above we should be mindful of two recent enhancements, listed below, which enable greater integration of non-java management plugins. Assuming a java and/or jboss AS stack for development may no longer be a valid assumption for all plugin developers.
http://pilhuhn.blogspot.de/2012/07/jar-less-plugins-for-rhq.html
http://pilhuhn.blogspot.de/2012/07/introducing-no-op-plugin-for-rhq.html
What do others people think about this? Any other enhancements areas that we should add to this list? I think such a fundamental part of the RHQ management framework should be easier to use and the productivity gains will be substantial.
-Simeon
11 years, 7 months
RHQ 4.5 scheduled for around Sept 20th
by Heiko W.Rupp
Hey,
we are thinking of releasing RHQ 4.5 around September 20th and the 4.6 just before X-mas.
For the RHQ 4.5 we should try to find a set of things we still want to do until then. From my
side this clearly includes working on the REST-api. What else do we have?
NOTE: For RHQ 4.6 we will remove the JAX-WS based web services.
--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschaeftsführer: Mark Hegarty, Charlie Peters, Michael Cunningham, Charles Cachera
11 years, 7 months
[GSoC] PyAgent summary
by Krzysztof Kwaśniewski
Hi all!
As the GSoC 2012 program is about to finish, I would like to thank
all the great people I could work with during this summer. Especially
Heiko, who did an excellent job being my mentor for the PyAgent project.
Although the tasks sketched at the beginning of the program were
finished, in my opinion there is still some space for further
development of PyAgent. I would like to still contribute to this
project. Probably not that intensively as so far, as in a short time I
will need to spend time on studying and I should start thinking about my
master's thesis as well. All in all, this was a splendid and fruitful
summer!
Thanks
Krzysiek
11 years, 7 months
automated testing for inventory sync code
by John Sanda
Inventory sync is the process by which an agent reconciles or synchronizes its local inventory with the server's inventory. The server's inventory is canonical. This means that once an inventory sync has finished the resources in the agent's local inventory should match the server's inventory (for that agent). For each resource the agent does some additional work to synchronize metric collection schedules and drift definitions. Again the goal is to make sure that agent's version of metric schedules and drift definitions is consistent with the ones on the server. The agent needs to ensure that collection intervals are correct and that any metric schedules or drift definitions that should be disabled are in fact disabled. These are just a couple examples of things that need to be kept in sync with respect to metric schedules and drift definitions.
Inventory sync is complex in large part due to the number of scenarios or permutations involved. For instance, there are times when the agent initiates a sync, and there are times when the server initiates the sync. Testing becomes even more of a challenge due to the necessary agent/server integration. This is an area (or I should say areas) of the code base that suffers from inadequate automated test coverage. I want to suggest a two phase process for increasing test coverage.
In the first phase we need to set up some integration tests. These tests will involve three components - a running server, a running agent, and a test driver. The test driver will have two main responsibilities. It will coordinate work flows like starting server and agent, and it will perform verification. Let me illustrate with an example.
Test Scenario: Resource exist in server inventory with drift definitions. Agent is started with --purgedata.
Test Driver:
Start RHQ server
When server is initialized, start agent with --purgedata
Wait for for agent to initialize and complete inventory sync
Verify that resource along with drift definitions exist in agent's inventory
There are some subtle details I left out here like how the test will know when the server has initialized or how it will know when the inventory sync has finished. I will save my thoughts on that for another discussion and instead focus on the bigger picture for now. Getting this infrastructure set up would require a good deal of work; however, it provides us with broad test coverage not possible with unit tests. This could also lay the foundation for other integration tests for things like provisioning.
For phase two, we focus on unit tests. These tests will involve a combination of state and behavioral verification. The state verification should be pretty similar to the verification done in the integration tests - making sure the local inventory matches expected values. The behavioral verification will cover server-side calls. For example, when the agent is merging a unknown resource into its inventory (as would be the case when starting with --purgedata), we want to verify that a call is made to MeasurementServerService to obtain the metric collection schedules for that resource.
- John
11 years, 8 months