Need advice about agent plugin design
by Steven North
I am trying to design an RHQ/JON agent plugin to manage a software
resource with the following characteristics:
- there is the software itself (the installation);
- there are a variable number of "bundles" of configuration information
about 250KB in size each which need to be read from and written to the
agent; and
- there are "log" files which can 10-50MB in size each which need to be
read from the agent.
I think I am pretty clear on how to handle the software itself--just
like any number of other agents.
I am not sure how to handle the configuration bundles and the large log
files.
We might want to have the RHQ/JON server manage different versions of
these configuration files and distribute them to multiple remote agents.
Is there some existing domain object that would handle the read/write
aspect of the configuration bundles (zip files)? Could the "package"
concept be used for these? Would we need to create a new domain object
on the server side for these bundles? If so, is there an example of
this kind of thing?
For the log files, I see some mention of the SupportFacet. Would this
be appropriate for retrieving large log files? Is there an example of this?
We expect to access the configuration bundles and the log files using
remote client operations because we have a separate GUI tool to
build/edit the configuration bundles and to correlate and analyze the
log files. Is there an example of using a remote client to pull files
from and push files to remote agents?
Thanks in advance for any advice you can give or examples you can point to.
Steve
10 years, 1 month
MeasurementUnits (EPOCH_MILLISECONDS, EPOCH_SECONDS)
by Jiri Kremser
Hi,
in plugin descriptor, there can be a metric definition with unit type "epoch_milliseconds" or "epoch_seconds" (rhq-configuration.xsd allows it). What kind of metric should could be represented by epoch_milliseconds? Shouldn't the exact time moments (what values epoch_milliseconds represent) be addressed rather by traits?
I am asking because the values of this type are not formatted (https://bugzilla.redhat.com/show_bug.cgi?id=857144).
I think these two unit types should be removed from MeasurementUnits and xml schema, however there might be some plugins out there using it, so what about deprecation? Are there any edge cases, when these unit types do make sense?
JK
10 years, 2 months
Rename manageRootDir property in bundles
by Lukas Krejci
Hi all,
In BZ 801464 (and 801926) (https://bugzilla.redhat.com/show_bug.cgi?id=801464), we're discussing the problems of manageRootDir property on the deployment units in bundles.
We have decided to take 2 actions on it:
1) A short term remedy to prevent people from shooting themselves in the foot
2) A long term overhaul of how bundles work to make them more similar to RPM/APT so that people are less surprised by the behavior.
I won't discuss 2) here because there will be plenty of opportunities to do that later when we start concentrating on it, but I'd like to discuss 1) with you.
As the subject suggests, one of the things we intend to do is to rename the manageRootDir property to something that more clearly expresses the true meaning of the setting. The other thing that we want to do is to make the parameter mandatory so that people are forced to think about the value to put in there when creating a new bundle (this will only affect the upload of new bundles. The system will be able to revert to bundle versions that are already uploaded).
So the one thing I'm gathering input for is the new name for manageRootDir.
But what manageRootDir (which defaults to true) really means? If it is true (the default), the bundle deployer will delete any and all files (recursively) in the destination directory before deploying the bundle files into it. This means that it is guaranteed that the bundle destination only contains the bundle files. When manageRootDir is false, it means that that the files and directories at directly under the destination directory are left intact - this means that the bundle deployer will only overwrite the files that are also part of the bundle and, also, will replace any directories that are present in the bundle with the contents from the bundle - this means that the sub-directories are always wiped to conform to the contents of the bundle - there is currently no way of overriding that (and we don't search for a mechanism to provide that in this email).
There already have been some suggestions for the new name:
manageRootDir=true
could also be expressed like:
manageDestDirFully=true
doNotTouchTopLevelDirectoriesFoundInDestDirButNotFoundInBundle=false
ignoreTopLevelDirectoriesInDestDirButNotFoundInBundle=false
cleanTopLevelDirectoriesFoundInDestDirAndNotFoundInBundle=true
bundleToManageAllTopLevelDirectoriesInDestDir=true
manageAllTopLevelDirectoriesInDestDir=true
I'll add a couple of suggestions, too:
cleanDestinationFully=true
wipeDestinationDir=true
alsoWipeUnmanagedFilesAndDirectoriesInDestination=true
Which of the above do you think expresses the best what manageRootDir does? Do you have some other suggestions for the name?
Cheers,
Lukas
10 years, 9 months
Agent / Server Communications Options
by Van Dillon
Hello All,
I'm new to RHQ and am interested in using it in several different
ways. Some of the ways I want to use it require the agent to be on
platforms that do not have a public IP address. I read the Design
Documentation wiki entries: "Design-Agent-to-Server Unidirectional
Communications" and "Design-Agent-Server Communications 3", both of
which indicate that you guys have put some serious thought into adding
this ability to RHQ.
Is this feature being added to the roadmap anytime soon? If not, is
there a way to hook our own handlers into the agent and server
communications layers?
Thanks,
Van Dillon
10 years, 9 months
Summary report for 'RHQ Overnight Full Report'
by Michael Foley
Hi,
For transparency, I am sharing information about QE test automation for RHQ :
* the tests are available here -->> https://github.com/RedHatQE/jon-tests
* the test results are below
Regards,
Michael Foley
QE Lead, JBoss Operations Network
----- Original Message -----
From: noreply(a)redhat.com
To: jon-qa-list(a)redhat.com
Sent: Monday, July 29, 2013 9:00:02 AM
Subject: [jon-qa-list] Summary report for 'RHQ Overnight Full Report'
Report Engine - Email Report Hello, This is auto generated email from Report Engine. It's based on RHQ Overnight Full Report
Test Suite Name Total Passed Failed Skipped Build Start Time End Time Duration RHQ REST JAVA Client Test 40 (0) 20 (0) 20 (0) 0 (0) 4.9.0-SNAPSHOT (9674555) Jul 28 23:38:08 IST 2013 Jul 28 23:38:26 IST 2013 00:00:17
RHQ REST Groovy Client Test 8 (0) 8 (0) 0 (0) 0 (0) 4.9.0-SNAPSHOT (9674555) Jul 28 23:37:29 IST 2013 Jul 28 23:37:35 IST 2013 00:00:05
RHQ 4.x core UI Test 276 (0) 267 (-1) 8 (+1) 1 (0) 4.9.0-SNAPSHOT (9674555) Jul 28 22:18:38 IST 2013 Jul 28 23:36:40 IST 2013 01:18:01
Import Resources 4 (0) 4 (0) 0 (0) 0 (0) 4.9.0-SNAPSHOT (9674555) Jul 28 22:16:47 IST 2013 Jul 28 22:17:56 IST 2013 00:01:08
RHQ_4.4_UI_AS7Plugin_Test 210 (0) 134 (+14) 35 (+7) 41 (-21) 4.9.0-SNAPSHOT (9674555) EAP 6.1.0.GA Jul 28 15:46:17 IST 2013 Jul 28 21:07:11 IST 2013 05:20:52
RHQ CLI EAP6 44 (0) 39 (+5) 5 (-1) 0 (-4) 4.9.0-SNAPSHOT (9674555) EAP 6.1.0.GA Jul 28 13:58:39 IST 2013 Jul 28 15:45:24 IST 2013 01:46:33
RHQ CLI JS/JAVA Test 92 (0) 54 (+1) 1 (-1) 37 (0) 4.9.0-SNAPSHOT (59fa894) Jun 30 22:12:46 IST 2013 Jun 30 22:56:26 IST 2013 00:43:39
Fresh Installation of RHQ 1 (0) 0 (0) 1 (0) 0 (0) - Dec 24 23:45:58 IST 2012 Dec 24 23:47:26 IST 2012 00:01:28
Check Resource Configuration 1161 (+5) 1158 (+3) 2 (+2) 0 (0) - Aug 22 16:14:05 IST 2012 - - Summary: 1836 1684 72 79 09:12:07
Test Suite Detaild View:
Test Suite: RHQ REST JAVA Client Test
Group Name Total Passed Failed Skipped Login Test Negative 2 2 0 0
check New Rest Api 1 1 0 0
Login Test Positive 1 1 0 0
List Status 1 1 0 0
Visit Platforms 1 0 1 0
Visit Platforms By Id 1 0 1 0
get Resource By Platform Name 1 0 1 0
get Resource By Platform Name And Category 1 0 1 0
get Resource By Category 1 0 1 0
get Resource By Paging 1 0 1 0
get Platform Xml 1 1 0 0
Validate Children 1 0 1 0
get Alerts For Resources 1 0 1 0
get Schedules For Resources 1 0 1 0
get Availability For Resources 1 0 1 0
get Availability History For Resources 1 0 1 0
get Root URI 1 1 0 0
get Json PWrapper 1 1 0 0
verify AS Server Presence 1 0 1 0
Check Platform Operations 1 0 1 0
Validate Schedules 1 0 1 0
Validate Metric Data Default 1 0 1 0
Validate Metric Data 1 0 1 0
Visit Alerts 1 1 0 0
Redirect From Definition 1 1 0 0
Alert Definitions 1 1 0 0
Alert Count 1 1 0 0
AlertSender 1 1 0 0
AlertSender_DirectEmails 1 1 0 0
AlertSender_Mobicents 1 1 0 0
AlertSender_NotExisting 1 1 0 0
get Event Sources 1 0 1 0
get Event Definitions 1 0 1 0
get Group 1 1 0 0
get Group With Params 1 1 0 0
get Metrics Schedule 1 0 1 0
get Metrics Schedule By Id 1 0 1 0
get Favorite Resources 1 1 0 0
get Favorite Groups 1 1 0 0 Total: 40 20 20 0 Test Suite: RHQ REST Groovy Client Test
Group Name Total Passed Failed Skipped Status 2 2 0 0
Login 3 3 0 0
Resource 3 3 0 0 Total: 8 8 0 0 Test Suite: RHQ 4.x core UI Test
Group Name Total Passed Failed Skipped GuiLoginTest 1 1 0 0
Version Information 1 1 0 0
Metric Collection Schedules validation 39 39 0 0
AlertDefinitionCreationTest 13 12 1 0
InventoryTest 11 11 0 0
BundleTest 4 4 0 0
TopLevelMenusTest 12 12 0 0
ReportsTest 7 7 0 0
DashboardTest 12 12 0 0
ResourcesTest 5 5 0 0
AdministrationTest 6 6 0 0
HelpTest 10 10 0 0
RecentOperationsTest 5 3 1 1
FavouriteTest 4 4 0 0
PluginsTest 52 52 0 0
Individualconfig 4 4 0 0
Configuration History 2 2 0 0
Group Config 3 3 0 0
SubNavigationMenuTests 2 2 0 0
Administration JSP page validation 6 6 0 0
Drift management-Create-[add/change/remove]-Delete 5 5 0 0
AlertHistoryValidationTest 13 13 0 0
AlertHistoryDeletionTest 13 13 0 0
AlertDefinitionDeletionTest 13 13 0 0
Scheduling Groups 4 4 0 0
Search groups... 6 6 0 0
Permissions Test 7 6 1 0
Alert Definition Template 6 6 0 0
CheckAlertDefTempCreated 1 0 1 0
Drift Definition Template 3 3 0 0
StorageAutoInventoryTest 1 0 1 0
StorageNode 2 0 2 0
Setup LDAP 1 1 0 0
GuiLDAPUserLoginTest 1 0 1 0
GuiLogoutTest 1 1 0 0 Total: 276 267 8 1 Test Suite: Import Resources
Group Name Total Passed Failed Skipped GuiLoginTest 1 1 0 0
Version Information 1 1 0 0
GImportResourcesTest 1 1 0 0
GuiLogoutTest 1 1 0 0 Total: 4 4 0 0 Test Suite: RHQ_4.4_UI_AS7Plugin_Test
Group Name Total Passed Failed Skipped GuiLoginTest 1 1 0 0
Version Information 1 1 0 0
Standalone_AutodetectConfiguration 1 1 0 0
DetectServerVersion 1 1 0 0
Standalone_SocketBindingGroups 3 3 0 0
Standalone_Logging_Subsystem 16 9 3 4
Standalone_SecuritySubsystem 6 4 2 0
Standalone_Naming_Subsystem 1 1 0 0
Standalone_SignalServerReload 2 1 1 0
Standalone_Mail_Subsystem 13 3 4 6
Standalone_InfinispanSubsystem 9 3 3 3
Standalone_Web_Subsystem 7 6 0 1
Standalone_ResourceCreation 2 2 0 0
Standalone_Monitoring 3 1 1 1
Standalone_Deployment 20 18 2 0
Standalone_JMSDestinations 4 4 0 0
Standalone_Datasources 16 3 2 11
Standalone_SystemProperties 3 3 0 0
Standalone_ResourceConfiguration 5 5 0 0
Standalone_OSGI_Child_Configuration_Test 6 0 0 6
Standalone_Transactions_Test 1 1 0 0
Standalone_JAXR_Test 4 3 1 0
Standalone_Operations 4 4 0 0
Standalone_ManualImport 28 22 5 1
Domain_AutodetectConfiguration 1 1 0 0
Domain_ResourceConfiguration 3 3 0 0
Domain_SystemProperties 12 7 2 3
Domain_ResourceCreation 2 2 0 0
Domain_Monitoring 3 1 2 0
Domain_JMSDestinations 4 2 1 1
Domain_Deployment 6 2 1 3
Domain_Operations 4 4 0 0
Domain_ManagedServersManagement 7 7 0 0
Domain_ServerGroupsManagement 10 4 5 1
GuiLogoutTest 1 1 0 0 Total: 210 134 35 41 Test Suite: RHQ CLI EAP6
Group Name Total Passed Failed Skipped Standalone_Discovery_StartCommand_Parameters 15 15 0 0
Domain_Discovery_StartCommand_Parameters 4 4 0 0
Uninventory_EAP_Servers 1 1 0 0
Inventory_EAP_Servers 1 1 0 0
Detect_ServerVersion 1 1 0 0
Standalone_Deployment 5 4 1 0
Standalone_BundleDeployment 1 1 0 0
Standalone_NetworkInterfaces 1 0 1 0
Standalone_MailSubsystemTest 1 0 1 0
Standalone_BZ918677Test 1 1 0 0
DeployAndRestartSampleTest 1 1 0 0
Domain_Deployment 11 10 1 0
Domain_NetworkInterfaces 1 0 1 0 Total: 44 39 5 0 Test Suite: RHQ CLI JS/JAVA Test
Group Name Total Passed Failed Skipped Import all Resources 2 2 0 0
Examples from jon documentation 22 21 0 1
bz reproduction test 3 1 0 2
Examples from rhq documentation 4 3 0 1
Samples from CLI client 3 3 0 0
operation scheduling test 3 3 0 0
manager test 6 6 0 0
permission test 15 0 0 15
role test 2 1 0 1
subject test 2 2 0 0
drift test 1 1 0 0
DynaGroup definitions test 7 5 0 2
Bundles 15 5 0 10
schedule_Management_For_RHQ_Agent 1 1 0 0
stress test 5 0 0 5
Uninventory-Platforms-End 1 0 1 0 Total: 92 54 1 37 Test Suite: Fresh Installation of RHQ
Group Name Total Passed Failed Skipped GuiInstallationTest 1 0 1 0 Total: 1 0 1 0 Test Suite: Check Resource Configuration
Group Name Total Passed Failed Skipped GuiLoginTest 1 1 0 0
Version Information 1 1 0 0
CheckResourceConfiguration 1159 1156 2 0 Total: 1161 1158 2 0 Report Engine URL: https://mercury.lab.eng.pnq.redhat.com:8443/re
Report Generated on: Jul 29 18:30:00 IST 2013
_______________________________________________
jon-qa-list mailing list
jon-qa-list(a)redhat.com
https://post-office.corp.redhat.com/mailman/listinfo/jon-qa-list
10 years, 9 months
Minor change to ConfigurationEditor
by Jiri Kremser
Hi devs,
I would like to do a minor change to our ConfigurationEditor UI component. Just a reminder, it takes a configuration object and creates a rich UI out of it and we use this component in many places in RHQ. The configuration may be divided into multiple so called group definitions clustering related properties into logical units. For each group the section is created (accordion like component).
The problem is if there is only one section on the page, there is still that "Jump to Section" toolbar. It is totally useless to have it there if there is only 1 section on the page. I would like to hide it in that case. See the attached screenshot to better see the difference.
jk
10 years, 9 months
JavaFX
by Heiko W.Rupp
Helpers /plugin* is not enabled by default in the parents, so does not influence a normal
build.
Also JavaFX is normally bundled with the (Oracle)JRE, it looks like OpenJDK does
not contain that and you are thus seeing the issues.
There is an OpenJFX project, but that is not mature enough.
I will take that stuff and move it to github, so that it is out of the "normal" repo
[19:48:43] <mazz> so, I'm getting some errors in eclipse, something about javafx. I see this is a pom:
[19:48:52] <mazz> <groupId>com.oracle</groupId>
[19:48:53] <mazz> <artifactId>javafx</artifactId>
[19:48:53] <mazz> <version>2.0</version>
[19:48:53] <mazz> <systemPath>${java.home}/lib/jfxrt.jar</systemPath>
[19:48:53] <mazz> <scope>system</scope>
[19:49:01] <mazz> are we requiring Oracle's JDK7 to build RHQ now?
[19:49:10] <mazz> specifically, *oracle* JDK
[19:52:56] <LdapPRo> The URL to report HRQ defects
[20:07:22] <jshaughn> https://bugzilla.redhat.com
[20:15:17] mtho11-lunch is now known as mtho11
[23:06:55] <jshaughn> mazz, I'm also seeing that javafx thing and I don't exactly know how to get that helper stuff to build. At the moment I have a lot of errors in eclipse
10 years, 9 months
Storage Node - Disk Space Metric
by Stefan Negrea
Hello Everybody,
I would need feedback for the percentage of disk space used metric for RHQ Storage Nodes. The metric already went through three iterations and I think we could refine it more before releasing next RHQ version.
Background
- The metric is Disk Used Percentage
- It is a calculated metric
- Data files are Cassandra's largest disk space consumers
- Data files could be stored in one or more data directories, this is configurable by the user
- If multiple directories are configured, Cassandra will select the partition with most disk space available
- The size of all data files is available via the JMX interface
- Disk space available is a system/platform property but to report anything for Cassandra the focus needs to be on the partitions where data files are stored
Iteration 1 - delivered with RHQ 4.8
- The metric name: Calculated.DiskSpaceUsedPercentage
- Calculation:
+ based on the overall disk usage, not just Cassandra (total disk used/total disk available)
+ if multiple directories used for data files, just return max percentage
- Problems:
+ the metric can be misleading, since it is reported in Cassandra sub-resource, yet it represents a platform metric
+ the selection of max is not representative on how Cassandra works, since it distributes data files across all the partitions available
+ using max will work correctly only in cases where there is only one data files location
Iteration 2 - Post RHQ 4.8
- The metric name: Calculated.PartitionDiskSpaceUsedPercentage
- Calculation:
+ an aggregate percentage of disk space used for all the partitions where Cassandra stores data files
+ similar to iteration 1, but it looks at the disk space used across all the partitions
+ example: for data files on two partitions: metric value = (disk space used of partition_1 + partition_2)/(total disk space of partition_1 + partition_2)
- Problems:
+ like Iteration 1, the metric can be misleading since it is a platform metric but reported on Cassandra sub-resource
+ seeing a high percentage for disk utilization when you just installed a new RHQ Storage Node could be confusing - "Cassandra is using 42% of the disk already?"
+ the only way for users to understand what is going on is to look at the metric description, just looking at the table of values reported is not sufficient
- Positives:
+ representative to how Cassandra uses disk partitions
+ an alert on this metric is guaranteed to trigger every time the free disk space gets low
Iteration 3 - in Master
- The metric name: Calculated.PartitionDiskSpaceUsedPercentage
- Calculation:
+ an aggregate percentage of disk space used by Cassandra data files for all the partitions where Cassandra stores data files
+ similar to iteration 2, but it will not look at the overall disk space used, just at the disk space used by data files
+ eg. for data files on two partitions: metric value = (disk space used by data files)/(total disk space of partition_1 + partition_2)
- Problems:
+ it is hard to design an alert for this metric because there is no percentage to alert on that will guarantee trigger
+ for example: if total disk space utilization pre-Cassandra is 60%, a value of 50% of the alert on this metric will never be trigger the alert because there is only 40% available disk
+ Cassandra data could grow until disk is full and never trigger an alert
- Positives:
+ representative to how Cassandra uses disk partitions
+ the value will no longer be misleading, when a new RHQ Storage Node is installed the disk used will low (eg. under 1%)
+ the user will not have to dig in documentation or report that the metric is wrong out of the box
Right now Iteration 3 is in master but I would need feedback since there is no perfect solution (both Iteration 2 and Iteration 3 have downsides). I like the correctness of the calculation for Iteration 3 but the alerting capability of Iteration 2. I do not see any easy solution to get both...
1) Should we include both metrics? Overall disk space use & data files disk space used? We could only hope that having the two metrics will make it easier for users to understand what they are.
2) Is iteration 2 good? The confusion concern is not warranted?
3) Could we place the calculation done for Iteration 2 on a sub-resource that would not guarantee confusion?
4) Any other solution to have a self-documenting metric and a good alert template?
Thank you,
Stefan Negrea
Software Engineer
10 years, 9 months
Enterpriser Server Jar Naming - Fix
by Stefan Negrea
Hello Everybody,
Mazz helped me find the problem with rhq-enterprise-server (modules/enterprise/server/jar) artifact not being properly re-deployed to the dev container after a module rebuild. It turns out it was related to a set of changes done late last week ([1] & [3]). I placed a temporary fix on master [5] but I would like to ask for feedback for a long term fix. This email is based on my discussion with Mazz.
I understand the commits [1] and [3] were done to shorten the names of rhq-enterprise-server and server-startup modules. For example, the old deployment name was rhq-enterprise-server-ejb3, new one is rhq-server. While the new names are shorter they are no longer representative of the module name. In the big picture I do not like this because we introduce yet another variable in the packaging process. If something breaks during packaging it is harder for the person running the release process to trace and fix things. I really prefer a slightly longer name that makes tracing easier during packaging.
An option would be to change the artifact name from rhq-enterprie-server to rhq-server. But I do not like this for server jar because it will change the artifact name and linage in public maven repositories. rhq-enterprise-server artifact has a long and established history. Another issue with changing the artifact name is external usage. This module would likely be used by other people extending the project.
The second change that caught my eye is the name update for the startup subsystem [3]. Since this artifact is relatively new, it does not have a long history. Plus this module is less likely to be used by external project consumers (unlike server jar). So overall the name change is ok. However, I would like to have the name consistent with the rest of the modules by being prefixed with "rhq-". See [4] for the list of RHQ artifacts in Nexus.
So here is my proposal for a long term fix:
1) Change the server startup module name to rhq-server-startup
2) Leave the server jar module name as is rhq-enterprise-server
3) Use the artifact names unchanged as the deployment names in the ear, dev container, and any other place.
This will give us project consistency and no name changes between building and packaging.
Any thoughts?
[1] https://git.fedorahosted.org/cgit/rhq/rhq.git/commit/?id=957781e62543ff50...
[2] https://repository.jboss.org/nexus/content/groups/public-jboss/org/rhq/rh...
[3] https://git.fedorahosted.org/cgit/rhq/rhq.git/commit/modules/enterprise/s...
[4] https://repository.jboss.org/nexus/content/groups/public-jboss/org/rhq/
[5] https://git.fedorahosted.org/cgit/rhq/rhq.git/commit/?id=375f6f66fcc9214c...
Thank you,
Stefan Negrea
Software Engineer
10 years, 9 months