BZ 1069648 - Synchronize JVM with standalone.conf - Engineering Review
by Stefan Negrea
The following is a new meeting request:
Subject: BZ 1069648 - Synchronize JVM with standalone.conf - Engineering Review
Organizer: "Stefan Negrea" <snegrea(a)redhat.com>
Location: Google Hangout
Time: Monday, June 2, 2014, 10:30:00 AM - 11:30:00 AM GMT -06:00 US/Canada Central
Invitees: hrupp(a)redhat.com; rhq-devel(a)lists.fedorahosted.org
*~*~*~*~*~*~*~*~*~*
Hello Everybody,
I would like to schedule an engineering review for BZ 1069648 (Synchronize JVM heap arguments between JON Console and JBoss EAP standalone.conf). This will be a code oriented meeting to discuss the proposed changes.
BZ:
https://bugzilla.redhat.com/show_bug.cgi?id=1069648
Pull Request:
https://github.com/rhq-project/rhq/pull/42
Google Hangout:
Link provided before the meeting on #rhq
9 years, 10 months
AUTO: David A. Webster is out of the office (returning 05/27/2014)
by David A. Webster
I am out of the office until 05/27/2014.
On Vacation, will be back Tuesday morning. Baseball tournament in Las
Vegas....
Note: This is an automated response to your message "Re: dev-container
rebuild/upgrade required (another one)" sent on 5/23/2014 3:13:04 PM.
This is the only notification you will receive while this person is away.
**
This email and any attachments may contain information that is confidential and/or privileged for the sole use of the intended recipient. Any use, review, disclosure, copying, distribution or reliance by others, and any forwarding of this email or its contents, without the express permission of the sender is strictly prohibited by law. If you are not the intended recipient, please contact the sender immediately, delete the e-mail and destroy all copies.
**
9 years, 10 months
dev-container rebuild/upgrade required
by John Sanda
I just pushed to master a squash commit of the jsanda/metrics-schema branch. The changes require running the server installer so that the new schema is applied. You can either rebuild your dev-container or do an upgrade on it. I tested upgrading from a 4.10.0 installation. Please let me know if you any questions or run into any problems.
Thanks
- John
9 years, 10 months
AS7/EAP6 deployment resources for domain mode
by Thomas Segismont
Hi,
In an effort to improve integration of RHQ with AS7/EAP6 domain mode,
I'm looking right now at the different "Deployment" resource types.
In domain mode, deployments appear at three different levels:
(1) Domain deployments under the host controller resource
(2) Deployments under server group resources
(3) Managed deployments under managed server resources
In the plugin descriptor, both resource type (1) and (2) have
createDeletePolicy="both" (with creationDataType="content"), while (3)
has createDeletePolicy="neither".
This is confusing because when you upgrade the content of a Domain
deployment resource, you see a stale history of content in the Server
Group Deployment resource (see Bug 889250 [1]).
Unless I missed something, users should only upgrade content of Domain
deployment resource, and use the server group deployment resource to
execute enable/disable/restart operations (deploy/undeploy/redeploy in
AS7/EAP6 terminology).
I'm for creating a new deployment resource type, server group specific
and unaware of the content aspects. Is there any major objection before
I start the pull request?
Thanks,
Thomas
[1] https://bugzilla.redhat.com/show_bug.cgi?id=889250
9 years, 10 months
Resource Subcategories - Simplified
by Stefan Negrea
Hello Everybody,
I've been working for a while on adding resource subcategories to the AS7 plugin to simplify the resource tree. While on surface this would be simple change, just add subcategories to the plugin descriptor, the change is very very complicated. The current subcategory design is outdated due to the addition of the run-inside (multiple parents) feature. I've been trying to fix the current subcategory design from different angles but none worked so far. Mazz helped me along with reviews to my design progressions. This morning while on a call with Mazz, we had a small epiphany, let's solve everything by completely removing the current implementation.
So here are the changes that I am currently working on:
1) Do not require parent resources to pre-declare all subcategories for children. This was done today with a top level <subcategories> tag.
2) The <subcategories> tag will be deprecated for the next release and will eventually be dropped in subsequent releases.
3) While deprecated, just ignore the <subcategories> tag.
4) The subcategories will be declared only the resources themselves via the subCategory attribute
5) For hierarchical categories allow a pipe delimited syntax subCategory="Subsystems|Test"
6) In the UI apply use the camel case syntax to make it more readable, TestSubsystems = Test Subsystems
7) Drop the entire subcategories entities and related tables. For now deprecated, and later completely remove the functionality.
8) Work on a database migration task to fold subcategories text into the resource types.
9) Update current RHQ plugins to remove the <subcategories> tag.
Why the changes:
1) The current implementation is almost unused, none of the complicated structure is fully taken advantage of.
2) The current implementation is broken in so many ways, the run-inside really made the design obsolete and almost unfixable.
3) The current validation for subcategories (= they are declared on the parent) is really not needed. The only the actual resources that is placed on a subcategory needs to know about the subcategory.
4) The UI operates under the assumptions of a the simplified model. If a resource belongs to a category then create the UI resource tree accordingly.
5) Simple plugin structures makes it easy for community members to implement/fix plugins.
Removing the current subcategory implementation is relatively easy because very little is actually used; it is mostly a liability. There were a couple of fixes applied over time around transactional boundaries that made the code very brittle. And after all this is done there is absolutely no change from a user perspective.
To summarize the change:
1) Deprecate <subcategories> tag
2) Drop <subcategories> tag in future releases
2) Subcategories are set only via subCategory attribute
3) Pipe delimit hierarchies of subcategories and use camel case to improve legibility
4) Clean backend and existing plugins completely
5) No change for the users
BZ:
https://bugzilla.redhat.com/show_bug.cgi?id=1069545
Working pull request:
https://github.com/rhq-project/rhq/pull/22
Thank you,
Stefan Negrea
Software Engineer
9 years, 10 months
AlertAvailabilityDuration Job problem
by Costel Cosman
Hi,
I found out that AlertAvailabilityDurationJob has critical problems in some use-cases.
At least in our environment happens, the alert was not triggered.
In the implementation of the AlertAvailabilityDurationJob, the algorithm detects from database the list of the availabilities that matches the last duration interval: [durationStart = (currentTime - duration_sec * 1000), durationEnd=currentTime]
The problem is that the availabilities are stored in database with the measurement startTime that comes from the RHQ agent (rhq-agent time), but the AlertAvailabilityDurationJob gets the list of availabilities using the current RHQ server time.
We could have here uses-cases like:
- the rhq-agent time is not synchronized with the rhq-server time
- the availability reported by the rhq-agent reaches the server with a delay and therefore the Job is started with delay
- other...
The problem I saw is that sometimes the AlertAvailabilityDurationJob detects more availabilities, usually includes the previous availability which is the Availability UP. Therefore the algorithm considers the availability fluctuated and no alert is triggered.
My use case is : Availability Measurement Schedule = 1 min, Availability Duration Stays NOT UP for 5 min.
Importunately I couldn't figure out exactly why this happens, it occurred only in production environment, in a test environment I could not reproduce.
The fix that I propose would be to provide the availability start time when scheduling the
availability duration check and this to be used as durationStart by the AlertAvailabilityDurationJob
It's needed to do the following modifications in the source code:
1. GlobalConditionCache.java :
- provide the availability start time
AvailabilityDurationCacheElement.checkCacheElements(durationCacheElements, resource, availabilityType, availability.getStartTime());
2. AvailabilityDurationCacheElement.java
- add a new parameter startTime and provide the value to the AvailabilityManager
public static void checkCacheElements(List<AvailabilityDurationCacheElement> cacheElements, Resource resource, AvailabilityType providedValue, long startTime)
LookupUtil.getAvailabilityManager().scheduleAvailabilityDurationCheck(cacheElement, resource,startTime);
3. AvailabilityManagerLocal.java, AvailabilityManagerBean.java:
- add a new parameter durationStartTime and provide the value to the Job in the infoMap
public void scheduleAvailabilityDurationCheck(AvailabilityDurationCacheElement cacheElement, Resource resource,long durationStartTime);
infoMap.put(AlertAvailabilityDurationJob.DATAMAP_DURATION_START_TIME, String.valueOf(durationStartTime)); // in ms
3. AlertAvailabilityDurationJob.java:
- use the provided duration start time
long durationStart = Long.valueOf(infoMap.get(DATAMAP_DURATION_START_TIME)); // in ms
long durationEnd = durationStart + duration * 1000;
criteria.addFilterInterval(durationStart + 1, durationEnd - 1); // reduce by 1 ms to fake exclusive an interval filter.
=======================================================================================
Another problem I found is that having more availabilities doesn't exclude the possibility to have an "Availability Stays Not UP" alert as long we have anything but "Availability UP". We could have {DOWN, UNKNOWN, DISABLED} and the Availability Stays not UP should happen.
I attach the source code for AlertAvailabilityDurationJob.java
Please have a look, maybe is useful.
Regards,
Costel
9 years, 10 months
BZ 1070326, PullRequest 26 : feedback requested
by Jay Shaughnessy
Folks, there is BZ 107032, "RFE: Allow AS7 deployments to provide
version in the artifact name", and the related PR 26 for the changes and
discussion. This has to do with potential changes in the AS7 plugin's
Deployment discovery. In short, it would allow for the same logical
deployment (EAR, WAR...) to be discovered as different versions of the
same resource, based on versioning in the artifact name.
For example, myapp-1.0.war and myapp-2.0.war would result in two
versions of the same resource, myapp.war. Currently this would result
in two resources, and likely the 1.0 variant would actually have been
removed, leaving its resource "dead" and needing to be uninventoried.
This would prevent inventory from getting littered with missing/dead
resources when the user's approach is to generate and deploy artifacts
versioned in this way.
If interested in this topic please take a look at the BZ and/or PR
comments, and feel free to add your opinions here or there. Thanks.
https://bugzilla.redhat.com/show_bug.cgi?id=1070326
https://github.com/rhq-project/rhq/pull/26
9 years, 10 months
[BZ 1069648] Synchronize JVM heap arguments between in EAP standalone.conf
by Stefan Negrea
Hello Everybody,
I've been working on a design for synchronizing JVM heap arguments between RHQ and managed JBoss AS7 standalone.conf. On the surface this is looks like a simple task but it is far more complicated because of stability and security implications. I got a few ideas on how to accomplish this and then had a design session with Thomas. We narrowed it down to two options (see below) but in the end I think only one is really possible to implement.
BZ - https://bugzilla.redhat.com/show_bug.cgi?id=1069648
The issue:
1) The BZ is applicable only to standalone servers. Domain mode has a very nice built-in mechanism to update JVM settings (already implemented in RHQ).
2) The official way to update JVM settings in standalone is via configuration files: https://docs.jboss.org/author/display/AS7/JVM+settings
3) JVM settings are stored in two platform dependent files: standalone.conf and standalone.conf.bat
4) The configuration files have logic in them, shell and batch logic.
5) The configuration files are sourced in the start scripts.
6) Users can update these files outside of RHQ and they can have really important information
7) So far the AS7 plugin does not touch these files
Design Idea:
1) Expose JAVA_OPTS in the Connection Settings for the server resource as a configuration field
2) If the user sets then update the two config files simultaneously, regardless of current platform
3) Add the four lines of configuration at the top of each file
Line 1: == RHQ Configuration ==
Line 2: == BIG WARNING HERE ==
Line 3: If JAVA_OPTS not set already
Line 4: set JAVA_OPTS to user value
4) if RHQ detects these lines at the top of each file then update them when the user updates the configuration
5) if RHQ does not detect them (user tampered with the configuration, or any other reason) then add them
Some problems with this design:
1) Because the config files are sourced scripts in the startup scripts for each platform, RHQ code needs to be very conservative since updates to these files have big security implications.
2) Anything wrong the lines added in the configuration could render the AS7 server unusable/unstartable. And the only solution for the user would be to manually fix the file.
3) If the configuration files become complex via AS7 updates, or the user completely changes them per their environment the RHQ updates can lead to errors (see 1).
4) If there is any tampering with the start of the file then the users would lose the value even if it still is in the file.
5) The JAVA_OPTS setting would still be overriden by the Start Script Environment Variables value if set in the Connection Settings.
6) On Windows, if the user adds anything at the top of the file, the RHQ updates will force ignore the user configuration. There is no IF in batch, everything is accomplished via GOTO. So the update for Windows would force a jump from start of the file to a portion of the config file after the JAVA_OPTS configuration. And that place is currrently at the bottom of the Windows config file.
The other design idea was around exposing the entire configuration file as a sub-resource in the EAP resource tree. But that is not easily achievable with the current UI. And this is not very secure since the config scripts are sourced; somebody could inject malicious code and then restart EAP.
Any feedback would be greatly appreciated.
Thank you,
Stefan Negrea
Software Engineer
9 years, 10 months
upgrade to 4.11 has problem - should we create new db upgrade step?
by John Mazzitelli
OK, as you saw in my forum post and rhq-user email, I found a very bad problem if you did an upgrade from a previous version to 4.11.
I fixed it in master for future versions. But Jay had a point that needs to be discussed.
Read the BZ for what I did to fix and why: https://bugzilla.redhat.com/show_bug.cgi?id=1095970
Right now, those that upgraded to 4.11 have no recourse but to do a manual SQL rename of the bad sequence. I did NOT add a new db upgrade step because it doesn't help those that have the problem now. They will have to do a rename now via manual SQL. So adding a new db upgrade step is essentially useless to those people.
Because I fixed the original db upgrade step, those that wait to upgrade to 4.12 (when we release) won't have a problem. They will go from RHQ 4.10 (or previous) directly to 4.12, which means they will get the correct seq name due to my fix. The problem will only be with those that upgraded NOW to 4.11. (doesn't affect those that did a clean install of 4.11).
However, it does makes sense to create a new db upgrade step IF we want to immediately release a 4.11.1 - people who upgraded to 4.11 already could download 4.11.1, upgrade to it, and get the fixed sequence name. All this does is free them from having to do a manual SQL execution of the sequence rename.
Its not enough to say "well, what happens if they just wait to upgrade 4.12? How will they get fixed?" because they are dead in the water now (agents won't be able to register). So those people can't wait for however long for 4.12 to be released, their 4.11 just won't operation properly. They have to fix it now. So if we want to fix it "now" it has to be with a 4.11.1 release, or we just leave it as-is where the user has to issue the one manual SQL statement to rename the bad sequence.
Sorry, I dropped the ball on this one. I didn't do enough upgrade testing.
9 years, 10 months