Răspuns: AlertAvailabilityDuration Job problem

Costel Cosman costelcsmn at yahoo.ro
Tue May 20 10:49:41 UTC 2014



Hi,

I created a Bugzilla issue with this :
Bug 1099114 -AlertAvailabilityDurationJob interrogates a wrong duration interval

https://bugzilla.redhat.com/show_bug.cgi?id=1099114

Regards,
Costel



________________________________
 De la: John Mazzitelli <mazz at redhat.com>
Către: rhq-devel at lists.fedorahosted.org 
Trimis: Luni, 19 Mai 2014 15:15:10
Subiect: Re: AlertAvailabilityDuration Job problem
 

can you write this up as a bug in Bugzilla? 

https://bugzilla.redhat.com/enter_bug.cgi?product=RHQ%20Project

And feel free to make those code changes in a github pull request and we'll take a look and merge if possible.

https://github.com/rhq-project/rhq


----- Original Message -----
> 
> Hi,
> 
> I found out that AlertAvailabilityDurationJob has critical problems in some
> use-cases.
> 
> At least in our environment happens, the alert was not triggered.
> 
> In the implementation of the AlertAvailabilityDurationJob, the algorithm
> detects from database the list of the availabilities that matches the last
> duration interval: [durationStart = (currentTime - duration_sec * 1000),
> durationEnd= currentTime]
> 
> The problem is that the availabilities are stored in database with the
> measurement startTime that comes from the RHQ agent (rhq-agent time), but
> the AlertAvailabilityDurationJob gets the list of availabilities using the
> current RHQ server time.
> We could have here uses-cases like:
> - the rhq-agent time is not synchronized with the rhq-server time
> - the availability reported by the rhq-agent reaches the server with a delay
> and therefore the Job is started with delay
> - other...
> 
> The problem I saw is that sometimes the AlertAvailabilityDurationJob detects
> more availabilities, usually includes the previous availability which is the
> Availability UP. Therefore the algorithm considers the availability
> fluctuated and no alert is triggered.
> 
> My use case is : Availability Measurement Schedule = 1 min, Availability
> Duration Stays NOT UP for 5 min.
> Importunately I couldn't figure out exactly why this happens, it occurred
> only in production environment, in a test environment I could not reproduce.
> 
> The fix that I propose would be to provide the availability start time when
> scheduling the
> availability duration check and this to be used as durationStart by the
> AlertAvailabilityDurationJob
> 
> It's needed to do the following modifications in the source code:
> 
> 1. GlobalConditionCache.java :
> 
> - provide the availability start time
> AvailabilityDurationCacheElement.checkCacheElements(durationCacheElements,
> resource, availabilityType, availability.getStartTime() );
> 
> 2. AvailabilityDurationCacheElement.java
> - add a new parameter startTime and provide the value to the
> AvailabilityManager
> 
> public static void checkCacheElements(List<AvailabilityDurationCacheElement>
> cacheElements, Resource resource, AvailabilityType providedValue, long
> startTime )
> 
> LookupUtil.getAvailabilityManager().scheduleAvailabilityDurationCheck(cacheElement,
> resource, startTime );
> 
> 3. AvailabilityManagerLocal.java, AvailabilityManagerBean.java:
> - add a new parameter durationStartTime and provide the value to the Job in
> the infoMap
> 
> public void
> scheduleAvailabilityDurationCheck(AvailabilityDurationCacheElement
> cacheElement, Resource resource, long durationStartTime );
> 
> infoMap.put(AlertAvailabilityDurationJob.DATAMAP_DURATION_START_TIME,
> String.valueOf(durationStartTime)); // in ms
> 
> 3. AlertAvailabilityDurationJob.java:
> 
> - use the provided duration start time
> 
> long durationStart = Long.valueOf(infoMap.get(DATAMAP_DURATION_START_TIME));
> // in ms
> long durationEnd = durationStart + duration * 1000;
> 
> criteria.addFilterInterval(durationStart + 1, durationEnd - 1); // reduce by
> 1 ms to fake exclusive an interval filter.
> 
> =======================================================================================
> Another problem I found is that having more availabilities doesn't exclude
> the possibility to have an "Availability Stays Not UP" alert as long we have
> anything but "Availability UP". We could have {DOWN, UNKNOWN, DISABLED} and
> the Availability Stays not UP should happen.
> 
> I attach the source code for AlertAvailabilityDurationJob.java
> 
> Please have a look, maybe is useful.
> 
> Regards,
> Costel
> 
> 
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
> 
_______________________________________________
rhq-devel mailing list
rhq-devel at lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fedorahosted.org/pipermail/rhq-devel/attachments/20140520/7f533b98/attachment-0001.html>


More information about the rhq-devel mailing list