Hi,

I created a Bugzilla issue with this :
Bug 1099114 - AlertAvailabilityDurationJob interrogates a wrong duration interval

https://bugzilla.redhat.com/show_bug.cgi?id=1099114

Regards,
Costel


De la: John Mazzitelli <mazz@redhat.com>
Către: rhq-devel@lists.fedorahosted.org
Trimis: Luni, 19 Mai 2014 15:15:10
Subiect: Re: AlertAvailabilityDuration Job problem

can you write this up as a bug in Bugzilla?

https://bugzilla.redhat.com/enter_bug.cgi?product=RHQ%20Project

And feel free to make those code changes in a github pull request and we'll take a look and merge if possible.

https://github.com/rhq-project/rhq

----- Original Message -----
>
> Hi,
>
> I found out that AlertAvailabilityDurationJob has critical problems in some
> use-cases.
>
> At least in our environment happens, the alert was not triggered.
>
> In the implementation of the AlertAvailabilityDurationJob, the algorithm
> detects from database the list of the availabilities that matches the last
> duration interval: [durationStart = (currentTime - duration_sec * 1000),
> durationEnd= currentTime]
>
> The problem is that the availabilities are stored in database with the
> measurement startTime that comes from the RHQ agent (rhq-agent time), but
> the AlertAvailabilityDurationJob gets the list of availabilities using the
> current RHQ server time.
> We could have here uses-cases like:
> - the rhq-agent time is not synchronized with the rhq-server time
> - the availability reported by the rhq-agent reaches the server with a delay
> and therefore the Job is started with delay
> - other...
>
> The problem I saw is that sometimes the AlertAvailabilityDurationJob detects
> more availabilities, usually includes the previous availability which is the
> Availability UP. Therefore the algorithm considers the availability
> fluctuated and no alert is triggered.
>
> My use case is : Availability Measurement Schedule = 1 min, Availability
> Duration Stays NOT UP for 5 min.
> Importunately I couldn't figure out exactly why this happens, it occurred
> only in production environment, in a test environment I could not reproduce.
>
> The fix that I propose would be to provide the availability start time when
> scheduling the
> availability duration check and this to be used as durationStart by the
> AlertAvailabilityDurationJob
>
> It's needed to do the following modifications in the source code:
>
> 1. GlobalConditionCache.java :
>
> - provide the availability start time
> AvailabilityDurationCacheElement.checkCacheElements(durationCacheElements,
> resource, availabilityType, availability.getStartTime() );
>
> 2. AvailabilityDurationCacheElement.java
> - add a new parameter startTime and provide the value to the
> AvailabilityManager
>
> public static void checkCacheElements(List<AvailabilityDurationCacheElement>
> cacheElements, Resource resource, AvailabilityType providedValue, long
> startTime )
>
> LookupUtil.getAvailabilityManager().scheduleAvailabilityDurationCheck(cacheElement,
> resource, startTime );
>
> 3. AvailabilityManagerLocal.java, AvailabilityManagerBean.java:
> - add a new parameter durationStartTime and provide the value to the Job in
> the infoMap
>
> public void
> scheduleAvailabilityDurationCheck(AvailabilityDurationCacheElement
> cacheElement, Resource resource, long durationStartTime );
>
> infoMap.put(AlertAvailabilityDurationJob.DATAMAP_DURATION_START_TIME,
> String.valueOf(durationStartTime)); // in ms
>
> 3. AlertAvailabilityDurationJob.java:
>
> - use the provided duration start time
>
> long durationStart = Long.valueOf(infoMap.get(DATAMAP_DURATION_START_TIME));
> // in ms
> long durationEnd = durationStart + duration * 1000;
>
> criteria.addFilterInterval(durationStart + 1, durationEnd - 1); // reduce by
> 1 ms to fake exclusive an interval filter.
>
> =======================================================================================
> Another problem I found is that having more availabilities doesn't exclude
> the possibility to have an "Availability Stays Not UP" alert as long we have
> anything but "Availability UP". We could have {DOWN, UNKNOWN, DISABLED} and
> the Availability Stays not UP should happen.
>
> I attach the source code for AlertAvailabilityDurationJob.java
>
> Please have a look, maybe is useful.
>
> Regards,
> Costel

>
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel@lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
>
_______________________________________________
rhq-devel mailing list
rhq-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/rhq-devel