Thanks really!
It did help me to understand the way it works.
Unfortunately I do not perfectly understand the idea behind. It delays the
alerts significantly, and we here usually monitor parameters which does not
really recover by-itself.
We usually want:
- monitor frequently
- alert ASAP
- do not repeat the same alert for all the trigger event
Even the simplest syslog apps can do this. Doing the other way round could
also be interesting for many types of services ... but in many cases there
is just no chance that things get better without action - but it's just
strange to get the same alert mail in each and every 10/20 minutes - or
delay the alert - and no other way.
Thanks again!
Regards,
Attila
2013/12/2 Jay Shaughnessy <jshaughn(a)redhat.com>
Attila,
OK, looking at your dampening definition this is actually a
misunderstanding of dampening, the behavior is as expected (even if not as
desired). When you say "N Occurrence in X Hours" that means that if the
conditions are met N times in X hours that you will generate an alert. If
N = 1 the dampening is effectively useless. That just means the conditions
have to match 1 time in X hours. As soon as they match you get the
alert. After the alert fires the whole thing *resets*. The intent of
that dampening rule is to avoid alerting on spikes in activity. Instead,
alert only if a certain aberration is consistent.
For example, if "number of open files" sometimes spikes you may not want
an alert. But if say you are taking measurements every 10 minutes, and 3
times in an hour you see the same spike , maybe there is a problem. In
that case you'd say, "3 occurrences in 1 Hour".
What you want is "Fire no more than 1 times in an hour". We don't really
have that sort of dampening, although it may be a nice RFE for a time-based
recovery.
We have various dampening, we have the ability to disable-after-fire and
we have recovery alerts. Perhaps you will find something that works for
you, otherwise feel free to create an RFE BZ with your exact requirement.
For more on dampening and recovery alerting see
https://docs.jboss.org/author/display/RHQ/Alerts#Alerts-Dampening.
On 11/28/2013 3:21 AM, Attila Heidrich wrote:
It happens to all alerts, does not matter if it is derived from a
template, or directly defined for a resource.
f.e. I had to define an alert for the postgres servers on the number of
open files. This is defined as [Metric value threshold (Open file
descriptors) > 10000]
Dempening settings: Time period: 1 Occurence in 1 Hour.
I receive all alerts in every 10 minutes. I have also tried other
dampening methods, but none worked for me.
Attila
2013/11/27 Jay Shaughnessy <jshaughn(a)redhat.com>
>
> No, all of the metric conditions should use dampening just fine.
> Availability is a bit different because of the way we only signal changes
> in availability and don't supply a constant stream of availability (this is
> for efficiency). But metrics are reported as scheduled, whether the value
> changes or not.
>
> Does this happen only when the alert definition is derived from a
> template? If you define the exact same alert def at the resource level
> does the dampening work? That would be very strange but I'm just checking.
>
> Can you post the exact alert def conditions and the dampening rule you
> have in effect.
>
>
> On 11/27/2013 10:38 AM, Attila Heidrich wrote:
>
> Hi!
>
> Thanks a lot, it does help me to understand things! My problem may be
> rooted in the concept of "discrete events"!
> anyway... is it possible, that the alert "process CPU consumption
> (percent) is higher than a given value" is affected by "being a discrete
> event"? I receive all alerts in every 20 minutes (or whatever the
> measurement interval is).
>
> The templates are itemized correctly!
>
> Regards,
>
> Attila
>
>
> 2013/11/27 Jay Shaughnessy <jshaughn(a)redhat.com>
>
>>
>> Hi Attila, two things. First, what type of alert conditions are you
>> using? Certain conditions, most notably Availability conditions like GOES
>> DOWN, ignore dampening because they are discrete events. Second, there was
>> a recent bug fix in the area of dampening. The hourly data purge was
>> negating some of the dampening state, so dampening that crossed the top of
>> the hour may have been affected. This fix will be in 4.10.
>>
>> Do either of these things, maybe explain what you are seeing?
>>
>> Also, you are saying that this is dampening at the alert template
>> level? Check to ensure the dampening is also reflected at the resource
>> level by looking at the alert definition that was propagated down from the
>> template to one of the resources of the relevant type.
>>
>>
>>
>>
>> On 11/27/2013 4:35 AM, Attila Heidrich wrote:
>>
>> No one encounters similar? It is proven, that I can set anything in
>> the dampening section, I will get each and every alerting mail! Very
>> annoying, tbh...
>>
>> What to chack if I set it correctly (I think there's nothing I can
>> miss, it's quite simple - and I have also seen this work)? Or any log that
>> should be correlated?
>>
>> Attila
>>
>>
>>
>> 2013/11/8 Attila Heidrich <attila.heidrich(a)gmail.com>
>>
>>> Hi!
>>>
>>> It seems that there's no use to set any dampening for the alert
>>> tamplates, I got all alerts for any of them.
>>>
>>> I usually use "one for several hours"... practically I only use
>>> templates, so the servers inherits the settings.
>>>
>>> Is there any trick I miss? I have used 4.8 only a few days long, but
>>> as far as I can remember, it worked as I expected.
>>>
>>> Attila
>>>
>>
>>
>>
>> _______________________________________________
>> rhq-users mailing
listrhq-users@lists.fedorahosted.orghttps://lists.fedorahosted.org/mailman/listinfo/rhq-users
>>
>>
>>
>> _______________________________________________
>> rhq-users mailing list
>> rhq-users(a)lists.fedorahosted.org
>>
https://lists.fedorahosted.org/mailman/listinfo/rhq-users
>>
>>
>
>
> _______________________________________________
> rhq-users mailing
listrhq-users@lists.fedorahosted.orghttps://lists.fedorahosted.org/mailman/listinfo/rhq-users
>
>
>
> _______________________________________________
> rhq-users mailing list
> rhq-users(a)lists.fedorahosted.org
>
https://lists.fedorahosted.org/mailman/listinfo/rhq-users
>
>
_______________________________________________
rhq-users mailing
listrhq-users@lists.fedorahosted.orghttps://lists.fedorahosted.org/mailman/listinfo/rhq-users
_______________________________________________
rhq-users mailing list
rhq-users(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/rhq-users