+1
On 7 November 2016 4:35:51 pm GMT-06:00, Kevin Fenzi <kevin(a)scrye.com>
wrote:
>
> Greetings.
>
> This has been alerting a fair bit:
>
> Nov 02 08:15:25 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 5274 (noc01)
> Nov 02 08:24:46 <zodbot> ACKNOWLEDGEMENT -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 5274 (noc01)
http://kevin.id.fedoraproject.org/
> Nov 02 18:15:16 <zodbot> RECOVERY -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is OK: OK: fedmsg consumer FedoraBadgesConsumer backlog value is 4440
> (noc01)
> Nov 02 21:17:19 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 5715 (noc01)
> Nov 02 21:27:20 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 5440 (noc01)
> Nov 02 22:07:19 <zodbot> RECOVERY -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is OK: OK: fedmsg consumer FedoraBadgesConsumer backlog value is 4207
> (noc01)
> Nov 03 05:59:23 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers
> backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog
> value is 5250 (noc01)
> Nov 03 06:03:17 <zodbot> ACKNOWLEDGEMENT -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 5250 (noc01)
http://puiterwijk.id.fedoraproject.org/
> Nov 03 07:49:19 <zodbot> RECOVERY -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is OK: OK: fedmsg consumer FedoraBadgesConsumer backlog value is 4964
> (noc01)
> Nov 03 08:22:11 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog
> value is 5185 (noc01)
> Nov 03 11:53:24 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 5621 (noc01)
> Nov 03 12:03:21 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 5578 (noc01)
> Nov 03 13:17:07 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 6395 (noc01)
> Nov 03 13:17:07 <zodbot> ACKNOWLEDGEMENT -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 6395 (noc01)
http://kevin.id.fedoraproject.org/
> Nov 03 22:43:19 <zodbot> RECOVERY -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is OK: OK: fedmsg consumer FedoraBadgesConsumer backlog value is 4574
> (noc01)
> Nov 05 22:45:19 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 5206 (noc01)
> Nov 05 22:55:21 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 5834 (noc01)
> Nov 05 23:15:22 <zodbot> RECOVERY -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is OK: OK: fedmsg consumer FedoraBadgesConsumer backlog value is 0 (noc01)
> Nov 07 15:07:22 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is
> 5178 (noc01)
> Nov 07 15:27:19 <zodbot> PROBLEM -
>
badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog
> is WARNING: WARNING: fedmsg consumer
> FedoraBadgesConsumer backlog value is 5392 (noc01)
>
> +1s?
>
> I would apply the below and run a master -t nagios_client to make sure
> all clients are updated.
>
> From a2c969c4ed4d3f80f9307859807040222b3a4b25 Mon Sep 17 00:00:00 2001
> From: Kevin Fenzi <kevin(a)scrye.com>
> Date: Mon, 7 Nov 2016 22:33:02 +0000
> Subject: [PATCH] increase the badges backlog warning to 7000 to avoid
> constant
> alerts
>
> ---
> roles/nagios/client/templates/check_fedmsg_consumers.cfg.j2 | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/roles/nagios/client/templates/check_fedmsg_consumers.cfg.j2
> b/roles/nagios/client/templates/check_fedmsg_consumers.cfg.j2
> index 2805772..0b4b973 100644
> --- a/roles/nagios/client/templates/check_fedmsg_consumers.cfg.j2
> +++ b/roles/nagios/client/templates/check_fedmsg_consumers.cfg.j2
> @@ -47,7 +47,7 @@
>
> command[check_fedmsg_cbacklog_app]={{libdir}}/nagios/plugins/check_fedmsg_consum
>
>
command[check_fedmsg_cbacklog_value]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py
> fedmsg-irc IRCBotConsumer 10 50
>
>
command[check_fedmsg_cbacklog_pkgs]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py
> fedmsg-hub GenACLsConsumer 10 50
>
>
command[check_fedmsg_cbacklog_summershum]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py
> fedmsg-hub SummerShumConsumer 100 500
>
>
-command[check_fedmsg_cbacklog_badges_backend]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py
> fedmsg-hub FedoraBadgesConsumer 5000 10000
>
>
+command[check_fedmsg_cbacklog_badges_backend]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py
> fedmsg-hub FedoraBadgesConsumer 7000 10000
>
>
>
command[check_fedmsg_cbacklog_notifs_backend]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py
> fedmsg-hub FMNConsumer 15000 20000
>
>
command[check_fedmsg_cbacklog_bugzilla2fedmsg]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py
> moksha-hub BugzillaConsumer 10 100
>
>
command[check_fedmsg_cbacklog_fedimg_backend]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py
> fedmsg-hub FedimgConsumer 2000 5000
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________
infrastructure mailing list -- infrastructure(a)lists.fedoraproject.org
To unsubscribe send an email to infrastructure-leave(a)lists.fedoraproject.org