I'd like to try to implement this, and possibly add app-specific monitoring of queues afterwards.

I've made a few changes to Jeremy's proposal, because I wanted to make use of the configuration file that the NRPE plugin already deploys.
Attached is my proposed change to the Ansible repo.

If that works I'll add more checks later on.