infrastructure June 2019

infrastructure@lists.fedoraproject.org

24 participants
25 discussions

About JS framework
by Pierre-Yves Chibon 17 Oct '22

17 Oct '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

10 12

Re: Deprecating Autocloud
by Sinny Kumari 06 Dec '19

06 Dec '19

On Mon, Apr 29, 2019 at 4:47 PM Kamil Paral <kparal(a)redhat.com> wrote: > On Mon, Apr 29, 2019 at 11:39 AM Sinny Kumari <ksinny(a)gmail.com> wrote: > >> >> >> On Wed, Apr 24, 2019 at 12:19 AM Kevin Fenzi <kevin(a)scrye.com> wrote: >> >>> Or could we move f29+ all to whatever is replacing it? (taskotron?) >>> >> >> It will be nice but I am not aware of any other system in place which >> would >> replace checks performed by autocloud. >> >> (CC'ed tflink and kparal) >> Does taskotron provides capability to perform tests on Fedora cloud >> Images like booting images and other basic checks? >> > > Theoretically it is possible using nested virt. However, Taskotron is > going away as well. The replacement is Fedora CI: > https://docs.fedoraproject.org/en-US/ci/ > Thanks kamil! yeah, it doesn't make sense to move to Taskotron if it is going to be deprecated as well. > I recommend to ask in the CI list: > https://lists.fedoraproject.org/archives/list/ci%40lists.fedoraproject.org/ > > It should be possible for them to provide the infrastructure you need. > Hmm, I am not very sure if we should spend time investigating and setting up alternative to autocloud unless we have usecases for long run. Fedora Atomic Host Two Week releases ends with F29 EOL. -- http://sinny.io/

5 11

[PATCH] Add a nagios check for each rabbitmq server
by Jeremy Cline 03 Aug '19

03 Aug '19

Commit eae92f73e95 installed the nagios scripts[0] that are packaged for epel7-infra on the RabbitMQ hosts. This is an attempt to use them with nagios. I don't know anything about nagios though, so I have no idea if this is even close to right, or if it works. [0] https://github.com/nagios-plugins-rabbitmq/nagios-plugins-rabbitmq Signed-off-by: Jeremy Cline <jcline(a)redhat.com> --- roles/nagios_client/tasks/main.yml | 15 +++++++++++++ .../templates/check_rabbitmq_server.cfg.j2 | 1 + roles/nagios_server/tasks/main.yml | 22 +++++++++++++++++++ .../nrpe/check_rabbitmq_server.cfg.j2 | 1 + 4 files changed, 39 insertions(+) create mode 100644 roles/nagios_client/templates/check_rabbitmq_server.cfg.j2 create mode 100644 roles/nagios_server/templates/nrpe/check_rabbitmq_server.cfg.j2 diff --git a/roles/nagios_client/tasks/main.yml b/roles/nagios_client/tasks/main.yml index 0259a43ca..65a60a7c3 100644 --- a/roles/nagios_client/tasks/main.yml +++ b/roles/nagios_client/tasks/main.yml @@ -233,6 +233,21 @@ tags: - nagios_client +- name: install nrpe checks for the RabbitMQ cluster + template: + src: "{{ item }}.j2" + dest: "/etc/nrpe.d/{{ item }}" + owner: root + group: root + mode: 0600 + with_items: + - check_rabbitmq_server.cfg + when: inventory_hostname.startswith('rabbitmq') + notify: + - restart nrpe + tags: + - nagios_client + - name: nrpe service start service: name=nrpe state=started enabled=true tags: diff --git a/roles/nagios_client/templates/check_rabbitmq_server.cfg.j2 b/roles/nagios_client/templates/check_rabbitmq_server.cfg.j2 new file mode 100644 index 000000000..498a8757e --- /dev/null +++ b/roles/nagios_client/templates/check_rabbitmq_server.cfg.j2 @@ -0,0 +1 @@ +command[check_nrpe_check_rabbitmq_server]=exec /usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_server -H localhost --password "{{ (env == 'production')|ternary(rabbitmq_monitoring_password_production, rabbitmq_monitoring_password_staging) }}" --user nagios-monitoring diff --git a/roles/nagios_server/tasks/main.yml b/roles/nagios_server/tasks/main.yml index 2d6d50c31..f61584ed6 100644 --- a/roles/nagios_server/tasks/main.yml +++ b/roles/nagios_server/tasks/main.yml @@ -110,6 +110,28 @@ - config - nagios_server +- name: Ensure the nrpe.d directory exists for nrpe configs + file: + dest: /etc/nrpe.d + state: directory + tags: + - nagios_server + +- name: Add nrpe configurations to /etc/nrpe.d + template: + src: "/nrpe/{{ item }}.j2" + dest: "/etc/nrpe.d/{{ item }}" + owner: root + group: root + mode: 0600 + with_items: + - check_rabbitmq_server.cfg + notify: + - restart nrpe + tags: + - config + - nagios_server + ## Copy over the nagios configs - name: Copy /etc/nagios/conf.d/*cfg files copy: src=nagios/configs/{{ item }} dest=/etc/nagios/conf.d/{{ item }} diff --git a/roles/nagios_server/templates/nrpe/check_rabbitmq_server.cfg.j2 b/roles/nagios_server/templates/nrpe/check_rabbitmq_server.cfg.j2 new file mode 100644 index 000000000..498a8757e --- /dev/null +++ b/roles/nagios_server/templates/nrpe/check_rabbitmq_server.cfg.j2 @@ -0,0 +1 @@ +command[check_nrpe_check_rabbitmq_server]=exec /usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_server -H localhost --password "{{ (env == 'production')|ternary(rabbitmq_monitoring_password_production, rabbitmq_monitoring_password_staging) }}" --user nagios-monitoring -- 2.21.0

3 7

New website or ... ?
by Cătălin George Feștilă 30 Jun '19

30 Jun '19

I found this on my email ... https://fedoramag.wpengine.com/ This is a new magazine?

2 1

Re: ** PROBLEM alert - copr-frontend01.stg.phx2.fedoraproject.org/Disk_Space_/ is WARNING **
by Pavel Raiskup 28 Jun '19

28 Jun '19

Can anyone help me to find the box this is about? [root@copr-fe ~][PROD]# ping copr-frontend01.stg.phx2.fedoraproject.org ping: copr-frontend01.stg.phx2.fedoraproject.org: Name or service not known $ ssh root(a)copr-fe-dev.cloud.fedoraproject.org Last login: Wed Jun 26 04:35:35 2019 from 213.175.37.10 [root@copr-fe-dev ~][STG]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 2.0G 0 2.0G 0% /dev tmpfs 2.0G 8.0K 2.0G 1% /dev/shm tmpfs 2.0G 520K 2.0G 1% /run tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup /dev/vda1 40G 11G 28G 27% / /dev/vdb1 12G 4.1G 7.1G 37% /srv/copr-fe tmpfs 395M 0 395M 0% /run/user/0 Pavel On Wednesday, June 26, 2019 2:11:53 AM CEST Nagios Monitoring User wrote: > ***** Nagios ***** > > Notification Type: PROBLEM > > Service: Disk_Space_/ > Host: copr-frontend01.stg.phx2.fedoraproject.org > Address: 10.5.128.49 > State: WARNING > Source: noc01.phx2.fedoraproject.org > > Date/Time: Wed Jun 26 00:11:53 UTC 2019 > > Additional Info: > > DISK WARNING - free space: / 739 MB (14.93% inode=71%): >

2 1

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 26 Jun '19

26 Jun '19

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2019-06-27 from 15:00:00 to 16:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before or view it at https://infinote.fedoraproject.org/cgit/infinote/tree/fedora-infrastructure… Source: https://apps.fedoraproject.org/calendar/meeting/22/

1 0

[Fedocal] Reminder meeting : Bodhi stakeholders' meeting
by bowlofeggs＠fedoraproject.org 26 Jun '19

26 Jun '19

Dear all, You are kindly invited to the meeting: Bodhi stakeholders' meeting on 2019-06-26 from 11:00:00 to 11:50:00 US/Eastern At fedora-meeting(a)irc.freenode.net The meeting will be about: Source: https://apps.fedoraproject.org/calendar/meeting/9545/

1 0

Fedocal orphaned package?
by Justin W. Flory 25 Jun '19

25 Jun '19

Hi all, I noticed that Fedocal was on the upcoming orphaned packages list on a recent email to the devel-announce mailing list. I was curious if this would have an impact on Fedocal in Fedora Infra currently since I hadn't heard anything about it prior. I wasn't sure what this meant for Fedocal, so I was hoping to get some clarification if it's relevant for end-users or not. Thanks! -------- Forwarded Message -------- Subject: Orphaned packages need you Date: Mon, 24 Jun 2019 18:57:23 +0200 From: Miro Hrončok <mhroncok(a)redhat.com> Reply-To: Development discussions related to Fedora <devel(a)lists.fedoraproject.org> Organization: Red Hat To: Development discussions related to Fedora <devel(a)lists.fedoraproject.org>, devel-announce(a)lists.fedoraproject.org The following packages are orphaned and will be retired when they are orphaned for six weeks, unless someone adopts them. If you know for sure that the package should be retired, please do so now with a proper reason: https://fedoraproject.org/wiki/How_to_remove_a_package_at_end_of_life Note: If you received this mail directly you (co)maintain one of the affected packages or a package that depends on one. Please adopt the affected package or retire your depending package to avoid broken dependencies, otherwise your package will be retired when the affected package gets retired. This e-mail was shortened, see the full report at: https://churchyard.fedorapeople.org/orphans-2019-06-24.txt Packages retired today: clpbar compat-openssl10-pkcs11-helper rubygem-commander rubygem-paranoia Request package ownership via: https://pagure.io/releng/issues Package (co)maintainers Status Change ================================================================================ [snip] fedocal infra-sig, orphan 3 weeks ago [snip] The following packages require above mentioned packages: Too longs, see https://churchyard.fedorapeople.org/orphans-2019-06-24.txt Grep the content for your username and follow the dependencies. infra-sig: fedocal, python-fmn-lib -- The script creating this output is run and developed by Fedora Release Engineering. Please report issues at its pagure instance: https://pagure.io/releng/ The sources of this script can be found at: https://pagure.io/releng/blob/master/f/scripts/find_unblocked_orphans.py -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

2 1

ansible style and checking
by Kevin Fenzi 24 Jun '19

24 Jun '19

Greetings. An item that came out of our meetings last week was to try and standardize style and checks on our ansible repo. This has a number of goals, including making things more readable for newcomers as well as reducing the number of errors we commit and have to correct. Toward that end, we decided to start using 'ansible-review' on our playbooks/roles/commits. ansible-review is packaged in fedora and comes with a set of default rules which is easy to expand or reduce. We decided to start with the standard rules and asjudt from there. I setup: https://pagure.io/cpe/ansible-review-templates (which currently doesn't have anything in it) as a central place to hold any rules when/if we change from the default. We don't yet have ansible-review as a hook, but we hope to add it soon. It would only reject on errors, not warnings (of which we have a lot). Once we get ansible setup to handle PR's we can add it in there as a CI step. In the mean time you can run it over any changes you are about to make easily enough (just cat whatever things into 'ansible-review'). Additionally, we came up with a few other items: * Use 'yml' instead of 'yaml' * do add '.j2' to the end of templates * in general let readablity trump grepability, ie: DONT: - name: This is some play module: name=thing arg=thing2 anotherlongerarg=thing arg4=anodheranth DO: - name: This is some play module: - name: thing - arg: thing2 ..etc I have added a "STYLEGUIDE" text file to the root on the ansible repo with this infrmation also. If you can think of other things we should standardize on, please do follow up on this thread or file a ticket and we can discuss. Thoughts? Thanks, kevin

2 2

Re: How to consume fedora-messaging?
by Adam Williamson 20 Jun '19

20 Jun '19

On Mon, 2019-06-10 at 09:24 +0900, Tristan Cacqueray wrote: > On Mon, Jun 10, 2019 at 01:24 Igor Gnatenko wrote: > > Hello, > > > > I have been trying to write some script which would listen on > > generation of new repository / successful build is tagged in Koji and > > do some actions locally. Or basically whenever someone pushes commits, > > I want to fetch repo locally. > > > > I was reading https://fedora-messaging.readthedocs.io/en/latest/consuming.html, > > but it is not very clear to me where I can find list of topics and > > what data messages will contain... As well as the doc, there is a sample consumer: https://github.com/fedora-infra/fedora-messaging/blob/master/fedora_messagi… and a couple of sample config files: https://github.com/fedora-infra/fedora-messaging/blob/master/configs/fedora… https://github.com/fedora-infra/fedora-messaging/blob/master/configs/fedora… that you will probably find useful. > > Hi Igor, > > you can find the list of topics and their associated schema here: > https://fedora-fedmsg.readthedocs.io/en/latest/topics.html Treat this with caution, because it is...sometimes a lie. It is in fact auto-generated from the test cases for the fedmsg_meta_fedora_infrastructure project: https://github.com/fedora-infra/fedmsg_meta_fedora_infrastructure/ which is where the metadata providers that produce things like the human-readable summaries you get from FMN live. If you maintain a thing that publishes fedmsgs, you are *supposed to* also maintain a provider in fedmsg_meta that can interpret all the fedmsgs your thing publishes, and a test suite for that provider which includes tests for every topic your provider publishes on and useful docstrings for each test. That doc page is then actually built more or less just by gluing all the test docstrings together. However, there is no enforcement of any of this. Which means that it quite often isn't really all in sync. I think there may still be some fedmsg publishers which don't have a metadata provider or tests at all. There are definitely some where a provider and tests exist, but the fedmsg publisher's behaviour has changed a lot since they were written and no-one has updated the provider or the tests, so the doc no longer actually lists the topics on which messages are now being published. I don't really use this list any more because of issues like that... > And you can also find samples on this website: > https://apps.fedoraproject.org/datagrepper/raw ...this is much more useful, because it is just a log of all the *actual* fedmsgs that have ever been published. It can be a bit unwieldy to work with, but it's not too bad. You can see all Koji messages from the last two days here: https://apps.fedoraproject.org/datagrepper/raw?category=buildsys&delta=1728… You can tweak the 'delta' arg (given in seconds), or use 'start' and 'end' (given as Unix timestamps) instead, to get messages from different periods. Note, datagrepper consumes and publishes *fedmsgs*, not fedora- messaging messages (at least at present). We are currently in the middle of trying to move things over from fedmsg (the old 0MQ-based system) to fedora-messaging (the new AMQP-based system). To aid this, there are bridges in place in both directions: messages published to fedora-mesaging are re-published as fedmsgs, and messages published as fedmsgs are re-published to fedora-messaging. Even for messages that are actually natively published to fedora-messaging, what you see on datagrepper is the result of the AMQP->0MQ bridge: it's not the actual original message, but the re-published fedmsg. AFAIK there is no publicly available 'datagrepper-for-fedora-messaging' ATM, though, so you can't really look at an archive of the actual messages as they're seen on the new system. There is one very important wrinkle you should bear in mind thanks to this: the message format. If you're writing an AMQP consumer in Python, what you'll ultimately get for your consumer to process is a `message` object which is an instance of a fedora_messaging.api.Message() (or a subclass of it - a message schema class). This will have a `body` attribute which should be a dict of the message 'body' - the main meat of the message. If the message was natively published as a fedora-messaging AMQP message, that is indeed what it will be. However, if it was published as a fedmsg, what you get as `message.body` is actually the *entire* fedmsg dict - the whole dict as you see it when you view a message on datagrepper, with a bunch of headers and a 'msg' dict with the real body in it, e.g. https://apps.fedoraproject.org/datagrepper/id?id=2019-633e9879-8974-4ad4-8f… . To get to what is effectively the 'body' of a fedmsg, you need to take message.body['msg']. This wrinkle is currently under discussion at https://github.com/fedora-infra/fedmsg-migration-tools/issues/20 . I came across this while converting fedmsg consumers to fedora- messaging, and I wrote a little helper to deal with it: https://pagure.io/fedora-qa/fedora_openqa/blob/fedora-messaging/f/fedora_op… Whenever you want the body of a message you just run it through that and it should give you the 'true' body. That also means you won't have to worry if the bridge gets changed to fix the bug, that function will still give the correct result. If you want to look at some real-world fedora-messaging consumers, that file contains three consumers for scheduling openQA jobs and reporting their results. Also see the reference config files (the '.toml' files) and the README, at top level of the fedora-messaging branch: https://pagure.io/fedora-qa/fedora_openqa/tree/fedora-messaging You can also look at Bodhi, which has also been converted to fedora- messaging: https://github.com/fedora-infra/bodhi/blob/fde8eefc016b55be76ec6a5298b8b400… the config file for Bodhi is in infra ansible: https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/bodhi2… Oh, one more wrinkle I had fun with: if you want to use the 'public' credentials for the brokers, be aware that the amqp_url setting and [tls] block settings in the sample config files are correct, but the necessary certificate and key files for the staging broker are not included in the `fedora-messaging` package at present. If you look in /etc/fedora-messaging after installing it you'll see only the files for the production broker are there. If you want to listen to the staging broker, you have to grab the files from a git checkout (they're in the 'configs' dir) and copy them to /etc/fedora-messaging . -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net

4 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure June 2019