infrastructure June 2017

infrastructure@lists.fedoraproject.org

41 participants
56 discussions

About JS framework
by Pierre-Yves Chibon 17 Oct '22

17 Oct '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

10 12

Big HyperKitty schema change
by Aurelien Bompard 07 Aug '17

07 Aug '17

Hey folks! There's a pretty big DB schema change that I'd like to do in HyperKitty, and since it involves a primary key I expect a downtime of several hours, possibly a day (more precise testing going on) The good news it that mailman will keep running, only the web UIs will be down (HyperKitty & Postorius, the admin UI). It's a change that I've put off for a while, but the more we wait the longer the downtime will be (because it depends on the size of the email table). What time do you think would be the less worse for such a change? After freeze break next week? After General Availability (whenever that is) ? Another time? I have a feeling that after freeze break is better because the folks who use the web UIs to read email may also be those who will need to communicate more after the GA, to discuss communication and reviews. I don't mind starting the migration on a weekend of course, to impact less people. Thanks for your input. Aurélien

3 5

retrace / faf issues
by Kevin Fenzi 19 Jul '17

19 Jul '17

Greetings. I've seen some various retrace/faf issues of late, so I thought I would collect them into an email and see if you all could take a look and solve them. :) - retrace02.qa.fedoraproject.org has a 100% full disk. - retrace01.qa.fedoraproject.org is almost constantly alerting on swap being full. Not sure what to do about this, but perhaps we could add more swap or somehow limit it to use only memory for normal jobs? - faf01.stg has a aily cron that outputs: /etc/cron.daily/logrotate: error: skipping "/var/log/faf/create-problems.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/faf-celery-beat.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/faf-celery-worker.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/save-reports.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. - retrace01.qa.fedoraproject.org has a daily cron that outputs: /etc/cron.daily/logrotate: error: skipping "/var/log/faf/create-problems-core.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/create-problems.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/create-problems-oops.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/create-problems-python.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/create-problems-ruby.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/db_backup.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/export.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/faf-celery-beat.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/faf-celery-worker.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/find-components-centos.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/find-components-fedora.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/find-crashfn-core.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/find-crashfn-kerneloops.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/koops-to-xorg.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/mark-probably-fixed-f24.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/mark-probably-fixed-f25.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/mark-probably-fixed-f26.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/match-unknown-packages.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/pull-associates-f24.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/pull-associates-f25.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/pull-associates-f26.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/pull-components-centos.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/pull-components-fedora.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/pull-releases-centos.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/pull-releases-fedora.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-centos-7.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-epel-7.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-24-kernel-rt.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-24.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-24-testing.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-24-updates.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-25-kernel-rt.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-25.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-25-testing.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-25-updates.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-26-kernel-rt.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-26.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-26-testing.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-26-updates.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync-fedora-rawhide.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/reposync.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/retrace-core.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/retrace-kerneloops.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/retrace-oops.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/save-reports.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/update-bugs-centos.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. error: skipping "/var/log/faf/update-bugs.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. - in ansible commit f717d637 arm01-retrace01 was commented from the inventory. This means we havent hit it with any of the normal ansible based management we do (updates, etc). Can we re-add it, if only in another group? Thanks for looking at these. kevin

4 9

Re: Subscription lists
by Misha S. 19 Jul '17

19 Jul '17

I can not re-enable or change sending of mailing list mail to myself. https://lists.fedoraproject.org/admin/accounts/subscriptions Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. Please contact the server administrator at root@localhost to inform them of the time this error occurred, and the actions you performed just before this error. More information about this error may be available in the server error log. Misha https://fedoraproject.org/wiki/User:Shnurapet English / Español / Italiano / Русский

2 2

Fedora 26 Final Freeze now in effect
by Kevin Fenzi 12 Jul '17

12 Jul '17

Greetings. we are now in the infrastructure freeze leading up to the Fedora 26 Final release. This is a final-release freeze. We do this to ensure that our infrastructure is stable and ready to release Fedora 26 when it's available. You can see a list of hosts that do not freeze by checking out the ansible repo and running the freezelist script: git clone https://infrastructure.fedoraproject.org/infra/ansible.git/scripts/freezeli… -i inventory Any hosts listed as freezes is frozen until 2017-07-11 (or later if the release slips). Frozen hosts should have no changes made to them without a sign-off on the change from at least 2 sysadmin-main or rel-eng members, along with (in most cases) a patch of the exact change to be made to this list. Thanks, kevin

4 5

[Release] pagure: 3.0
by Pierre-Yves Chibon 03 Jul '17

03 Jul '17

Good Morning Everyone, I just cut a new release of pagure: 3.0. This is the version that we started working on during the beta freeze (and cut during final freeze, we may want to adjust our schedule a little better ^^). Here is its changelog: * Fri Jun 30 2017 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 3.0-1 - Update to 3.0 - Since 2.90 celery has become a requirement as well as one of the queueing system it supports (pagure defaults to using redis) - Multiple stability and performance improvements (mainly thanks to Patrick Uiterwijk) - Fix the assignee value in fedmsg when assigning a ticket (Ricky Elrod) - Make pagure support bleach 2.0.0 (Shengjing Zhu) - Fixes in CI support (Tim Flink) - Update the documentation - Fix plain readme html escape (Shengjing Zhu) - Refactor user existence code in API and UI (Abhijeet Kasurde) - Add an API to modify a Pagure project's owner (Matt Prahl) - Support for uploading multiple files to an issue at once - Introduce the external committer feature - Add the required groups feature - Add an API endpoint to get the git urls of a project (Matt Prahl) - Blacklist 'wait' as project name - Add a border to the search box on the side bar to the documentation - Add the list-id, list-archive and X-Auto-Response-Suppress email headers - Add ways to customize the gitolite configuration file with snippets - Return a 404 on private ticket if the user is not authenticated - cleanup: move static js/css to vendor dir - Limit the requests version as it conflicts with our chardet requirement - Rename all the services to pagure-* - Remove 'on <project name' - watch status dropdown (Vivek Anand) - Create references for pull-request in the git repo for local checkout - Use the entire list of users for the assignee field completion - Fix searching for groups - Make the search work when searching for project with namespaces or forks - Return a human-friendly error message when upload fails - Let acting on the status potentially set the close_status and vice versa - Multiple fixes to the SSE server - When forking a project, wait until the very end to let the user go through - Allow customizing the writing of gitolite's configuration file - Fix diffing the branch of a project against the target branch - Fix displaying the new PR button on the default branch - Do not send a notification upon merge conflicts - Do not let pagure return 500 when hit with bogus URL - When loading comment from JSON rely on username/comment rather than comment id - When deleting a comment, refresh the ticket git repo - Make patch_to_diff use lists instead of string concatenation (Patrick Uiterwijk) It is currently happily running in stg, I will be updating src.stg with this later today. Happy hacking! Pierre

2 1

Retroactive FBR: Apply string patch to Pagure
by Patrick Uiterwijk 03 Jul '17

03 Jul '17

Hi all, Last night I applied the patch from https://pagure.io/pagure/pull-request/2385#request_diff to proudction, which seems to have stabilized it completely. I just grabbed https://pagure.io/pagure/pull-request/2385.patch and applied it with "patch filter.py /root/2385.patch", and it can be reversed with the same command. I discussed this with Kevin on IRC, but for the record I'd still like to request retroactive FBR +1s. Anyone willing to give me those? Thanks! Patrick

3 2

[FREEZE BREAK REQUEST] Add nb to fama@fedoraproject.org alias
by Nick Bebout 30 Jun '17

30 Jun '17

I am now helping with Fedora Ambassadors Membership Administration (fama). I would like to add myself to that mail alias. From 66e996ec5e5746191605c9b718f405ce834357a7 Mon Sep 17 00:00:00 2001 From: Nick Bebout <nb(a)batcave01.phx2.fedoraproject.org> Date: Thu, 29 Jun 2017 16:47:18 +0000 Subject: [PATCH] Add myself to fama(a)fedoraproject.org alias --- roles/fas_client/files/aliases.template | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/fas_client/files/aliases.template b/roles/fas_client/files/aliases.template index b00bec5..4afae1d 100644 --- a/roles/fas_client/files/aliases.template +++ b/roles/fas_client/files/aliases.template @@ -225,7 +225,7 @@ info: marketing(a)lists.fedoraproject.org fedorarewards: famsco(a)lists.fedoraproject.org openvideo: tchung freemedia: tchung,susmit -fama: robyduck,giannisk +fama: robyduck,giannisk,nb # Firstname.lastname exceptions (preferrably only for people with a good reason) # History: these are people wishing to keep their firstname.lastname email -- 1.8.3.1

2 1

Plan for tomorrow's Fedora Infrastructure meeting 2017-06-29
by Kevin Fenzi 29 Jun '17

29 Jun '17

The infrastructure team will be having it's weekly meeting tomorrow, 2017-06-29 at 18:00 UTC in #fedora-meeting on the freenode network. We have a gobby document (see: https://fedoraproject.org/wiki/Gobby ) fedora-infrastructure-meeting-next is the document. Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of today is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. -- > This shared document is for the next fedora infrastructure meeting. > = Introduction = > We will use it over the week before the meeting to gather status and info and > discussion items and so forth, then use it in the irc meeting to transfer > information to the meetbot logs. > > = Meeting start stuff = > > #startmeeting Infrastructure (2017-06-29) > #meetingname infrastructure > #topic aloha > #chair smooge relrod nirik abadger1999 dgilmore threebean pingou puiterwijk pbrobinson > > = Let new people say hello = > > #topic New folks introductions > > = Status / information / Trivia / Announcements = > > (We put things here we want others on the team to know, but don't need to discuss) > (Please use #info <the thing> - your name) > > #topic announcements and information > #info F26 final freeze > #info things, more things, THINGS > #info nagios fully templated in Infrastructure > #info kojipkgs adjusted to hopefully prevent download failures - kevin > #info cloud upgraded and rebooted - relrod/patrick > #info > > > = Things we should discuss = > > We use this section to bring up discussion topics. Things we want to talk about > as a group and come up with some consensus /suor decision or just brainstorm a > problem or issue. If there are none of these we skip this section. > (Use #topic your discussion topic - your username) > > #topic > > = Apprentice office hours = > > #topic Apprentice Open office hours > > Here we will discuss any apprentice questions, try and match up people looking > for things to do with things to do, progress, testing anything like that. > > = Learn about some application or gsetup in infrastructure = > > (This section, each week we get 1 person to talk about an application or setup > that we have. Just going over what it is, how to contribute, ideas for improvement, > etc. Whoever would like to do this, just add the i/nfo in this section. In the > event we don't find someone to teach about something, we skip this section > and just move on to open floor.) > > #topic Learn about: > > = Meeting end stuff = > > #topic Open Floor > > #endmeeting

1 0

Freeze break request: increase max workers on kojipkgs apache
by Kevin Fenzi 29 Jun '17

29 Jun '17

Greetings. We saw two issues in composes this morning (an ostree that failed to download things and a server dvd compose that failed to download a package). looking at the time this happened, the apache on kojipkgs02 hit maxrequestworkers. So, I think when this happens, varnish sends back a 502 to new requests for a few seconds while the queue processes down. I would like to increase the limit from 256 (the default) to 512. +1s? kevin -- From 6733ba83e129157eb93e2ac5c24b79aebc2ac0b8 Mon Sep 17 00:00:00 2001 From: Kevin Fenzi <kevin(a)scrye.com> Date: Wed, 28 Jun 2017 20:00:45 +0000 Subject: [PATCH] up the apache limit on kojipgks from default 256 to 512. This should hopefully stop issues with 256 requests at a time causing download errors for composes Signed-off-by: Kevin Fenzi <kevin(a)scrye.com> --- roles/kojipkgs/files/kojipkgs.conf | 2 ++ 1 file changed, 2 insertions(+) diff --git a/roles/kojipkgs/files/kojipkgs.conf b/roles/kojipkgs/files/kojipkgs.conf index cf8ea56..73174bd 100644 --- a/roles/kojipkgs/files/kojipkgs.conf +++ b/roles/kojipkgs/files/kojipkgs.conf @@ -1,4 +1,6 @@ ServerName https://kojipkgs.fedoraproject.org +MaxRequestWorkers 512 +ServerLimit 512 RequestHeader unset Accept-Encoding early -- 1.8.3.1

4 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure June 2017