infrastructure February 2016

infrastructure@lists.fedoraproject.org

32 participants
38 discussions

Support levels and RFR adjustments
by Kevin Fenzi 07 Apr '16

07 Apr '16

So, I sent an email a while back about this to get people thinking, but I didn't get too much feedback from my questions, so this time I am going to actually outline a proposal for people to look at. ;) Currently users expect pretty much any public service we have is fully supported. This means things like updating status when it's down, working anytime something is down to fix it as quickly as we can. New applications/services currently all pass through the (somewhat long) RFR process which we setup to make sure we could support the service moving forward. This is great and all, but some services just aren't as sustainable, or don't really fit into our RFR process very well. Also, our RFR process makes us pretty slow to bring a new service online properly. In order to have support levels, we need a way to communicate that to our users easily and the only/best way I can think of to do that easily is via domain name. If we try and have a table or something it could get pretty confusing for people. Tying it to domain names would make it much easier. So: fedoraproject.org - Anything with this domain is something that has passed though our RFR process and we support fully. This means we update status, we alert on them anytime they have issues, we work on them anytime they are down, etc. fedorainfracloud.org - This comes with a lesser level of support, simply because our cloud doesn't have any kind of HA setup, so it will be down when doing maint or when there's problems. Services in this domain may be down when there is scheduled cloud maint. We monitor, but don't page off hours, we may work on issues only during business hours, etc. Services here may not have passed through our RFR process (perhaps we should have a parallel cloud process) stg.fedoraproject.org - These can be down anytime and we monitor on them, but may not work on them off hours, etc. someother domain that sounds fedora related (fedorarelated.org? fedoralinks.org? ?) - These are things that are fedora related, but not fully controlled by fedora infrastructure. Things like the fedora bootstrap site or the porting python3 in fedora site, or possibly cloud instances that aren't managed by us. These we don't monitor or have status on, and direct people to contact the managers directly. Any other types of sites / domains people can think of? Any general thoughts on the idea? kevin

7 13

[release] mdapi: 2.4
by Pierre-Yves Chibon 29 Feb '16

29 Feb '16

Good Morning Everyone, I just released a new version of mdapi: 2.4 Here is the corresponding changelog: * Mon Feb 29 2016 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 2.4-1 - Update to 2.4 - Adjust copyright year - Expand mdapi to return dependencies information (conflicts, enhances, recommends, suggests, supplements, requires, provides and obsoletes) - Improve the cron script to retry after 30 seconds waiting, 3 times max if it fails to get info for a repo Happy querying, Pierre

1 0

[release] the-new-hotness: 0.7.0
by Ralph Bean 29 Feb '16

29 Feb '16

1 0

Fwd: Fwd: Re: Very late Koji certificate expiration warnings
by Corey Sheldon 29 Feb '16

29 Feb '16

-------- Forwarded Message -------- Delivered-To: sheldon.corey(a)gmail.com Received: by 10.25.1.212 with SMTP id 203csp981473lfb; Mon, 29 Feb 2016 01:39:32 -0800 (PST) X-Received: by 10.37.231.70 with SMTP id e67mr8421643ybh.57.1456738772077; Mon, 29 Feb 2016 01:39:32 -0800 (PST) Return-Path: <infrastructure-bounces(a)lists.fedoraproject.org> Received: from bastion.fedoraproject.org (bastion01.fedoraproject.org. [209.132.181.2]) by mx.google.com with ESMTP id l190si7997053ywg.52.2016.02.29.01.39.31 for <sheldon.corey(a)gmail.com>; Mon, 29 Feb 2016 01:39:32 -0800 (PST) Received-SPF: pass (google.com: domain of infrastructure-bounces(a)lists.fedoraproject.org designates 209.132.181.2 as permitted sender) client-ip=209.132.181.2; Authentication-Results: mx.google.com; spf=pass (google.com: domain of infrastructure-bounces(a)lists.fedoraproject.org designates 209.132.181.2 as permitted sender) smtp.mailfrom=infrastructure-bounces(a)lists.fedoraproject.org Received: from mailman01.phx2.fedoraproject.org (mailman01.phx2.fedoraproject.org [10.5.126.36]) by bastion01.phx2.fedoraproject.org (Postfix) with ESMTP id 81E0160C9833 for <sheldon.corey(a)gmail.com>; Mon, 29 Feb 2016 09:39:31 +0000 (UTC) Received: from mailman01.phx2.fedoraproject.org (localhost [IPv6:::1]) by mailman01.phx2.fedoraproject.org (Postfix) with ESMTP id 7E685245DC26D for <sheldon.corey(a)gmail.com>; Mon, 29 Feb 2016 09:39:31 +0000 (UTC) Subject: Fwd: Re: Very late Koji certificate expiration warnings From: infrastructure-owner(a)lists.fedoraproject.org To: sheldon.corey(a)gmail.com List-Administrivia: poster MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2375863589195699255==" Message-ID: <20160229093930.26287.49947(a)mailman01.phx2.fedoraproject.org> Date: Mon, 29 Feb 2016 09:39:30 +0000 List-Id: Fedora Infrastructure <infrastructure.lists.fedoraproject.org> Precedence: bulk X-Mailman-Version: 3.1.0 Your message to the infrastructure mailing-list was rejected for the following reasons: The message is not from a list member The original message as received by Mailman is attached.

1 0

Fwd: Fwd: Re: Very late Koji certificate expiration warnings
by Corey Sheldon 29 Feb '16

29 Feb '16

1 0

non free builders
by Kevin Fenzi 26 Feb '16

26 Feb '16

Greetings. So, we pride ourselves on having a 100% free and open infrastructure software wise. However, we have had some exceptions (the autocloud vbox test machine, etc). For f24, the plan is to make workstation downloads at least prefer to download a usb media creator, and then that downloads the images, verifies them and writes them to usb in a sane/best manner. Producing the Fedora/linux version of this is no problem, it's just a package. However, windows and osx are a different matter. For windows, we could run a windows vm and attach it to koji (it supports windows builders I think). This would of course mean that we would need to run a windows builder vm. ;( Or we could have a isolated hw instance that we only power on for windows builds. For osx, I don't think you can virtualize it at all, and koji has no support, so we would need to have an isolated osx instance available for building the osx media creator. (possibly only running on demand). We could of course just say: sorry, no non free oses in our infrastructure, but I fear this will result in builds done ad-hock on someones test machine that aren't reproducible or all the other things we expect from products we release. ;( Mostly this is just a heads up, but if anyone has ideas, feel free to bring them up. ;) kevin

4 3

Plan for tomorrow's Fedora Infrastructure meeting - 2016-02-25
by Kevin Fenzi 25 Feb '16

25 Feb '16

The infrastructure team will be having it's weekly meeting tomorrow, 2016-02-25 at 18:00 UTC in #fedora-meeting on the freenode network. We have a gobby document (see: https://fedoraproject.org/wiki/Gobby ) fedora-infrastructure-meeting-next is the document. Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of today is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. kevin -- = Introduction = This shared document is for the next fedora infrastructure meeting. We will use it over the week before the meeting to gather status and info and discussion items and so forth, then use it in the irc meeting to transfer information to the meetbot logs. = Meeting start stuff = #startmeeting Infrastructure (2016-02-25) #meetingname infrastructure #topic aloha #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk pbrobinson #topic New folks introductions / Apprentice feedback Can apprentice work on Ticket #5126 ? - aldapetr = Status / information / Trivia / Announcements = (We put things here we want others on the team to know, but don't need to discuss) (Please use #info <the thing> - your name) #topic announcements and information #info builders almost all re-installed - kevin #info aarch64 cartridges almost ready for service - kevin #info mass branching happened tuesday for f24 - pingou / dgilmore #info kojij database / iscsi woes - kevin/patrick/smooge #info rawhide/branched composers reinstalled as f23 - kevin #info = Things we should discuss = We use this section to bring up discussion topics. Things we want to talk about as a group and come up with some consensus or decision or just brainstorm a problem or issue. If there are none of these we skip this section. (Use #topic your discussion topic - your username) #topic = Learn about some application or setup in infrastructure = (This section, each week we get 1 person to talk about an application or setup that we have. Just going over what it is, how to contribute, ideas for improvement, etc. Whoever would like to do this, just add the info in this section. In the event we don't find someone to teach about something, we skip this section and just move on to open floor.) Schedule: 2016-02-25 - RFR - patrick 2016-03-03 - ??? #topic Learn about: = Meeting end stuff = #topic Open Floor #endmeeting

1 0

koji db issues (2016-02-23 and 24)
by Kevin Fenzi 24 Feb '16

24 Feb '16

I thought I would send out a note on what we know so far, and status on koji db. Last night the vm that runs koji's database went unresponsive. Rebooting it made it come up in a very degraded state where it couldn't find some of it's disks and never fully started. An additional reboot brought it back up, but as soon as it started serving database requests it went into a state where it was waiting for i/o and not processing. We rebooted the virthost that runs that vm a number of times as well as engaged networking and storage folks to look at those things. In working on this issue we: * ran a database vacuum on the koji db. * fixed a misconfiguration in our multipathd config. * fixed a configuration issue on a related virthost that made it's vm's unable to connect to the storage network. Finally things seemed to settle down early this morning and we were able to bring the database back online. Later in the morning there was another short period of heavy i/o wait, but it recovered without intervention on our part. The root cause seems to be the iscsi netapp volume that the instance was defined on had some connectivity or loading issues and wasn't able to handle the load for the vm. We have storage folks looking for issues on the netapp side of things and we are closely watching the server end. Hopefully we are all back on track now. kevin

1 0

[release] pagure: 1.1.1
by Pierre-Yves Chibon 24 Feb '16

24 Feb '16

Good Morning everyone, I just cut a new (bugfix) release of pagure: 1.1.1 Here is the corresponding changelog: * Wed Feb 24 2016 Pierre-Yves Chibon <pingou(a)pingoured.fr> - 1.1.1-1 - Update to 1.1.1 - Fix showing some files where decoding to UTF-8 was failing - Avoid adding a notification to a PR for nothing - Show notifications correctly on the PR page when received via SSE The UTF-8 fix should reduce the number of emails I was getting about this error, the other are more fixing inconvenience. This is currently happily running in stg and prod. Happy hacking! Pierre

1 0

FAS3
by Ryan Lerch 23 Feb '16

23 Feb '16

HI all! Just wondering if there has been any recent work on the new FAS lately. I am interested in jumping in and helping out from a UX perspective. cheers, ryanlerch

3 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure February 2016