infrastructure September 2016

infrastructure@lists.fedoraproject.org

51 participants
54 discussions

About JS framework
by Pierre-Yves Chibon 17 Oct '22

17 Oct '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

10 12

Proposal to mirror Docker images
by Randy Barlow 27 Oct '16

27 Oct '16

During Flock 2016, I had the opportunity to talk with Adam Miller, Dennis Gilmore, and Pierre-Yves Chibon about the technical challenges with distributing Docker images with Fedora's extensive mirror network. These conversations helped me to solidify a proposal for how Fedora could solve this problem, outlined below. High level view =============== In summary, the proposal is to write a patch for the docker client that will give it the capability to accept metalink responses upon docker pull operations. We would also need to add support for Docker images to mirror list and mirror manager. Additionally, we will need a small tool to pull the content to be mirrored out of a docker registry and write them to disk in a format that can be mirrored, as well as some Ansible code to run the tool when there is new content to be mirrored. Background ========== The Fedora project wishes to begin distributing new types of content than it has in the past. One of the types that has been identified as a goal is the Docker image. Adam Miller has already done the work that will allow packagers to build Docker images, but we still need a way to distribute those builds to Fedora's users. Adam Miller's implementation helpfully drops the builds we want into a Docker registry. Proposed Changes ================ Mirror List ----------- Users will be pointing their docker clients at Mirror List when they docker pull Fedora's Docker images. In order for this to work, we will need to make two changes to Mirror List so that it can respond to the docker client properly. The first change is that Mirror List will need to respond with a special header and a body of "{}" when the docker client sends a GET request for /v2/. The second change is that it will need to return a metalink document when the client makes additional requests so that the clients can be redirected to a list of mirrors near their locations, just as it does with the dnf client today. The docker client typically connects to port 5000. We could run a second instance of Mirror List on port 5000 if we wanted to isolate it from the current instance. We can also have the docker client pull from 443 as dnf does if we want to keep the deployment simpler. Mirror Manager -------------- We will need to make a few changes to Mirror Manager as well. We will need to provide an interface to allow mirror admins to opt in/out of mirroring Docker content. We will also need to modify the curler to detect whether a given mirror is up to date or not. We will need to make sure that UMDL is updated when content changes. docker ------ The most significant work required will likely be modifying the docker client to enable it to properly handle the metalink responses it will be receiving from Mirror List. When requesting the manifest, it will receive a metalink document that will give it a priority ordered list of mirrors. It will need to work through the list in order until it reaches a mirror that has the correct checksum for the requested manifest. It will then use that same mirror for the subsequent blob requests. There is some concern that such a feature would not be accepted by the upstream docker project. If we were to proceed with this proposal, we would propose this patch to the upstream Docker project. If upstream were not willing to accept the feature, we would need to have the Fedora docker packager carry this patch as a downstream add on. New Tool -------- The last piece that is needed is a tool that can create the filesystem tree that we want to synchronize out to the mirrors. The mirrors only need to carry manifests and blobs, so the tool needs only to pull these documents out of the registry that Adam Miller has set up and write them to disk in a particular structure. For optimization, we could use hardlinks for blobs that are common across the various images (for example, the Fedora base blob will be the same in all images) to save rsync time and mirror disk space. Additionally, we will need a playbook to run this new tool in response to fedmsgs. We may be able to use Adam Miller's loopabull project to run such a playbook at the right times. Conclusion ========== Thanks for reading, and please respond with any comments or questions you have about this proposal. I'm happy to clarify any points further, and if you have any alternative proposals I'd love to hear those as well.

12 26

Fedora 25 Beta freeze now in effect
by Kevin Fenzi 12 Oct '16

12 Oct '16

Greetings. we are now in the infrastructure freeze leading up to the Fedora 25 Beta release. This is a pre-release freeze. We do this to ensure that our infrastructure is stable and ready to release the Fedora 25 Beta when it's available. You can see a list of hosts that do not freeze by checking out the ansible repo and running the freezelist script: git clone https://infrastructure.fedoraproject.org/infra/ansible.git scripts/freezelist -i inventory Any hosts listed as freezes is frozen until 2016-10-11. (or later if Beta slips). Frozen hosts should have no changes made to them without a sign-off on the change from at least 2 sysadmin-main or rel-eng members, along with (in most cases) a patch of the exact change to be made to this list. Thanks, kevin

1 1

Freeze break request: add diversity and magazine fedmsg bots
by Kevin Fenzi 01 Oct '16

01 Oct '16

Greetings. For https://pagure.io/fedora-infrastructure/issue/5506 and https://pagure.io/fedora-infrastructure/issue/5507 we need to adjust the fedmsg irc bot config. +1s? kevin -- From 5f7ebfbbbcf0109e2d41853439d32767f525d0d5 Mon Sep 17 00:00:00 2001 From: Kevin Fenzi <kevin(a)scrye.com> Date: Fri, 30 Sep 2016 20:05:44 +0000 Subject: [PATCH] add diversity and magazine fedmsg bots. issues 5506, 5507 --- roles/fedmsg/irc/templates/ircbot.py | 38 ++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/roles/fedmsg/irc/templates/ircbot.py b/roles/fedmsg/irc/templates/ircbot.py index 4723938..34227fa 100644 --- a/roles/fedmsg/irc/templates/ircbot.py +++ b/roles/fedmsg/irc/templates/ircbot.py @@ -386,6 +386,44 @@ config = dict( body=['^((?!(modularity|Modularity)).)*$'], ), ), + + # And #fedora-diversity + dict( + network='chat.freenode.net', + port=6667, + make_pretty=True, + make_terse=True, + + {% if env == 'staging' %} + nickname='fedmsg-diversity-stg', + {% else %} + nickname='fedmsg-diversity-bot', + {% endif %} + channel='#fedora-diversity', + # If the word diversity appears in any message, forward it. + filters=dict( + body=['^((?!(diversity|Diversity)).)*$'], + ), + ), + + # And #fedora-magazine + dict( + network='chat.freenode.net', + port=6667, + make_pretty=True, + make_terse=True, + + {% if env == 'staging' %} + nickname='fedmsg-magazine-stg', + {% else %} + nickname='fedmsg-magazine-bot', + {% endif %} + channel='#fedora-magazine', + # If the word modularity appears in any message, forward it. + filters=dict( + body=['^((?!(magazine|Magazine)).)*$'], + ), + ), ], ### Possible colors are ### -- 1.8.3.1

3 2

[patch] Badges: Oldschool mapping for Summer Coder
by Justin W. Flory 30 Sep '16

30 Sep '16

Hi all, This patch is another FAS group mapping to the corresponding badge in Fedora Badges. It maps the 'summer-coding' FAS group to the 'summer-coder' badge. Patch file is attached. Thanks! -- Cheers, Justin W. Flory jflory7(a)gmail.com

2 1

Weekly Koji Intra tags
by marc84 30 Sep '16

30 Sep '16

dist-5E-epel-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- corosync dist-5E-epel-infra ausil postgresql8.3 dist-5E-epel-infra toshio net-snmp dist-5E-epel-infra ausil buildsys-macros dist-5E-epel-infra ausil qpidc dist-5E-epel-infra ausil koji-theme-fedora dist-5E-epel-infra ausil postgresql dist-5E-epel-infra toshio xz dist-5E-epel-infra ausil postgresql8.3-pgpool-II dist-5E-epel-infra toshio openais dist-5E-epel-infra ausil postgresql-pgpool-II dist-5E-epel-infra toshio cluster dist-5E-epel-infra ausil moksha dist-5E-epel-infra lmacken rpm dist-5E-epel-infra ausil epel6-infra (no matching packages) epel7-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- python-robosignatory epel7-infra puiterwijk pdc-updater epel7-infra ralph f23-infra (no matching packages) f24-infra Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- mediawiki-FedoraBadges f24-infra kevin mediawiki-Lockdown f24-infra kevin mediawiki-RSS f24-infra kevin mediawiki-openid f24-infra kevin basset f24-infra puiterwijk f25-infra (no matching packages)

1 0

Weekly Koji Intra tags
by marc84 30 Sep '16

30 Sep '16

Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- corosync dist-5E-epel-infra ausil postgresql8.3 dist-5E-epel-infra toshio net-snmp dist-5E-epel-infra ausil buildsys-macros dist-5E-epel-infra ausil qpidc dist-5E-epel-infra ausil koji-theme-fedora dist-5E-epel-infra ausil postgresql dist-5E-epel-infra toshio xz dist-5E-epel-infra ausil postgresql8.3-pgpool-II dist-5E-epel-infra toshio openais dist-5E-epel-infra ausil postgresql-pgpool-II dist-5E-epel-infra toshio cluster dist-5E-epel-infra ausil moksha dist-5E-epel-infra lmacken rpm dist-5E-epel-infra ausil Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- python-robosignatory epel7-infra puiterwijk pdc-updater epel7-infra ralph Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- --------------- mediawiki-FedoraBadges f24-infra kevin mediawiki-Lockdown f24-infra kevin mediawiki-RSS f24-infra kevin mediawiki-openid f24-infra kevin basset f24-infra puiterwijk

1 0

#5504 weekly script to report packages in the koji infra* tags to infrastructure list
by Marc Bagatela 29 Sep '16

29 Sep '16

#!/bin/bash koji list-pkgs --tag=dist-5E-epel-infra --noinherit > /tmp/dist-5E-epel-infra koji list-pkgs --tag=epel6-infra --noinherit > /tmp/epel6-infra koji list-pkgs --tag=epel7-infra --noinherit > /tmp/epel7-infra koji list-pkgs --tag=f23-infra --noinherit > /tmp/f23-infra koji list-pkgs --tag=f24-infra --noinherit > /tmp/f24-infra koji list-pkgs --tag=f25-infra --noinherit > /tmp/f25-infra cat /tmp/dist-5E-epel-infra /tmp/epel6-infra /tmp/epel7-infra /tmp/f23-infra /tmp/f24-infra /tmp/f25-infra > /tmp/koji-infra-tags-weekly

1 0

Freeze Break Request: update koji-builder
by Adam Miller 29 Sep '16

29 Sep '16

Hello all, There has been a new patch for Koji that adds the option '--new-chroot' to the runroot plugin which fixes an issue with building ostrees in koji via pungi. This is needed by Release Engineering to successfully compose Fedora 25 Atomic Host for Beta. The patch has been tested in stage koji. Also, the patch has been accepted upstream and will be included in the next release of Koji[0]. Packages are in Koji: http://koji.fedoraproject.org/koji/buildinfo?buildID=805252 http://koji.fedoraproject.org/koji/buildinfo?buildID=805251 http://koji.fedoraproject.org/koji/buildinfo?buildID=805250 http://koji.fedoraproject.org/koji/buildinfo?buildID=805249 Updates are in bodhi: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-f66e4486db https://bodhi.fedoraproject.org/updates/FEDORA-2016-cf7725e102 https://bodhi.fedoraproject.org/updates/FEDORA-2016-d26923757a https://bodhi.fedoraproject.org/updates/FEDORA-2016-5c132f6a26 Thank you, -AdamM [0] - https://pagure.io/koji/pull-request/162

4 7

Meeting Agenda Item: Introduction Ankit Raj Ojha
by Ankit Raj Ojha 29 Sep '16

29 Sep '16

Hello, everybody My name is Ankit Raj Ojha, My IRC handle is:- ankit01ojha Programming languages that I am familiar with are C, C++, Python, HTML, CSS and a bit with AngularJS and I am currently working on linux. I am associated with FOSS@amrita group. I want to join because I want to contribute to Open-Source and I think that I can increase my skills by contributing to Fedora and found that Fedora-infrastructure has lots of new possibilities. Right now, I am studying B.Tech in Computer Science and Engineering and I am in my first year so, I can work upto 50 hours per week and I am a Open-Source newbie.I would like to do some bugs fixing and programming. With Regards Ankit Raj Ojha B.Tech CSE *|Blog <https://ankitrajmastermind.wordpress.com/>*|*Github* <https://github.com/ankit123legend>|

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure September 2016