I have been tasked with helping to deploy coreos.fedoraproject.org, and
so I have a patch to review, Clement told me to send it here for merge/review.
1 patch is for the DNS repo, the other for ansible.
I can't test any of them right now, so maybe I overlooked something
(and as said on irc, the deadline is a bit tight, but I will take care of any errors
= Preamble =
The infrastructure team will be having its weekly meeting tomorrow,
2018-06-14 at 14:00 UTC in #fedora-meeting-1 on the freenode network.
We have a gobby document
(see: https://fedoraproject.org/wiki/Gobby )
fedora-infrastructure-meeting-next is the document.
Please try and review and edit that document before the meeting and we
will use it to have our agenda of things to discuss. A copy as of today
is included in this email.
If you have something to discuss, add the topic to the discussion area
with your name. If you would like to teach other folks about some
application or setup in our infrastructure, please add that topic and
your name to the learn about section.
= Introduction =
We will use it over the week before the meeting to gather status and info and
discussion items and so forth, then use it in the irc meeting to transfer
information to the meetbot logs.
= Meeting start stuff =
#startmeeting Infrastructure (2018-06-14)
#chair nirik pingou puiterwijk relrod smooge tflink threebean
= Let new people say hello =
#topic New folks introductions
#info This is a place where people who are interested in Fedora
Infrastructure can introduce themselves
= Status / Information / Trivia / Announcements =
(We put things here we want others on the team to know, but don't need
(Please use #info <the thing> - your name)
#topic announcements and information
#info relrod PTO 9 Jun - 19 Jun # this is ongoing.
#info smooge PTO 15 Jun -> 18 Jun
#info Office hours
= Things we should discuss =
We use this section to bring up discussion topics. Things we want to talk about
as a group and come up with some consensus /suor decision or just brainstorm a
problem or issue. If there are none of these we skip this section.
(Use #topic your discussion topic - your username)
#info Nirik is on call from 2018-06-07->2018-06-19
#info Relrod is on call from 2018-06-20->2018-06-26
#info Smooge is on call from 2018-06-27->2018-07-05
#info ?? is on call from 2018-07-06->2018-07-12
#topic Flock - nirik
#topic Tickets discussion
Go thru each ticket one by one
= Apprentice office hours =
#topic Apprentice Open office minutes
#info A time where apprentices may ask for help or look at problems.
Here we will discuss any apprentice questions, try and match up people looking
for things to do with things to do, progress, testing anything like that.
= Learn about some application or setup in infrastructure =
(This section, each week we get 1 person to talk about an application or setup
that we have. Just going over what it is, how to contribute, ideas for
etc. Whoever would like to do this, just add the i/nfo in this section. In the
event we don't find someone to teach about something, we skip this section
and just move on to open floor.)
= Meeting end stuff =
#topic Open Floor
Stephen J Smoogen.
I've finally finished writing patches to integrate zchunk support into
dnf/libsolv/librepo, and I'd greatly appreciate some code review. A
vast majority of the code is in librepo, but libsolv has been expanded
to support zchunk files and dnf has a tiny patch that passes the base
cache directory to librepo to find source zchunk files to delta
With these patches and a zchunk-enabled repository, you will download
only the differences in your primary/filelists/other metadata. Partial
downloads are validated and then continued, and each chunk is validated
as it's downloaded, ending the download and moving to a new mirror if
any chunk is corrupt.
I would love to get these changes into Fedora 29, and the code is
testable now, but with only three weeks until System-Wide change
proposals are due, I'm not sure if I'm being ambitious.
On another note, I would also like to finalize zchunk's API and make a
stable ABI promise, but before I take that step I'd really love some
feedback on its usability.
I would like to get feedback about starting a weekly infrastructure office
hours which would be used to help anyone with contributing to Fedora's
This would be a dedicated IRC meeting where people would be available to
answer questions, help setting up development environment or even maybe
work together on an issue. It could also be used to add more details to the
issues we have in our backlog to help new contributor to get started.
Note that we already have a general "ask whatever, whenever" on
#fedora-apps and #fedora-admin where anybody can "ask whatever, whenever"
:-), but this does not seems to work since we have very few requests.
So do you think that having a dedicated meeting where you know that people
are available would improve the situation ?
Good Morning Everyone,
We have been informed thst week at the upstream devs working on the
product-definition-center (PDC) are moving away from the project and are going
to leave it without a maintainer. Since we adopted PDC for a variety of data
flows, this puts us in an awkward position. :(
Ralph and I met up on Tuesday to brainstorm the list of things we actively use
PDC for today and to come up with a contingency plan for how to handle them. One
overarching option is for us (fedora-infra) to take ownership of the PDC
codebase as a whole. We didn't fully explore this option, figuring that the
codebase is large and contains lots of tables, endpoints, and codepaths that we
didn't use nor which we plan to use.
Instead, below we've got the four things we use PDC for and some options for
what to do with each.
With the exception of /modules/, one common pattern that we like is to
investigate splitting out the "django apps" that make up PDC into their own
projects. We're calling these "pdc-lite", for fun. See more below.
The data in the /modules/ PDC endpoint is *also* in the MBS db. Ralph's
team is going to just use that and stop using pdc anything for modules.
We're going to need to patch pungi, mbs for local builds, and a few other
places. This should be a relatively low-pain transition.
* Stream branches, branch ownership, retirement dates?
- SLA/EOL are currently stored in PDC.
- Queried by releng scripts for retirement, fedrepo-req for new branches,
git repo full of yaml file similar to the override repo
compiled into a single JSON blob
Single place for all retired packages
This feels like the lowest tech option.
git gives us change control for free... but people easily get lost in the
"UX" of navigating a gigantic git repo full of plaintext files.
pagure knows nothing about branches currently, so that would be bigger
PDC internally is composed of ~20 "django apps"
We could pick the 2 or 3 that comprise the branches feature, copy them
out, and turn them into their own service: the "branch definition center":
That would be the "pdc-lite" approach mentionned above, ie: PDC with only
the "app" of interest
* release/life-circle tracking?
PDC lite with just that app of interest
JSON/yaml file on the proxies
PDC-lite with just that app of interest
Drop this entirely?
Adam probably really wants to keep the record of composes.
The "pdc-lite" options are attractive, across the board. One thing we get from
this is greater clarity when discussing things formerly in PDC. If something is
in the branch-definition-center, the compose-definition-center, or the
release-definition-center.. you know what you're talking about. Today, when
talking about whether or not something should be or is in "PDC", it is easy to
I propose we start the discussion on the list and plan for a meeting sometime
late next week to discuss it further with the interested parties (please signal
What do you think?
Pierre and Ralph
My IRC nick is janslow. I've volunteered in the infrastructure team in the past as an apprentice, but had to drop out for personal reasons.
I'm hoping that I can start to contribute again and on a more regular basis.
My purpose for wanting to volunteer is to keep my infrastructure knowledge relevant, and to expand it to new technologies and applications. I previously worked full time as a software developer and later DevOps engineer, mainly programming in PHP and generic web development languages and frameworks, with mostly Jenkins and Bamboo for CI. On the infra side I have worked maintaining CentOS and Debian boxes in both traditional bare-metal and IaaS/"Cloud" environments, both for smaller projects and at scale for larger blue chip corporates.
Hope to see you at today's meeting.
Outage: db-qa server - 2018-06-07 23:00 UTC
There will be an outage starting at 2018-06-07 23:00 UTC, which will last
approximately 4 hours.
To convert UTC to your local time, take a look at
date -d '2018-06-07 23:00 UTC'
Reason for outage:
Currently the database server on the fedora qa network has only 2 CPUs
and limited ram. This is causing problems with waiverdb/resultsdb and
other tools needing to get builds in place. The databaseserver will be
moved to a different virtual server and given 8 additional cpus and
Please join #fedora-admin in irc.freenode.net or add comments to the
ticket for this outage above.
Stephen J Smoogen.
Jeremy and I have been working on a proposal to migrate fedmsg from our
current brokerless architecture to a broker-based architecture.
The overview and reasons for the migration are described on this page:
Head there if you want the details. The plan has the following requirements:
* No flag day.
* Don't disrupt any services or applications.
* Don't break any services outside of Fedora's infrastructure relying on
The first step is to deploy a broker in Fedora to use. In order to avoid a
flag day, bridges from AMQP to ZeroMQ and ZeroMQ to AMQP have been
implemented and will be deployed.
In order to validate that the bridges are functioning, a small service will
be run during the transition period that connects to fedmsg and to the AMQP
queues to compare messages.
After the bridges are running, applications are free to migrate. There are
several options when migrating and each has advantages and disadvantages.
We have written a new library called fedora-messaging that has the
* A method to define message schemas and offer automatic validation of
messages using those schemas.
* Boilerplate for typical publishers and consumers.
Head over to the document for a demo!
What do you think of this proposal? Any blind spots?