infra ticket stats
by Kevin Fenzi
Just FYI, I gathered some real simple stats on our infrastructure
tickets. Just counting the number opened and the number closed per
month. So this doesn't include existing tickets, just new and closed.
And of course some tickets are easier than others, etc.
open | closed
2018-01: 57 | 42
2018-02: 98 | 85
2018-03: 65 | 44
2018-04: 92 | 105
2018-05: 113 | 96
2018-06: 67 | 60
2018-07: 52 | 66
2018-08: 68 | 85
2018-09: 72 | 99
2018-10: 77 | 76
2018-11: 81 | 66
2018-12: 61 | 54
2019-01: 81 | 84
2019-02: 108 | 59
2019-03: 117 | 85
2019-04: 103 | 96
2019-05: 123 | 132
2019-06: 72 | 70
2019-07: 94 | 111
2019-08: 97 | 118
2019-09: 113 | 118
2019-10: 102 | 103
2019-11: 103 | 84
2019-12: 90 | 78
2020-01: 94 | 80
2020-02: 137 | 115
2020-03: 141 | 126
2020-04: 29 | 51 (so far)
I think we are improving our close rate, but also incoming rate is
increasing (which may be due to us asking everyone to actually file
tickets instead of pinging us, etc).
Also, we can see we handle usually around 100 tickets a month, but have
been increasing that rate too.
Just thought I would toss it out there to see if there's anything we can
learn from it. :)
kevin
4 years
[FBR] Add kerneltest.fp.o reverse proxy + post-fact FBR
by Pierre-Yves Chibon
Good Morning,
While working on https://pagure.io/fedora-infrastructure/issue/8035 to help
Kevin I ended up adding the following commit to the proxies and running the
proxy playbook for it:
commit d6fe2e08a6897470cdaf3ecd191c5329dca49b3c
Author: Pierre-Yves Chibon <pingou(a)pingoured.fr>
Date: Tue Apr 14 11:25:17 2020 +0200
proxies-website: Add kerneltest to the proxies
Signed-off-by: Pierre-Yves Chibon <pingou(a)pingoured.fr>
diff --git a/playbooks/include/proxies-websites.yml b/playbooks/include/proxies-websites.yml
index a18f7a3a3..5159ecdff 100644
--- a/playbooks/include/proxies-websites.yml
+++ b/playbooks/include/proxies-websites.yml
@@ -996,6 +996,13 @@
cert_name: "{{wildcard_cert_name}}"
tags: calendar
+ - role: httpd/website
+ site_name: kerneltest.fedoraproject.org
+ sslonly: true
+ server_aliases: [kerneltest.stg.fedoraproject.org]
+ cert_name: "{{wildcard_cert_name}}"
+ tags: kerneltest
+
# fedorahosted is retired. We have the site here so we can redirect it.
- role: httpd/website
I had totally forgot proxies were frozen, so my apologies for pushing this
through and I'd like to ask for a post-fact FBR.
With this I got kerneltest mostly working expect that it doesn't send back the
expected html page.
Clément took a look at it and pointed me to the reverse proxy which haven't been
configured for kerneltest.
So I'd like to apply the following patch and run the corresponding playbook:
commit d6fe2e08a6897470cdaf3ecd191c5329dca49b3c
Author: Pierre-Yves Chibon <pingou(a)pingoured.fr>
Date: Tue Apr 14 11:25:17 2020 +0200
proxies-website: Add kerneltest to the proxies
Signed-off-by: Pierre-Yves Chibon <pingou(a)pingoured.fr>
diff --git a/playbooks/include/proxies-websites.yml b/playbooks/include/proxies-websites.yml
index a18f7a3a3..5159ecdff 100644
--- a/playbooks/include/proxies-websites.yml
+++ b/playbooks/include/proxies-websites.yml
@@ -996,6 +996,13 @@
cert_name: "{{wildcard_cert_name}}"
tags: calendar
+ - role: httpd/website
+ site_name: kerneltest.fedoraproject.org
+ sslonly: true
+ server_aliases: [kerneltest.stg.fedoraproject.org]
+ cert_name: "{{wildcard_cert_name}}"
+ tags: kerneltest
+
# fedorahosted is retired. We have the site here so we can redirect it.
- role: httpd/website
+1s?
Thanks,
Pierre
4 years
CPE Weekly: 2020-04-14
by Aoife Moloney
# CPE Weekly: 2020-04-14
---
title: CPE Weekly status email
tags: CPE Weekly, email
---
Hi All,
Apologies for the delayed weekly mail, I enjoyed a lovely four-day
weekend with my family over Easter which was important and didn't get
around to sending this email.
The upside is you get two emails from me this week instead :)
Background:
The Community Platform Engineering group is the Red Hat team combining
IT and release engineering from Fedora and CentOS.Check out our teams
info here https://docs.fedoraproject.org/en-US/cpe/
## GitForge Updates
* Initial conversations have started with GitLab to investigate
options and weight them against more refined requirements to make sure
we reach the best possible outcome
* We are still engaging with Fedora Council
https://pagure.io/Fedora-Council/tickets/issue/292
* We are also engaging with the CentOS board too
* And we are looking at ways to have open conversations/Q&A sessions
in public forums too as we move through this journey.
## Fedora Updates
* F32 release first go/no-go meeting is scheduled for this week to
review the estimated release date of 21st April
* Calendar for meetings here
https://apps.fedoraproject.org/calendar/Fedora%20release/
* F32 release schedule here https://fedoraproject.org/wiki/Releases/32/Schedule
### Data Centre Move
* CommuniShift is now down until May 8th est. to facilitate the data
centre move that has now officially begun.
* Please see our list of affected https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA
* Our first few sets of hardware have been deracked and are awaiting
collection from PHX2 this week.
* Please check our schedule here - we are on Week 06
https://hackmd.io/R3EkjzVyTG2TYwQvkfzYrA
### AAA Replacement
* The team are planning phase two of the replacement which will target
UX/UI improvements,
### CI/CD
* rpmautospec
* Call for testers has been sent to the devel-announce list:
https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedora...
* Documentation is also being refined on:
* How it works
* What are its peculiarities
* How to opt-in
* What is working and what isn't yet
* What remains to do before this gets pushed to production
* Thank you for your feedback!
### Sustaining Team
* Mbbox Upgrade
* The team are making good progress on the koji-builder & koji-hub
CRD https://github.com/fedora-infra/mbbox/pull/20
* They also have resolved the issue with PVC in the mbox-operator
https://github.com/fedora-infra/mbbox/pull/23
* We are now looking into CI support and the work that this will involve
* Bodhi
* Investigation database lockups, seems related to celery worker
not completing their tasks
* Working on a 5.3 release
* Releng
* fedscm-admin 1.0.13 released
* Future openh264 composes will use ODCS, script is added to releng repo
* Infra
* The daily standup the team has has helped a lot with managing
infra tickets - they are down to 99 tickets!
* Mass update of stg and prod
* Please note you may experience some Kojira slowness
* New review-stats application deployed -
(https://fedoraproject.org/PackageReviewStatus/)
## CentOS Updates
### CentOS
* CentOS CI is stable and all working fine
* Investigation for cloud vms addition to cico is also underway
* RHEL 8.1 batch update 07-Apr
### CentOS Stream
* We have now ~200 packages built in CentOS Stream & we expect a few
hundred more to become available to us soon!
* The team are also still working on general compose config, koji tag
& housekeeping :)
As always, feedback is welcome, and we will continue to look at ways
to improve the delivery and readability of this weekly report.
Have a great weekend!
Aoife
Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ
-
Aoife Moloney
Product Owner
Community Platform Engineering Team
Red Hat EMEA
Communications House
Cork Road
Waterford
4 years
Packit Pagure support (was: [RFC] Optionally using git repositories
instead of the lookaside cache)
by Neal Gompa
On Thu, Apr 9, 2020 at 11:41 AM Tomas Tomecek <ttomecek(a)redhat.com> wrote:
>
> On Thu, Apr 9, 2020 at 4:34 PM Jeremy Cline <jeremy(a)jcline.org> wrote:
> >
> > We actually are. I assume you're asking about why we're not using
> > packit. We're not on GitHub so the service isn't (as far as I can tell)
> > useful to us. There's huge piles of existing bash scripts and makefiles
> > that achieve about what I think packit-the-cli would give us so it
> > would be some amount of work with no obvious benefit to move at the
> > moment. Generally speaking, though, I'm not against the idea.
>
> When we started packit, we played with the scripts and makefiles in
> your kernel repo and I hope it's not too bold of me to say that it
> shouldn't be that hard to integrate the two now.
>
> I'm assuming you're using pagure.io - at this point, we're not going
> to integrate packit with pagure (for obvious reasons). What are your
> future plans for the git forge?
What the heck are you talking about? pagure.io is not going away
anytime soon. And there will be other pagure instances that host
source code that packit integration support would be useful for.
Hell, I've already offered to help with the pagure.io service.
--
真実はいつも一つ!/ Always, there's only one truth!
4 years
Re: Issue with Fedora GeoIP service
by Kevin Fenzi
On Fri, Dec 06, 2019 at 04:49:15PM +0000, Tom Hughes wrote:
> On 06/12/2019 16:41, Martin Kolman wrote:
> > On Fri, 2019-12-06 at 08:38 -0600, Chris Adams wrote:
> >
> > > I also installed the Fedora 31 GeoIP packages and ran the geoipupdate,
> > > and that DB has the correct info.
> >
> > IIRC the infra team mentioned some issues with the new geoip database
> > being incompatible with how the service is currently implemented,
> > resulting in being stuck with an outdated database until this is resolved.
>
> Sounds like it maybe doesn't have support for GeoLite2 and is using the
> old MaxMind GeoLite Legacy databases which haven't been updated since
> the start of this year.
Yes, that is exactly the case.
We recently looked at this to see if we could retire the service, but it
looks like it's still needed, so we need to figure out how to get cycles
to update it.
If someone wants to work on this, let us know!
kevin
4 years
[RFC] Optionally using git repositories instead of the lookaside
cache
by Jeremy Cline
Hi folks,
The Fedora kernel is moving to maintaining the package in a source
(sometimes people refer to it as an "exploded") tree. Basically just a
fork of upstream. This makes a lot of packager tasks easier, but has
introduced a minor issue with respect to the lookaside cache.
Right now, it's configured to create a tarball from the git tree and
upload it to the lookaside cache for each build. We build the rawhide
kernel every weekday (give or take) and the xz compressed source
tarball is ~110MB. This works out to about 28GB per year for Rawhide
alone (if this is a drop in the bucket and no one cares please let me
know and we'll just do this). The old approach uploaded a release
tarball and then incremental tarballs on top of that.
If, however, Fedora allowed packagers to optionally generate tarballs
from a git repository we could just push the linux git repository. The
entire repository with history going back 15 years is under 4GB total,
which is pretty good when compared to ~419GB which is the space
required for the equivalent time using the lookaside cache.
What would need to change:
* Fedora offers a git repository to push source trees to.
* A new file in the dist-git repository could be added if the packager
wishes called "source-repos". In it, it contains a git url and commit
identifier. For example, an entry might look like:
"
https://src.fedoraproject.org/sources/kernel.git v5.6"
where v5.6 is a tag in the repository. We can restrict it so the git
repository must be hosted by Fedora so we keep all the sources
forever.
* fedpkg and fedpkg-minimal would need to be updated to pull the
source tree if the "source-repos" file is found and run
"git archive". Fortunately this work is actually already done since
Red Hat's version of fedpkg already supports this.
I'm happy do to all the work for fedpkg/fedpkg-minimal to make this
possible because the other option is to add a bunch of hacks to the
kernel tooling to spit out a bunch of incremental tarballs to reduce
what we have to upload.
I assume this is something that will need to go through the packaging
SIG, but from an infra side of things are there any thoughts/concerns?
Regards,
Jeremy
4 years
Hello world!
by David Kirwan
Hello all,
My name is David Kirwan, I'm originally from Kilkenny, Ireland, now living
in Waterford, Ireland. I've recently transferred to the CPE team at Red
Hat, where I'll likely be working on CentOS and Fedora infrastructure in a
devops capacity. I also hope to get involved in some of the packaging work
for the Ruby/ML SIGs, although that might be in my spare time!
To share a little background about me, I spent a few years as a software
researcher, working on projects in the energy sector, security and software
defined networking. In my previous role at Red Hat, I worked as a
devops/releng engineer on an operations team responsible for maintaining a
hosted SaaS product. Most recently, I've been working as a regular software
engineer on a team developing and improving Red Hat's middleware products
to work on Openshift 3/4 using Kubernetes "Operators" (Custom resource
controllers).
When I'm not working, I try to spend as much time as possible getting
involved in the local Makerspace (South East Makerspace), working on
personal art, astronomy, amateur radio, diy, electronics and woodworking
projects. I'm delighted to be here, and I'm looking forward to working with
everyone!
--
David Kirwan
SOFTWARE ENGINEER
Red Hat Cloud Services
Communications House
Cork Road, Waterford
dkirwan(a)redhat.com T: +(353) 86-8624108 IM: @dkirwan
4 years