planet
by Kevin Fenzi
Hey folks. I thought I would open a discussion about fedoraplanet and
possibly some plans for it.
Right now:
fedoraplanet.org runs on people02.fedoraproject.org (aka fedorapeople).
To add a blog/rss feed you have to login there and edit your .planet
file, then scripting pulls all those .planet files and tries to fetch
all the feeds and then serves them up at http://fedoraplanet.org.
It uses a app called 'venus' to do this. venus is written in very old
python2 and very very dead upstream.
We run into the following problems with it:
* Sometimes it gets stuck and just stops processing until it's killed.
* It's serving on a http site, which causes people to ask us to make it
https, but that would just change the errors because many feeds it pulls
are still http since they were added back before letsencrypt existed.
* We have a handy 'website' field in our new account system, but aren't
using it at all.
* The .planet parsing is poor, any number of things can cause it to
break.
We have two open tickets on it:
- https://pagure.io/fedora-infrastructure/issue/10383 (upgrade to pluto,
a ruby based, but maintained thing)
- https://pagure.io/fedora-infrastructure/issue/10490
( planet not served via ssl) Which I am just going to close now.
So, I can think of a number of options and would love everyone who has
thoughts on it to chime in:
1. Do nothing. Venus "works" and .planet files are cool and retro.
2. Switch to pluto and use account system 'website' fields of
contributors. We could likely shove it in openshift and serve it
directly from there to avoid fedorapeople entirely.
(This would likely break anyone who has multiple feeds in there)
3. Switch to something better/bigger. I would think (although I don't
know) that there might be something that would not only aggregate rss
feeds for contributors, but perhaps mastodon/twitter/whatever also.
4. Planets are old and tired, just drop the entire thing. People can
maintain their own rss lists.
5. Planets are old and tired, just drop the entire thing.
But also, get our social media people to maintain contributor /
interesting lists. ie, the fedoraproject twitter account could maintain
a list of 'fedora contributors' and 'fedora packagers' or whatever.
6. Switch to pluto as in 2, but also setup some curators. Have a
'firehose' of all feeds, but the main fedora planet would be just
curated things that are known to be related to fedora and not off topic
or unrelated.
6. Get someones (not it!) to take in all the
twitter/facebook/mastodon/blog posts/rss feeds and post some kind of
curated round up every week or something.
7. Your brilliant idea here!
So, thoughts? this is not at all urgent, but we should end up doing
something with it sometime. :)
kevin
1 month, 2 weeks
About JS framework
by Pierre-Yves Chibon
Good Morning Everyone,
Our infrastructure is mostly a python store, meaning almost all our apps are
written in python and most using wsgi.
However in python we are using a number of framework:
* flask for most
* pyramid for some of the biggest (bodhi, FAS3)
* Django (askbot, Hyperkitty)
* TurboGears2 (fedora-packages)
* aiohttp (python3, async app: mdapi)
While this makes sometime things difficult, these are fairly standard framework
and most of our developers are able to help on all.
However, as I see us starting to look at JS for some of our apps (fedora-hubs,
wartaa...), I wonder if we could start the discussion early about the different
framework and eventually see if we can unify around one.
This would also allow those of us not familiar with any JS framework to look at
the recommended one instead of picking one up semi-randomly.
So has anyone experience with one or more JS framework? Do you have one that
would you recommend? Why?
Thanks for your inputs,
Pierre
7 months, 3 weeks
openshift application container versions
by Kevin Fenzi
I whipped up a brute force little script and ran it on our staging
openshift 4 cluster. It just gets all running pods and does oc rsh into
each and looks at /etc/os-release. For a few of these that aren't
running yet in stg, I looked at prod (like toddlers).
Here's the results and some comments in between.
I can work on upgrading some of these, but would be nice if application
owners update their own applications. ;)
asknot
"platform:f34"
blockerbugs
"platform:el8"
bodhi
"platform:f36"
"platform:f36"
"platform:f36"
"platform:f36"
bugzilla2fedmsg
"platform:el8"
community-metrics
compose-tracker
"platform:f36"
coreos-cincinnati
"platform:f35"
datagrepper
"platform:el8"
datanommer
"platform:el8"
discourse2fedmsg
"platform:el8"
docsbuilding
"platform:f34"
^ This should get updated.
docstranslation
^ No running pods, but cronjobs use f34. Should be updated
elections
"platform:el8"
fasjson
"platform:f35"
fedocal
"platform:el8"
fedora-packages-static
^ debian 11 in the solr pod
^ fedora 34 in the static pod, should get updated
flask-oidc-dev
"platform:el8"
flatpak-indexer
"platform:el8"
"platform:el8"
"platform:f36"
greenwave
"platform:f33"
"platform:f33"
"platform:f33"
^ Needs update.
ipsilon-website
^ Alpine Linux v3.16 (is this current? no idea)
koschei
"platform:f35"
"platform:f35"
"platform:f35"
"platform:f35"
"platform:f35"
"platform:f35"
languages
"platform:el8"
mdapi
"platform:f31"
"platform:f31"
^ Definitely needs update.
message-tagging-service
"platform:f29"
^ Definitely needs update.
messaging-bridges
"platform:el8"
"platform:el8"
"platform:el8"
monitor-gating
"platform:f31"
^ Definitely needs update.
mote
"platform:f36"
"platform:f36"
"platform:el9"
noggin
"platform:el8"
noggin-centos
"platform:el8"
oraculum
"platform:el8"
"platform:el8"
"platform:el8"
"platform:el8"
"platform:el8"
release-monitoring
"platform:f36"
"platform:f36"
"platform:f36"
resultsdb
"platform:f35"
"platform:f35"
resultsdb-ci-listener
"platform:f36"
review-stats
^ just cronjobs, fedora:36
silverblue
PRETTY_NAME="Fedora 27 (Twenty Seven)"
^ very much needs updating.
testdays
"platform:el8"
"platform:el8"
the-new-hotness
"platform:f36"
toddlers
PRETTY_NAME="Alpine Linux v3.16"
^ is this current?
transtats
"platform:f34"
"platform:f34"
^ needs updating.
waiverdb
"platform:f34"
"platform:f34"
^ needs updating.
websites
^ cron jobs, fedora:34. Needs updating.
zezere
"platform:el8"
11 months, 1 week
openshift 3->4 moves status and info needed
by Kevin Fenzi
Greetings everyone.
I finally had a quiet afternoon to really dig into the ocp3->4
migration.
We have a total of 64 unique applications/projects over the 4 clusters
(ocp3 prod/stg and ocp4 prod/stg).
Of those 20 have been moved to ocp4 (either stg or prod or both).
There's 4 that are sort of in the midst of moving to ocp4 prod, but need
some more tweaking (flatpak-indexer, release-monitoring,
discourse2fedmsg, transtats).
I'd like to propose removing the following 9 entirely:
REMOVE: accountsystem.yml
( an old proof of concept of noggin/caapi/etc)
REMOVE: distgit_bugzilla_sync.yml
( we do this in a toddler anymore)
REMOVE: fas-changes.yml
( I think this was just needed for a short time for the account system
migration, please correct me if it's got some better use)
REMOVE: fas-stg.yml
( Old account system staging, we can use the fas.yml playbook if we
have to keep fas)
REMOVE: ipsilon.yml
( we moved this to vm's because we couldn't run pam_sssd in openshift.
Has that changed?)
REMOVE: kerneltest.yml
( This never went into service and doesn't do exactly what kernel
maintainers want )
REMOVE: sanitarium.yml
( some test project? )
REMOVE: simple-koji-ci.yml
( This was retired a long time ago)
REMOVE: sso.yml
( another test?)
These 3 are broken in staging and need to be fixed before we can move
them (I really like to see things run in stg before we move them to
prod):
BROKEN: compose-tracker.yml
BROKEN: monitor_gating.yml
BROKEN: transtats.yml
I'll try and ping owners of these to look and fix them soon.
That leaves:
datagrepper.yml
datanommer.yml
docsbuilding.yml
docstranslation.yml
elections.yml (will wait for current elections to finish)
fas.yml
fedocal.yml
fedora-packages-static.yml
greenwave.yml
koschei.yml
languages.yml
mdapi.yml
messaging-bridges.yml
nuancier.yml (I think this was a staging only proof of concept)
oraculum.yml
review-stats.yml
testdays.yml
toddlers.yml
waiverdb.yml
websites.yml
zezere.yml
I'll keep working on the list this week and hopefully we can get them
all moved soon. :)
kevin
11 months, 2 weeks
CPE Weekly Update – Week 25 2022
by Akashdeep Dhar
Hi everyone,
This is a weekly report from the CPE (Community Platform Engineering) Team.
If you have any questions or feedback, please respond to this report or
contact us on #redhat-cpe channel on libera.chat (https://libera.chat/).
Week: 20th - 24th June 2022
If you wish to read this in form of a blog post, check the post on the
Fedora community blog:
https://communityblog.fedoraproject.org/cpe-weekly-update-week-25-2022/
Highlights of the week Infrastructure & Release Engineering Goal of this
Initiative
The purpose of this team is to take care of day-to-day business regarding
CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS infrastructure
and preparing things for the new Fedora release (mirrors, mass branching,
new namespaces etc.).
The ARC (which is a subset of the team) investigates possible initiatives
that CPE might take on.
Link to planning board: https://zlopez.fedorapeople.org/I&R-2022-06-22.pdf
Link to docs: https://docs.fedoraproject.org/en-US/infra/
Update Fedora Infra
- Most apps have moved over to the OpenShift4 cluster. Hopefully, the
transition should be finishing up this week.
- Wiki: All upgraded in production and working (thanks Ryan!)
- Resultsdb: All moved over to OpenShift 4 in prod and working (thanks
Leo!)
- Business proceeding as usual
CentOS Infra including CentOS CI
- Kerberos settings switch for git.centos.org (kcm on el8 vs keyring on
el7) for lookaside upload CGI <https://pagure.io/centos-infra/issue/811>
[1]
- Issue on iad2 hosted reference mirror
<https://pagure.io/centos-infra/issue/812>[2] (epel.next and
mirrormanager), all fixed now
- Duffy CI ongoing tasks and deployments (all announced)
- Equinix nodes migration <https://pagure.io/centos-infra/issue/816>[3]
(on their request)
- Business proceeding as usual
Release Engineering
- Compose-tracker updated to f36 in staging, production happening
tomorrow
- Python 3.11 merged to rawhide
- MBS randomly fails to process builds
- Rawhide compose failures recently (syslinux retirement, then python
3.11 merge) all fixed now
- Business proceeding as usual
CentOS Stream Goal of this Initiative
This initiative is working on CentOS Stream/Emerging RHEL to make this new
distribution a reality. The goal of this initiative is to prepare the
ecosystem for the new CentOS Stream.
Updates
- CentOS Stream 8: Manually keeping regular RPMs and module RPMs updated
on the koji.stream server as current updates are composed and released.
CentOS Duffy CI Goal of this Initiative
Duffy is a system within CentOS CI infrastructure allowing tenants to
provision and access machines (physical and/or virtual, of different
architectures and configurations) for the purposes of CI testing.
Development of Duffy is largely finished, we're currently planning and
testing deployment scenarios.
Updates
- Release version 3.2.1
- Docs, docs, docs and a Dojo
Package Automation (Packit Service) Goal of this initiative
Automate RPM packaging of infra apps/packages
Updates
- Mostly business as usual
- Thanks again to all who are reviewing our PRs
- Most of our GitHub critical apps are enabled now or close to being
enabled
Flask-oidc: oauth2client replacement Goal of this initiative
Flask-oidc is a library used across the Fedora infrastructure and is the
client for ipsilon for its authentication. flask-oidc uses oauth2client.
This library is now deprecated and no longer maintained. This will need to
be replaced with authlib.
Updates:
- POC working using authlib, tidying up code to prepare to submit a PR
back to upstream
EPEL Goal of this initiative
Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest
Group that creates, maintains, and manages a high-quality set of additional
packages for Enterprise Linux, including, but not limited to, Red Hat
Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux
(OL).
EPEL packages are usually based on their Fedora counterparts and will never
conflict with or replace packages in the base Enterprise Linux
distributions. EPEL uses much of the same infrastructure as Fedora,
including a build system, Bugzilla instance, updates manager, mirror
manager and more.
Updates
- This week we have 6442 (+127) packages, from 2882 (+76) source
packages
- Containerd and puppet retired from EPEL7 because of upstream EOL and
multiple CVEs.
- Caddy was updated, fixing 4 CVEs in EPEL9
*Index*
[1] https://pagure.io/centos-infra/issue/811
[2] https://pagure.io/centos-infra/issue/812
[3] https://pagure.io/centos-infra/issue/816
Thanks and Regards,
Akashdeep Dhar (he/him),
Objective Representative, Fedora Council
Red Hat Community Platform Engineering
t0xic0der(a)fedoraproject.org
akashdeep(a)redhat.com
11 months, 2 weeks
openshift applications best practices
by Kevin Fenzi
So, when moving openshift apps this week, I really came to the
conclusion we should look at writing up some guidelines for openshift
apps. Perhaps it could have 'MUST' items and 'SHOULD' items and
'SUGGESTION' items.
I know I am coming at this from the operations side of things, so please
do share other viewpoints, but let me mention a few issues I see:
It's very difficult to tell what an application is running on:
- We should probibly forbid 'X:latest' because what the application is
running depends on when it was last built/deployed. (unless: see
below)
- A number of apps use images elsewhere (quay.io, etc). We have very
little way to tell whats in those or how they are made or if anyone is
still making them.
We have a few apps that rebuild/deploy on git changes, which is fine,
but doesn't do anything to keep their container env up to date/secure.
We have a number of apps that tigger on imagestream changes, but as far
as I can tell thats only changes in their image ('bodhi-base') not the
base image that it was built from ('fedora:36'). I'd like to see us
track base images and build/deploy when one changes. If we do that then
:latest might be ok to use, since we know automation took care of
updating it for us.
There's a number of applications using ubi (rhel's base image), which I
think might be worth recommending, especially if we start build/deploy
on base image changes. Likely Fedora base images will change more
quickly.
Currently we have some apps that were setup and probibly have not
changes in years, happily running the same old container image since
there was no reason to redeploy them.
Similarly, python versions are all over the place... python3.6, 3.8,
3.9, etc. Of course different apps will target different versions
sometimes, but it might be nice to make sure we are using a supported
version by whatever container image the app is using. (ie, gets security
fixes).
Our naming conventions are a bit all over the place too. Some apps have
$appname-build for buildconfig, some just $appname, some seperate
configs for seperate containers ($app-web, $app-frontend). Same for
deploymentconfigs.
Anyhow, I'll stop there... but hopefully everyone would be ok with such
a doc and then changing things to match it?
kevin
11 months, 3 weeks
[RFC] Build tag in RPM: from NVR to NVRB
by Aleksandra Fedorova
Hi, all,
I'd like to discuss how we can add Build tag in the RPM.
As one of the key points is to turn it into a common standard for rpm
packages across the ecosystem, the conversation is currently opened
upstream [1] and in RHEL Engineering. And I'd like to get Fedora
community on board.
This is not a finalized proposal and I think it is not ready yet to be
a Fedora Change.
But I want to start a conversation and ask for opinions. There are
also some open questions which need help, especially the requirements
around build reason. And alternative suggestions are welcome as well.
I've posted long version at
https://discussion.fedoraproject.org/t/rfc-build-tag-in-rpms/39954
You can comment there (simply login with your FAS id) or here on the
mailing list.
And I am going to update that post with the new feedback.
Shorter version:
================================================
We'd like to introduce Build Number/Tag/Id in the rpm metadata.
By this change we would like to:
* Provide a possibility to change build environment and rebuild rpm
packages without changing their content: neither sources nor spec
files.
* Set a common standard for the RPM-based ecosystem, which can be used
not just within Fedora, but also by Remixes, downstreams, SIGs and
other distributions.
The most visible impact of the proposal would be the filename change:
Current: dnf-4.9.0-12.fc36.noarch.rpm
Proposed: dnf-4.9.0-12.fc36.34897715.noarch.rpm
It can be implemented in two steps:
1) Now → Compatibility mode
* Introduce Build tag in the rpm metadata
* Introduce “build reason” to be added to rpm changelog as a top entry
* Enable passing Build tag value to the build in Koji. The value of
the Build tag will be set from Koji build id.
* Introduce macro in Release field of the rpm spec files, which adds
Build tag after the usual disttag
Release: 12.%{?dist}%{?build:.%{build}}
* Introduce option to pass “build reason” to a Koji build via koji cli
and fedpkg/centpkg tooling.
2) Compatibility mode → Final
* Implement support for the upgrade path on the rpm side in a
compatible way. So that NV(R’=R+B) and NVRB are treated the same.
* Remove %{build} part from Release tag and use independent Build tag
for versioning.
================================================
[1] https://github.com/rpm-software-management/rpm/discussions/2031
--
Aleksandra Fedorova
CI for Fedora/CentOS Stream/RHEL
bookwar on Matrix/IRC
11 months, 3 weeks
CPE Weekly Update – Week 24 2022
by Michal Konecny
Hi everyone,
This is a weekly report from the CPE (Community Platform Engineering)
Team. If you have any questions or feedback, please respond to this
report or contact us on #redhat-cpe channel on libera.chat
(https://libera.chat/).
Week: 13th - 17th June 2022
If you wish to read this in form of a blog post, check the post on
Fedora community blog:
https://communityblog.fedoraproject.org/cpe-weekly-update--week-24-2022/
# Highlights of the week
## Infrastructure & Release Engineering
Goal of this Initiative
-----------------------
Purpose of this team is to take care of day to day business regarding
CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS
infrastructure and preparing things for the new Fedora release (mirrors,
mass branching, new namespaces etc.).
The ARC (which is a subset of the team) investigates possible
initiatives that CPE might take on.
Link to planning board: https://zlopez.fedorapeople.org/I&R-2022-06-15.pdf
Link to docs: https://docs.fedoraproject.org/en-US/infra/
Update
------
### Fedora Infra
* Resultsdb almost moved to ocp4 in prod, just a few parts to finish
(Thanks Leo!)
* Ocp4 cluster now on our vpn, so all proxies can reach apps (thanks
darknao!)
* Wiki upgrade looking good in staging, prod to come (thanks ryan!)
* Some more vm’s to f36
* About 50% done moving apps to ocp4.
* Image builder prod move blocked due to firewall issues
### CentOS Infra including CentOS CI
* https://git.centos.org went live last monday
* Trying to resume discussion with RH IT for Stream storage migration
* New CI infra deployment tasks to be ready
* https://lists.centos.org/pipermail/ci-users/2022-June/004547.html
### Release Engineering
* ELN composes were broken over the weekend because of ODCS backend /
front end version mismatch
* Nodejs-sig removed as the default assignee on a bunch of components in BZ
* we have discovered workflow in bodhi that locks update in a weird
state more info https://github.com/fedora-infra/bodhi/issues/4566
## CentOS Stream
Goal of this Initiative
-----------------------
This initiative is working on CentOS Stream/Emerging RHEL to make this
new distribution a reality. The goal of this initiative is to prepare
the ecosystem for the new CentOS Stream.
Updates
-------
* We imported all RPMs for modules (CentOS Stream 8) to the shared
buildsystem
* All sources imported to GitLab (CentOS Stream 8)
## CentOS Duffy CI
Goal of this Initiative
-----------------------
Duffy is a system within CentOS CI infrastructure allowing tenants to
provision and access machines (physical and/or virtual, of different
architectures and configurations) for the purposes of CI testing.
Development of Duffy is largely finished, we're currently planning and
testing deployment scenarios.
Updates
-------
* Test and polish `duffy client …` experience
* Docs and CentOS Dojo talk prep
## Package Automation (Packit Service)
Goal of this initiative
-----------------------
Automate RPM packaging of infra apps/packages
Updates
-------
* Almost finished, only mirrormanager2 remaining from our critical apps
on Github
* Couple of outliers (fasjson, flask-mod-auth) need downstream repos created
* Datanommer.models manually packaged so datagrepper can be automated
* Noggin now fully automated
## Flask-oidc: oauth2client replacement
Goal of this initiative
-----------------------
Flask-oidc is a library used across the Fedora infrastructure and is the
client for ipsilon for its authentication. flask-oidc uses oauth2client.
This library is now deprecated and no longer maintained. This will need
to be replaced with authlib.
Updates:
--------
* Working [poc
app](https://app-flask-oidc-dev.apps.ocp.stg.fedoraproject.org/oidc/)
which authenticates against noggin/ipa using authlib and OIDC.
* Working on an upstream PR with the working code now.
## EPEL
Goal of this initiative
-----------------------
Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special
Interest Group that creates, maintains, and manages a high quality set
of additional packages for Enterprise Linux, including, but not limited
to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL),
Oracle Linux (OL).
EPEL packages are usually based on their Fedora counterparts and will
never conflict with or replace packages in the base Enterprise Linux
distributions. EPEL uses much of the same infrastructure as Fedora,
including buildsystem, bugzilla instance, updates manager, mirror
manager and more.
Updates
-------
* This week we have 6315 (+297) packages, from 2806 (+76) source packages
* [XFCE now available in
epel9-testing](https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-5...
* Removed epel9 packages that were included in rhel9.0
* oniguruma: [backported
fix](https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-a9236c0113)
for CVE-2019-13225 (moderate) from rhel8 to epel7
* drbd: pull request to [switch pacemaker dependency from requires to
recommends](https://src.fedoraproject.org/rpms/drbd/pull-request/2) to
comply with policy
Kindest regards,
CPE Team
11 months, 3 weeks