planet
by Kevin Fenzi
Hey folks. I thought I would open a discussion about fedoraplanet and
possibly some plans for it.
Right now:
fedoraplanet.org runs on people02.fedoraproject.org (aka fedorapeople).
To add a blog/rss feed you have to login there and edit your .planet
file, then scripting pulls all those .planet files and tries to fetch
all the feeds and then serves them up at http://fedoraplanet.org.
It uses a app called 'venus' to do this. venus is written in very old
python2 and very very dead upstream.
We run into the following problems with it:
* Sometimes it gets stuck and just stops processing until it's killed.
* It's serving on a http site, which causes people to ask us to make it
https, but that would just change the errors because many feeds it pulls
are still http since they were added back before letsencrypt existed.
* We have a handy 'website' field in our new account system, but aren't
using it at all.
* The .planet parsing is poor, any number of things can cause it to
break.
We have two open tickets on it:
- https://pagure.io/fedora-infrastructure/issue/10383 (upgrade to pluto,
a ruby based, but maintained thing)
- https://pagure.io/fedora-infrastructure/issue/10490
( planet not served via ssl) Which I am just going to close now.
So, I can think of a number of options and would love everyone who has
thoughts on it to chime in:
1. Do nothing. Venus "works" and .planet files are cool and retro.
2. Switch to pluto and use account system 'website' fields of
contributors. We could likely shove it in openshift and serve it
directly from there to avoid fedorapeople entirely.
(This would likely break anyone who has multiple feeds in there)
3. Switch to something better/bigger. I would think (although I don't
know) that there might be something that would not only aggregate rss
feeds for contributors, but perhaps mastodon/twitter/whatever also.
4. Planets are old and tired, just drop the entire thing. People can
maintain their own rss lists.
5. Planets are old and tired, just drop the entire thing.
But also, get our social media people to maintain contributor /
interesting lists. ie, the fedoraproject twitter account could maintain
a list of 'fedora contributors' and 'fedora packagers' or whatever.
6. Switch to pluto as in 2, but also setup some curators. Have a
'firehose' of all feeds, but the main fedora planet would be just
curated things that are known to be related to fedora and not off topic
or unrelated.
6. Get someones (not it!) to take in all the
twitter/facebook/mastodon/blog posts/rss feeds and post some kind of
curated round up every week or something.
7. Your brilliant idea here!
So, thoughts? this is not at all urgent, but we should end up doing
something with it sometime. :)
kevin
7 months, 2 weeks
About JS framework
by Pierre-Yves Chibon
Good Morning Everyone,
Our infrastructure is mostly a python store, meaning almost all our apps are
written in python and most using wsgi.
However in python we are using a number of framework:
* flask for most
* pyramid for some of the biggest (bodhi, FAS3)
* Django (askbot, Hyperkitty)
* TurboGears2 (fedora-packages)
* aiohttp (python3, async app: mdapi)
While this makes sometime things difficult, these are fairly standard framework
and most of our developers are able to help on all.
However, as I see us starting to look at JS for some of our apps (fedora-hubs,
wartaa...), I wonder if we could start the discussion early about the different
framework and eventually see if we can unify around one.
This would also allow those of us not familiar with any JS framework to look at
the recommended one instead of picking one up semi-randomly.
So has anyone experience with one or more JS framework? Do you have one that
would you recommend? Why?
Thanks for your inputs,
Pierre
1 year, 1 month
nest infra&releng mini barcamp
by Kevin Fenzi
I put in for a 1/2 day mini hackfest for infra and releng at nest.
See:
https://pagure.io/flock/issue/393
My thought was to collect ideas for cool things we could work on or do,
vote on them before the event and do them in most popular order.
(bar camp style).
Please do chime in here or in the flock issue if you have ideas or would
rather do something else. :)
kevin
1 year, 4 months
Where should mirroring docs live?
by Ben Cotton
Hi folks,
adamw noticed that we have two pages that claim to give information
about setting up a public mirror: a wiki page[1] and a Quick Docs
page[2] which is actually blank. This is one more than we need, so the
question is how to fix it.
We can either leave the wiki page as-is and remove the Quick Docs
page. Or we can port the wiki to Docs and set up a redirect. If we do
that, I think it's probably better suited for the infra docs than the
Quick Docs.
But I leave this up to the infra team because it's your content. I'm
happy to help make whichever answer you pick happen.
[1] https://fedoraproject.org/wiki/Infrastructure/Mirroring
[2] https://docs.fedoraproject.org/en-US/quick-docs/mirroring/
Thanks,
BC
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
1 year, 4 months
CPE Weekly Update - Week 30 2022
by Michal Konecny
Hi everyone,
This is a weekly report from the CPE (Community Platform Engineering)
Team. If you have any questions or feedback, please respond to this
report or contact us on #redhat-cpe channel on libera.chat
(https://libera.chat/).
Week: 25th July - 29th July 2022
If you wish to read this in form of a blog post, check the post on
Fedora community blog:
https://communityblog.fedoraproject.org/cpe-weekly-update-week-30-2022/
# Highlights of the week
## Infrastructure & Release Engineering
Goal of this Initiative
-----------------------
Purpose of this team is to take care of day to day business regarding
CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS
infrastructure and preparing things for the new Fedora release (mirrors,
mass branching, new namespaces etc.).
The ARC (which is a subset of the team) investigates possible
initiatives that CPE might take on.
Link to planning board: https://zlopez.fedorapeople.org/I&R-2022-07-27.pdf
Link to docs: https://docs.fedoraproject.org/en-US/infra/
Update
------
### Fedora Infra
* New Fedora Media Writer release (mostly translation fixes) 5.0.2
* Rabbitmq cluster instability last week due to both ocp3 and ocp4
clusters running monitor-gating and them obsoleting each other over and
over. ;(
* Worked around redhat.com SPF issue by accepting email from redhat.com,
expanding fedoraproject.org alias and sending it right back out through
redhat.com mx to deliver.
### CentOS Infra including CentOS CI
* Preparing duffy.ci migration ([now announced for next
week](https://lists.centos.org/pipermail/ci-users/2022-July/004582.html))
### Release Engineering
* F37 Mass rebuild finished
## CentOS Stream
Goal of this Initiative
-----------------------
This initiative is working on CentOS Stream/Emerging RHEL to make this
new distribution a reality. The goal of this initiative is to prepare
the ecosystem for the new CentOS Stream.
Updates
-------
* Imported c8s modules into the new mbs.
* CentOS Linux 7.9 ISOs are respun
## EPEL
Goal of this initiative
-----------------------
Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special
Interest Group that creates, maintains, and manages a high quality set
of additional packages for Enterprise Linux, including, but not limited
to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL),
Oracle Linux (OL).
EPEL packages are usually based on their Fedora counterparts and will
never conflict with or replace packages in the base Enterprise Linux
distributions. EPEL uses much of the same infrastructure as Fedora,
including buildsystem, bugzilla instance, updates manager, mirror
manager and more.
Updates
-------
* EPEL9 is up to 6986 (+378) packages from 3137 (+54) source packages
* Backported fix for CVE-2021-21419 to EPEL8's
[python-eventlet](https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022...
* Retired several rawhide branches of EPEL-only packages
* Updated KDE (5.24.6) is available in epel9-next and epel8-next testing.
* Will not go out to regular epel until RHEL 8.7 and 9.1
* Working on an EPEL survey that will go out in August
## FMN replacement
Goal of this initiative
-----------------------
FMN (Fedora-Messaging-Notification) is a web application allowing users
to create filters on messages sent to (currently) fedmsg and forward
these as notifications on to email or IRC.
The goal of the initiative is mainly to add fedora-messaging schemas,
create a new UI for a better user experience and create a new service to
triage incoming messages to reduce the current message delivery lag
problem. Community will profit from speedier notifications based on own
preferences (IRC, Matrix, Email), unified fedora project to one message
service and human-readable results in Datagrepper.
Also, CPE tech debt will be significantly reduced by dropping the
maintenance of fedmsg altogether.
Updates
-------
* Basic repo structure in place (Python + JS side), more boilerplate
work ongoing
* Fedora-messaging schemas checkup commenced
Kindest regards,
CPE Team
1 year, 4 months
CPE Weekly Update – Week 29 2022
by Lenka Segura
Hi everyone,
This is a weekly report from the CPE (Community Platform Engineering) Team.
If you have any questions or feedback, please respond to this report or
contact us on #redhat-cpe channel on libera.chat (https://libera.chat/).
Week: 18th - 22nd of July 2022
If you wish to read this in form of a blog post, check the post on Fedora
community blog:
https://communityblog.fedoraproject.org/cpe-weekly-update--week-29-2022/
# Highlights of the week
## Infrastructure & Release Engineering
Goal of this Initiative
-----------------------
Purpose of this team is to take care of day to day business regarding
CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS infrastructure
and preparing things for the new Fedora release (mirrors, mass branching,
new namespaces etc.).
The ARC (which is a subset of the team) investigates possible initiatives
that CPE might take on.
Link to planning board: https://zlopez.fedorapeople.org/I&R-2022-07-20.pdf
Link to docs: https://docs.fedoraproject.org/en-US/infra/
Update
------
### Fedora Infra
* New version of FMN released - removal of python-fedora requirement and
all FAS2 calls
* (almost) all builders reinstalled with f36 and bvmhosts moved to RHEL9 or
Fedora36. There’s a few arm buildhw’s to be done after the mass rebuild.
* FMN (py2) caught up finally, still needs fas2. Looking to switch to FMN
(py3) soon.
* Some recent rabbitmq / fedora-messaging cluster problems. Gave it more
memory and hopefully it’s happy again.
* Finally got *.fedoraproject.org ssl cert working with “FUTURE” encryption.
* 5 new aarch64 machines and ½ new shelf of netapp storage arriving soon.
### CentOS Infra including CentOS CI
* Mirror requests
* Duffy migration tasks
* Business as usual
### Release Engineering
* Mass rebuild starts later this morning (Wed 20th) unless any last minute
delays.
## CentOS Stream
Goal of this Initiative
-----------------------
This initiative is working on CentOS Stream/Emerging RHEL to make this new
distribution a reality. The goal of this initiative is to prepare the
ecosystem for the new CentOS Stream.
Updates
-------
* A Stream 8 compose has been pushed to correct some repoclosure issues
with the kernel
* Lots of PTO
## Package Automation (Packit Service)
Goal of this initiative
-----------------------
Automate RPM packaging of infra apps/packages.
Updates
-------
* Datagrepper is with packit
* Mirrormanager2 in progress
## Flask-oidc: oauth2client replacement
Goal of this initiative
-----------------------
Flask-oidc is a library used across the Fedora infrastructure and is the
client for ipsilon for its authentication. flask-oidc uses oauth2client.
This library is now deprecated and no longer maintained. This will need to
be replaced with authlib.
Updates:
--------
* Pull request is in https://github.com/fedora-infra/flask-oidc/pull/8
* Worked through the comments/suggestions from Aurelien
* NEED EXPERIENCED TESTERS (to help write tests, need to update existing
tests, scenario is complex, mocking out Ipsilon responses etc feel free to
join)
## EPEL
Goal of this initiative
-----------------------
Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest
Group that creates, maintains, and manages a high quality set of additional
packages for Enterprise Linux, including, but not limited to, Red Hat
Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux
(OL).
EPEL packages are usually based on their Fedora counterparts and will never
conflict with or replace packages in the base Enterprise Linux
distributions. EPEL uses much of the same infrastructure as Fedora,
including buildsystem, bugzilla instance, updates manager, mirror manager
and more.
Updates
-------
* EPEL9 is up to 6608 (+266) packages from 3083 (+127) source packages
* Sponsoring of new packagers with interest in EPEL
* Add python-tenacity to resolve ansible-lint fail to install bug
* Notable package additions:
* keepassxc
* asciinema
## FMN replacement
Goal of this initiative
-----------------------
FMN (Fedora-Messaging-Notification) is a web application allowing users to
create filters on messages sent to (currently) fedmsg and forward these as
notifications on to email or IRC.
The goal of the initiative is mainly to add fedora-messaging schemas,
create a new UI for a better user experience and create a new service to
triage incoming messages to reduce the current message delivery lag
problem. Community will profit from speedier notifications based on own
preferences (IRC, Matrix, Email), unified fedora project to one message
service and human-readable results in Datagrepper.
Also, CPE tech debt will be significantly reduced by dropping the
maintenance of fedmsg altogether.
Updates
-------
* Early days - pre-planning talks done today
* Scoping/research/fact finding from the arc investigation
https://fedora-arc.readthedocs.io/en/latest/fmn/april2022/index.html
* Possibility of using Vue instead of React on frontend, typescript vs
javascript also a possibility, FastAPI for the backend API
Kindest regards,
CPE Team
1 year, 4 months
Email replies to src.fpo notifications
by Frank R Dana Jr.
Recently, after receiving an email notification regarding a pull request at src.fedoraproject.org, I attempted[1] to update the discussion via email reply to the notification email.
That attempt was not successful. Moreover, I was only made AWARE that the attempt was unsuccessful 24 hours later, when Google's MTA responded with an undeliverable notification saying that it couldn't contact the recipient email, pagure(a)pkgs.fedoraproject.org.
If email reply to src.fpo notifications is supposed to be working, it's not. If it's INTENDED not to be working, then perhaps the reply address for notifications could be set to something that bounces immediately (donotreply(a)fedoraproject.org or similar, perhaps?), rather than an address at a non-responsive hostname that's subject to lengthy timeouts before the status of the attempted reply is updated?
[1]: https://discussion.fedoraproject.org/t/should-pagure-actually-src-fedorap...
1 year, 4 months
Software Bill of Materials for Fedora
by Jason Shepherd
Hello Fedora Infrastructure team,
Red Hat Product Security are building an application called Component
Registry to meet the requirements set out in the recent Executive Order
14028 [1], "Improving the Nation's Cybersecurity". The executive order
requires that software producers and suppliers should take steps to report
and validate a listing of all components included in or used by their
software products, aka a Software Bill of Materials. We'd like to build our
application in the open by providing the source code to the
opensource community.
Since all the Red Hat build infrastructure is internal to Red Hat, we'd
like also provide this service to Fedora so that our open source project
can have a life outside of Red Hat's corporate firewall. I suspect we are
close to being able to provide an example of the Software Bill of Materials
(SBOM) for Fedora, since it is built in a very similar way to Red Hat
Enterprise Linux. The reason for reaching out is to find out if you are
interested in hosting an SBOM for Fedora or not. We could build it inside
the Red Hat firewall, and provide a static file for each target release of
Fedora, undated periodically. Alternatively we could run the application
somewhere on your infrastructure in order to make the data available via an
API on demand. In which case we'd probably need to help to maintain that
infrastructure.
Let me know your thoughts. I didn't provide a link to the code repository
as it's currently private to Red Hat associates. But we expect to
opensource the project in the coming weeks. At that time we'll be able to
provide the source code.
Regards,
Jason Shepherd
Red Hat Product Security
[1] https://www.cisa.gov/executive-order-improving-nations-cybersecurity
1 year, 4 months
Re: Once again, more than 8 days delayed notifications
by Kevin Fenzi
crossposting to infra list to keep folks there in the loop...
On Tue, Jul 12, 2022 at 02:53:24PM +0100, Jonathan Wakely wrote:
> On Mon, 11 Jul 2022 at 15:06, Stephen Smoogen wrote:
> > That said, IRC is actually one of the fastest ones we can push to.
>
> Is https://apps.fedoraproject.org/notifications/ really still sending
> notifications to freenode though?
>
> Hasn't everybody moved to libera?
yes. It's just incorrect/out of date there.
Let me sum up what I know and perhaps I can point people to this post
later. :)
The current state is bad. We know it's bad. We have known for a long
time that it's bad. It's bad for all of the following reasons:
* It's running a python2 codebase. Upstream development/PR's have long
ago moved to python3, but thats not the version we current have
deployed.
* It's heavily tied to the old account system. Several of us spent many
hours last week trying to untangle it. I think we might be getting close
now, but it's really hard to tell.
* It's pretty heavily tied to fedmsg (not fedora-messaging). fedmsg
still works of course, but it's another layer of confusion.
* It's rules/interface lets you do all kinds of cool things, but it's
complex and confusing to most everyone that tries to use it.
* It's running on a end of life OS version. ;(
* In order to try to scale it has a number of layers of things which
makes it hard to debug. For example it uses redis, it's own rabbitmq
instance with a bunch of queues, multiple workers talking to all those
things and multiple backends for irc and email. You might think
performance shouldn't be a big deal, but when you have thousands of
users, each with their own custom rulesets, that means you have to
potentially match a incoming message against every single ruleset of
every user and you have to do it fast enough to keep up with the
incoming flow of messages and the outgoing flow of notifications. :(
Short term, I would like to hope that the python2/current version can
catch mostly up. (Its currently 4 days behind). Once it does, I would
very much like to try switching to the python3 version. It has a lot of
the problems the existing one does, but it should also have some
advantages, like running on a supported OS, much easier to release and
deploy new versions, etc.
Longer term, we are just now starting an iniative to re-write FMN from
the ground up. It's going to be a while until it's ready, but I really
hope it will be much better for everyone involved. Better/easier
interface, much better handling of messages, etc.
Hope that helps clarify things...
kevin
1 year, 4 months