infrastructure April 2023

infrastructure@lists.fedoraproject.org

23 participants
18 discussions

VDO
by Miroslav Suchý 04 Jul '26

04 Jul '26

After I seen the talk about VDO: https://www.youtube.com/watch?v=7CGr5LEAfRY I went ahead and tried it. I tried it on small (12GB) sample of Copr data and I saved 20-30% data. I then deployed it on production server retrace.fedoraproject.org and saved there 15% out of 2TB. Several notes: The RPM packages are only available for RHEL. Recently the public github has been updated with version of VDO, which should work on Fedora. No RPM available for Fedora yet though. Here are my notes: # vdo create --name=vdo1 --device=/dev/vdb --vdoLogicalSize=1T # mkfs.ext4 /dev/mapper/vdo1 Some slides suggest to use -E discard. This is not needed for new volumes. See vdo-devel mailing list for reasoning. # mount -o discard /dev/mapper/vdo1 /mnt/a Probably best scenario is to create vdo on whole disk and then make PV from whole VDO disk and put LVM on top of that. When you already have data, you cannot convert your disk to VDO. You need create new VDO disk, format it, mount it and copy the data. In my case, due limited space for migration I created a small LVM volume "vdosrv": # lvcreate -L 80G -n vdosrv vol0 I converted it to VDO which claims to have 2TB of vitual total size # vdo create --name=srv18 --device=/dev/vol0/vdosrv --vdoLogicalSize=2T # mkfs.ext4 /dev/mapper/srv18 See how many space is *actually* free. # vdostats # mkdir /mnt/srv18 # mount -o discard /dev/mapper/srv18 /mnt/srv18 Now I moved some data from old volume to /mnt/srv18. I shrank the FS of old volume, I shrank the old LV. And then I enlarged the new LV. # lvresize -L +400G /dev/vol0/vdosrv and let know vdo that it can use the new space: # vdo growPhysical -n srv18 I repeated this several times. I have to say that my experience is so far good. I hope that my notes help someone. Miroslav

10 14

AWS gp2 -> gp3
by Miroslav Suchý 13 Sep '23

13 Sep '23

tl;dr I want to change **all** storage types in AWS from gp2 to gp3 next week. The recent Change proposal https://fedoraproject.org/wiki/Changes/CloudEC2gp3 brought to my attention that gp3 is actually faster and cheaper than gp2. The comparision is as follows: gp2: * $0.10 per GB-month * IOPS: 100 IOPB, burstable to 3000 IOPS * Throughput: Throughput limit is between 128 MiB/s and 250 MiB/s, depending on the volume size. gp3: * IOPS: 3000 for free, addional IOPS can be bought * Throughput 125 MiB/s for free, addional throughput can be bought For more information see https://aws.amazon.com/blogs/storage/migrate-your-amazon-ebs-volumes-from-g… We have about 150 volumes of gp2 type in AWS. Copr is the most prominent user, but there are other: libravatar, taiga, TF, and other... The type of the storage can be changed on the fly - even for attached and volumes in-use. I already tried that with my private server. Unless somebody will have objections I plan to change volume types to gp3 for Copr next week and for all other services in week following Easter. I also modified fedimg template to allow gp3 https://pagure.io/fedora-infra/ansible/c/034cf953db5d0519eb327136abd72a2ee6… If you have scripts outside of fedora-infra/ansible to provision instances that uses gp2 for volumes, it will be great if you can move it to gp3. Miroslav

4 18

This list and discussion.fedoraproject.org
by Kevin Fenzi 04 May '23

04 May '23

As some of you who follow the devel list may know, Matthew put forward the idea of starting to move some discussion over to discussion.fedoraproject.org (our hosted dicourse instance) instead of the mailing list. For those of you who don't know: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… (and the long thread after it). Please do read that. I'll wait. ok. So, my question is: Should we look at moving to discussion.fedoraproject.org instead of this list? We have a tag there already. I've personally setup discourse to email me things, so it actually ends up being very similar to a list, with a few corner cases. I'm happy to help anyone who has trouble with that setup. Thoughts? kevin

7 7

clean up on aisle pagure.io
by Kevin Fenzi 26 Apr '23

26 Apr '23

Hey folks. So, a few weeks back I noticed some spam projects on pagure.io. So, I cleaned up about 165 of them and deactivated 165 spam users. Now, I see there's another pile of them. :( So, I started to look at cleaning things up again, but I think we need a better solution that doesn't involve admins manually cleaning things up. ;( Additionally, the clean up is not really very scripted and takes a long time to do. So, thoughts on a longer term solution? I can think of a few: 1. only allow fedora 'contributors' to make new projects. (ie, people in at least one non cla/non base ipa group Pros: - Would very likely cut off the spam or at least cut it way down. - Might be easy to implement? (not sure tho!) Cons: - Would block legit people who aren't fedora contributors. 2 Some kind of moderation for new projects Pros: - Would let non fedora folks make new projects. - Would likely cut spam Cons: - Would require someone to moderate things - Would requite us to make some kind of moderation code 3. A script to do all the cleanup so we could do it easier and some kind of 'bad words' blocklist we could put in place to stop obvious spammers (most of these are bogus "exam answers" ones) Pros: - Will cut down on spam some, but not fully. Cons: - Will have to write the script and implement the blocklist - It's likely spammers will use different words over time and avoid the block. Or perhaps someone has further clever ideas? Happy to hear em. ;) kevin

8 11

awscli2 and Fedora sso
by Miroslav Suchý 26 Apr '23

26 Apr '23

Has anyone use awscli2? How did you configure it to use Fedora SSO? I played with it, but with not luck. BTW awcli2 is now in package review process and is available at https://copr.fedorainfracloud.org/coprs/g/fedora-review/fedora-review-21894… Miroslav

2 1

AWS volume cleanup
by Miroslav Suchý 26 Apr '23

26 Apr '23

When I have been working in the AWS console I noticed we have lots of volumes that are: not described, not owned and not attached. Sometimes even combination of all of these. And some of such volumes have been created in 2020. Here is screenshot of suspicious volumes: https://k00.fr/1cn48fze This link to console with Name = Not tagged https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Volumes:… Please check your volumes and add these tags: * Owner: * FedoraGroup * Name In a week I will write longer and detailed mail and everything that is not properly tagged (in a reasonable timeframe) will be proposed for deletion. Miroslav

2 3

Orphaning python-simplemediawiki
by Mattia Verga 25 Apr '23

25 Apr '23

I'm going to orphan python-simplemediawiki. Upstream is long time dead and the github repository was archived in 2018. It was used by Bodhi backend to fetch TestCases information from Fedora Wiki, but Bodhi 7.1 switched to python-pymediawiki. I cannot find any other use in fedora infrastructure. infra-sig is listed as co-maintainer, let me know if you want me to remove the group before orphaning the package. Mattia

2 1

Re: planet
by Pedro Moura 25 Apr '23

25 Apr '23

> On 2023-04-18 15:07, Pedro Moura wrote: > > Random thought: does the http server have any metrics or logs that could > be counted to give an indication of how many requests there are for each > feed (including the main one) over a period of time? For each feed, I don't think so About the survey results: 100% agreed to drop subplanets [image: Screenshot from 2023-04-25 14-46-49.png] I made a PR on noggin creating a new rss field https://github.com/fedora-infra/noggin/pull/1216 This is a work in progress, because Im having this error "Unknown option: fasrssurl" So If any of you knows how to fix it, the help is appreciated regards, Pedro Moura he/him/his Software Engineer Red Hat <https://www.redhat.com> pmoura(a)redhat.com <https://www.redhat.com> On Wed, Apr 19, 2023 at 5:59 PM Gerard Ryan <gerard(a)ryan.lt> wrote: > On 2023-04-18 15:07, Pedro Moura wrote: > > Thanks for the suggestions :) > > > > Just sent to announce mailing list. > > We could get planet users and send to them directly, but them would > > have an opinion only of those who are posting their blogs there and > > sending in announce we might have an opinion of those who are using > > planet also to read? > > Random thought: does the http server have any metrics or logs that could > be counted to give an indication of how many requests there are for each > feed (including the main one) over a period of time? > >

1 0

FMN replacement deployment
by Aurelien Bompard 21 Apr '23

21 Apr '23

Hey folks! I have a few questions about the final deployment of the FMN replacement: - There's been a request to handle a user being disabled in IPA, which should trigger their rules being disabled (FMN#826). We can do that but we have questions about re-enablement: should the rules be auto-enabled when the user is re-enabled? What about rules that the user may have disabled previously? Should we just leave things disabled? - How do you see the transition to the new system? We were thinking: - move the current FMN to a different URL, such as notifications-old.fp.o. It will still be processing messages and sending notifications - run the new system in notifications.fp.o (in place of the old) - add a small banner to the new system to point people to the old in case they want to change their rules there Is it too quick? Should we deploy the new one to a notifications-new URL first? That's one more step to get to the final setup, so more work for you. You get to decide :-) → Michal & Kevin are fine with this plan. - IRC account: we can't connect to libera.chat with the same account twice, we'll need a second account. I can create it, but do you prefer that we: 1. run the old FMN on the new account (fedora-notifs-old) and the new FMN on the usual account 2. run the old FMN on the usual account and the new FMN on the new account (fedora-notifs-new) and then switch back to the usual one when we retire the old FMN 3. run the old FMN on the usual account and the new FMN on a new account (fedora-fmn) and just drop the old account when we retire the old FMN (I guess that's connected to the URL transition issue) → Kevin prefers option 3. - Let's do an "ask us anything" session for infra people. We can do that on IRC or you can ask all your questions in this email thread. - When should we do the switch? → Kevin suggests next week, with an email to devel-announce. What do you think? I've had a few answers from Michal & Kevin already and added them inline. Kevin also notes that we may need a RHIT ticket to allow IRC out from OpenShift. I'll test it as soon as I've created the new IRC account. He also suggests a sunset date for the old FMN after the F39 release. Thanks! Aurélien

4 6

Failure to download metadata for “CentOS-PowerTools” repo
by Laura Flores 20 Apr '23

20 Apr '23

Hello, Around two weeks ago, I started seeing this error after running “sudo yum install -y kernel”. It seems to be an issue with the CentOS-PowerTools repo. Is this a known issue, or does anyone have any suggestions? This is happening on Centos Stream 8 and RHEL 8.6. Multiple mirrors are tried with no success. ``` Curl error (7): Couldn't connect to server for https://download-cc-rdu01.fedoraproject.org/pub/centos/8-stream/PowerTools/… [Failed to connect to download-cc-rdu01.fedoraproject.org port 443: Connection timed out] 2023-04-20T13:06:39.243 ``` Let me know if any additional information is needed. Thanks, Laura Flores

4 4

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure April 2023