infrastructure August 2020

infrastructure@lists.fedoraproject.org

30 participants
29 discussions

VDO
by Miroslav Suchý 04 Jul '26

04 Jul '26

After I seen the talk about VDO: https://www.youtube.com/watch?v=7CGr5LEAfRY I went ahead and tried it. I tried it on small (12GB) sample of Copr data and I saved 20-30% data. I then deployed it on production server retrace.fedoraproject.org and saved there 15% out of 2TB. Several notes: The RPM packages are only available for RHEL. Recently the public github has been updated with version of VDO, which should work on Fedora. No RPM available for Fedora yet though. Here are my notes: # vdo create --name=vdo1 --device=/dev/vdb --vdoLogicalSize=1T # mkfs.ext4 /dev/mapper/vdo1 Some slides suggest to use -E discard. This is not needed for new volumes. See vdo-devel mailing list for reasoning. # mount -o discard /dev/mapper/vdo1 /mnt/a Probably best scenario is to create vdo on whole disk and then make PV from whole VDO disk and put LVM on top of that. When you already have data, you cannot convert your disk to VDO. You need create new VDO disk, format it, mount it and copy the data. In my case, due limited space for migration I created a small LVM volume "vdosrv": # lvcreate -L 80G -n vdosrv vol0 I converted it to VDO which claims to have 2TB of vitual total size # vdo create --name=srv18 --device=/dev/vol0/vdosrv --vdoLogicalSize=2T # mkfs.ext4 /dev/mapper/srv18 See how many space is *actually* free. # vdostats # mkdir /mnt/srv18 # mount -o discard /dev/mapper/srv18 /mnt/srv18 Now I moved some data from old volume to /mnt/srv18. I shrank the FS of old volume, I shrank the old LV. And then I enlarged the new LV. # lvresize -L +400G /dev/vol0/vdosrv and let know vdo that it can use the new space: # vdo growPhysical -n srv18 I repeated this several times. I have to say that my experience is so far good. I hope that my notes help someone. Miroslav

10 14

About JS framework
by Pierre-Yves Chibon 17 Oct '22

17 Oct '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

10 12

Another Rust MirrorManager experiment
by Adrian Reber 03 Nov '20

03 Nov '20

Our MirrorManager setup exports the current state of all mirrors every hour at :30 to a protobuf based file which is then used by the mirrorlist servers to answer the requests from yum and dnf. The Python script requires up to 10GB of memory and takes between 35 and 50 minutes. The script does a lot of SQL queries and also some really big SQL queries joining up to 6 large MirrorManager tables. I have rewritten this Python script in Rust and now it only needs around 1 minute instead of 35 to 50 minutes and only 600MB instead of 10GB. I think the biggest difference is that I am almost not doing any joins in my SQL request. I download all the tables once and then I do a lot of loops over the downloaded tables and this seems to be massively faster. As the mirrorlist-server in Rust has proven to be extremely stable over the last months we have been using it I would also like to replace the mirrorlist protbuf input generation with my new Rust based code. I am planing to try out the new protobuf file in staging in the next days and would then try to get my new protobuf generation program into Fedora. Once it is packaged I would discuss here how and if we want to deploy in Fedora's infrastructure. Having the possibility to generate the mirrorlist input data in about a minute would significantly reduce the load on the database server and enable us to react much faster if broken protobuf data has been synced to the mirrorlist servers on the proxies. Adrian

4 17

Fedora 33 Beta Freeze now in effect
by kevin 30 Sep '20

30 Sep '20

Greetings. We are now in the infrastructure freeze leading up to the Fedora 33 Beta release. This is a pre release freeze. We do this to ensure that our infrastructure is stable and ready to release the Fedora 33 Beta when it's available. You can see a list of hosts that do not freeze by checking out the ansible repo and running the freezelist script: git clone https://infrastructure.fedoraproject.org/infra/ansible.git ansible/scripts/freezelist -i inventory Any hosts listed as freezes is frozen until 2020-09-15 (or later if release slips). Frozen hosts should have no changes made to them without a sign-off on the change from at least 2 sysadmin-main or rel-eng members, along with (in most cases) a patch of the exact change to be made to this list. Thanks, Kevin

2 1

What is our technical debt?
by Pierre-Yves Chibon 21 Sep '20

21 Sep '20

Good Morning Everyone, Just like every team we have technical debt in our work. I would like your help to try to define what it is for us. So far, I've come up with the following: - python3 support/migration - fedora-messaging - fedora-messaging schema - documentation - (unit-)tests - OpenID Connect What else would we want in there? Looking forward to your thoughts, Pierre

22 55

IPAM/DCIM for Fedora Infrastructure
by Pierre-Yves Chibon 05 Sep '20

05 Sep '20

ignatenkobrain wanted to share this with us as something potentially interesting to look at. He opened a ticket but agreed that an email thread would be better suited to discuss this, so here it is and here below is Igor's original comment: ---- Hi folks, This is not really request to do anything but rather heads-up from work I've been doing in GoodData for some time. We've been searching for something that helps us to have list of devices, their IPs and generally some kind of IP management in one place. So we started to use netbox (I think it is originating from DigitalOcean). I've finally managed to package it https://src.fedoraproject.org/rpms/netbox in Fedora. It supports LDAP and I guess can be somehow paired with Kerberos. I did not check if it is possible to pair it with OIDC, but since it is Django should be not very hard. So in case you were looking for something like this - feel free to try it out and let me know if you have any issues. ---- Original ticket: https://pagure.io/fedora-infrastructure/issue/9279 Pierre

5 9

The approach to extract all Fedora srpm?
by Jean-Baptiste Holcroft 02 Sep '20

02 Sep '20

Hello, I would like to provide translation memories for translators and measure localization progress over Fedora versions (Linux wide). I need to get upstream's content to get translation files such as po/gettext files. To do that, I need to learn how to interact with Fedora packages to reach the SRPM. The easiest way would probably be to run `dnf list --all` in a virtual machine or container for each Fedora release. An alternative could be to use datagrepper using org.fedoraproject.prod.buildsys.build.state.change? Using this method would allow me to catch new changes, but how do I load previous events or do the first initialization? Is there other ways to get all the SRPM of a Fedora release? thanks a lot for your help, I had a discussion on fedora-i18n [0], and Ben Cotton suggested me to ask my question here. I tried to document what I want to do in this wiki page page [1] [0] https://lists.fedoraproject.org/archives/list/i18n@lists.fedoraproject.org/… [1] https://fedoraproject.org/wiki/User:Jibecfed/LinuxLocalizationMeasurement

4 4

Freeze Break Request (Fix 2factor on builders)
by Stephen John Smoogen 28 Aug '20

28 Aug '20

Currently the builders are trying to contact os-node boxes on port 8443 but that is not where fas-all lives. diff --git a/roles/base/templates/iptables/iptables.kojibuilder b/roles/base/templates/iptables/iptables.kojibuilder index a3819777c..805cf735f 100644 --- a/roles/base/templates/iptables/iptables.kojibuilder +++ b/roles/base/templates/iptables/iptables.kojibuilder @@ -78,10 +78,12 @@ -A OUTPUT -p tcp -m tcp -d 10.3.163.76 --dport 443 -j ACCEPT -A OUTPUT -p tcp -m tcp -d 10.3.163.77 --dport 80 -j ACCEPT -A OUTPUT -p tcp -m tcp -d 10.3.163.77 --dport 443 -j ACCEPT -# for 2 facter auth --A OUTPUT -p tcp -m tcp -d 10.3.163.69 --dport 8443 -j ACCEPT --A OUTPUT -p tcp -m tcp -d 10.3.163.70 --dport 8443 -j ACCEPT --A OUTPUT -p tcp -m tcp -d 10.3.163.71 --dport 8443 -j ACCEPT +# for 2 facter auth (fas-all) +-A OUTPUT -p tcp -m tcp -d 10.3.163.74 --dport 8443 -j ACCEPT +-A OUTPUT -p tcp -m tcp -d 10.3.163.75 --dport 8443 -j ACCEPT +-A OUTPUT -p tcp -m tcp -d 10.3.163.76 --dport 8443 -j ACCEPT +-A OUTPUT -p tcp -m tcp -d 10.3.163.77 --dport 8443 -j ACCEPT + #nfs to vtap-fedora-nfs01.storage.phx2.redhat.com - a little to wide-open - but # kinda necessary -- Stephen J Smoogen.

4 3

Meeting Agenda Item: Introduction Manu Hernandez
by Manu Hernandez 28 Aug '20

28 Aug '20

FAS: manher IRC: miausX or manher Hi there! My name is Manu. I have been the sysadmin/tech support for a small manufacturing company (about 100 employees) for twelve years. Our IT team is rather small (two developers and me) and our infrastructure is very simple: just a few servers and networking gear. Nothing fancy. As in most companies in the area, ours relies on Microsoft products, so the core services are Windows based. We use VMware vSphere to manage our virtual infrastructure. I love FLOSS. Not only the software (the product), but the culture. It's amazing what a community can achieve. So many people giving away it's free time, effort and expertise... I try to learn about and use FLOSS solutions as much as I can (and time permits). I have being a Linux user since 2003 (with some hiatus and lots of distro-hopping), and I use Fedora on all my computers, both at home and at work. My current homelab runs CentOS as a KVM hypervisor, and I plan to run all the workloads with Fedora CoreOS and podman. Also, I try to automate as much as I can, being Ansible my tool of choice. I use Kickstart to deploy to bare metal and virtual machines. I don't have experience with programming languages (very basic scripting skills at most) but I'd love to learn Python. So, why do I want to join the Fedora infrastructure team? For two reasons: I want to give back to this amazing community and I want to learn as much as I can from the team. I'm a Jack-of-all-trades and I know I don't have much to offer to this team, but my curiosity, tenacity and willingness to learn. If I'm not a good fit for this team, don't worry! Just let me know. I'll try to help in another way to the project. Pointers and notes to improve my skill set are more than welcome. Nice to meet you all, and thanks for reading! Manu PS - English is not my native language. I can read it without problems but I'm a bit slow with the writing and have difficulties with the listening. I'll try to improve that too! -- Email domain proudly hosted at https://migadu.com

1 0

Freeze Break Request: koji update for all builders
by kevin 27 Aug '20

27 Aug '20

The koji 1.22.0 update added handling for the '--extra-boot-args' kickstart option. UNfortunately, this support wasn't working right and causes all our images specifying such options to fail. The good news is that the fix is just a one-liner: - cmd.extend(['--extra-boot-args', - '--append=\"%s\"' % b_append]) + cmd.extend(['--extra-boot-args', '\"%s\"' % b_append]) I've built a new rpm with this patch applied and would like to do a freeze break to apply it to the 'builders' ansible group. (This applies to kojid). See: https://pagure.io/koji/pull-request/2452 https://pagure.io/fedora-infrastructure/issue/9271 +1s? kevin

5 4

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure August 2020