infrastructure January 2021

infrastructure@lists.fedoraproject.org

23 participants
20 discussions

VDO
by Miroslav Suchý 04 Jul '26

04 Jul '26

After I seen the talk about VDO: https://www.youtube.com/watch?v=7CGr5LEAfRY I went ahead and tried it. I tried it on small (12GB) sample of Copr data and I saved 20-30% data. I then deployed it on production server retrace.fedoraproject.org and saved there 15% out of 2TB. Several notes: The RPM packages are only available for RHEL. Recently the public github has been updated with version of VDO, which should work on Fedora. No RPM available for Fedora yet though. Here are my notes: # vdo create --name=vdo1 --device=/dev/vdb --vdoLogicalSize=1T # mkfs.ext4 /dev/mapper/vdo1 Some slides suggest to use -E discard. This is not needed for new volumes. See vdo-devel mailing list for reasoning. # mount -o discard /dev/mapper/vdo1 /mnt/a Probably best scenario is to create vdo on whole disk and then make PV from whole VDO disk and put LVM on top of that. When you already have data, you cannot convert your disk to VDO. You need create new VDO disk, format it, mount it and copy the data. In my case, due limited space for migration I created a small LVM volume "vdosrv": # lvcreate -L 80G -n vdosrv vol0 I converted it to VDO which claims to have 2TB of vitual total size # vdo create --name=srv18 --device=/dev/vol0/vdosrv --vdoLogicalSize=2T # mkfs.ext4 /dev/mapper/srv18 See how many space is *actually* free. # vdostats # mkdir /mnt/srv18 # mount -o discard /dev/mapper/srv18 /mnt/srv18 Now I moved some data from old volume to /mnt/srv18. I shrank the FS of old volume, I shrank the old LV. And then I enlarged the new LV. # lvresize -L +400G /dev/vol0/vdosrv and let know vdo that it can use the new space: # vdo growPhysical -n srv18 I repeated this several times. I have to say that my experience is so far good. I hope that my notes help someone. Miroslav

10 14

About JS framework
by Pierre-Yves Chibon 17 Oct '22

17 Oct '22

Good Morning Everyone, Our infrastructure is mostly a python store, meaning almost all our apps are written in python and most using wsgi. However in python we are using a number of framework: * flask for most * pyramid for some of the biggest (bodhi, FAS3) * Django (askbot, Hyperkitty) * TurboGears2 (fedora-packages) * aiohttp (python3, async app: mdapi) While this makes sometime things difficult, these are fairly standard framework and most of our developers are able to help on all. However, as I see us starting to look at JS for some of our apps (fedora-hubs, wartaa...), I wonder if we could start the discussion early about the different framework and eventually see if we can unify around one. This would also allow those of us not familiar with any JS framework to look at the recommended one instead of picking one up semi-randomly. So has anyone experience with one or more JS framework? Do you have one that would you recommend? Why? Thanks for your inputs, Pierre

10 12

Introducing the ARC sub-team in CPE - and first research topic
by Pierre-Yves Chibon 26 Feb '21

26 Feb '21

Good Morning Everyone, While planning work, the CPE team has realized that a number of our initiatives actually start with a research phase to find the most appropriate technical solution. This leads to some issues with planning as without knowing the technical solution we want to take, it's hard to evaluate the amount of work needed and thus the time it'll take to do it. In order to help with this, we're creating a small sub-team in CPE, called the ARC team for Advance Reconaissance Crew*. The goal of this team will be to investigate what we believe to be the possible technical solutions for initiatives and advise the team on what they believe would be the appropriate solution. To this end, we will reach out when we start looking for ideas as you may have ideas that we did not think about. The first investigation, led by Will Woods, Mark O'Brien and I, will be around datanommer and datagrepper. datanommer is an application listening to fedmsg and filling a (postgresql) database with all the messages passing on the bus. datagrepper is a web application exposing these messages and offering a way to filter or search them. available at: https://apps.fedoraproject.org/datagrepper/ Currently our ideas are: - for datanommer: - port it to fedora-messaging - adjust it to whichever solution we chose to replace datagrepper - for datagrepper: - keep it as is - Replace by - postgres https://postgrest.org/ - prest https://github.com/prest/prest - kinto https://docs.kinto-storage.org/en/stable/ - Swagger/OpenAPI https://swagger.io/ - Add support for Graphql - for the postgresql server - Split messages per year in different table - Unite them using a postgresql view - Kick out the old messages per year - Keep the current year + n-1 in the current DB - Kick the other to another DB? - Kick the other to a tarball somewhere? - Output the database daily dump to file / year - TimescaleDB a postgresql plugin for time-series data - https://alibaba-cloud.medium.com/postgresql-time-series-database-plug-in-ti… - https://dev.t-matix.com/blog/postgresql-as-a-time-series-database/ - https://docs.timescale.com/latest/introduction - Make the msg field in the message table be a JSON field Would you have any other ideas of things we could look at? Looking forward for your input, Thanks, Pierre, Will and Mark * Our notes and documentation are hosted at: https://fedora-arc.readthedocs.io/en/latest/index.html

3 6

Re: Fedora packages site down
by Anatoli Babenia 02 Feb '21

02 Feb '21

Hi. I've deployed the static website to GitHub Pages at https://abitrolly.github.io/fedora-packages-static/ The search doesn't work, and there is no index to browse, so it is not very useful. Looks like the search is supposed to be Solr, which is not static.

2 3

Anitya (release-monitoring.org) 1.0.0 available
by Michal Konecny 01 Feb '21

01 Feb '21

Hi everyone, today I deployed a new version of Anitya on production [0]. I decided that Anitya is mature enough to have version 1.0.0. So here it is. And what this versions brings? Plenty of changes, here is the list of the most interesting ones: * Add preview mode Now you can try your changes before submitting them, on the edit and add project page is a new button "Test check" which will take the fields from the form and do a check for releases above them. Nothing is changed in the database during test check. * Flag pre-release versions Yes, you are reading it right. Anitya is now flagging versions that are considered unstable, it uses the version scheme recognition and above that you can add your own filter when editing project. * Message schema 2.0.0 The Anitya message schema now contains a new topic "anitya.project.version.update.v2". This topic will send message that has "upstream_versions" field which contains all the newly found versions, not only the latest one. And it also contains a "stable_versions" field, so you can look if some of the newly versions is stable or not. With this version "anitya.project.version.update" is now deprecated! * Add version filter for project Anitya now allows user to add their own version filter, if you see any bogus version, you can just edit project and add the string to filter (This will not delete any version that was already retrieved, but you can flag a project and ask admin to do it for you and it will never be retrieved again). * Project archiving Anitya 1.0.0 allows admins to archive projects if it seems reasonable (project dead upstream) for the sake of history. Archived projects can't be edited and are not checked for new versions, but still could be found in Anitya. * Projects menu is rewritten The projects menu now contains items that are more sensible to current state of Anitya, you can see projects that were successfully updated (sorted by the time of update from newest), failed to update (sorted by the number of failed attempts from highest number), never updated (incorrectly set up projects, where update was never successful, sorted by the date of creation from oldest) and archived projects. * Updated documentation The documentation was fully rewritten to reflect the current state of Anitya. User guide was added containing use cases that could be done by user. User admin guide was added for users in Anitya with Admin rights. And the Admin guide and Contribution guide was verified that these steps are working with current version of Anitya. If you want to see whole list of changes, see Anitya 1.0.0 release on GitHub [1]. I hope this release will bring joy to your life and solve at least some of the pain points people had with Anitya. Michal Mage from release-monitoring.org P.S.: If you want to try something in Anitya without the fear of breaking anything, you can try it on staging instance [2]. [0] - https://release-monitoring.org/ [1] - https://github.com/fedora-infra/anitya/releases/tag/1.0.0 [2] - https://stg.release-monitoring.org/

3 5

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 27 Jan '21

27 Jan '21

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2021-01-28 from 08:00:00 to 09:00:00 America/Los_Angeles At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

LISA CfP open
by Ben Cotton 25 Jan '21

25 Jan '21

Hi everyone, The CfP for the LISA Conference is open through 23 February: https://www.usenix.org/conference/lisa21/call-for-participation It is a virtual event this year, held the first three days of June. I was involved with this conference for several years in a past life and it's a great venue for sharing sysadmin/DevOps knowledge. -- Ben Cotton He / Him / His Senior Program Manager, Fedora & CentOS Stream Red Hat TZ=America/Indiana/Indianapolis

1 0

Self introduction email
by Mauro Filanti 24 Jan '21

24 Jan '21

Hi guys! I already sent an email to the fedora-join mailing list so here my self introduction: - https://lists.fedoraproject.org/archives/list/fedora-join@lists.fedoraproje… Anyway I'm writing here in the infrastructure mailing list because it's one of my interests and where I think I can start to help/contribute in some way (I'll send it to the cloud one too, sorry if you receive it from both the mailing lists). My experience as Sysadmin it's new, just a few months old, but I've always been interested in the admin side of the system so please if someone can help me in order to get the next steps, would be great! Thanks and stay safe! Mauro

1 0

[Fedocal] Reminder meeting : Fedora Infrastructure
by smooge＠gmail.com 20 Jan '21

20 Jan '21

Dear all, You are kindly invited to the meeting: Fedora Infrastructure on 2021-01-21 from 08:00:00 to 09:00:00 America/Los_Angeles At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Weekly Fedora Infrastructure meeting. See infrastructure list for agenda a day before. Source: https://apps.fedoraproject.org/calendar/meeting/9675/

1 0

Resources for a hosting an alternative Fedora buildroot
by Tom Stellard 18 Jan '21

18 Jan '21

Hi, I am considering making a proposal for an alternative buildroot (similar to ELN) in Fedora, and I want to try to understand what Fedora resources might be available for this. My requirements are: + A koji server and builders for doing 'shadow' builds of ~2000 Fedora packages. 'shadow' builds means that every time a build is made in the official Fedora koji then one would be done for this alternative buildroot. + Builders for all architectures supported by Fedora. + Developer access to the koji server using Fedora kerberos credentials. + Permissions to create and modify tags on the koji server for at least 3 people. + A DNF repo generated from these package builds that is updated once per day. + Jenkins server that can host a CI job that listens for Fedora builds and submits builds to this alternative buildroot. My questions are: Do we have enough machine resources and people time to implement something like this, and if not, how much additional resources would be required? How and where to implement this, e.g. can we use the staging koji server or something else? Can we build something that is generic enough, so that other people who want to make similar proposals would be able to easily set up their own alternative buildroot? Thanks, Tom

6 8

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure January 2021