On Fri, 4 Aug 2023 at 06:16, Miro Hrončok <mhroncok@redhat.com> wrote:
Hello folks,

With the retirement of modularity, the modular dnf repositories for Fedora 39
no longer exist. However, this will introduce a problem during upgrades. When
users try to upgrade from previous Fedora releases with fedora-repos-modular
installed, they will hit fatal errors that will probably look like this:

Errors during downloading metadata for repository 'fedora-modular':
   - Status code: 404 for
https://mirrors.fedoraproject.org/metalink?repo=fedora-modular-39&arch=x86_64
(IP: ...)
   - Status code: 404 for
https://mirrors.fedoraproject.org/metalink?repo=fedora-modular-39&arch=x86_64
(IP: ...)
   - Status code: 404 for
https://mirrors.fedoraproject.org/metalink?repo=fedora-modular-39&arch=x86_64
(IP: ...)
Error: Failed to download metadata for repo 'fedora-modular': Cannot prepare
internal mirrorlist: Status code: 404 for
https://mirrors.fedoraproject.org/metalink?repo=fedora-modular-39&arch=x86_64
(IP: ...)

Or:

Error: Failed to download metadata for repository 'fedora-modular': Cannot
download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

(The actual error might differ depending on the exact state of the removal of
the modular repos and their mirrorlist etc.)


This is caused by the combination of the following facts:

  - the modular repo configuration in Fedora 37/38 has skip_if_unavailable=False
  - when the releasever is set to 39, the URLs of the repos give error 404

This is a big deal, because even users who don't use modularity at all (but
have not uninstalled fedora-repos-modular) will not be able to upgrade to
Fedora 39+ without reaching for help.

Adam outlined 3 options to solve this problem in the bugzilla where he reported
this: https://bugzilla.redhat.com/2228827

I slept on it and have found a fourth option (but I don't think it's very
viable). I'll present all 4 options here:


I think we are going to need to do multiple of these solutions because there are too many ways people upgrade their systems (usually because someone told them there method would work years ago or they just found a doc which was written years ago, etc). As Petr Pisar points out elsewhere this is not a new problem. We regularly see someone on IRC for years asking why an upgrade didn't work and it comes down to something like skip_if_unavailable=true or some similar issue where a repository is not properly set up (say a copr or third party repo) for the current release. 

1. We need to document that this failure could and can occur.  We need to point out that we are only testing N methods for upgrades and anything else will need manual work arounds. The way to maybe find those will be to ask on <probably discussion.fedoraproject.org>.
2. We need to work out 1-2 methods we 'support' for upgrading which of the ones below I would say D and A where certain tooling will check to see if the upgrade is possible and then alert if it isn't and try a method of turning things off if ok. [If that is even possible with code paths not yet written.]
3. ....
4. Profit. 

I don't like B because it could be the most fragile as it will need work in pungi to make sure that the repositories are ALWAYS properly created in a way that doesn't cause problems. [They can't just be 'empty'. They need to be empty with repodata that allows for updates to always work. If this requires hacking pungi then any upgrade of the source code on the releng boxes could cause breakage weeks later or releases later.] It is also hard to ever turn this off. As much as we like to have people upgrade from 36->37 or 36->38, there is a large enough 'hump' of people who do a 36->40 turn that means we would be doing this til maybe 44.

Of course if it is just a simple pungi config which doesn't make other things break, B may be the easiest.. 

 

A. Set skip_if_unavailable=True in the modular repo configuration on old Fedora



B. Mirror empty modular repositories for the time being



C. Document this in the upgrade documentation



D. Change our upgrade tools to disable modular repos on upgrades to F39+


========================

What do you think we should do? My preference would be to go with option (B)
and fallback to option (A) if (B) isn't finished in reasonable time.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue


--
Stephen Smoogen, Red Hat Automotive
Let us be kind to one another, for most of us are fighting a hard battle. -- Ian MacClaren