I have been thinking about proposing a Change to Fedora 39, which would disable yum modular repos by default in installs. I thought I would float the idea here first.
I suspect the vast majority of Fedora users don't use the modular repos, so I don't see the point of enabling them by default anymore. Does this make sense?
I know dnf5 is coming with performance improvements but I still think turning off the modular repos would speed up dnf and save users a lot of time.
Jens (pulling his flame-wear closer :)
ps I think it would be a good idea to disable the cisco-h264 repo too by default in the fedora container image, and maybe also for headless Fedora editions.
On Tue, May 09, 2023 at 12:31:54PM +0800, Jens-Ulrik Petersen wrote:
I have been thinking about proposing a Change to Fedora 39, which would disable yum modular repos by default in installs. I thought I would float the idea here first.
I suspect the vast majority of Fedora users don't use the modular repos, so I don't see the point of enabling them by default anymore. Does this make sense?
Yep. Removing or disabling the modular repos provides a nice speedup that is quite noticable if you have a spotty network. Users who want to install modules can easily add the repos back with a simple one-line command.
ps I think it would be a good idea to disable the cisco-h264 repo too by default in the fedora container image, and maybe also for headless Fedora editions.
Also yep.
Zbyszek
On Tue, May 9, 2023 at 6:32 AM Jens-Ulrik Petersen petersen@redhat.com wrote:
I have been thinking about proposing a Change to Fedora 39, which would disable yum modular repos by default in installs. I thought I would float the idea here first.
I suspect the vast majority of Fedora users don't use the modular repos, so I don't see the point of enabling them by default anymore. Does this make sense?
It makes sense. I don't use modularity at all. I am curious to know who is actually using modularity now.
On Tue, May 9, 2023 at 12:32 AM Jens-Ulrik Petersen petersen@redhat.com wrote:
I have been thinking about proposing a Change to Fedora 39, which would disable yum modular repos by default in installs. I thought I would float the idea here first.
I suspect the vast majority of Fedora users don't use the modular repos, so I don't see the point of enabling them by default anymore. Does this make sense?
I know dnf5 is coming with performance improvements but I still think turning off the modular repos would speed up dnf and save users a lot of time.
I'm okay with this. The only case I had for using modular repos was Nodejs, but that is demodularizing now.
ps I think it would be a good idea to disable the cisco-h264 repo too by default in the fedora container image, and maybe also for headless Fedora editions.
Please don't do this, as ffmpeg will pull the codec in if available, and there are tons of headless use-cases for ffmpeg that make the availability of this codec useful. It's also a really tiny repository anyway...
-- 真実はいつも一つ!/ Always, there's only one truth!
On Tue, May 9, 2023 at 1:27 PM Neal Gompa ngompa13@gmail.com wrote:
On Tue, May 9, 2023 at 12:32 AM Jens-Ulrik Petersen petersen@redhat.com wrote:
I have been thinking about proposing a Change to Fedora 39, which would disable yum modular repos by default in installs. I thought I would float the idea here first.
I suspect the vast majority of Fedora users don't use the modular repos, so I don't see the point of enabling them by default anymore. Does this make sense?
I know dnf5 is coming with performance improvements but I still think turning off the modular repos would speed up dnf and save users a lot of time.
I'm okay with this. The only case I had for using modular repos was Nodejs, but that is demodularizing now.
ps I think it would be a good idea to disable the cisco-h264 repo too by
default in the fedora container image, and maybe also for headless Fedora editions.
Please don't do this, as ffmpeg will pull the codec in if available, and there are tons of headless use-cases for ffmpeg that make the availability of this codec useful. It's also a really tiny repository anyway...
I'd like to second to what Neal said. I think it makes sense to disable the modular repos by default, but I'd rather keep the cisco openh264 repo enabled.
V Tue, May 09, 2023 at 12:31:54PM +0800, Jens-Ulrik Petersen napsal(a):
I have been thinking about proposing a Change to Fedora 39, which would disable yum modular repos by default in installs. I thought I would float the idea here first.
I suspect the vast majority of Fedora users don't use the modular repos
Probably. But I don't know. We will know from countme statistics after we disable them.
, so I don't see the point of enabling them by default anymore. Does this make sense?
That makes sense.
I know dnf5 is coming with performance improvements but I still think turning off the modular repos would speed up dnf and save users a lot of time.
How much is a lot of time?
I measured cached "dnf upgrade" (i.e. DNF4) on rawhide without and with the modular repository 5 times and the times are 1.022 vs. 1.090 seconds. I.e. 6.2% speedup.
Then I removed caches and looked at download times. I again did 5 tries, but the variance in dowload times reported by DNF was significantly useless. So I can only say that a size of the transmitted data is 71.0 MB for nonmodular repository and 1.6 MB modular repository. However, even this comparison is not fair as a different compression algorithm is used.
-- Petr
On Tue, May 9, 2023 at 2:22 PM Petr Pisar ppisar@redhat.com wrote:
I measured cached "dnf upgrade" (i.e. DNF4) on rawhide without and with the modular repository 5 times and the times are 1.022 vs. 1.090 seconds. I.e. 6.2% speedup.
Just a note here that the speedup might be a bit larger on released Fedora versions as there are two modular repositories that'd get disabled: base repo and updates repository, where rawhide has just one unified base and updates.
Also (no raw data at hand), lower perf devices/SBCs can benefit even more from this as repo parsing takes significantly longer time there than on desktops/laptops.
On Tue, May 09, 2023 at 02:21:46PM +0200, Petr Pisar wrote:
V Tue, May 09, 2023 at 12:31:54PM +0800, Jens-Ulrik Petersen napsal(a):
I have been thinking about proposing a Change to Fedora 39, which would disable yum modular repos by default in installs. I thought I would float the idea here first.
I suspect the vast majority of Fedora users don't use the modular repos
Probably. But I don't know. We will know from countme statistics after we disable them.
, so I don't see the point of enabling them by default anymore. Does this make sense?
That makes sense.
I know dnf5 is coming with performance improvements but I still think turning off the modular repos would speed up dnf and save users a lot of time.
How much is a lot of time?
I measured cached "dnf upgrade" (i.e. DNF4) on rawhide without and with the modular repository 5 times and the times are 1.022 vs. 1.090 seconds. I.e. 6.2% speedup.
Then I removed caches and looked at download times. I again did 5 tries, but the variance in dowload times reported by DNF was significantly useless. So I can only say that a size of the transmitted data is 71.0 MB for nonmodular repository and 1.6 MB modular repository. However, even this comparison is not fair as a different compression algorithm is used.
Those results will depend a lot on the network. (I *did* say "spotty network" in my original message.) At home I have the problem that most dnf download operations are OK, but every once in a while I get a slow connection (I don't know if it's a question of a different mirror, or just some temporary congestion, etc.) So doing each extra metadata download increase the chances of hitting a bad connection. This doesn't scale proportionally to the size of the downloads, but rather is some complex function that grows with the number of downloads and their sizes.
Another case is network with large latency: even if the data is fresh, just checking if it needs to be re-downloaded takes some time that grows with the number of repos.
(FWIW, I did some measurements when writing this, and essentially the standard deviation of the download time is larger than the download time on this machine, so I'm not posting those meaningless numbers.)
Zbyszek
On Mon, May 8, 2023, at 23:31, Jens-Ulrik Petersen wrote:
I suspect the vast majority of Fedora users don't use the modular repos, so I don't see the point of enabling them by default anymore. Does this make sense?
I could get behind that. One of my first tasks on a new system is to disable them.
Jens-Ulrik Petersen wrote:
I have been thinking about proposing a Change to Fedora 39, which would disable yum modular repos by default in installs. I thought I would float the idea here first.
I suspect the vast majority of Fedora users don't use the modular repos, so I don't see the point of enabling them by default anymore. Does this make sense?
Yes, finally! It is time to shelve the last vestiges of the failed "Modularity by default" project.
Users who really want Modularity can just opt in. But I think it usually makes more sense to deliver alternative versions in different ways that are not subject to the kind of version conflicts modules can cause (i.e., libraries as parallel-installable compatibility libraries, leaf packages through Copr).
Kevin Kofler
Thanks for all the helpful comments and feedback, though more is still welcome of course. So I am leaning now towards not installing fedora-repos-modular by default (rather than disabling the modular repos by default): also for upgrade compatibility.
I have started a Change proposal draft, which I think should be ready soon, with current working title *"Turn off modular dnf repos by default"*. Perhaps *"No default fedora-repos-modular"* would be more precise. Or any better suggestions (that don't sound like modularity is being removed)?
I think the actual work (PR's) required is not huge, but if someone is particularly keen to join the effort let me know.
Jens
On Wed, May 10, 2023 at 12:21:51PM +0800, Jens-Ulrik Petersen wrote:
Thanks for all the helpful comments and feedback, though more is still welcome of course. So I am leaning now towards not installing fedora-repos-modular by default (rather than disabling the modular repos by default): also for upgrade compatibility.
I have started a Change proposal draft, which I think should be ready soon, with current working title *"Turn off modular dnf repos by default"*. Perhaps *"No default fedora-repos-modular"* would be more precise. Or any better suggestions (that don't sound like modularity is being removed)?
"No fedora-repos-modular in default installation" ?
I think the actual work (PR's) required is not huge, but if someone is particularly keen to join the effort let me know.
The discussion is the work ;)
Zbyszek
"No fedora-repos-modular in default installation" ?
I feel that's a better way than disabling (enabled=0) module repos by default.
/etc/yum.repos.d/fedora-modular.repo /etc/yum.repos.d/fedora-updates-modular.repo
#enabled=1 enabled=0
For people who want to use the modularity, once they install the RPM package fedora-repos-modular, they can install the modules without the dnf command line option to enable the module repos.
``` $ sudo dnf install fedora-repos-modular $ sudo dnf install <modules/*> ```
Jun
On Wed, May 10, 2023 at 10:42 AM Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
On Wed, May 10, 2023 at 12:21:51PM +0800, Jens-Ulrik Petersen wrote:
Thanks for all the helpful comments and feedback, though more is still welcome of course. So I am leaning now towards not installing fedora-repos-modular by default (rather than disabling the modular repos by default): also for upgrade compatibility.
I have started a Change proposal draft, which I think should be ready soon, with current working title *"Turn off modular dnf repos by default"*. Perhaps *"No default fedora-repos-modular"* would be more precise. Or any better suggestions (that don't sound like modularity is being removed)?
"No fedora-repos-modular in default installation" ?
I think the actual work (PR's) required is not huge, but if someone is particularly keen to join the effort let me know.
The discussion is the work ;)
Zbyszek _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Wed, May 10, 2023 at 01:07:55PM +0200, Jun Aruga (he / him) wrote:
"No fedora-repos-modular in default installation" ?
I feel that's a better way than disabling (enabled=0) module repos by default.
To clarify: I fully agree. My earlier comment was about how to better reflect this in the name. ("Disable" is not a good word because suggests "enabled=0".)
Zbyszek
On Wed, May 10, 2023 at 12:21:51PM +0800, Jens-Ulrik Petersen wrote:
Thanks for all the helpful comments and feedback, though more is still welcome of course. So I am leaning now towards not installing fedora-repos-modular by default (rather than disabling the modular repos by default): also for upgrade compatibility.
I have started a Change proposal draft, which I think should be ready soon, with current working title *"Turn off modular dnf repos by default"*. Perhaps *"No default fedora-repos-modular"* would be more precise. Or any better suggestions (that don't sound like modularity is being removed)?
I think the actual work (PR's) required is not huge, but if someone is particularly keen to join the effort let me know.
So, at the risk of opening a larger can of worms...and increasing the scope here beyond what you have any desire to do...
Should we consider just retiring modular repos entirely?
I do know there's a number of active module builds, but I am not sure how much users use them. While RHEL still uses modules, EPEL has retired their epel8 modular repos.
Can the things using modules now switch to compat packages in the main repository?
I realize we may want to just say 'no, not now' here, but I thought I would bring it up.
kevin
Am 10.05.2023 um 18:45 schrieb Kevin Fenzi kevin@scrye.com:
On Wed, May 10, 2023 at 12:21:51PM +0800, Jens-Ulrik Petersen wrote:
Thanks for all the helpful comments and feedback, though more is still welcome of course. So I am leaning now towards not installing fedora-repos-modular by default (rather than disabling the modular repos by default): also for upgrade compatibility.
I have started a Change proposal draft, which I think should be ready soon, with current working title *"Turn off modular dnf repos by default"*. Perhaps *"No default fedora-repos-modular"* would be more precise. Or any better suggestions (that don't sound like modularity is being removed)?
I think the actual work (PR's) required is not huge, but if someone is particularly keen to join the effort let me know.
So, at the risk of opening a larger can of worms...and increasing the scope here beyond what you have any desire to do...
Should we consider just retiring modular repos entirely?
I do know there's a number of active module builds, but I am not sure how much users use them. While RHEL still uses modules, EPEL has retired their epel8 modular repos.
Can the things using modules now switch to compat packages in the main repository?
I realize we may want to just say 'no, not now' here, but I thought I would bring it up.
I suppose I would belong to the latter. From a Server admin’s POV the modules are a great chance to keep some backwards compatibility with our fast pacing distro. The most prominent example is PostgreSQL. It is not so rare that the new major version requires an adjustment in the application programme or at least an extensive test. And every major update also requires a migration of the entire data stock. Modularisation is a welcome opportunity to quickly switch to a new release and use new capabilities, but to carry out the adaptation / tests with PostgreSQL at your leisure.
And that is just one example. Another example is changes in languages such as Ruby, where application programmes sometimes cannot keep up with the changes. Same is true for httpd or tomcat - just some other examples.
If we could manage compat packages or parallel use of different major versions (as, for example, the Java runtime environment has managed to provide perfectly for decades), then modularisation would probably no longer be needed.
With Fedora Server, we are not only aiming at the home lab, but also at an enterprise deployment. And there, backwards compatibility and long-term usability are an indispensable must.
-- Peter Boy https://fedoraproject.org/wiki/User:Pboy PBoy@fedoraproject.org
Timezone: CET (UTC+1) / CEST /UTC+2)
Fedora Server Edition Working Group member Fedora Docs team contributor and board member Java developer and enthusiast
And that is just one example. Another example is changes in languages such as Ruby, where application programmes sometimes cannot keep up with the changes. Same is true for httpd or tomcat - just some other examples.
I am in the Ruby SIG. In the case of Ruby, I don't think we have the capacity of the older Ruby modules in the rawhide and supported platforms. You can see how the current Module Ruby builds look like from the Koji build page below. In my memory, we are struggling to build old Ruby, Ruby 2.7 to build on the new environment, rawhide. https://koji.fedoraproject.org/koji/packageinfo?packageID=125
If you want to use multiple versions of the Ruby in Linux, RVM (Ruby Version Manager) can be an alternative. https://rvm.io/
On Wed, May 10, 2023 at 07:09:49PM +0200, Peter Boy wrote:
Am 10.05.2023 um 18:45 schrieb Kevin Fenzi kevin@scrye.com:
On Wed, May 10, 2023 at 12:21:51PM +0800, Jens-Ulrik Petersen wrote:
Thanks for all the helpful comments and feedback, though more is still welcome of course. So I am leaning now towards not installing fedora-repos-modular by default (rather than disabling the modular repos by default): also for upgrade compatibility.
I have started a Change proposal draft, which I think should be ready soon, with current working title *"Turn off modular dnf repos by default"*. Perhaps *"No default fedora-repos-modular"* would be more precise. Or any better suggestions (that don't sound like modularity is being removed)?
I think the actual work (PR's) required is not huge, but if someone is particularly keen to join the effort let me know.
So, at the risk of opening a larger can of worms...and increasing the scope here beyond what you have any desire to do...
Should we consider just retiring modular repos entirely?
I do know there's a number of active module builds, but I am not sure how much users use them. While RHEL still uses modules, EPEL has retired their epel8 modular repos.
Can the things using modules now switch to compat packages in the main repository?
I realize we may want to just say 'no, not now' here, but I thought I would bring it up.
I suppose I would belong to the latter. From a Server admin’s POV the modules are a great chance to keep some backwards compatibility with our fast pacing distro. The most prominent example is PostgreSQL. It is not so rare that the new major version requires an adjustment in the application programme or at least an extensive test. And every major update also requires a migration of the entire data stock. Modularisation is a welcome opportunity to quickly switch to a new release and use new capabilities, but to carry out the adaptation / tests with PostgreSQL at your leisure.
And that is just one example. Another example is changes in languages such as Ruby, where application programmes sometimes cannot keep up with the changes. Same is true for httpd or tomcat - just some other examples.
If we could manage compat packages or parallel use of different major versions (as, for example, the Java runtime environment has managed to provide perfectly for decades), then modularisation would probably no longer be needed.
Debian has parallel installation with PostgreSQL. It does mean the major versions show up in the DB paths, but I don't think that's a bad thing. I think various problems that modularity tries to solve could also be solved with parallel installs and sometimes provide a better user experience.
On Wed, May 10, 2023 at 07:09:49PM +0200, Peter Boy wrote:
I suppose I would belong to the latter. From a Server admin’s POV the modules are a great chance to keep some backwards compatibility with our fast pacing distro. The most prominent example is PostgreSQL. It is not so rare that the new major version requires an adjustment in the application programme or at least an extensive test. And every major update also requires a migration of the entire data stock. Modularisation is a welcome opportunity to quickly switch to a new release and use new capabilities, but to carry out the adaptation / tests with PostgreSQL at your leisure.
There's a proposal to provide parallel postgresql versions via plain old normal traditional boring packages: https://src.fedoraproject.org/rpms/postgresql/pull-request/59
Zbyszek
V Wed, May 10, 2023 at 09:45:35AM -0700, Kevin Fenzi napsal(a):
Should we consider just retiring modular repos entirely?
I do know there's a number of active module builds, but I am not sure how much users use them. While RHEL still uses modules, EPEL has retired their epel8 modular repos.
Speaking selfishly for my use case, not yet now. They are used as a development phase for RHEL modules. The upstream-downstream relation where downstream tries to push as much things to upstream to get early development and wider integration testing.
Can the things using modules now switch to compat packages in the main repository?
Compat packages suffer from distinct package (and RPM Provides) names. That's something what modules neatly solved. I would like too see different solution which does not regress from modularity first.
-- Petr
Hey Jens,
On Tue, 2023-05-09 at 12:31 +0800, Jens-Ulrik Petersen wrote:
ps I think it would be a good idea to disable the cisco-h264 repo too by default in the fedora container image, and maybe also for headless Fedora editions.
If we do decide to disable the fedora-cisco-openh264 repository on the base fedora OCI image, then we might have to enable it separately for the fedora-toolbox images, because the OpenH264 codec is something that applications running inside the container might want to use.
Coincidentally, changes like these is one of the motivations for: https://fedoraproject.org/wiki/Changes/ToolbxReleaseBlocker :)
Cheers, Rishi
On Wed, May 10, 2023 at 8:39 PM Debarshi Ray via devel < devel@lists.fedoraproject.org> wrote:
On Tue, 2023-05-09 at 12:31 +0800, Jens-Ulrik Petersen wrote: If we do decide to disable the fedora-cisco-openh264 repository on the base fedora OCI image, then we might have to enable it separately for the fedora-toolbox images, because the OpenH264 codec is something that applications running inside the container might want to use.
Right, so I already dropped that idea based on the feedback given so far, thanks. :+1:
An initial draft (I will rename the page later before submitting) is here btw for those really interested: https://fedoraproject.org/wiki/Changes/TurnOffModularRepos (At this point I would probably prefer critical feedback/suggestions in private until it is submitted. :-)
Jens