I've been thinking a little about how Go is updated in Fedora. I would like to hear other opinions about the current state of the releases and improve it.
This is not related to the Fedora proposal that I'm planning to submit today regarding the update of Go. I do not pretend to change anything for the next Fedora release, but at least have an idea of if what we are doing right now can be improved.
The current state:
Each Fedora release has a major release of Go available.
For example, Fedora 33 had 1.15, and Fedora 34 had 1.16.
The problem:
I see two main issues with this approach.
1. Both Go and Fedora have release cycles of 6 months, so the schedule is a little tight. Go releases can be delayed and have issues in the mass rebuild phase. 2. A user needs to wait for the next Fedora release to get a new major version of Go. So consequentially, they will tend to download from upstream, making the Go package just necessary for dependencies but with little use on its own. Also, backport bugs to Fedora releases might be a problem. Releasing packages that depend on new features are an issue too [0].
What I think is an improvement:
Suppose a new Go major version is released in the middle of a Fedora life cycle. In that case, I think it is an improvement for the user to be able to update to the following Go release.
A hypothetical new release cycle would look like this:
- Fedora N release follows Go upstream as close as we can. - Fedora N-1 sticks with the latest major version of Go that was available on it until the release of Fedora N.
Another hypothetical approach could be using modules with each upstream supported release in a stream.
To help in the decision, I made a report tool/web page [1] that shows the current state of the packages that depend on Go (Thanks to the COPR API).
It compares the builds of every single Go dependent package in Fedora 35 using the current available Go package with the same list of Go packages but using the Go package is available on Fedora Rawhide. To rephrase it, it compares Fedora 35 packages built with Go 1.16 with Fedora 35 packages built with Go 1.17.
As you can see, right now, from the 1901 packages that depend on Go, 196 have some sort of change and 1705 built exactly the same. You can search for "Same results" or "Something has changed" to see this. Or by the name of the package.
The report is not smart enough to say what happened right now so some "issues" are in fact improvements like golang-github-cactus-statsd-client, others like golang-github-briandowns-spinner are real issues.
My idea is to improve this report with your suggestions. Hence, if a new Go major release is available, we can confidently say that the package can be updated in the middle of a Fedora release.
[0] https://github.com/containers/podman/pull/12544
[1] https://alexsaezm.fedorapeople.org/report.html -> let it load, it has a JS that allows you to search
On Mon, Dec 13, 2021 at 02:22:24PM +0100, Alejandro Saez Morollon wrote:
A hypothetical new release cycle would look like this:
- Fedora N release follows Go upstream as close as we can.
- Fedora N-1 sticks with the latest major version of Go that was
available on it until the release of Fedora N.
Another hypothetical approach could be using modules with each upstream supported release in a stream.
This seems like the thing Modularity was invented to do, and would have the advantage of being able to be consistent across a release with a "baseline" version but also provide options.
On Tue, Dec 14, 2021 at 5:01 AM Matthew Miller mattdm@fedoraproject.org wrote:
On Mon, Dec 13, 2021 at 02:22:24PM +0100, Alejandro Saez Morollon wrote:
A hypothetical new release cycle would look like this:
- Fedora N release follows Go upstream as close as we can.
- Fedora N-1 sticks with the latest major version of Go that was
available on it until the release of Fedora N.
Another hypothetical approach could be using modules with each upstream supported release in a stream.
This seems like the thing Modularity was invented to do, and would have the advantage of being able to be consistent across a release with a "baseline" version but also provide options.
But AFAIK, only users can select a module stream, right? I mean, packages can't be build on top of a module stream so new needs of package maintainers cannot be satisfy with modules.
I'm curious about how other SIGs deal with these scenarios...
-- Matthew Miller mattdm@fedoraproject.org Fedora Project Leader _______________________________________________ golang mailing list -- golang@lists.fedoraproject.org To unsubscribe send an email to golang-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/golang@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On Sat, Dec 18, 2021 at 09:11:08AM +0100, Alejandro Saez Morollon wrote:
But AFAIK, only users can select a module stream, right? I mean, packages can't be build on top of a module stream so new needs of package maintainers cannot be satisfy with modules.
Packages _can_ build on top of a module stream, but only if they themselves are in other modules. Partly, this is a consequence of us deciding not to have a "default module" functionality in Fedora, but the _general_ thing is that modularity conceptually isn't really good for having arbitrary versions of low-level tools.
It's best for whole language stacks meant for end-user selection, and for big (like, "that's what this container or vm is _for_") applications (a database server or similar, or a web app). For example, this is a problem that it'd be ideal for: https://ask.fedoraproject.org/t/shipped-versions-of-php-and-zabbix-frontend-...
It can also be used for desktop apps that have separation through Flatpak.
It's not good for "many multi-purpose utilities and small tools in Fedora Linux need to be built with one version, others need another version. (And I think part of our misstep in bringing modularity to Fedora was opening it up for folks to try basically anything... like the Jurassic Park thing, it turns out that just because you _can_ doesn't mean you _should_, and I think we'd have been more successful drawing those lines. But, ah well.)
So anyway, back to Go — I think that's basically the line on deciding how to package this:
A) Are multiple versions primarily for _users_ and for a few "large" (in the sense of "likely to be deployed as sole tool in a container or vm") apps, or
B) Is it important to have multiple versions to build many different packages in Fedora Linux itself?
In case A, modularity _should_ be a solution (and although I know people are despairing about the technology overall, it's still being worked on and we should figure out and fix anything that's less than ideal). In case B, it's probably best to offer parallel "compat" packages, or use one of the other "alternatives to modularity" approches people have suggested.
On 18. 12. 21 19:04, Matthew Miller wrote:
On Sat, Dec 18, 2021 at 09:11:08AM +0100, Alejandro Saez Morollon wrote:
But AFAIK, only users can select a module stream, right? I mean, packages can't be build on top of a module stream so new needs of package maintainers cannot be satisfy with modules.
Packages _can_ build on top of a module stream, but only if they themselves are in other modules. Partly, this is a consequence of us deciding not to have a "default module" functionality in Fedora...
Just a note here: Even with default modular streams, non-modular packages would still only be able to be built with one modular stream -- the default one. So this isn't a consequence (not even partly) of us deciding not to have default modular streams.
...snip... So anyway, back to Go — I think that's basically the line on deciding how to package this:
A) Are multiple versions primarily for _users_ and for a few "large" (in the sense of "likely to be deployed as sole tool in a container or vm") apps, or
B) Is it important to have multiple versions to build many different packages in Fedora Linux itself?
In case A, modularity _should_ be a solution (and although I know people are despairing about the technology overall, it's still being worked on and we should figure out and fix anything that's less than ideal). In case B, it's probably best to offer parallel "compat" packages, or use one of the other "alternatives to modularity" approches people have suggested.
In my opinion, both situations are best solved with regular packages. While it is *possible* to solve A with modularity, it is not *necessary*. In fact, it is discouraged:
https://docs.fedoraproject.org/en-US/modularity/policies/#_requirements_for_...
(And while I am open to discuss whether it should be discouraged or not when we "figure things out", I would prefer if we would not recommend this approach to anyone now when we agreed alternatives SHOULD be preferred.)
On Sat, Dec 18, 2021 at 07:50:34PM +0100, Miro Hrončok wrote:
Just a note here: Even with default modular streams, non-modular packages would still only be able to be built with one modular stream -- the default one. So this isn't a consequence (not even partly) of us deciding not to have default modular streams.
The "can build on the default stream, at least" is the "partly" I meant. But I don't want to quibble about that; I agree that if we have a lot of things _in the distro_ that need to build against different versions of something _and_ coexist, modularity isn't the right tool.
In my opinion, both situations are best solved with regular packages. While it is *possible* to solve A with modularity, it is not *necessary*. In fact, it is discouraged:
You snipped my example, which is "ripped from the headlines" https://bugzilla.redhat.com/show_bug.cgi?id=2020292. We can argue about "best", but ... these things _aren't_ being solved in Fedora right now, so it hardly matters. Even though Remi is actually actively maintaining a PHP 7.4 stack (in his own repos), Fedora hasn't provided a way to do that _here_ nicely.
Maybe the Go situation is simpler (because of the static builds and other differences), but... overall this is remains an unsolved problem, and our current recommendations aren't adequate.
On 18. 12. 21 9:11, Alejandro Saez Morollon wrote:
On Tue, Dec 14, 2021 at 5:01 AM Matthew Miller <mattdm@fedoraproject.org mailto:mattdm@fedoraproject.org> wrote:
On Mon, Dec 13, 2021 at 02:22:24PM +0100, Alejandro Saez Morollon wrote: > A hypothetical new release cycle would look like this: > > - Fedora N release follows Go upstream as close as we can. > - Fedora N-1 sticks with the latest major version of Go that was > available on it until the release of Fedora N. > > > Another hypothetical approach could be using modules with each upstream > supported release in a stream. This seems like the thing Modularity was invented to do, and would have the advantage of being able to be consistent across a release with a "baseline" version but also provide options.
But AFAIK, only users can select a module stream, right? I mean, packages can't be build on top of a module stream so new needs of package maintainers cannot be satisfy with modules.
I'm curious about how other SIGs deal with these scenarios...
In Python, we have one "main" Python version that remains the same within one Fedora release. E.g. in Fedora 33 and 34 this was Python 3.9, while in Fedora 35 and 36 it is Python 3.10.
Other Python versions are provided for users to install via other python3.X packages. Users of Fedora 35 can dnf install python3.9 and users of Fedora 34 can dnf install python3.10 (or python3.11 or python3.6 etc.).
https://developer.fedoraproject.org/tech/languages/python/multiple-pythons.h...
The packages are parallel-installable -- users need to explicitly select what Python they use, not what Python they install.
All Fedora Python packages are built with the "main" Python version, but the reason for that is maintainability rather than necessity -- building for multiple Python versions would mean we would need to build many (if not all) libraries for different Python versions as well.
Go has an advantage because AFAIK it statically builds everything, so if packagers choose to BuildRequire a different version of Go, they can do that in a way that does not require mass changes of all the used libraries.
Similarly, Python needs to be installed in order to run Python code while Go does not, so even if packaging parallel-installable versions of Go compiler is not possible or reasonable, they can be packaged as conflicting packages.
There is no reason to package the alternate (newer or older) versions of Go as modules. In fact, if you'd like other non-modular packages to be able to BuildRequire different Go versions, modules won't work.
As a side note, modules are discouraged by policy if they can be avoided, packagers SHOULD prefer "compatibility packages" (which is a term for what I have described above):
https://docs.fedoraproject.org/en-US/modularity/policies/#_requirements_for_...
Other languages do the same or similar thing as Python. E.g. clang and ghc packages:
https://fedoraproject.org/wiki/Changes/GHC_parallel_version_installs
On Sat, Dec 18, 2021 at 3:35 AM Alejandro Saez Morollon asm@redhat.com wrote:
On Tue, Dec 14, 2021 at 5:01 AM Matthew Miller mattdm@fedoraproject.org wrote:
On Mon, Dec 13, 2021 at 02:22:24PM +0100, Alejandro Saez Morollon wrote:
A hypothetical new release cycle would look like this:
- Fedora N release follows Go upstream as close as we can.
- Fedora N-1 sticks with the latest major version of Go that was
available on it until the release of Fedora N.
Another hypothetical approach could be using modules with each upstream supported release in a stream.
This seems like the thing Modularity was invented to do, and would have the advantage of being able to be consistent across a release with a "baseline" version but also provide options.
But AFAIK, only users can select a module stream, right? I mean, packages can't be build on top of a module stream so new needs of package maintainers cannot be satisfy with modules.
I'm curious about how other SIGs deal with these scenarios...
I can't think of anyone I know personally who actually uses "modularity" since its introduction, As Zathros said in the old Babylon Five episode "this is wrong tool!"
On Mon, Dec 13, 2021 at 2:25 PM Alejandro Saez Morollon asm@redhat.com wrote:
I've been thinking a little about how Go is updated in Fedora. I would like to hear other opinions about the current state of the releases and improve it.
(snip)
Sorry for not seeing this email earlier. I have been sending all mail from the golang list to /dev/null for my sanity's sake, and this thread has fallen victim to that.
Both Go and Fedora have release cycles of 6 months, so the schedule is a little tight. Go releases can be delayed and have issues in the mass rebuild phase. A user needs to wait for the next Fedora release to get a new major version of Go. So consequentially, they will tend to download from upstream, making the Go package just necessary for dependencies but with little use on its own. Also, backport bugs to Fedora releases might be a problem. Releasing packages that depend on new features are an issue too [0].
If the release cycles of Golang and Fedora are conflicting, there's nothing we can do. It might not be the "best" outcome considering the "First" foundation, but what about just ... *not* bothering with making the tight squeeze? If it's too tight to get Go X into Fedora Y in time, then just update to Go X when it's ready, only in rawhide (Fedora Y+1). That way, Fedora releases get the most recent, stable Go version that was available at the time. (This is already what happened with Go 1.17 in rawhide earlier this year).
And is keeping older branches really such a problem? I see that upstream keeps releasing bugfix / point releases for older golang branches. Fedora 34 and 35 have golang 1.17.11 (!) right now, so that should not be a problem for our 1-year+one-month lifecycle?
What I think is an improvement:
Suppose a new Go major version is released in the middle of a Fedora life cycle. In that case, I think it is an improvement for the user to be able to update to the following Go release.
A hypothetical new release cycle would look like this:
Fedora N release follows Go upstream as close as we can. Fedora N-1 sticks with the latest major version of Go that was available on it until the release of Fedora N.
I think this is a *very bad* idea. Thankfully, I now maintain only one package that uses golang (though with a lot of vendored dependencies), but it still breaks with almost every major release of Golang for some inexplicable reason (even though Go promises forwards-compatibility ...). Updating the Go compiler in the middle of a stable release would break numerous packages, and I don't think we should put even more maintenance and unnecessary build-fixing busywork on the already stretched-thin golang package maintainers in Fedora.
Just update the Go compiler to a new major version when it is ready, only in rawhide. There, package build issues can be resolved without impacting or blocking updates on stable releases, and then let the new Go trickle down into the next stable release. That's how the Rawhide -> Fedora train works for a reason (GCC, Python, Ruby, LLVM/Clang, etc. all work this way; the only different toolchain is Rust, which is updated - thanks to very very strong forward-compatibility guarantees - every 6 weeks).
Honestly, I do not see the problem with how things currently work. Isn't it normal that "the new hotness" is first baked to perfection in rawhide, until it is then released with the next stable Fedora release?
Fabio
On Sat, Dec 18, 2021 at 10:34:34PM +0100, Fabio Valentini wrote:
Honestly, I do not see the problem with how things currently work. Isn't it normal that "the new hotness" is first baked to perfection in rawhide, until it is then released with the next stable Fedora release?
Well, going back to the first post, the example is https://github.com/containers/podman/pull/12544. Some quotes from there:
vrothberg:
Looks like K8s went on to using Go 1.17 features.
@containers/podman-maintainers FYI ... not sure what we should do. Go 1.17 is not in F35 and below, so we may have to wait a while.
rhatdan:
Yup I see it only in Rawhide. We should hold this off until Fedora gets updated. @alexsaezm when do you think f34 and F35 will go to golang 1.17?
alexsaezm:
The current policy of the Go SIG is to have one Go version per Fedora release. So neither both Fedora 34 and Fedora 35 will have Go 1.17. But it's something that of course can be reevaluated
@jcajka what do you think? Perhaps we can reevaluate this and have the latest Go version in every Fedora release that is currently supported instead of fixed versions per release.
... and then, this conversation follows.
Basically, it seems like we're moving too slowly to keep up with changes in Kubernetes, with trickle-down consequences.
On Sat, Dec 18, 2021 at 11:01 PM Matthew Miller mattdm@fedoraproject.org wrote:
On Sat, Dec 18, 2021 at 10:34:34PM +0100, Fabio Valentini wrote:
Honestly, I do not see the problem with how things currently work. Isn't it normal that "the new hotness" is first baked to perfection in rawhide, until it is then released with the next stable Fedora release?
Well, going back to the first post, the example is https://github.com/containers/podman/pull/12544. Some quotes from there:
vrothberg:
Looks like K8s went on to using Go 1.17 features.
@containers/podman-maintainers FYI ... not sure what we should do. Go 1.17 is not in F35 and below, so we may have to wait a while.
rhatdan:
Yup I see it only in Rawhide. We should hold this off until Fedora gets updated. @alexsaezm when do you think f34 and F35 will go to golang 1.17?
alexsaezm:
The current policy of the Go SIG is to have one Go version per Fedora release. So neither both Fedora 34 and Fedora 35 will have Go 1.17. But it's something that of course can be reevaluated
@jcajka what do you think? Perhaps we can reevaluate this and have the latest Go version in every Fedora release that is currently supported instead of fixed versions per release.
... and then, this conversation follows.
Basically, it seems like we're moving too slowly to keep up with changes in Kubernetes, with trickle-down consequences.
Sure, I saw that ticket. But I fail to see how this is this a "new problem". If you use, for example, some shiny, new features that are only going to be in GCC 12 or LLVM 14, you'd be out of luck on stable Fedora branches, as well.
The only problem I see here is that Go 1.17 was not pushed to Fedora 35 in time, which is why it is stuck at Go 1.16, same as Fedora 34.
Fabio
On Sat, Dec 18, 2021 at 11:06:45PM +0100, Fabio Valentini wrote:
Basically, it seems like we're moving too slowly to keep up with changes in Kubernetes, with trickle-down consequences.
Sure, I saw that ticket. But I fail to see how this is this a "new problem". If you use, for example, some shiny, new features that are only going to be in GCC 12 or LLVM 14, you'd be out of luck on stable Fedora branches, as well.
I don't think it's a new problem at all.
On Sat, Dec 18, 2021, at 5:06 PM, Fabio Valentini wrote:
Sure, I saw that ticket. But I fail to see how this is this a "new problem". If you use, for example, some shiny, new features that are only going to be in GCC 12 or LLVM 14,
There's a *big* difference between Go and C/C++/Rust though, which is that the version of the Go compiler you use at build time becomes the version of the Go *runtime* statically linked into your binary, with implications for things like GC performance.
Go's 6-month cadence is also much faster than C/C++ (though also much slower than Rust's, but the Rust 6 week releases are also smaller, and anyways generally the ecosystem is OK with "older" Rust compilers). Also relating to the release cycle, Go releases tend to add new features that parts of the ecosystem adapt relatively quickly. That seems likely to continue with 1.18 and the early generics.
you'd be out of luck on stable Fedora branches, as well.
Not quite; one doesn't need to use the single Go compiler version in RPM form from Fedora except currently when shipping software built as Fedora RPMs. (I know this was implicit in this discussion, but it's worth noting because people can and do get the Go compiler from e.g. upstream on Fedora systems too to build and ship software outside of the Fedora package collection itself)
That said, I don't quite understand why it's so complex to use modularity as part of building non-modular RPMs. (But I know this was extensively discussed, I didn't follow it really) It seems like it's more about maintaining multiple builds of the compiler/runtime.
It's interesting to note that in CentOS Stream 8, go-toolset is modularized, but that's not true in CentOS Stream 9.
Also related to this, it's worth looking at what others do; e.g. NixOS seems to maintain a few versions of Go: https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/compilers/go But they only have one Rust version. Although there are a ton of derivations for gcc and llvm, I suspect only a few are used.
It looks like Debian ships package+versions, e.g. `golang` is 1.17, and there's `golang-1.15` in unstable.
On Mon, 20 Dec 2021 at 10:16, Colin Walters walters@verbum.org wrote:
On Sat, Dec 18, 2021, at 5:06 PM, Fabio Valentini wrote:
Sure, I saw that ticket. But I fail to see how this is this a "new problem". If you use, for example, some shiny, new features that are only going to be in GCC 12 or LLVM 14,
There's a *big* difference between Go and C/C++/Rust though, which is that the version of the Go compiler you use at build time becomes the version of the Go *runtime* statically linked into your binary, with implications for things like GC performance.
Go's 6-month cadence is also much faster than C/C++ (though also much slower than Rust's, but the Rust 6 week releases are also smaller, and anyways generally the ecosystem is OK with "older" Rust compilers). Also relating to the release cycle, Go releases tend to add new features that parts of the ecosystem adapt relatively quickly. That seems likely to continue with 1.18 and the early generics.
you'd be out of luck on stable Fedora branches, as well.
Not quite; one doesn't need to use the single Go compiler version in RPM form from Fedora except currently when shipping software built as Fedora RPMs. (I know this was implicit in this discussion, but it's worth noting because people can and do get the Go compiler from e.g. upstream on Fedora systems too to build and ship software outside of the Fedora package collection itself)
That said, I don't quite understand why it's so complex to use modularity as part of building non-modular RPMs. (But I know this was extensively discussed, I didn't follow it really) It seems like it's more about maintaining multiple builds of the compiler/runtime.
The following is based on the lessons learned from EPEL-8. Koji does not really know much about modules or even RPMs beyond a src.rpm. So when koji is putting together a build root it will pull in things which best satisfy a spec files requires.
Case 1: Module A: - golang-17 - golang-foo-3.4-<fill_in_hash>
Module B: - golang-18 - golang-foo-3.4<fill_in_hash_bigger than ModuleA's>
In this case, you could define a spec which requires golang-17 but koji's dep resolver pulls in golang-foo- from Module B for non-modular packages. This breaks the build.
Case 2: Module A : - golang-17 - golang-foo-3.4-<fill_in_hash>
Module B: - golang-18 - golang-foo-3.4<fill_in_hash_smaller than ModuleA's> or - golang-foo-3.3 <because it works better with 18>
Koji's dep resolver in this case will then pull in Module A's golang-foo for any 'non-modular' package which required golang-foo.
You can configure koji to do a slightly different depsolving using dnf versus the koji algorithms, which fix things in that only default modules may get enabled in mock. [However may is a strong word.. it sometimes does not.]
The fixes to this are: a) use a tool like grobisplitter which breaks out all modules into separate repositories versus 'virtual repositories'. This allows koji and mock better ability to sort out what is the right package to put into the buildroot. b) use a script program like ursa major which basically hardwires in MBS determinations. This works for EL8/EL9 because the changes in modules has to go through a lot of internal checklists and such to make sure the script is updated and doesn't break in builds. It doesn't work that well in Fedora unless we put in similar 'please mark this package as to be used for this and get a PR signoff' [or someone writes and maintains a tool to do this which they did in MBS...] c) everything which uses a module, requires to be a module. Then MBS does all this determination and tells koji -> I don't care what you think, tag in Module-B's golang-foobar for this build. d) someone replaces koji with a build system which knows about MBS and other meta-tools.
It's interesting to note that in CentOS Stream 8, go-toolset is modularized, but that's not true in CentOS Stream 9.
Also related to this, it's worth looking at what others do; e.g. NixOS seems to maintain a few versions of Go: https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/compilers/go But they only have one Rust version. Although there are a ton of derivations for gcc and llvm, I suspect only a few are used.
It looks like Debian ships package+versions, e.g. `golang` is 1.17, and there's `golang-1.15` in unstable. _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
V Mon, Dec 20, 2021 at 10:39:11AM -0500, Stephen John Smoogen napsal(a):
You can configure koji to do a slightly different depsolving using dnf versus the koji algorithms, which fix things in that only default modules may get enabled in mock. [However may is a strong word.. it sometimes does not.]
The fixes to this are: a) use a tool like grobisplitter which breaks out all modules into separate repositories versus 'virtual repositories'. This allows koji and mock better ability to sort out what is the right package to put into the buildroot. b) use a script program like ursa major which basically hardwires in MBS determinations. This works for EL8/EL9 because the changes in modules has to go through a lot of internal checklists and such to make sure the script is updated and doesn't break in builds. It doesn't work that well in Fedora unless we put in similar 'please mark this package as to be used for this and get a PR signoff' [or someone writes and maintains a tool to do this which they did in MBS...] c) everything which uses a module, requires to be a module. Then MBS does all this determination and tells koji -> I don't care what you think, tag in Module-B's golang-foobar for this build. d) someone replaces koji with a build system which knows about MBS and other meta-tools.
Koji uses Mock for building packages. Mock allows enabling modules. Users can define their own build roots using side-tags (*). There are only two missing pieces to allow building non-modular packages with modules: Allow side-tags to pass the enabled modules to Mock, and make composes in Koji modular. Then all the installation magic around modules would be heavy lifted by DNF.
* That reminds that actually users can do it manually even now: If they tag a modular go compiler into their side-tag, they simply get it installed as a build dependency for any package built in the side tag.
-- Petr