Hi Maxwell,
FYI the singularity-ce & apptainer packages have been using vendoring all along. I
don't know if there's anything from our tooling that would help you or not. Maybe
we should switch to yours when you are finished.
go mod vendor is run here:
https://github.com/apptainer/apptainer/blob/main/mlocal/frags/Makefile.st...
Here's the script that updates license dependencies:
https://github.com/apptainer/apptainer/blob/main/scripts/update-license-d...
There's an automated check for every PR to make sure it is up to date:
https://github.com/apptainer/apptainer/blob/main/.github/workflows/ci.yml...
Bundled provides are inserted here:
https://github.com/apptainer/apptainer/blob/main/mconfig#L890
Dave
On Mon, Mar 25, 2024 at 06:17:55PM -0500, Maxwell G wrote:
Hi everyone,
Intro
There have been on-and-off discussions within the Go SIG about vendoring dependencies. To
put my thoughts into words, I wrote a blog post [0] about the issues with Go dependency
de-vendoring that Fedora currently does.
Docker stack vendoring
I worry that the current approach to dependency management is unfeasible—especially for
complex package stacks like Containerd/Docker/Moby. These packages and the underlying
libraries have huge dependency trees and circular dependencies and have been out-of-date
for months or longer. moby-engine already takes a complicated, hybrid approach to
vendoring (part of the package uses vendored dependencies, part does not), while
containerd is all un-bundled. Getting everything up-to-date will require a significant
amount of work that nobody has stepped up to do, and in my view, is ultimately
unsustainable. Last year, I onboarded new contributors, as I no longer could dedicate time
to maintain these packages, but it was also difficult for them to keep up with the web of
dependencies.
I propose we start with fully vendoring the Docker stack. As I said, parts of moby-engine
are already bundled, and so are podman, kubernetes, cri-o, containernetworking-plugins,
and other applications in the written-in-Go containerization stack. I have been working on
revamped Docker stack packages at [1]. I believe that the simplified packaging approach
will entice new maintainers to come onboard—I have already reached out to one. I also
wrote specfiles for Docker Buildx and Docker Compose v2 that were not feasible to package
with the previous approach.
Overall Go ecosystem
Then, we can re-evaluate the overall state of the Go library ecosystem. I am not
proposing that we immediately mass retire all Go libraries and vendor everything, but I
think we need to consider the overall health of Go applications in Fedora and consider
vendoring in at least some cases.
I will also note the Go Packaging Guidelines' current stance on the vendoring[2]:
At the moment golang projects packaged in Fedora SHOULD be unbundled by default. It means
projects are built from dependencies packaged in Fedora.
For some project it can be reasonable to build from bundled dependencies. Every bundling
needs a proper justification.
Vendoring the Docker stack is allowed under this guideline. Any more drastic steps to
mass de-vendor certain packages or use vendoring for any new packages would obviously
require guidelines changes—but again, we are not there yet. There is more tooling work to
be done and more discussion to be had.
go-vendor-tools
I have been working on go-vendor-tools [3], a tool to enable packaging vendored Go
applications in a Guidelines-compliant way, for the past couple weeks. go-vendor-tools
aims to make creating fully reproducible vendor archives and handling licensing a
relatively frictionless process. The tool also natively supports regenerating vendor
tarballs to apply security updates[4].
See [5] for instructions to test the latest go-vendor-tools and the current iteration of
the go2rpm code to allow automatically generating vendored Go package specfiles. I look
forward to your feedback.
***
If you have read this far, thank you! Any feedback about the Docker stack or Go vendoring
overall is welcome.
Best,
Maxwell
[0]
https://gtmx.me/blog/fedora-go-unbundling-is-broken/
[1]
https://git.sr.ht/~gotmax23/docker-ng
[2]
https://docs.fedoraproject.org/en-US/packaging-guidelines/Golang/#_bundle...
[3]
https://fedora.gitlab.io/sigs/go/go-vendor-tools/
[4]
https://fedora.gitlab.io/sigs/go/go-vendor-tools/scenarios/#security-updates
[5]
https://gitlab.com/fedora/sigs/go/go2rpm/-/merge_requests/4
--
Maxwell G (@gotmax23)
Pronouns: He/They
--
_______________________________________________
golang mailing list -- golang(a)lists.fedoraproject.org
To unsubscribe send an email to golang-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/golang@lists.fedoraproject.org
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue