Hi everyone,
Intro
There have been on-and-off discussions within the Go SIG about vendoring
dependencies. To put my thoughts into words, I wrote a blog post [0]
about the issues with Go dependency de-vendoring that Fedora currently does.
Docker stack vendoring
I worry that the current approach to dependency management is
unfeasible—especially for complex package stacks like
Containerd/Docker/Moby. These packages and the underlying libraries have
huge dependency trees and circular dependencies and have been
out-of-date for months or longer. moby-engine already takes a
complicated, hybrid approach to vendoring (part of the package uses
vendored dependencies, part does not), while containerd is all
un-bundled. Getting everything up-to-date will require a significant
amount of work that nobody has stepped up to do, and in my view, is
ultimately unsustainable. Last year, I onboarded new contributors, as I
no longer could dedicate time to maintain these packages, but it was
also difficult for them to keep up with the web of dependencies.
I propose we start with fully vendoring the Docker stack. As I said,
parts of moby-engine are already bundled, and so are podman, kubernetes,
cri-o, containernetworking-plugins, and other applications in the
written-in-Go containerization stack. I have been working on revamped
Docker stack packages at [1]. I believe that the simplified packaging
approach will entice new maintainers to come onboard—I have already
reached out to one. I also wrote specfiles for Docker Buildx and Docker
Compose v2 that were not feasible to package with the previous approach.
Overall Go ecosystem
Then, we can re-evaluate the overall state of the Go library ecosystem.
I am not proposing that we immediately mass retire all Go libraries and
vendor everything, but I think we need to consider the overall health of
Go applications in Fedora and consider vendoring in at least some cases.
I will also note the Go Packaging Guidelines' current stance on the
vendoring[2]:
At the moment golang projects packaged in Fedora SHOULD be unbundled
by default. It means projects are built from dependencies packaged in
Fedora.
For some project it can be reasonable to build from bundled
dependencies. Every bundling needs a proper justification.
Vendoring the Docker stack is allowed under this guideline. Any more
drastic steps to mass de-vendor certain packages or use vendoring for
any new packages would obviously require guidelines changes—but again,
we are not there yet. There is more tooling work to be done and more
discussion to be had.
go-vendor-tools
I have been working on go-vendor-tools [3], a tool to enable packaging
vendored Go applications in a Guidelines-compliant way, for the past
couple weeks. go-vendor-tools aims to make creating fully reproducible
vendor archives and handling licensing a relatively frictionless
process. The tool also natively supports regenerating vendor tarballs to
apply security updates[4].
See [5] for instructions to test the latest go-vendor-tools and the
current iteration of the go2rpm code to allow automatically generating
vendored Go package specfiles. I look forward to your feedback.
***
If you have read this far, thank you! Any feedback about the Docker
stack or Go vendoring overall is welcome.
Best,
Maxwell
[0]
https://gtmx.me/blog/fedora-go-unbundling-is-broken/
[1]
https://git.sr.ht/~gotmax23/docker-ng
[2]
https://docs.fedoraproject.org/en-US/packaging-guidelines/Golang/#_bundle...
[3]
https://fedora.gitlab.io/sigs/go/go-vendor-tools/
[4]
https://fedora.gitlab.io/sigs/go/go-vendor-tools/scenarios/#security-updates
[5]
https://gitlab.com/fedora/sigs/go/go2rpm/-/merge_requests/4
--
Maxwell G (@gotmax23)
Pronouns: He/They