Hi everyone,

Intro

There have been on-and-off discussions within the Go SIG about vendoring dependencies. To put my thoughts into words, I wrote a blog post [0] about the issues with Go dependency de-vendoring that Fedora currently does.

Docker stack vendoring

I worry that the current approach to dependency management is unfeasible—especially for complex package stacks like Containerd/Docker/Moby. These packages and the underlying libraries have huge dependency trees and circular dependencies and have been out-of-date for months or longer. moby-engine already takes a complicated, hybrid approach to vendoring (part of the package uses vendored dependencies, part does not), while containerd is all un-bundled. Getting everything up-to-date will require a significant amount of work that nobody has stepped up to do, and in my view, is ultimately unsustainable. Last year, I onboarded new contributors, as I no longer could dedicate time to maintain these packages, but it was also difficult for them to keep up with the web of dependencies.

I propose we start with fully vendoring the Docker stack. As I said, parts of moby-engine are already bundled, and so are podman, kubernetes, cri-o, containernetworking-plugins, and other applications in the written-in-Go containerization stack. I have been working on revamped Docker stack packages at [1]. I believe that the simplified packaging approach will entice new maintainers to come onboard—I have already reached out to one. I also wrote specfiles for Docker Buildx and Docker Compose v2 that were not feasible to package with the previous approach.

Overall Go ecosystem

Then, we can re-evaluate the overall state of the Go library ecosystem. I am not proposing that we immediately mass retire all Go libraries and vendor everything, but I think we need to consider the overall health of Go applications in Fedora and consider vendoring in at least some cases.

I will also note the Go Packaging Guidelines' current stance on the vendoring[2]:

At the moment golang projects packaged in Fedora SHOULD be unbundled by default. It means projects are built from dependencies packaged in Fedora.

For some project it can be reasonable to build from bundled dependencies. Every bundling needs a proper justification.

Vendoring the Docker stack is allowed under this guideline. Any more drastic steps to mass de-vendor certain packages or use vendoring for any new packages would obviously require guidelines changes—but again, we are not there yet. There is more tooling work to be done and more discussion to be had.

go-vendor-tools

I have been working on go-vendor-tools [3], a tool to enable packaging vendored Go applications in a Guidelines-compliant way, for the past couple weeks. go-vendor-tools aims to make creating fully reproducible vendor archives and handling licensing a relatively frictionless process. The tool also natively supports regenerating vendor tarballs to apply security updates[4].

See [5] for instructions to test the latest go-vendor-tools and the current iteration of the go2rpm code to allow automatically generating vendored Go package specfiles. I look forward to your feedback.

***

If you have read this far, thank you! Any feedback about the Docker stack or Go vendoring overall is welcome.

Best,

Maxwell


[0] https://gtmx.me/blog/fedora-go-unbundling-is-broken/

[1] https://git.sr.ht/~gotmax23/docker-ng

[2] https://docs.fedoraproject.org/en-US/packaging-guidelines/Golang/#_bundled_or_unbundled

[3] https://fedora.gitlab.io/sigs/go/go-vendor-tools/

[4] https://fedora.gitlab.io/sigs/go/go-vendor-tools/scenarios/#security-updates

[5] https://gitlab.com/fedora/sigs/go/go2rpm/-/merge_requests/4

-- 
Maxwell G (@gotmax23)
Pronouns: He/They