----- Mail original -----
De: "Owen Taylor"
Hi Owen,
Is there a guide for Fedora packagers about how to handle unbundling
for
golang packages? The draft guidelines don't seem to go into any details.
I don't think there is, nor that it is necessarily needed. The posted guidelines
should be sufficient technically, there are no magic I know of I didn't document, the
rest is just a lot of work (but see ↓)
I've looked at packaging a few golang packages unbundled, and
have
immediately run into:
A) lots of unpackaged dependencies
B) dependencies that are packaged in Fedora with different, often much
older versions
Yes the state of Go packages in Fedora is pretty sad right now. I wouldn't expect
anyone to be able to package anything but the most trivial app in unbundled mode. Too many
common parts are missing, when they are not missing they are too old, trying to update an
existing Go package is an exercise in frustration (too much package-specific shell code,
that is difficult to understand and does not really work with new code versions), and
trying to update or add missing parts just reveals more breakage and work to be done.
However, accepting to package complex Go apps in bundled mode is how Fedora got to this
state in the first place. In plain speak, bundling (vendoring in Go linguo) just does not
scale. You need an awful lot of manpower to audit the hundreds of other projects each app
bundles, bundling removes all the package tooling that may help you to do so, and the
result is not shared with any other package, or with other versions of the same package,
so you get zero positive network effects. Worse, big bundled apps that do not actually try
to work in unbundled mode, do not actually test the code they export (they are bundling,
remember) so the result is toxic to small Go packages that try to work with them.
So bundling parts is a direct road to bundling everything, bundling everything is a direct
road to bundling everything blindly, bundling everything blindly is error-prone and
dangerous (because upstreams are only human and do make lots of mistakes), and pretty much
removes any value Fedora can add to end users, to other parts of Fedora that would like to
integrate with Go software, or to the upstream projects themselves (no Fedora QA, no
stream of Fedora-originated fixes, no Fedora pressure to stabilize parts when upstream is
lost in tunnel effect mode and does not realize that it is wasting everyone's time
starting with its own).
Therefore, trying to get all this it a better state, requires the following steps IMHO:
1. completely refactor our Go packaging style it's less painful to update Go packages
and they do not need a packager with deep knowledge of package-specific shell glue. That
takes documentation, and factoring out common Go packaging functions in shared rpm code
(macros and autodeps) → that's what I posted
2. using the new documentation and tooling to clean up years of Fedora technical debt, and
create a new set of up-to-date Go packages that can serve as new baseline → I have
hundreds of specs that I'm waiting for step 1 to complete to submit. They won't
constitute a full baseline by themselves, they're not all 100% done, it's too much
work to do alone but working with others requires the common macros and documentation to
be merged and adopted. This step is going to be painful I'm afraid, Fedora dug itself
a deep hole, leaving it is going to be hard.
3. hopefully, the result will be streamlined enough it won't be too painful to keep up
to date, having an up to date baseline will help attract new Go packagers like you,
everyone will benefit and be happy. We had to package some ostree bits for example to
create this baseline, and I'm pretty sure ostree people within Fedora would prefer to
maintain those themselves, if the rest of the Fedora go universe didn't make it too
hard
B) more
intimidating. It seems like to package the new package, I have to get the
maintainer of the library to upgrade the version,
The guidelines and automation aim at making upgrading easy, and avoid one package or
packager blocking others
and someone needs to
rebuild everything that depends on the rebuilt library and test that the
rebuilt packages work.
I hope that normalizing Fedora Go packages means it will be possible to automate QA tests
such as "try to rebuild all the packages that depend on golang(foo) every time you
see the package providing golang(foo) changing, and report what broke"
That would be expensive computing-side but a lot less expensive and long than expecting
each Go packager to do it himself with his own means. And that is certainly not overkill,
given how lax Go projects are about maintaining API stability.
And then in case of breakage, revert or create a compat package. That's why there is a
long chapter dedicated to compat package creation in the proposed guidelines.
Regards,
--
Nicolas Mailhot