----- Original Message -----
From: "nicolas mailhot"
<nicolas.mailhot(a)laposte.net>
To: golang(a)lists.fedoraproject.org
Cc: "Development discussions related to Fedora"
<devel(a)lists.fedoraproject.org>, "Discussion of RPM packaging
standards and practices for Fedora" <packaging(a)lists.fedoraproject.org>
Sent: Thursday, February 1, 2018 2:51:13 PM
Subject: Re: <DKIM> [Fedora-packaging] Re: Proposed Fedora packaging guideline:
More Go packaging
De: "Jakub Cajka"
Hi Jakub,
> It depends (as everything) on available manpower, if you are willing to own
> your dependencies
> you can package anything and everything debundled.
Sure, but available manpower depends on how high the bar you put for people
wanting to join, and right now this bar is pretty high.
> On contrary, to this state you dislike(or seems to) Fedora got because of
> effort of many to de
> bundle Go based projects.
I'm *not* disparaging the past efforts of many. They did a lot of good.
Unfortunately, the result still falls short of the mark. The baseline is not
complete enough, the baseline is old and aging fast, would-be Go packagers
like Owen are ready to give up when they start getting an idea of the Go
state in Fedora.
It's time to invest in tooling and simplification so the few Go Fedora
packagers there is, can do more with less, so casual Go packagers can
contribute their little bit instead of being repulsed by complexity that can
be avoided and factored out, so people already familiar with Fedora
packaging can package the Go bits they need without needing a lengthy
formation on Go specifics.
There *are* lots of people that would like to do Go packaging. Many Go
software projects currently make headlines. That there are so few Go
packagers in Fedora, at the time Go is in the limelight, is a pretty good
indicator we are making it too hard.
> I don't see way how it makes it less painful(you have still the whole
> distro issue). It makes specs
> more abstract and streamlined when it works, but more opaque and hard to
> read when its not(and you
> have to debug some arcane srpm/lua macros).
It makes it less painful because there is no package-specific magic in each
and every Go spec file.
You do not need to debug code in each spec because code that already just
works for hundreds of packages is unlikely to not work on the next spec
file.
You do not need to comb each and every line of a Go spec file wondering if
the line is cut and pasting of a common template (multiply by all past
revisions of common templates) or if it is package-specific. The common
parts are in common macros not spread right and left.
You do not need to worry wether common code is complete or not because
factoring out common code removes the temptation to document 90% of what's
needed, leaving others to fill in the missing 10%. 10% amounts to an
astonishing level of work when you multiply it by the number of Go packagers
and the number of Go packages we would ideally need.
Plus, when you do find a bug in common code, the fix is shared with the whole
distribution, not hidden in a single spec file.
That's what successful factoring of common code earns you.
And given I have more than 470 Go spec files that just work with the proposed
macros with no special package-specific adjustment needed, and most time I
compare one of those to the Fedora equivalent I'm executing more unit tests
and building more binaries, with drastically shorter spec code, I'm pretty
sure the factoring out *is* successful.
>> This step is going to be painful I'm afraid, Fedora dug itself a
>> deep hole, leaving it is going to be hard.
> On contrary Fedora is trying to fill the hole that upstream Go projects dug
> them selves in to.
Filling upstream holes is pretty much the definition of an
integrator/distributor role. In Go they are particularly numerous and deep,
but Fedora users do want their docker and kubernetes (and Kubernetes, BTW,
is astonishingly free of the problems that plague many Go projects, proving
it *is* possible to do good release engineering in Go).
??
Fedora did dug itself a hole by trying the bundling shortcut around EL7 time.
Where are the EL7 Go packages? Why are Fedora Atomic people not spearheading
Go packaging in Fedora? Waiting for bundling to be solved upstream-side made
Fedora worse not better.
I'm not sure that Fedora. Go ask them don't ask me or other regular Fedora
maintainers. Fedora haven't been bundling and has been doing as much as possible to
un-bundle. Fedora can't solve others non related problems.
> IMNHO Go have traded any subjectively perceived "RPM/deb hell" for even
> deeper levels of "Go
> vendor) hell".
I fully agree. What I meant to convey is that replicating upstream vendoring
Fedora-side only makes the situation worse for users, for upstreams, and for
Fedora.
> Pain will not be eased in any significant way as you will still have to
> carefully evaluate every
> change so you don't break any dependent package.
Pain *will* be eased in a significant way because we will have the tooling to
detect breakage automatically, and because fixes once they're done will be
packaged and shared.
Your proposal doesn't provide this kind of tooling, if I'm not mistaken.
>> The guidelines and automation aim at making upgrading easy, and avoid one
>> package or packager blocking others
> This is not really and automation per se. It is minor re-factoring and spec
> streamlining.
> It might or might not enable easier implementation of packaging automation.
If it's minor let's do it. I'm quite sure you will be pleasantly surprised
by
the level of pain those minor changes remove.
?? I'm in no way opposing your proposal per se.
>> That would be expensive computing-side but a lot less expensive and long
>> than
>> expecting each Go packager to do it himself with his own means. And that
>> is
>> certainly not overkill, given how lax Go projects are about maintaining
>> API
>> stability.
> Until upstreams will be working in true CI/CD manners(when they do not make
> any releases). Fedora
> can't really fix this.
Fedora can detect the breakage which is the first step to fix it. It does not
take an army of Go experts, just unbundled packages that can be rebuilt
automatically to identify incompatibilities. Go experts are better employed
working on fixes than doing rebuild checks that can be automated.
Rebuilds could be automated, analysis of them not easily.
> Currently AFAIK we have GoFed and accompanying projects that provide you
> with possibility to gather
> such data.
And nothing stops you for continuing to use those. Except, the golden test
will always be to actually try to build the code, which is something that
can be automated reliably once the dependencies exported to Fedora tools are
themselves complete and reliable, and Go packages are exported as rpm
packages, not hidden in private vendor trees.
?? I bit don't understand you. If you can, please help Jan to improve them.
>> And then in case of breakage, revert or create a compat package. That's
>> why
>> there is a long chapter dedicated to compat package creation in the
>> proposed
>> guidelines.
> Compat packages do not scale.
They scale a lot better than bundling, when an API break results in dozens of
projects choosing to vendor a pre-break commit. With each of them choosing a
slightly different state, resulting in ten times the amount of code to audit
and maintain.
> Imagine that you will have in worst case maintain 10s of different
> versions of same package for every release, doing security backports,
> etc....
When you do not unbundle you are still shipping the vendored code, you are
still responsible for its bugs (security or not). Compat packages do not
augment the code surface you have to care about, on the contrary they offer
you the possibility to *reduce* the number of code states in the
distribution.
Please don't put words in to my mouth. I haven't been talking about not
un-bundling, just compat packages.
JC
>
> Plus, they are the occasion, to notify upstreams they should move to a more
> recent code state. And reduce the number of code states you have to care
> about yet more. Not all upstreams are dead set against updating their
> vendoring. A lot of them are nice and responsible and just didn't see they
> had an API break to manage yet because their use of vendoring hid the
> problem.
>
> Of course that supposes that creating a compat package does not add a
> significant package-creation burden. Which I *did* try to avoid.
> Organization-wise Go will still need a form of exemption to create many many
> packages easily, because Go code is spread over many many projects, and not
> consolidated in a few big libraries like in the C world.
>
> Regards,
>
> --
> Nicolas Mailhot
> _______________________________________________
> golang mailing list -- golang(a)lists.fedoraproject.org
> To unsubscribe send an email to golang-leave(a)lists.fedoraproject.org
>