Martin Pitt wrote:
Ludwig Nussel [2020-09-25 13:25 +0200]:
> So far the cockpit packages we have for openSUSE Tumbleweed use the
> upstream release tarball. That means all web assets are prebuilt.
Right, that's how we use them as well.
> Therefore cannot be patched, it's almost like bundling binaries.
Indeed we avoid that -- what we do for RHEL is to have stable branches
like
backport
fixes -- then they can benefit from the full CI machinery, and get
landed with
confidence. At an appropriate time we then can just `make dist`, or
even tag
them.
We actually did try applying patches in the past, but it was such a
mess and we
could never be confident that everything *really* worked.
Yes, I'm sure anyone who wants to actively develop and support
cockpit on some stable code base would create a similar setup around
a github fork.
If however I'm just a random downstream distro hacker and want to
adjust something for my own flavor, I probably wouldn't want to
setup all this machinery for each and every package I want to change
e.g some documentation link or color scheme.
Also, some products that could be candidates to have cockpit
included may still be in use when our names are long engraved on
tombstones already. So while using an additional, external CI system
to feed the actual distro build system works today, the workflow should
be prepared to work without. So plain old src.rpm should work for
rebuilding within the distro. Whether or not the poor souls that have to
do that in the future care about a bundle approach like cockpit-cache or
original sources I don't know. To me the pristine sources way looks like
the cleaner one, even if that means more effort to set up :-)
> In openSUSE a package build is fully unattended, automatic and
has
> no network access.
FTR, that's the same in Fedora/RHEL/Debian/Ubuntu.
> That means all that nodejs stuff needs to be available. When talking
to other
> developers that touched Nodejs before, they just recommended to
tar up
> node_modules/. That's not acceptable either IMO as it also would be
just
some
> magic tarball that we don't really know how it was created.
It's not any more magic than building the main release tarball -- on
https://github.com/cockpit-project/cockpit/releases we publish the
cockpit-cache-VERSION.tar.xz tarballs, so you *could* just include
them into
your source distribution packages and use them to build.
That's also just the node_modules directory. It does not contain
pristine upstream sources. That tree likely even contains binaries.
Probably ones that you didn't build yourself. IIRC it is sass that
tries hard use a binary downloaded from github for example.
> So I wrote a PoC tool that parses package-lock.json instead.
With
> that information we know what upstream nodejs modules npm decided to
> use and where to put them in node_modules/.
Note that `npm install` does exactly that already -- If a
package-lock.json is
present, it will reproduce the exact versions. That's why we
include
package-lock.json into our release tarballs, so that in situations with
internet access the entire build is reproducible.
`npm install` also does not re-download already existing packages.
However, it
is still not fully offline, as it will always check the
npmjs.com
package index
despite package-lock.json. Maybe that can be avoided somehow -- but
in
such a
case, just grabbing cockpit-cache*.tar.xz seems much easier?
Sure it would be great if npm itself would offer features to print the
url of all locations it wants to fetch as well as options to get those
files fully offline from a local directory.
I'm not a JS developer (yet ;-)) though so it was easier to quickly
hack those python and shell bits than "npm install".
>
https://build.opensuse.org/package/show/home:lnussel:branches:systemsmana...
>
>> The madness comes from the huge amount of
tarballs. Apart from that
>> looks quite straight forward, doesn't it? :-)
>
> Heh, yes -- in spirit that's kind of an
npmjs.com mirror. Are you
afraid of
license problems in the case
npmjs.com is going down? I can see the
need for
keeping all the original sources at the same downstream location.
(Again,
cockpit-cache* seems easier to me, but this is certainly a valid
approach).
I didn't even think of license problems yet but the ability to track
each tarball separately certainly helps the license reviewers, esp
when those components are used in several packages.
> Also, cockpit-podman wants to use bits of cockpit
> (pkg/lib/patternfly) and the Makefile gets that from git if not
> present.
Right, but that's just for upstream git. The `make dist-gzip` rule
explicitly
includes these files, so the are part of the released tarballs (I
just
validated cockpit-podman-23.tar.gz).
> So I had to create a cockpit-devel subpackage that can be consumed by
> cockpit-podman. Would that be something to consider for upstream too? Ie
> install those files and probably a pkgconfig file in cockpit, then
check for
> that in the Makefiles of modules?
Honestly, I find our current approach of checking out a validated tag
so much
simpler to maintain. For upstream developing against git you
absolutely need
internet, and for building packages out of release tarballs it will
only
complicate the packaging a lot. We really don't give *any* API
guarantees
about
these, and these files are not at all a product.
That's fine. Part of the exercise is building from a git tag resp
master rather than the release tarball. I could also solve that by
making OBS pull from several git repos.
cu
Ludwig
--
(o_ Ludwig Nussel
//\
V_/_
http://www.suse.com/
SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer
HRB 36809 (AG Nürnberg)