On Wed, Nov 27, 2019 at 12:18 PM Neal Gompa <ngompa13(a)gmail.com> wrote:
On Wed, Nov 27, 2019 at 2:12 PM Chris Murphy <lists(a)colorremedies.com> wrote:
> The order of work needed:
> A. Upstream squashfs needs zstd support merged. There's patches
> Fedora's squashfs-tools are carrying that add this support. But it's
> probably fair to say this is for testing purposes, because upstream
> squashfs may have a different implementation in mind. I'm not sure of
> the status of this.
squashfs-tools v4.4 has it included. The project moved to GitHub last
Ahh yes it does, cool. And the change log shows reproducible builds
support as well.
> B. Koji needs to learn about existing support for plain squashfs images in Lorax
> C. Releng needs to update build scripts to create plain squashfs images
livecd-tools probably needs that too...
It might need logic to handle a plain squashfs image, yes. I think
right now it expects to find a nested ext4.
> D. Releng needs to decide whether to use zstd instead of xz, and then
> koji needs to support it, but before that A. above must happen.
> I floated this idea to the Btrfs list. The discussion explores Btrfs
> and alternatives. A Btrfs approach is more work and coordination, flat
> out. But also offers more features for free: always on metadata and
> data checksumming could obviate the slow monolithic md5 ISO media
> checker; simple, consistent, transparent overlay for LiveOS (either
> transient in-memory, or persistent on-drive), seed/sprout fast
> replication option. All of that support is in-kernel so you don't need
> a sophisticated initramfs to do such assembly on the client, or
> complicated build system to create such images. There is a lot of
> *other* work to get there, but then I think it's a lot saner, less
> fragile, and a lot more consumable across distributions. Could that be
> mimicked with plain squashfs on dm-verity? Sure. And that's also
> mentioned in this thread.
I'd love to explore using Btrfs for doing it. I have no idea how to
get started with that...
Before Btrfs specific, I'd ask how much compression configurability is
needed in the compose system and where? That relates to plain squashfs
images as well as hypothetical Btrfs images. Is there any advantage to
having Rawhide use zstd:3 since these nightlies are throw away? These
would be much faster to create and use than xz or zstd:16, but the ISO
size would be a lot bigger, maybe 25% bigger. And then for branched
nightlies, RCs, and releases, use zstd:16 or even zstd:19? Could the
granularity be compress=low,medium,high? And that is then translated
by lmc or even the installer, into the specific level numbers? This is
in effect the question I'm posing here:
Btrfs specifically, you'd probably want the installer to learn how to
enable Btrfs compression, at least in kickstarts, so that it's
possible to create an ISO with a highly compressed Btrfs rootfs image
rather than nesting it in squashfs. And then that support has to go
all the way up: lorax, lmc, kojis - just like the plain squashfs
Next, create a Btrfs spin predicated on exactly cloning either Fedora
Workstation or Silverblue. The two differences would be: ISO contains
a Btrfs rootfs, and the installer default/autopartitioning uses Btrfs.
The first instance of such a spin would make Btrfs almost incidental
and pointless, but from there it could be tweeked in straightforward
fashion to take advantage of Btrfs specific things. And beyond that
I'd say we need a separate thread. :-D
I just thought of a cute wrench to just cavalierly throw in the
general direction of all that I just wrote: What if this future image
isn't based on ISO? What if it assumes it's going to be used only for
USB sticks and PXE? Do things get simper and more reliable? At what
expense? And what if it assumes Fedora CoreOS as the base?