On Fri, Oct 12, 2018 at 4:30 AM, Marek Marczykowski-Górecki
On Thu, Oct 11, 2018 at 09:24:08PM -0600, Chris Murphy wrote:
> I'm pretty sure the original reason was the default live
> dd to block copy the root file system into the fedora-root LV, and
> then resized the LV and ext4 file system.
How is it done now?
On Live media installs, anaconda does:
rsync -pogAXtlHrDx --exclude /dev/ --exclude /proc/ --exclude /sys/
--exclude /run/ --exclude /boot/*rescue* --exclude /etc/machine-id
On DVD and netinstalls, I'm guessing based on packaging.log that it's
a dnf+rpm installation even though I never see a dnf or rpm process in
either top or ps. In any case, the rpm packages are directly on the
iso9660 file system, not baked into the
> Why does efiboot.img have a 32MiB limit?
Because "32MB should be enough for everybody"...
Long story short, "El Torito" boot catalog structure have 16-bit field
for image size (expressed in 512-bytes sectors). For details see here:
OK. On Fedora 28 media, efiboot.img is ~9.2 MiB and does not contain
either the kernel or initramfs. The kernel and initramfs are found on
the iso9660 file system at images/pxeboot/ and also at isolinux/ where
GRUB UEFI uses the former, and isolinux BIOS uses the latter. Both
initrd's are 65M so they're already too big to go into bootefi.img -
and they kinda need to be because this particular initramfs is built
by dracut with --nohostonly flag so that hopefully we can boot
anything. (Curiously, the initramfs is 65M on DVD/netinstall and 50M
on LiveOS - I don't have an explanation for that. I'm looking at
Fedora 28 release images.)
From my understanding, efiboot.img only would need to contain shim,
grubia32, grubx64 and supporting bootloader only files.
BTW, trivia: Fedora's installer creates EFI System partitions that are
always FAT16. So far as I know, no computer has complained, only
humans. FAT12/16 is OK for removable media but the spec pretty clearly
expects FAT32 for ESPs on permanent installs. The installer team
doesn't want to use mkfs flags, they expect the defaults to work
unless they don't work, and they do work, so FAT16 it is.
I've spent a lot of time debugging this, because mkisofs doesn't
complain about it, just silently overflow higher bits to adjacent field,
which results in weird results, depending on where you boot it. Adding
isohybrid to the picture doesn't make it easier (there, higher bits are
truncated, or actually not copied to the MBR partition table, as wasn't
part of the original field).
I think we're stuck with isohybrid for a while. Having UEFI and BIOS
bootloaders, along with isohybrid supporting both as well as Macs, all
on one media image, that can be burned to optical media and written to
a USB stick - is hugely beneficial.
The compose process takes about 12 hours. That every ISO for all the
editions, and the spins, and the VM images, for all archs. Even having
separate UEFI and BIOS images, or splitting out Macs with their own
image, it'll increase compose times and complexity across the board.
I'm not sure which happens first: the end to optical media booting
support; or dropping support for BIOS and/or old Apple EFI Macs (only
this year did they start using UEFI, rather than their own variant of
Intel EFI pre-UEFI, so it'll take some time to see how that shakes out
which also involves whether and how Secure Boot can ever be supported
This talks a bit about isohybrid and all the very clever hacks
involved to make Fedora boot practically anything with a single ISO
9660 image. (I'm being x86_64 arch specific when I say that.)
> I did give all of these things some thought a long time ago when I ran
> into a lorax hack by Will Woods who used Btrfs as the root.img file
> system, I'm not sure why it was used. But it gave me the idea of using
> a few features built into Btrfs specifically for this use case:
> - seed/sprout feature can be used with zram block device for volatile
> overlay; and used with a blank partition on the stick for persistent
> overlay. Discovery is part of the btrfs kernel code.
> - Since metadata and data is always checksummed on every read, we
> wouldn't have to depend on the slow and transient ISO checksum
> (rd.live.check which uses checkisomd5) which likewise breaks when
> creating a stick with livecd-iso-to-disk.
> - Btrfs supports zstd compression. I did some testing and squashfs is
> still a bit more efficient because it compresses fs metadata, whereas
> Btrfs only compresses data extents.
> The gotcha here is the resulting image isn't going to be bit for bit
> reproducible: UUIDs and time stamps are strewn throughout the file
> system (similar to ext4 and XFS), but any sufficiently complex file
> system is going to have this problem.
I wouldn't worry about _files_ timestamps that much - in most cases this is
solvable problem by elaborate enough find+touch. But that's not all
obviously, there are various timestamps in superblock, and other
metadata. The most problematic part in "normal" filesystems, using
kernel driver is inode allocation, block allocation etc. This greatly
depends on timing, ordering, specific kernel version etc.
See  for details.
mkfs.btrfs has --rootdir and --shrink features to pre-allocate a
volume with files at mkfs time; I have no idea to what degree it
depends on kernel code. The main benefit with this is it's really easy
to implement full checksum matching for metadata and data on every
read, and user space ends up with EIO instead of corrupt data, and
super clear kernel complaints. And such corruption whether on optical
or USB sticks, is common. Even the more rare case of a stick that
passes md5 checksum, can later have transient and silent corruption
that ends up showing up in weird ways.
It's plausible squashfs could implement this, I think by default it
already checksums every file to look for duplicates, but it doesn't
retain the per file hash for integrity checking later on. It's also
possible with dm-verity or dm-integrity but then that adds back the dm