Cool. A few questions inline...
On Mon, Dec 21, 2020 at 11:28:51AM -0500, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/RPMCoW
== Summary ==
RPM Copy on Write provides a better experience for Fedora Users as it
reduces the amount of I/O and offsets CPU cost of package
decompression. RPM Copy on Write uses reflinking capabilities in
btrfs, which is the default filesystem in Fedora 33.
What happens if you enable this on non btrfs installs?
Does it just not work gracefully? Does it fail somehow?
I think we need to be sure it doesn't actually do anything bad for other
non CoW filesystems.
...snip...
### Signature 8 bytes at the end of the file, used to differentiate
between traditional RPMs and extent based.
So, there's no change to rpm building or signing, as all thats done in
transcoding them on download?
=== Notes ===
# The headers are preserved bit for bit during transcoding. This
preserves signatures. The signatures cover the main header blob, and
the main header blob ensures the integrity of data in two ways:
## Each file with content has a digest. Originally this was md5, but
today it’s usually sha256. In normal RPM this is only used to verify
the integrity of files, e.g. <code>rpm -V</code>. With CoW we use this
as a content key.
## There is/are one or two digests (<code>PAYLOADDIGEST</code> and
<code>PAYLOADDIGESTALT</code>) covering the payload archive
(compressed cpio). The header value is preserved, but transcoded RPMs
do not preserve the original structure so RPM’s pre-installation
verification (controlled by <code>%_pkgverify_level</code> will fail.
<code>dnf-plugin-cow</code> disables this check in dnf because it
verifies the whole file digest which is captured during
Could rpm learn about this and still do it's verify in this case?
download/transcoding. The second one is likely used for delta rpm.
# This is untested, and possibly incompatible with delta RPM (drpm).
The process for reconstructing an rpm to install from a delta is
expensive from both a CPU and I/O perspective, while only providing
marginal benefits on download size. It is expected that having delta
rpm enabled (which is the default) will be handled gracefully.
I imagine drpms could still be used, just once you have constructed the
final rpm, you transcode it as if you just downloaded it?
But in general perhaps we should decide how much value drpms provide
these days and either make sure we are making more of them, or drop
them.
...snip...
=== Performance Metrics ===
Ballpark performance difference is about half the duration for file
download+install time. A lot of rpms are very small, so it’s difficult
to see/measure. Larger RPMs give much clearer signal.
(Actual numbers/charts will be supplied in Jan 2021)
Nice!
kevin