On Fri, 2021-04-02 at 10:37 -0700, Kevin Fenzi wrote:
On Fri, Apr 02, 2021 at 05:19:15PM +0100, Jonathan Dieter wrote:
> Right now, we're not using zdicts for the F34 zchunk metadata because
> they were only added in fedora-repo-zdicts-2103.1-2 (which should now
> be in the updates repo in all current Fedora releases).
>
> If we could update fedora-repo-zdicts to 2103.1-2 on whichever
> servers
> generate the metadata (preferably before the 34 GA metadata is
> generated), that should significantly reduce the size of the
> metadata.
Which metadata are you looking at?
bodhi-backend01 has fedora-repo-zdicts-2103.1-2.fc32.noarch
compose-branched does not have that package installed.
But it should be using the one in the chroot right?
I'm not sure. Looking at the zchunk metadata from the latest compose,
there's no dictionary (Chunk 0 is 0 length):
$ zck_read_header -c
8ec4565f29e273b132baee435e7e3ef6cdae866ba9c9811f87caa5bc9a1f112a-primary.xml.zck | head -n
11
Overall checksum type: SHA-256
Header size: 512869
Header checksum: bcc8e5e5d152d6e7e0036a1b9e2661633ebd5e33cab55a976ca7baab3f1f4e1d
Data size: 29656818
Data checksum: eea9776727a3b5a9b2232f78d91cc8ef2104622ff5fc5a9785ce2f218b5b2dcb
Chunk count: 25580
Chunk checksum type: SHA-512/128
Chunk Checksum Start Comp size Size
0 00000000000000000000000000000000 512869 0 0
1 cfb57636818534e317930134427b8e4b 512869 122 160
Looking at the updates-testing metadata, there is a dictionary:
$ zck_read_header -c
df4f7320d337a2d90594e1c8a91bcd2448d18854a57753ee2f83edbeb9880ace-primary.xml.zck | head -n
11
Overall checksum type: SHA-256
Header size: 57227
Header checksum: 00e2c3bc5b9413e39d109f9d121a38e41076113df9113f02fbea43bfc950b99b
Data size: 3969888
Data checksum: 94ad299653a3b8f6d5428cbd2115579d38e435de86e9d280dc08dccb4ef46ae3
Chunk count: 2841
Chunk checksum type: SHA-512/128
Chunk Checksum Start Comp size Size
0 a130547b7e9263e4722be934af8d1552 57227 29904 112640
1 450a0b82e6b3499ec7996832bc091af6 87131 102 160
The package needs to be available in whatever environment createrepo_c
is run from, so if we're running createrepo_c from a chroot, that's
where the package should be.
I don't know if it helps, but looking at
https://kojipkgs.fedoraproject.org/compose/branched/Fedora-34-20210401.n....
it looks like the dictionaries aren't available during the compose.
If there's anything else I can do to help troubleshoot, please let me
know.
Thanks,
Jonathan