Why are COPR ARM builders so slow?
by Neal Gompa
Hey all,
Lately, I've been using the COPR ARM builders a fair bit: I've been
working on Fedora Asahi stuff with Davide Cavalca for the past several
months, and I've noticed that builds in COPR are easily at least 2x
slower than their equivalent builds in Koji, most of the time
considerably worse.
For a concrete example: I've been building kernel-asahi in
ngompa/fedora-asahi and ngompa/fedora-asahi-dev COPRs for some time
now, and each kernel build is basically 3 hours. Most recently, I
built a kernel based on 6.1-rc5
(https://copr.fedorainfracloud.org/coprs/ngompa/fedora-asahi-dev/build/504...)
that took 3 hours. The equivalent build in Koji
(https://koji.fedoraproject.org/koji/taskinfo?taskID=94172035) took
less than 1.25 hours.
I tried earlier today to build QtWebEngine in COPR for testing a fix,
and it couldn't complete in the 5 hour time limit it has by default. I
eventually did a build locally (which took about an hour to build) and
validated it before pushing it to Koji
(https://koji.fedoraproject.org/koji/taskinfo?taskID=94372924), which
took a little under 3.15 hours.
I thought we had access to super-fast Graviton-based builders for
COPR. Our builds should be *flying* compared to Koji or my local
resources. So what's going on?
--
真実はいつも一つ!/ Always, there's only one truth!
1 year, 3 months
Re: Using Pungi instead of Mash
by Tomas Kopecek
Maybe misleading, but isn't koji's dist-repo command sufficient here?
It looks to me that you're not doing any sophisticated rpm list
preparation, nor using multilib, etc. So, maybe using just koji could
work for you?
On Mon, Nov 7, 2022 at 12:14 PM Marta Vila Fernandes
<marta.vila.fernandes(a)cern.ch> wrote:
>
>
> Hello,
>
> I currently use Mash to generate repositories from Koji tags and I'd like to move to something that isn't abandoned.
> Pungi is recommended by the Koji documentation, but I can't get it to produce the same thing Mash does now.
> I can get it to generate repo metadata for a given tag, but it doesn't copy the RPMs themselves.
>
> The Pungi version that I'm using is pungi-4.1.38-1.el8.2.noarch and the configuration:
>
> # RELEASE
> release_name = "mytag8s-testing"
> release_short = "mytag8s-testing"
> release_version = "1.1.17"
>
> # GENERAL SETTINGS
> variants_file = "variants.xml"
> tree_arches = ["x86_64", "aarch64"]
> tree_variants = ["os", "debug"]
>
> # CREATEREPO
> createrepo_c = True
> createrepo_database = True
> createrepo_checksum = "sha256"
>
> # KOJI
> koji_profile = "kojitest"
> runroot_method = "koji"
> runroot_tag = "mytag8s-testing"
>
> # PKGSET
> sigkeys = ["..."]
> pkgset_source = "koji"
> pkgset_koji_tag = "mytag8s-testing"
>
> # GATHER
> gather_method = "nodeps"
> check_deps = False
>
> # OTHER SETTINGS
> skip_phases = ['buildinstall', 'createiso', 'extra_files', 'extra_isos', 'image_build', 'image_checksum', 'live_images', 'live_media', 'osbs', 'ostree_installer', 'productimg', 'test', 'ostree']
>
> with variants:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE variants PUBLIC "-//CentOS//DTD Variants info//EN" "variants.dtd">
> <variants>
> <variant id="os" name="os" type="variant">
> <arches>
> <arch>x86_64</arch>
> <arch>aarch64</arch>
> </arches>
> </variant>
> <variant id="debug" name="debug" type="variant">
> <arches>
> <arch>x86_64</arch>
> <arch>aarch64</arch>
> </arches>
> </variant>
> </variants>
>
> I'm calling pungi-koji: # pungi-koji --config pungi.config --target-dir "/staging/" --no-label
>
> This configuration creates the repositories, but doesn't populate them with the packages.
>
> During the pkgset phase:
>
> 1) First it will get the name of the rpms from koji via koji tag - via koji api (listTaggedRPMS)
>
> 2) Then it checks if these packages exists in the /mnt/koji path. If it exists, a dictionary with that info is created.
> The path is the default topdir in koji.conf - It can be changed, but there is a validation to ensure that the directory exists. There is no option to use the topurl and takes this info from koji on that stage.
>
> 3) Then it will create the repositories for each arch / variant.
>
> During the gather phase:
> 1) It creates a rpm.json with a empty rpm list
> 2) It doesn’t copy the rpms from /mnt/koji to the new repositories.
>
>
> Could you please help me to understand if I'm doing something wrong or what I can change in the configuration / variants file to make it work?
>
> Kind regards,
> Marta Vila Fernandes
> _______________________________________________
> buildsys mailing list -- buildsys(a)lists.fedoraproject.org
> To unsubscribe send an email to buildsys-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/buildsys@lists.fedoraprojec...
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
--
Tomas Kopecek <tkopecek(a)redhat.com>
RHEL Build Development, RedHat
1 year, 4 months
Using Pungi instead of Mash
by Marta Vila Fernandes
Hello,
I currently use Mash to generate repositories from Koji tags and I'd like to move to something that isn't abandoned.
Pungi is recommended by the Koji documentation, but I can't get it to produce the same thing Mash does now.
I can get it to generate repo metadata for a given tag, but it doesn't copy the RPMs themselves.
The Pungi version that I'm using is pungi-4.1.38-1.el8.2.noarch and the configuration:
# RELEASE
release_name = "mytag8s-testing"
release_short = "mytag8s-testing"
release_version = "1.1.17"
# GENERAL SETTINGS
variants_file = "variants.xml"
tree_arches = ["x86_64", "aarch64"]
tree_variants = ["os", "debug"]
# CREATEREPO
createrepo_c = True
createrepo_database = True
createrepo_checksum = "sha256"
# KOJI
koji_profile = "kojitest"
runroot_method = "koji"
runroot_tag = "mytag8s-testing"
# PKGSET
sigkeys = ["..."]
pkgset_source = "koji"
pkgset_koji_tag = "mytag8s-testing"
# GATHER
gather_method = "nodeps"
check_deps = False
# OTHER SETTINGS
skip_phases = ['buildinstall', 'createiso', 'extra_files', 'extra_isos', 'image_build', 'image_checksum', 'live_images', 'live_media', 'osbs', 'ostree_installer', 'productimg', 'test', 'ostree']
with variants:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE variants PUBLIC "-//CentOS//DTD Variants info//EN" "variants.dtd">
<variants>
<variant id="os" name="os" type="variant">
<arches>
<arch>x86_64</arch>
<arch>aarch64</arch>
</arches>
</variant>
<variant id="debug" name="debug" type="variant">
<arches>
<arch>x86_64</arch>
<arch>aarch64</arch>
</arches>
</variant>
</variants>
I'm calling pungi-koji: # pungi-koji --config pungi.config --target-dir "/staging/" --no-label
This configuration creates the repositories, but doesn't populate them with the packages.
During the pkgset phase:
1) First it will get the name of the rpms from koji via koji tag - via koji api (listTaggedRPMS)
2) Then it checks if these packages exists in the /mnt/koji path. If it exists, a dictionary with that info is created.
The path is the default topdir in koji.conf - It can be changed, but there is a validation to ensure that the directory exists. There is no option to use the topurl and takes this info from koji on that stage.
3) Then it will create the repositories for each arch / variant.
During the gather phase:
1) It creates a rpm.json with a empty rpm list
2) It doesn’t copy the rpms from /mnt/koji to the new repositories.
Could you please help me to understand if I'm doing something wrong or what I can change in the configuration / variants file to make it work?
Kind regards,
Marta Vila Fernandes
1 year, 4 months