Hello,
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
https://pagure.io/releng/issue/9308#comment-703106
Basically, if/when Koji uses the bootstrap mock mode, as soon as we start bootstrapping (different meaning) Python in a side tag, dnf becomes temporarily uninstallable in that side tag and we can no longer do any builds.
This can be worked around by not using the boostrap mock option in Koji (ideally only for Python upgrade side tags, but I am not yet sure if that's possible), or by eliminating Python libraries out of the bootstrap chroot, hence the idea about using microdnf.
I've started with this config:
include('/etc/mock/fedora-rawhide-x86_64.cfg') config_opts['root'] = 'fedora-rawhide-microdnf' config_opts['package_manager'] = 'dnf' config_opts['dnf_command'] = '/usr/bin/microdnf' config_opts['dnf_install_command'] = 'install microdnf' config_opts['system_dnf_command'] = '/usr/bin/dnf' config_opts['dnf_common_opts'] = ['--allowerasing']
But it fails pretty soon with:
error: (--setopt) Unknown tsflag: nocontexts
And when I patch that option out, I still get:
error: The "--installroot" argument must be used together with "--config", "--noplugins", "--setopt=cachedir=<path>", "--setopt=reposdir=<path>", "--setopt=varsdir=<path>" arguments.
But I guess that if I figure the right options for dnf_common_opts, this might work...? Is there some crucial functionality that microdnf might be missing that would prevent it from creating mock chroots like this?
On 11/29/20 1:28 AM, Miro Hrončok wrote:
Hello,
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
https://pagure.io/releng/issue/9308#comment-703106
Basically, if/when Koji uses the bootstrap mock mode, as soon as we start bootstrapping (different meaning) Python in a side tag, dnf becomes temporarily uninstallable in that side tag and we can no longer do any builds.
This can be worked around by not using the boostrap mock option in Koji (ideally only for Python upgrade side tags, but I am not yet sure if that's possible), or by eliminating Python libraries out of the bootstrap chroot, hence the idea about using microdnf.
I've started with this config:
include('/etc/mock/fedora-rawhide-x86_64.cfg') config_opts['root'] = 'fedora-rawhide-microdnf' config_opts['package_manager'] = 'dnf' config_opts['dnf_command'] = '/usr/bin/microdnf' config_opts['dnf_install_command'] = 'install microdnf' config_opts['system_dnf_command'] = '/usr/bin/dnf' config_opts['dnf_common_opts'] = ['--allowerasing']
But it fails pretty soon with:
error: (--setopt) Unknown tsflag: nocontexts
And when I patch that option out, I still get:
error: The "--installroot" argument must be used together with "--config", "--noplugins", "--setopt=cachedir=<path>", "--setopt=reposdir=<path>", "--setopt=varsdir=<path>" arguments.
But I guess that if I figure the right options for dnf_common_opts, this might work...? Is there some crucial functionality that microdnf might be missing that would prevent it from creating mock chroots like this?
I guess I should have searched a bit better before asking the questiong.
I found the microdnf option in mock:
https://github.com/rpm-software-management/mock/issues/76 https://github.com/rpm-software-management/microdnf/pull/28
But using config_opts['package_manager'] = 'microdnf' still installs dnf into the bootstrap chroot, which is confusing. But I got around that with:
include('/etc/mock/fedora-rawhide-x86_64.cfg') config_opts['root'] = 'fedora-rawhide-microdnf' config_opts['package_manager'] = 'microdnf' config_opts['microdnf_command'] = '/usr/bin/microdnf' config_opts['microdnf_install_command'] = 'dnf-install microdnf distribution-gpg-keys' config_opts['system_dnf_command'] = '/usr/bin/dnf' config_opts['microdnf_common_opts'] = ['--allowerasing']
However, it still ends with:
Curl error (37): Couldn't read a file:// file for file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary [Couldn't open file /usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary]
Running `mock -r fedora-rawhide-microdnf --init` twice works around that problem, and it successfully crates the boostraop chroot, but then, trying to create the actual one, it fails with:
Start: microdnf install Directory /var/lib/mock/fedora-rawhide-microdnf/root doesn't look like it has an OS tree. Refusing.
On Sat, Nov 28, 2020 at 7:44 PM Miro Hrončok mhroncok@redhat.com wrote:
On 11/29/20 1:28 AM, Miro Hrončok wrote:
Hello,
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
https://pagure.io/releng/issue/9308#comment-703106
Basically, if/when Koji uses the bootstrap mock mode, as soon as we start bootstrapping (different meaning) Python in a side tag, dnf becomes temporarily uninstallable in that side tag and we can no longer do any builds.
This can be worked around by not using the boostrap mock option in Koji (ideally only for Python upgrade side tags, but I am not yet sure if that's possible), or by eliminating Python libraries out of the bootstrap chroot, hence the idea about using microdnf.
I've started with this config:
include('/etc/mock/fedora-rawhide-x86_64.cfg') config_opts['root'] = 'fedora-rawhide-microdnf' config_opts['package_manager'] = 'dnf' config_opts['dnf_command'] = '/usr/bin/microdnf' config_opts['dnf_install_command'] = 'install microdnf' config_opts['system_dnf_command'] = '/usr/bin/dnf' config_opts['dnf_common_opts'] = ['--allowerasing']
But it fails pretty soon with:
error: (--setopt) Unknown tsflag: nocontexts
And when I patch that option out, I still get:
error: The "--installroot" argument must be used together with "--config", "--noplugins", "--setopt=cachedir=<path>", "--setopt=reposdir=<path>", "--setopt=varsdir=<path>" arguments.
But I guess that if I figure the right options for dnf_common_opts, this might work...? Is there some crucial functionality that microdnf might be missing that would prevent it from creating mock chroots like this?
I guess I should have searched a bit better before asking the questiong.
I found the microdnf option in mock:
https://github.com/rpm-software-management/mock/issues/76 https://github.com/rpm-software-management/microdnf/pull/28
But using config_opts['package_manager'] = 'microdnf' still installs dnf into the bootstrap chroot, which is confusing. But I got around that with:
include('/etc/mock/fedora-rawhide-x86_64.cfg') config_opts['root'] = 'fedora-rawhide-microdnf' config_opts['package_manager'] = 'microdnf' config_opts['microdnf_command'] = '/usr/bin/microdnf' config_opts['microdnf_install_command'] = 'dnf-install microdnf
distribution-gpg-keys' config_opts['system_dnf_command'] = '/usr/bin/dnf' config_opts['microdnf_common_opts'] = ['--allowerasing']
However, it still ends with:
Curl error (37): Couldn't read a file:// file for file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary [Couldn't open file /usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary]
Running `mock -r fedora-rawhide-microdnf --init` twice works around that problem, and it successfully crates the boostraop chroot, but then, trying to create the actual one, it fails with:
Start: microdnf install Directory /var/lib/mock/fedora-rawhide-microdnf/root doesn't look like it has an OS tree. Refusing.
This is a legitimate bug with microdnf and deserves a bug report in Red Hat Bugzilla or GitHub.
The installroot support in microdnf should generally work, as I verified it for image builds with kiwi[1] this past week.
I think the next release of microdnf should have enough for the bootstrap phase (especially with the modularity stuff now sorted out), but using it in mock in all stages requires getting a builddep implementation[2].
[1]: https://bodhi.fedoraproject.org/updates/FEDORA-2020-9ce3983748 [2]: https://github.com/rpm-software-management/microdnf/issues/82
On Sunday, November 29, 2020 1:44:37 AM CET Miro Hrončok wrote:
On 11/29/20 1:28 AM, Miro Hrončok wrote:
Hello,
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
https://pagure.io/releng/issue/9308#comment-703106
Basically, if/when Koji uses the bootstrap mock mode, as soon as we start bootstrapping (different meaning) Python in a side tag, dnf becomes temporarily uninstallable in that side tag and we can no longer do any builds.
This can be worked around by not using the boostrap mock option in Koji (ideally only for Python upgrade side tags, but I am not yet sure if that's possible), or by eliminating Python libraries out of the bootstrap chroot, hence the idea about using microdnf.
I've started with this config:
include('/etc/mock/fedora-rawhide-x86_64.cfg') config_opts['root'] = 'fedora-rawhide-microdnf' config_opts['package_manager'] = 'dnf' config_opts['dnf_command'] = '/usr/bin/microdnf' config_opts['dnf_install_command'] = 'install microdnf' config_opts['system_dnf_command'] = '/usr/bin/dnf' config_opts['dnf_common_opts'] = ['--allowerasing']
But it fails pretty soon with:
error: (--setopt) Unknown tsflag: nocontexts
And when I patch that option out, I still get:
error: The "--installroot" argument must be used together with "--config", "--noplugins", "--setopt=cachedir=<path>", "--setopt=reposdir=<path>", "--setopt=varsdir=<path>" arguments.
But I guess that if I figure the right options for dnf_common_opts, this might work...? Is there some crucial functionality that microdnf might be missing that would prevent it from creating mock chroots like this?
I guess I should have searched a bit better before asking the questiong.
I found the microdnf option in mock:
https://github.com/rpm-software-management/mock/issues/76 https://github.com/rpm-software-management/microdnf/pull/28
But using config_opts['package_manager'] = 'microdnf' still installs dnf into the bootstrap chroot, which is confusing.
It is likely we'll have to fix Mock too, dunno. Because of the missing functionality in microdnf nobody had enough motivation to spend more time on the microdnf support in mock so far.
I'm just not sure it is a valid use-case to fall-back to microdnf in this case. The point of bootstrap was to install the final chroot by the **default** tooling from that chroot. Not only to absorb the "RPM" differences (like sqlite vs BD) but also to stop playing with the small nuances between 'yum' in RHEL vs. 'yum' in Fedora, or even 'dnf' in F33 vs. 'dnf' in F31.
Going with microdnf is a bit against that principle. At least as long as microdnf isn't guaranteed to e.g. calculate 100% compatible transactions with DNF, or as long as 'microdnf' is not the default Mock installer for Fedora chroots.
While it is less likely, I'm afraid that similar problems may happen with microdnf too (microdnf still depends on some libraries). So perhaps we we shouldn't install packages from the side-tag into the bootstrap?
Also note there's --use-bootstrap-image option that could help here.
Pavel
But I got around that with:
include('/etc/mock/fedora-rawhide-x86_64.cfg') config_opts['root'] = 'fedora-rawhide-microdnf' config_opts['package_manager'] = 'microdnf' config_opts['microdnf_command'] = '/usr/bin/microdnf' config_opts['microdnf_install_command'] = 'dnf-install microdnf
distribution-gpg-keys' config_opts['system_dnf_command'] = '/usr/bin/dnf' config_opts['microdnf_common_opts'] = ['--allowerasing']
However, it still ends with:
Curl error (37): Couldn't read a file:// file for file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary [Couldn't open file /usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary]
Running `mock -r fedora-rawhide-microdnf --init` twice works around that problem, and it successfully crates the boostraop chroot, but then, trying to create the actual one, it fails with:
Start: microdnf install Directory /var/lib/mock/fedora-rawhide-microdnf/root doesn't look like it has an OS tree. Refusing.
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ buildsys mailing list -- buildsys@lists.fedoraproject.org To unsubscribe send an email to buildsys-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/buildsys@lists.fedoraproject.o...
On 11/30/20 10:14 AM, Pavel Raiskup wrote:
On Sunday, November 29, 2020 1:44:37 AM CET Miro Hrončok wrote:
On 11/29/20 1:28 AM, Miro Hrončok wrote:
Hello,
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
https://pagure.io/releng/issue/9308#comment-703106
Basically, if/when Koji uses the bootstrap mock mode, as soon as we start bootstrapping (different meaning) Python in a side tag, dnf becomes temporarily uninstallable in that side tag and we can no longer do any builds.
This can be worked around by not using the boostrap mock option in Koji (ideally only for Python upgrade side tags, but I am not yet sure if that's possible), or by eliminating Python libraries out of the bootstrap chroot, hence the idea about using microdnf.
I've started with this config:
include('/etc/mock/fedora-rawhide-x86_64.cfg') config_opts['root'] = 'fedora-rawhide-microdnf' config_opts['package_manager'] = 'dnf' config_opts['dnf_command'] = '/usr/bin/microdnf' config_opts['dnf_install_command'] = 'install microdnf' config_opts['system_dnf_command'] = '/usr/bin/dnf' config_opts['dnf_common_opts'] = ['--allowerasing']
But it fails pretty soon with:
error: (--setopt) Unknown tsflag: nocontexts
And when I patch that option out, I still get:
error: The "--installroot" argument must be used together with "--config", "--noplugins", "--setopt=cachedir=<path>", "--setopt=reposdir=<path>", "--setopt=varsdir=<path>" arguments.
But I guess that if I figure the right options for dnf_common_opts, this might work...? Is there some crucial functionality that microdnf might be missing that would prevent it from creating mock chroots like this?
I guess I should have searched a bit better before asking the questiong.
I found the microdnf option in mock:
https://github.com/rpm-software-management/mock/issues/76 https://github.com/rpm-software-management/microdnf/pull/28
But using config_opts['package_manager'] = 'microdnf' still installs dnf into the bootstrap chroot, which is confusing.
It is likely we'll have to fix Mock too, dunno. Because of the missing functionality in microdnf nobody had enough motivation to spend more time on the microdnf support in mock so far.
I'm just not sure it is a valid use-case to fall-back to microdnf in this case. The point of bootstrap was to install the final chroot by the **default** tooling from that chroot. Not only to absorb the "RPM" differences (like sqlite vs BD) but also to stop playing with the small nuances between 'yum' in RHEL vs. 'yum' in Fedora, or even 'dnf' in F33 vs. 'dnf' in F31.
Going with microdnf is a bit against that principle. At least as long as microdnf isn't guaranteed to e.g. calculate 100% compatible transactions with DNF, or as long as 'microdnf' is not the default Mock installer for Fedora chroots.
Thanks, Pavel.
That makes perfect sense.
While it is less likely, I'm afraid that similar problems may happen with microdnf too (microdnf still depends on some libraries).
While technically yes, it's easier to manage around a single C library bootstrap than around dozen of Python packages with many more build dependencies.
So perhaps we we shouldn't install packages from the side-tag into the bootstrap?
Yes, that would be ideal.
Also note there's --use-bootstrap-image option that could help here.
Do you think we could use that in Koji?
On Monday, November 30, 2020 10:41:20 AM CET Miro Hrončok wrote:
So perhaps we we shouldn't install packages from the side-tag into the bootstrap?
Yes, that would be ideal.
So we'd have to have two repositories in Koji's mock config, e.g.:
[build] enabled=0 baseurl=... ....
[side-tag] enabled=1 baseurl=... ...
And the config would have to have something like: config_opts['dnf_install_command'] += " --disablerepo=side-tag --enablerepo=build"
It should make sure that DNF from [build] repo is installed.
We have similar hack [1] in mock-core-configs for the el6/7 chroots with scl repositories, because some historically shipped SCLs had broken Provides: that were breaking the bootstrap chroot install transaction.
[1] https://github.com/rpm-software-management/mock/blob/70ab0b923c9d65e29c11337...
Also note there's --use-bootstrap-image option that could help here.
Do you think we could use that in Koji?
Technically it should be possible. But AFAIK it's not yet widely tested feature (can be newly tested in Copr).
The question is though what podman image to use in Koji. If we use the default one (e.g. fedora:33 for fedora-33-x86_64), it may not be updated often/fast enough. Also I'm not sure if we have podman support for all our architectures.
Pavel
Dne 29. 11. 20 v 1:44 Miro Hrončok napsal(a):
I guess I should have searched a bit better before asking the questiong.
I found the microdnf option in mock:
https://github.com/rpm-software-management/mock/issues/76 https://github.com/rpm-software-management/microdnf/pull/28
But is is documented only in site-defaults.cfg. Point taken. I started https://github.com/rpm-software-management/mock/wiki/Feature-package-manager... and I will try to add more content to that.
Though, you already get the information from these PRs. :)
* Miro Hrončok:
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
Would it be possible to use rpm-ostree with a vendored RPM implementation? That should provide even better isolation.
Thanks, Florian
On Mon, Nov 30, 2020 at 6:57 AM Florian Weimer fweimer@redhat.com wrote:
- Miro Hrončok:
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
Would it be possible to use rpm-ostree with a vendored RPM implementation? That should provide even better isolation.
We're making this too complicated. The only real thing we have to do is use OCI images to populate the bootstrap environment rather than having DNF create them on the fly.
* Neal Gompa:
On Mon, Nov 30, 2020 at 6:57 AM Florian Weimer fweimer@redhat.com wrote:
- Miro Hrončok:
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
Would it be possible to use rpm-ostree with a vendored RPM implementation? That should provide even better isolation.
We're making this too complicated. The only real thing we have to do is use OCI images to populate the bootstrap environment rather than having DNF create them on the fly.
But these images have to come from somewhere, too. Their build process still needs to solve the bootstrap issue.
Thanks, Florian
On Mon, Nov 30, 2020 at 7:39 AM Florian Weimer fweimer@redhat.com wrote:
- Neal Gompa:
On Mon, Nov 30, 2020 at 6:57 AM Florian Weimer fweimer@redhat.com wrote:
- Miro Hrončok:
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
Would it be possible to use rpm-ostree with a vendored RPM implementation? That should provide even better isolation.
We're making this too complicated. The only real thing we have to do is use OCI images to populate the bootstrap environment rather than having DNF create them on the fly.
But these images have to come from somewhere, too. Their build process still needs to solve the bootstrap issue.
For our purposes, it solves the problem. The main issue for us is that since Koji does not support generating the bootstrap itself (this is being passed to Mock), you cannot control the content going into bootstrap creation. Copr has this issue too. But we don't need to solve that problem to solve Miro's issue.
On Mon, 30 Nov 2020 at 06:57, Florian Weimer fweimer@redhat.com wrote:
- Miro Hrončok:
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
Would it be possible to use rpm-ostree with a vendored RPM implementation? That should provide even better isolation.
koji has a very strong idea of what it wants inside a chroot and what it does not (and may never) want. Getting rpm-ostree or other items to work inside of the koji mock interface is probably looking at a large project (going from how many other 'simple' things have ended up being much more complicated than expected by the original proposers). My usual way of figuring out how to make anything work in koji is to see what the oldest environment it needs to work on by a customer and see how I would do it in RHEL-5 (now maybe RHEL-6).
Thanks, Florian -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill _______________________________________________ buildsys mailing list -- buildsys@lists.fedoraproject.org To unsubscribe send an email to buildsys-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/buildsys@lists.fedoraproject.o...
On Mon, Nov 30, 2020 at 7:41 AM Stephen John Smoogen smooge@gmail.com wrote:
On Mon, 30 Nov 2020 at 06:57, Florian Weimer fweimer@redhat.com wrote:
- Miro Hrončok:
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
Would it be possible to use rpm-ostree with a vendored RPM implementation? That should provide even better isolation.
koji has a very strong idea of what it wants inside a chroot and what it does not (and may never) want. Getting rpm-ostree or other items to work inside of the koji mock interface is probably looking at a large project (going from how many other 'simple' things have ended up being much more complicated than expected by the original proposers). My usual way of figuring out how to make anything work in koji is to see what the oldest environment it needs to work on by a customer and see how I would do it in RHEL-5 (now maybe RHEL-6).
Then that's easy: we define a list of packages that are needed to get the package manager working, and Koji would just download and unpack them with no script execution, then it would manually run some predefined scripts and invoke the package manager after that.
Aside from me doing this before to do RHEL 5 -> RHEL 7 upgrades by hand, this is also the strategy used by the openSUSE Build Service to work around this problem.
On Monday, November 30, 2020 1:44:14 PM CET Neal Gompa wrote:
On Mon, Nov 30, 2020 at 7:41 AM Stephen John Smoogen smooge@gmail.com wrote:
On Mon, 30 Nov 2020 at 06:57, Florian Weimer fweimer@redhat.com wrote:
- Miro Hrončok:
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
Would it be possible to use rpm-ostree with a vendored RPM implementation? That should provide even better isolation.
koji has a very strong idea of what it wants inside a chroot and what it does not (and may never) want. Getting rpm-ostree or other items to work inside of the koji mock interface is probably looking at a large project (going from how many other 'simple' things have ended up being much more complicated than expected by the original proposers). My usual way of figuring out how to make anything work in koji is to see what the oldest environment it needs to work on by a customer and see how I would do it in RHEL-5 (now maybe RHEL-6).
Then that's easy: we define a list of packages that are needed to get the package manager working, and Koji would just download and unpack them with no script execution, then it would manually run some predefined scripts and invoke the package manager after that.
Can you elaborate a bit more please? What is the practical difference between mock's bootstrap chroot and the manual technique you propose?
Pavel
Aside from me doing this before to do RHEL 5 -> RHEL 7 upgrades by hand, this is also the strategy used by the openSUSE Build Service to work around this problem.
-- 真実はいつも一つ!/ Always, there's only one truth! _______________________________________________ buildsys mailing list -- buildsys@lists.fedoraproject.org To unsubscribe send an email to buildsys-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/buildsys@lists.fedoraproject.o...
On Mon, Nov 30, 2020 at 8:01 AM Pavel Raiskup praiskup@redhat.com wrote:
On Monday, November 30, 2020 1:44:14 PM CET Neal Gompa wrote:
On Mon, Nov 30, 2020 at 7:41 AM Stephen John Smoogen smooge@gmail.com wrote:
On Mon, 30 Nov 2020 at 06:57, Florian Weimer fweimer@redhat.com wrote:
- Miro Hrončok:
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
Would it be possible to use rpm-ostree with a vendored RPM implementation? That should provide even better isolation.
koji has a very strong idea of what it wants inside a chroot and what it does not (and may never) want. Getting rpm-ostree or other items to work inside of the koji mock interface is probably looking at a large project (going from how many other 'simple' things have ended up being much more complicated than expected by the original proposers). My usual way of figuring out how to make anything work in koji is to see what the oldest environment it needs to work on by a customer and see how I would do it in RHEL-5 (now maybe RHEL-6).
Then that's easy: we define a list of packages that are needed to get the package manager working, and Koji would just download and unpack them with no script execution, then it would manually run some predefined scripts and invoke the package manager after that.
Can you elaborate a bit more please? What is the practical difference between mock's bootstrap chroot and the manual technique you propose?
This does not require dependency resolution to be verified, unless you use the dependency resolver in the COPR/Koji code to create the list (that's what OBS does).
-- 真実はいつも一つ!/ Always, there's only one truth!
On Mon, 30 Nov 2020 at 08:01, Pavel Raiskup praiskup@redhat.com wrote:
On Monday, November 30, 2020 1:44:14 PM CET Neal Gompa wrote:
On Mon, Nov 30, 2020 at 7:41 AM Stephen John Smoogen smooge@gmail.com
wrote:
On Mon, 30 Nov 2020 at 06:57, Florian Weimer fweimer@redhat.com
wrote:
- Miro Hrončok:
I was wondering whether it might be possible to use microdnf instead of dnf in the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode now complicates upgrading Pythons to a newer version.
Would it be possible to use rpm-ostree with a vendored RPM implementation? That should provide even better isolation.
koji has a very strong idea of what it wants inside a chroot and what
it
does not (and may never) want. Getting rpm-ostree or other items to
work
inside of the koji mock interface is probably looking at a large
project
(going from how many other 'simple' things have ended up being much
more
complicated than expected by the original proposers). My usual way of figuring out how to make anything work in koji is to see what the
oldest
environment it needs to work on by a customer and see how I would do
it in
RHEL-5 (now maybe RHEL-6).
Then that's easy: we define a list of packages that are needed to get the package manager working, and Koji would just download and unpack them with no script execution, then it would manually run some predefined scripts and invoke the package manager after that.
Can you elaborate a bit more please? What is the practical difference between mock's bootstrap chroot and the manual technique you propose?
Pavel
Koji has to be programmed to know about the mock's bootstrap chroot and be able to use it. Koji 'handwrites' a mock config for every build and calls mock with specific instructions.
I think the manual technique is already built into koji for certain cases but takes configuration on releng to make work. [So the difference would be one is very old, awkward, but there.. the other is a project plan with the koji team.]
Aside from me doing this before to do RHEL 5 -> RHEL 7 upgrades by hand, this is also the strategy used by the openSUSE Build Service to work around this problem.
-- 真実はいつも一つ!/ Always, there's only one truth! _______________________________________________ buildsys mailing list -- buildsys@lists.fedoraproject.org To unsubscribe send an email to buildsys-leave@lists.fedoraproject.org Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:
https://lists.fedoraproject.org/archives/list/buildsys@lists.fedoraproject.o...
buildsys mailing list -- buildsys@lists.fedoraproject.org To unsubscribe send an email to buildsys-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/buildsys@lists.fedoraproject.o...
* Pavel Raiskup:
Can you elaborate a bit more please? What is the practical difference between mock's bootstrap chroot and the manual technique you propose?
Koji has access to a compatible RPM for the unpacking, something that may not be available on a developer workstation.
Thanks, Florian
Dne 30. 11. 20 v 13:44 Neal Gompa napsal(a):
Then that's easy: we define a list of packages that are needed to get the package manager working, and Koji would just download and unpack them with no script execution, then it would manually run some predefined scripts and invoke the package manager after that.
That is not needed.
This:
config_opts['update_before_build'] = True config_opts['dnf_install_command'] = '--help'
config_opts['use_bootstrap'] = True config_opts['use_bootstrap_image'] = True config_opts['bootstrap_image'] = 'fedora:rawhide'
The last 3 lines enables this feature: https://github.com/rpm-software-management/mock/wiki/Feature-container-for-b...
Then config_opts['dnf_install_command'] = '--help' just shortcut dnf install dnf dnf-plugins-core after downloading the image. It is there just to make sure that we have DNF in that container and not just microdnf. Rawhide image has it there. So this step is redundant.
Fedora-infra will need whitelist fedora registry on Koji builders so they are network reachable.
Additionally, the config in attachemnt will use fedora-33 for bootstrap and fedora-34 for building.
Here is the important part:
config_opts['use_bootstrap'] = True config_opts['use_bootstrap_image'] = True config_opts['bootstrap_image'] = 'fedora:33'
config_opts['target_arch'] = 'x86_64'
config_opts['root'] = 'fedora-rawhide-{{ target_arch }}' config_opts['bootstrap_root'] = 'fedora-33-{{ target_arch }}'
config_opts['chroot_name'] = 'fedora-rawhide-{{ target_arch }}' config_opts['bootstrap_chroot_name'] = 'fedora-33-{{ target_arch }}'
config_opts['releasever'] = 34 config_opts['bootstrap_releasever'] = 33
Disclaimer - I believe that this can be even simplier. Functional one is in the attachement.
If you will fiddle with: config_opts['bootstra_dnf.conf'] than you can skip that bootstrap image if that would be a problem for Koji.
buildsys@lists.fedoraproject.org