Hello Copr users and developers.
When we update packages in Fedora, we regularly use Copr to test the impact of the upgrade.
For me, the procedure usually goes like this:
1) create a new copr with Fedora rawhide x86_64 chroot (and added Koji repo)
$ copr create $COPR --repo='http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/' --chroot fedora-rawhide-x86_64 --delete-after-days 30
2) define and build the updated package in the copr
$ copr add-package-distgit $COPR --name $PKG --webhook-rebuild on --commit $BRANCH --namespace forks/$(whoami) $ copr build-package $COPR --name $PKG
3) get the list of dependent packages
$ repoquery -q --repo=rawhide{,-source} [--whatrequires $spkg for each subpackage] --recursive | grep src$ | pkgname
4) define and build the depended packages in the copr
$ parallel copr add-package-distgit $COPR --webhook-rebuild on --commit rawhide --name -- $(repoquery from above ...) $ parallel copr build-package $COPR --nowait --background --name -- $(repoquery from above ...)
5) analyze build failures, do a "control" rebuild in another copr if needed
However, this procedure has a flaw. Let's say I'm working on upgrading python-click from 7.x to 8.x. And let's say a package (even transitively) BuildRequires:
python3dist(click) < 8
The way that dnf dependency resolution works, that package will be built with Rawhide's python3-click 7.x and it will be marked as successful. However, I'd like to see a failure here to be notified that such package cannot be build and will be negatively impacted by the update.
Is there a way to solve this? I have couple ideas, but none of them is fully working:
A) Compose my own repo with the updated package and Rawhide content without it, use that repo in the copr.
Pros:
- this is similar to what will happen in Koji once the package is updated
Cons:
- this requires tooling that I don't think exists - this requires a place to put that repo to - the repo creation could take a lot of time and would need to be repeated on-demand each time rawhide changes - Copr's Fedora chroots always include Fedora repos (maybe I can use custom-1-x86_64 chroot?)
B) Create a Fedora side tag, explicitly block the package from it, use that side tag's Koji repo.
Pros:
- same as (A)
Cons:
- I don't think on-demand side tags allow users to block packages - Copr's Fedora chroots always include Fedora repos (same as (A)) - this wastes Koji's resources a bit - requires waiting for the initial Koji regen-repo
C) Block (exclude) python3-click < 8 from the chroot.
Pros:
- no custom repos required - no resources overhead - no time overhead
Cons:
- There is no way exclude packages in chroot settings. Mock settings possibly allow me to do this in config_opts['dnf.conf']. - The exclude could obfuscate root.log resolution problems logs. - Packager needs to figure out what exactly to exclude (possibly need to exclude all subpackage's NEVRAs from rawhide compose (and Koji buildroot if they differ))
Is there another way? If not, I think (C) is easiest to actually implement, if the chroot settings page in copr gains an "excludes" option.
On Thursday, July 8, 2021 3:47:56 PM CEST Miro Hrončok wrote:
Hello Copr users and developers.
When we update packages in Fedora, we regularly use Copr to test the impact of the upgrade.
For me, the procedure usually goes like this:
create a new copr with Fedora rawhide x86_64 chroot (and added Koji repo)
$ copr create $COPR
--repo='http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/' --chroot fedora-rawhide-x86_64 --delete-after-days 30
define and build the updated package in the copr
$ copr add-package-distgit $COPR --name $PKG --webhook-rebuild on --commit
$BRANCH --namespace forks/$(whoami) $ copr build-package $COPR --name $PKG
get the list of dependent packages
$ repoquery -q --repo=rawhide{,-source} [--whatrequires $spkg for each
subpackage] --recursive | grep src$ | pkgname
define and build the depended packages in the copr
$ parallel copr add-package-distgit $COPR --webhook-rebuild on --commit
rawhide --name -- $(repoquery from above ...) $ parallel copr build-package $COPR --nowait --background --name -- $(repoquery from above ...)
- analyze build failures, do a "control" rebuild in another copr if needed
However, this procedure has a flaw. Let's say I'm working on upgrading python-click from 7.x to 8.x. And let's say a package (even transitively) BuildRequires:
python3dist(click) < 8
The way that dnf dependency resolution works, that package will be built with Rawhide's python3-click 7.x and it will be marked as successful. However, I'd like to see a failure here to be notified that such package cannot be build and will be negatively impacted by the update.
Is there a way to solve this?
I thought that 'best=1' is about to help here, but since we use that config in mock-core-configs it is not.
I have couple ideas, but none of them is fully working:
A) Compose my own repo with the updated package and Rawhide content without it, use that repo in the copr.
Pros:
- this is similar to what will happen in Koji once the package is updated
Cons:
- this requires tooling that I don't think exists
- this requires a place to put that repo to
- the repo creation could take a lot of time and would need to be repeated
on-demand each time rawhide changes
- Copr's Fedora chroots always include Fedora repos (maybe I can use
custom-1-x86_64 chroot?)
B) Create a Fedora side tag, explicitly block the package from it, use that side tag's Koji repo.
Pros:
- same as (A)
Cons:
- I don't think on-demand side tags allow users to block packages
- Copr's Fedora chroots always include Fedora repos (same as (A))
- this wastes Koji's resources a bit
- requires waiting for the initial Koji regen-repo
C) Block (exclude) python3-click < 8 from the chroot.
Pros:
- no custom repos required
- no resources overhead
- no time overhead
Cons:
- There is no way exclude packages in chroot settings. Mock settings possibly
allow me to do this in config_opts['dnf.conf'].
- The exclude could obfuscate root.log resolution problems logs.
- Packager needs to figure out what exactly to exclude (possibly need to
exclude all subpackage's NEVRAs from rawhide compose (and Koji buildroot if they differ))
Is there another way? If not, I think (C) is easiest to actually implement, if the chroot settings page in copr gains an "excludes" option.
This is pre-production experiment, right? Couldn't you just specify "Provides: python3dist(click) = 7.9"?
We have an RFE https://pagure.io/copr/copr/issue/1336 that could help you if you could specify dnf.conf excludepkgs= option for the rawhide repo. But this is not an easy one.
Pavel
On 09. 07. 21 9:46, Pavel Raiskup wrote:
This is pre-production experiment, right? Couldn't you just specify "Provides: python3dist(click) = 7.9"?
It is, but it would not help me. By manually providing this, I would:
- confuse %pyproject_buildrequires generator -- Python would think 'click < 8' is not satisfied, by RPM/dnf would insist it is. - still not get a root.log resolution failure I want - still not "satisfy" packages that BR "python3click" instead of "python3dist(click)"
I wonder if adding Obsoletes: python3-click < %{version} helps, but I don't think so.
We have an RFEhttps://pagure.io/copr/copr/issue/1336 that could help you if you could specify dnf.conf excludepkgs= option for the rawhide repo. But this is not an easy one.
Yes, this would work.
On Friday, July 9, 2021 10:19:57 AM CEST Miro Hrončok wrote:
On 09. 07. 21 9:46, Pavel Raiskup wrote:
This is pre-production experiment, right? Couldn't you just specify "Provides: python3dist(click) = 7.9"?
It is, but it would not help me. By manually providing this, I would:
- confuse %pyproject_buildrequires generator -- Python would think 'click <
8' is not satisfied, by RPM/dnf would insist it is.
- still not get a root.log resolution failure I want
- still not "satisfy" packages that BR "python3click" instead of
"python3dist(click)"
I wonder if adding Obsoletes: python3-click < %{version} helps, but I don't think so.
We have an RFEhttps://pagure.io/copr/copr/issue/1336 that could help you if you could specify dnf.conf excludepkgs= option for the rawhide repo. But this is not an easy one.
Yes, this would work.
Another idea... What if you had a syntetic package, say "click-blocker", with 'Conflicts: python3-click < XYZ'? That package could be added as always-installed package into "Packages" field (chroot Edit form).
Or maybe it could simply require 'python3-click >= XYZ', too?
Pavel
On 09. 07. 21 11:30, Pavel Raiskup wrote:
On Friday, July 9, 2021 10:19:57 AM CEST Miro Hrončok wrote:
On 09. 07. 21 9:46, Pavel Raiskup wrote:
This is pre-production experiment, right? Couldn't you just specify "Provides: python3dist(click) = 7.9"?
It is, but it would not help me. By manually providing this, I would:
- confuse %pyproject_buildrequires generator -- Python would think 'click <
8' is not satisfied, by RPM/dnf would insist it is.
- still not get a root.log resolution failure I want
- still not "satisfy" packages that BR "python3click" instead of
"python3dist(click)"
I wonder if adding Obsoletes: python3-click < %{version} helps, but I don't think so.
We have an RFEhttps://pagure.io/copr/copr/issue/1336 that could help you if you could specify dnf.conf excludepkgs= option for the rawhide repo. But this is not an easy one.
Yes, this would work.
Another idea... What if you had a syntetic package, say "click-blocker", with 'Conflicts: python3-click < XYZ'? That package could be added as always-installed package into "Packages" field (chroot Edit form).
Or maybe it could simply require 'python3-click >= XYZ', too?
Is "Packages" always-installed or initially-installed? What prevents dnf to downgrade it?
I'll test this approach thou, seems reasonable.
On Friday, July 9, 2021 11:36:09 AM CEST Miro Hrončok wrote:
On 09. 07. 21 11:30, Pavel Raiskup wrote:
On Friday, July 9, 2021 10:19:57 AM CEST Miro Hrončok wrote:
On 09. 07. 21 9:46, Pavel Raiskup wrote:
This is pre-production experiment, right? Couldn't you just specify "Provides: python3dist(click) = 7.9"?
It is, but it would not help me. By manually providing this, I would:
- confuse %pyproject_buildrequires generator -- Python would think 'click <
8' is not satisfied, by RPM/dnf would insist it is.
- still not get a root.log resolution failure I want
- still not "satisfy" packages that BR "python3click" instead of
"python3dist(click)"
I wonder if adding Obsoletes: python3-click < %{version} helps, but I don't think so.
We have an RFEhttps://pagure.io/copr/copr/issue/1336 that could help you if you could specify dnf.conf excludepkgs= option for the rawhide repo. But this is not an easy one.
Yes, this would work.
Another idea... What if you had a syntetic package, say "click-blocker", with 'Conflicts: python3-click < XYZ'? That package could be added as always-installed package into "Packages" field (chroot Edit form).
Or maybe it could simply require 'python3-click >= XYZ', too?
Is "Packages" always-installed or initially-installed? What prevents dnf to downgrade it?
I'll test this approach thou, seems reasonable.
You are right, it is always "initially installed".. It is definitely interesting topic, please share the results of the experiment :-)
If DNF really removes the artificial package because it depends on a _newer_ click package (than the following transaction requires) -- we could probably consider an upstream report..
Pavel
On 09. 07. 21 13:27, Pavel Raiskup wrote:
On Friday, July 9, 2021 11:36:09 AM CEST Miro Hrončok wrote:
On 09. 07. 21 11:30, Pavel Raiskup wrote:
On Friday, July 9, 2021 10:19:57 AM CEST Miro Hrončok wrote:
On 09. 07. 21 9:46, Pavel Raiskup wrote:
This is pre-production experiment, right? Couldn't you just specify "Provides: python3dist(click) = 7.9"?
It is, but it would not help me. By manually providing this, I would:
- confuse %pyproject_buildrequires generator -- Python would think 'click <
8' is not satisfied, by RPM/dnf would insist it is. - still not get a root.log resolution failure I want - still not "satisfy" packages that BR "python3click" instead of "python3dist(click)"
I wonder if adding Obsoletes: python3-click < %{version} helps, but I don't think so.
We have an RFEhttps://pagure.io/copr/copr/issue/1336 that could help you if you could specify dnf.conf excludepkgs= option for the rawhide repo. But this is not an easy one.
Yes, this would work.
Another idea... What if you had a syntetic package, say "click-blocker", with 'Conflicts: python3-click < XYZ'? That package could be added as always-installed package into "Packages" field (chroot Edit form).
Or maybe it could simply require 'python3-click >= XYZ', too?
Is "Packages" always-installed or initially-installed? What prevents dnf to downgrade it?
I'll test this approach thou, seems reasonable.
You are right, it is always "initially installed".. It is definitely interesting topic, please share the results of the experiment :-)
If DNF really removes the artificial package because it depends on a _newer_ click package (than the following transaction requires) -- we could probably consider an upstream report..
Finally got to it. It actually downgrades it:
https://copr.fedorainfracloud.org/coprs/thrnciar/python-platformdirs/build/5...
Installing: python3-platformdirs noarch 3.5.1-1.fc39 copr_base 38 k ... Downgrading: python3-platformdirs noarch 2.6.0-2.fc38 fedora 42 k
Dne 08. 07. 21 v 15:47 Miro Hrončok napsal(a):
However, this procedure has a flaw. Let's say I'm working on upgrading python-click from 7.x to 8.x. And let's say a package (even transitively) BuildRequires:
python3dist(click) < 8
The way that dnf dependency resolution works, that package will be built with Rawhide's python3-click 7.x and it will be marked as successful. However, I'd like to see a failure here to be notified that such package cannot be build and will be negatively impacted by the update.
Is there a way to solve this?
Hmm, the root.log contains list of package actually installed. But the decision what you are looking for is on you and will not be easy. :(
Miroslav
copr-devel@lists.fedorahosted.org