In my local Koji deployment I have one particular package (mdct-aos-flash) whose spec %build has "%{__python3} -m unittest discover" so that my builds will fail if any of the unit tests fail. This has worked well for some time now. Recently I added another unit test which requires another locally built package (python3-mdct) to assist with the test. My first build submission failed and I immediately realized that I of course needed to add this other package as a BuildRequires. I did that and was surprised to find the build still failing. The root.log shows:
DEBUG package_manager.py:77: ['/usr/bin/yum-builddep', '--installroot', '/var/lib/mock/f21-build-1274-3319/root/', '/var/lib/mock/f21-build-1274-3319/root//builddir/build/SRPMS/mdct-aos-flash-3.1-1.git.5.401b5db.fc21.src.rpm'] DEBUG util.py:508: child environment: None DEBUG util.py:441: Executing command: ['/usr/bin/yum-builddep', '--installroot', '/var/lib/mock/f21-build-1274-3319/root/', '/var/lib/mock/f21-build-1274-3319/root//builddir/build/SRPMS/mdct-aos-flash-3.1-1.git.5.401b5db.fc21.src.rpm', '--setopt=tsflags=nocontexts'] with env {'LANG': 'en_US.UTF-8', 'TERM': 'vt100', 'SHELL': '/bin/bash', 'LC_MESSAGES': 'C', 'PROMPT_COMMAND': 'printf "\x1b]0;<mock-chroot>\x07<mock-chroot>"', 'PATH': '/usr/bin:/bin:/usr/sbin:/sbin', 'HOME': '/builddir', 'HOSTNAME': 'mock'} and shell False DEBUG util.py:377: Getting requirements for mdct-aos-flash-3.1-1.git.5.401b5db.fc21.src DEBUG util.py:377: --> python3-devel-3.4.1-18.fc21.x86_64 DEBUG util.py:377: --> python3-libs-3.4.1-18.fc21.x86_64 DEBUG util.py:377: --> systemd-216-25.fc21.x86_64 DEBUG util.py:377: Error: No Package found for python3-mdct DEBUG util.py:488: Child return code was: 1
Now I'm stumped. Koji builds python3-mdct for f21 so I would expect that it's just available to be pulled in as needed. I have Koji configured to use our local mirrors of Fedora as external-repos because we certainly depend on much of that, but I wouldn't think it necessary to do so for our own repo. My understanding of the external-repos is to give Koji access to rpms which it may need, but itself did not build.
Is my understanding wrong or have I missed a step here?
-- John Florian
I think I've figured this out. I had to "koji add-external-repo ..." to give the koji access to our mashed repositories to satisfy the BuildRequires. I had been under the impression that Kojira and /mnt/koji/repos somehow worked in concert to implicitly make anything that koji built available to itself for satisfying build dependencies. Rather, it appears that this must be explicitly defined through the external repos.
On Wed, 27 Jan 2016 21:25:02 -0000 "John Florian" jflorian@fedoraproject.org wrote:
I think I've figured this out. I had to "koji add-external-repo ..." to give the koji access to our mashed repositories to satisfy the BuildRequires. I had been under the impression that Kojira and /mnt/koji/repos somehow worked in concert to implicitly make anything that koji built available to itself for satisfying build dependencies. Rather, it appears that this must be explicitly defined through the external repos.
it is automatic if the builds done are tagged so they are accessible (usually via inheritance) by the build-tag in the target, please look at the Fedora setup
http://koji.fedoraproject.org/koji/buildtargetinfo?targetID=151 http://koji.fedoraproject.org/koji/taginfo?tagID=315 http://koji.fedoraproject.org/koji/taginfo?tagID=308
Dan
If that's the case, I'm not sure why I had to add the external repo given that I have:
$ koji list-tag-inheritance --reverse f21 f21 (6) ├─f21-build (7) ├─f21-candidates (8) ├─f21-testing (9) ├─f21-released (10) └─f22 (12) ├─f22-build (13) ├─f22-candidates (14) ├─f22-testing (15) └─f22-released (16)
Argh... the formatting of that isn't ideal. As you might guess, from f22-build on down should be indented.
Argh... the formatting of that isn't ideal. As you might guess, from f22- build on down should be indented.
I just realized the formatting looks fine in my MUA, it's HyperKitty that doesn't show this well:
https://lists.fedoraproject.org/archives/list/buildsys@lists.fedoraproject.o...
But at least there I don't have to fight with Outlook's insistence on top-posting
-- John Florian
On Thu, 28 Jan 2016 16:14:03 -0000 "John Florian" jflorian@fedoraproject.org wrote:
If that's the case, I'm not sure why I had to add the external repo given that I have:
$ koji list-tag-inheritance --reverse f21 f21 (6) ├─f21-build (7) ├─f21-candidates (8) ├─f21-testing (9) ├─f21-released (10) └─f22 (12) ├─f22-build (13) ├─f22-candidates (14) ├─f22-testing (15) └─f22-released (16)
if for your target the build tag is f22-build and the destination tag is f22-candidates, but even if it is f22-released, then the f22-build tag won't see builds until they are tagged as f22. You should have f22-build inherit from f22-released in your case if I understand your setup correctly.
You can check the situation with "koji latest-build f22-build foo" to see if the needed build of foo is available for other builds.
Understanding the tag hierarchy and its impacts is the most difficult thing in koji I think.
Dan
Oh, now my light bulb has a glow, dim it may be but it's not utter darkness any more. Tags in general in Koji are hard to understand since they can take on so many uses.
Let's start from scratch. Here's what I am hoping to accomplish:
All builds initially are dumped into fN-candidates. These get mashed into a repo, but I rarely use it -- it's mostly proof the build can succeed at all. (Though, I'm now becoming aware of scratch/test builds from HEAD via Koji so perhaps fN-candidates is unnecessary.) If I want to actually try a build out, I "promote" it by untagging from fN-candidates and tagging it with fN-testing. If functional testing proves it good, then I'll promote again by untagging from fn-testing and tagging with fN-released.
In the Fedora Project sense, I don't have any distinction between "base" and "updates". They're effectively squashed into my fN-released as we've no need for anything but the newest functionally tested version.
So, if IUYC, I should have my fN-build inherit from fN-released. Am I correct?
Will I be able to change my inheritance hierarchy for f21 & f22 or will I have to wait and change my model once I start supporting f23?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 01/28/2016 11:52 AM, John Florian wrote:
Oh, now my light bulb has a glow, dim it may be but it's not utter darkness any more. Tags in general in Koji are hard to understand since they can take on so many uses.
Let's start from scratch. Here's what I am hoping to accomplish:
All builds initially are dumped into fN-candidates. These get mashed into a repo, but I rarely use it -- it's mostly proof the build can succeed at all. (Though, I'm now becoming aware of scratch/test builds from HEAD via Koji so perhaps fN-candidates is unnecessary.) If I want to actually try a build out, I "promote" it by untagging from fN-candidates and tagging it with fN-testing. If functional testing proves it good, then I'll promote again by untagging from fn-testing and tagging with fN-released.
In the Fedora Project sense, I don't have any distinction between "base" and "updates". They're effectively squashed into my fN-released as we've no need for anything but the newest functionally tested version.
So, if IUYC, I should have my fN-build inherit from fN-released. Am I correct?
Will I be able to change my inheritance hierarchy for f21 & f22 or will I have to wait and change my model once I start supporting f23? -- buildsys mailing list buildsys@lists.fedoraproject.org http://lists.fedoraproject.org/admin/lists/buildsys@lists.fedoraproject.org
Can you list the inheritance of your various tags without the - --reverse argument?
- - Jay
$ koji list-tag-inheritance f22 f22 (12) └─f21 (6) └─f20 (1)
Basically, f22 (may newest) inherits from f21 which inherits from f20 (my oldest).
On Thu, 28 Jan 2016 16:52:32 -0000 "John Florian" jflorian@fedoraproject.org wrote:
Oh, now my light bulb has a glow, dim it may be but it's not utter darkness any more. Tags in general in Koji are hard to understand since they can take on so many uses.
Let's start from scratch. Here's what I am hoping to accomplish:
All builds initially are dumped into fN-candidates. These get mashed into a repo, but I rarely use it -- it's mostly proof the build can succeed at all. (Though, I'm now becoming aware of scratch/test builds from HEAD via Koji so perhaps fN-candidates is unnecessary.) If I want to actually try a build out, I "promote" it by untagging from fN-candidates and tagging it with fN-testing. If functional testing proves it good, then I'll promote again by untagging from fn-testing and tagging with fN-released.
yes, that's quite common workflow
In the Fedora Project sense, I don't have any distinction between "base" and "updates". They're effectively squashed into my fN-released as we've no need for anything but the newest functionally tested version.
So, if IUYC, I should have my fN-build inherit from fN-released. Am I correct?
yes
Will I be able to change my inheritance hierarchy for f21 & f22 or will I have to wait and change my model once I start supporting f23?
with "koji edit-tag-inheritance" you can change it at any time
Dan
Woot; success! After several failed attempts, I got to:
$ koji add-tag-inheritance --priority=2 f21-build f21-released $ koji list-tag-inheritance f21-build f21-build (7) ├─f21 (6) │ └─f20 (1) └─f21-released (10)
I tried a scratch build targeting f21 and now my BuildRequires is satisfied through inheritance of f21-released and it worked. So to test my understanding, I believe:
* having f21 inherit from f20 is mostly useful so that you don't have to "koji add-pkg ..." (and likely more) everything again.
* inheritance also allows the parent tags of a build tag help resolve build deps
Please correct me if I'm wrong.
Also the priorities involved in inheritance are a bit mysterious. I merely guessed at a priority when adding the inheritance above. I don't see any means from the CLI with which to show the inheritance priorities, nor any indication if a lower number indicates a lower or a higher priority.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 01/28/2016 02:13 PM, John Florian wrote:
Woot; success! After several failed attempts, I got to:
$ koji add-tag-inheritance --priority=2 f21-build f21-released $ koji list-tag-inheritance f21-build f21-build (7) ├─f21 (6) │ └─f20 (1) └─f21-released (10)
I tried a scratch build targeting f21 and now my BuildRequires is satisfied through inheritance of f21-released and it worked. So to test my understanding, I believe:
- having f21 inherit from f20 is mostly useful so that you don't
have to "koji add-pkg ..." (and likely more) everything again.
- inheritance also allows the parent tags of a build tag help
resolve build deps
Please correct me if I'm wrong.
Looks like a good set up to me.
Also the priorities involved in inheritance are a bit mysterious. I merely guessed at a priority when adding the inheritance above. I don't see any means from the CLI with which to show the inheritance priorities, nor any indication if a lower number indicates a lower or a higher priority.
When Koji looks for a package, it does a depth-first search. (f21-build, f21, f20, f21-released is the order in your case) The priority influences the ordering. They're just arbitrary numbers. I usually use multiples of 10 to start, so that if I ever need to inject a tag, I can pick a priority in between the tags that are already there.
The numbers you see in list-tag-inheritance in the parentheses are tag IDs, not priority, in case that is confusing you.
- - Jay
-- buildsys mailing list buildsys@lists.fedoraproject.org http://lists.fedoraproject.org/admin/lists/buildsys@lists.fedoraproject.org
When Koji looks for a package, it does a depth-first search. (f21-build, f21, f20, f21-released is the order in your case) The priority influences the ordering.
Hmm.. I think I would want that order to be f21-build, f21-released, f21, f20 instead. So I assume I need to change the priority somewhere.
They're just arbitrary numbers. I usually use multiples of 10 to start, so that if I ever need to inject a tag, I can pick a priority in between the tags that are already there.
The numbers you see in list-tag-inheritance in the parentheses are tag IDs, not priority, in case that is confusing you.
That makes sense, but I still don't know how to *see* the priority values (short of diving into the DB, which I hope is unnecessary). The "koji list-external-repos" command shows priorities for those, but the only mention of priorities I see with tags and/or inheritance is via "--priority=N".
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 01/28/2016 03:28 PM, John Florian wrote:
When Koji looks for a package, it does a depth-first search. (f21-build, f21, f20, f21-released is the order in your case) The priority influences the ordering.
Hmm.. I think I would want that order to be f21-build, f21-released, f21, f20 instead. So I assume I need to change the priority somewhere.
They're just arbitrary numbers. I usually use multiples of 10 to start, so that if I ever need to inject a tag, I can pick a priority in between the tags that are already there.
The numbers you see in list-tag-inheritance in the parentheses are tag IDs, not priority, in case that is confusing you.
That makes sense, but I still don't know how to *see* the priority values (short of diving into the DB, which I hope is unnecessary). The "koji list-external-repos" command shows priorities for those, but the only mention of priorities I see with tags and/or inheritance is via "--priority=N". -- buildsys mailing list buildsys@lists.fedoraproject.org http://lists.fedoraproject.org/admin/lists/buildsys@lists.fedoraproject.org
koji taginfo <tag>
The priorities are attributes on the tag doing the inheritance. Counter-intuitive, I know.
- - Jay
Thanks Jay and Dan! While I didn't find using "koji edit-tag-inheritance" a whole lot of fun, I did manage to get things working. I really appreciate the time you took to answer my questions.
buildsys@lists.fedoraproject.org
-
Dan Horák -
Jay Greguske -
John Florian -
John Florian