Re: [Marketing] Re: [MAGAZINE PROPOSAL] Fwd: [DRAFT] Why we're retiring 32-bit Images (was Re: Retiring 32-bit images)
On 04/07/2016 06:59 PM, Joe Brockmeier wrote:
On 04/07/2016 06:47 PM, Ryan Lerch wrote:
Will it just be a featured image? Or are there other graphics required?
Just the featured image. Thanks!
OK, looks like we have the featured image here:
https://pagure.io/fedoramagazine-images/issue/12
Is this ready to go? Is there a specific date to target or run it now, or...?
Best,
jzb
On Tue, Apr 12, 2016 at 11:18:05AM -0400, Joe Brockmeier wrote:
On 04/07/2016 06:59 PM, Joe Brockmeier wrote:
On 04/07/2016 06:47 PM, Ryan Lerch wrote:
Will it just be a featured image? Or are there other graphics required?
Just the featured image. Thanks!
OK, looks like we have the featured image here:
https://pagure.io/fedoramagazine-images/issue/12
Is this ready to go? Is there a specific date to target or run it now, or...?
Hi Joe,
I didn't see this called out in the meeting notes from today (I missed the meeting, sorry), but we could either:
* run this next Tuesday, in addition to the scheduled articles; or * run it on Wednesday, push Wednesday's article to Friday, and give nirik a little more time for his OpenVPN piece
Anyone have preferences?
On Thursday, April 14, 2016 9:14:51 PM CDT Paul W. Frields wrote:
On Tue, Apr 12, 2016 at 11:18:05AM -0400, Joe Brockmeier wrote:
On 04/07/2016 06:59 PM, Joe Brockmeier wrote:
On 04/07/2016 06:47 PM, Ryan Lerch wrote:
Will it just be a featured image? Or are there other graphics required?
Just the featured image. Thanks!
OK, looks like we have the featured image here:
https://pagure.io/fedoramagazine-images/issue/12
Is this ready to go? Is there a specific date to target or run it now, or...?
Hi Joe,
I didn't see this called out in the meeting notes from today (I missed the meeting, sorry), but we could either:
- run this next Tuesday, in addition to the scheduled articles; or
- run it on Wednesday, push Wednesday's article to Friday, and give nirik a little more time for his OpenVPN piece
Anyone have preferences?
I would like us to demote them to secondary.
Dennis
On 04/15/2016 10:38 AM, Dennis Gilmore wrote:
I would like us to demote them to secondary.
Why? We've already decided to drop. I'm not opposed, just curious why. IIRC we were hitting a major problem with kernel compat as well?
On Fri, 2016-04-15 at 17:28 -0400, Joe Brockmeier wrote:
On 04/15/2016 10:38 AM, Dennis Gilmore wrote:
I would like us to demote them to secondary.
Why? We've already decided to drop. I'm not opposed, just curious why. IIRC we were hitting a major problem with kernel compat as well?
32-bit F24 kernels just don't boot, in at least a lot of cases (maybe at all, ever) right now. Rawhide ones do boot, though anaconda on Rawhide can't see any disks. Yeah, 32-bit is pretty bad right now.
On 04/15/2016 05:28 PM, Joe Brockmeier wrote:
On 04/15/2016 10:38 AM, Dennis Gilmore wrote:
I would like us to demote them to secondary.
Why? We've already decided to drop. I'm not opposed, just curious why. IIRC we were hitting a major problem with kernel compat as well?
Pinging on this - I thought we'd reached a decision and wanted to publicize that sooner than later.
If there's a reason to prefer move to secondary, let's discuss.
Best,
jzb
On Monday, April 18, 2016 2:59:18 PM CDT you wrote:
On 04/15/2016 05:28 PM, Joe Brockmeier wrote:
On 04/15/2016 10:38 AM, Dennis Gilmore wrote:
I would like us to demote them to secondary.
Why? We've already decided to drop. I'm not opposed, just curious why. IIRC we were hitting a major problem with kernel compat as well?
Pinging on this - I thought we'd reached a decision and wanted to publicize that sooner than later.
If there's a reason to prefer move to secondary, let's discuss.
Best,
jzb
I prefer to move it to secondary because people could be relying on it still, it gives us a way to move forward and not be blocked on 32 bit x86. If it does not work then it will not get shipped. Just dropping them on the floor does not give as smooth a transition, nor does it give people that want it still the chance to pick it up and continue to carry it forward.
Dennis
On Mon, Apr 18, 2016 at 5:31 PM, Dennis Gilmore dennis@ausil.us wrote:
On Monday, April 18, 2016 2:59:18 PM CDT you wrote:
On 04/15/2016 05:28 PM, Joe Brockmeier wrote:
On 04/15/2016 10:38 AM, Dennis Gilmore wrote:
I would like us to demote them to secondary.
Why? We've already decided to drop. I'm not opposed, just curious why. IIRC we were hitting a major problem with kernel compat as well?
Pinging on this - I thought we'd reached a decision and wanted to publicize that sooner than later.
If there's a reason to prefer move to secondary, let's discuss.
Best,
jzb
I prefer to move it to secondary because people could be relying on it still, it gives us a way to move forward and not be blocked on 32 bit x86. If it does not work then it will not get shipped. Just dropping them on the floor does not give as smooth a transition, nor does it give people that want it still the chance to pick it up and continue to carry it forward.
Is the context Cloud, or in general? I think going from primary for all products to totally dropping it is a problem, even if install media is non-blocking. I have no stake in i686 at all, and I think Cloud and Server are less affected by totally dropping i686 than Workstation; but I think quitting i686 cold turkey needs reconsideration.
Anyway I think no one has done anything wrong here, but the warnings of the kernel team were maybe considered something like, "oh, we'll get by one more release or two by the skin of our teeth before it blows up" and yet it just turned out that it's blowing up already.
If the idea is we should block on i686 in general for upgrading, I'd agree, even though it's a pain.
For Cloud, maybe the way forward at worst is to support Cloud Atomic. And the images are i686 only? Of course that assumes any problems with binutil and kernel, or whatever else comes up, is sanely fixable with a best effort.
?
On Monday, April 18, 2016 9:34:35 PM CDT Chris Murphy wrote:
On Mon, Apr 18, 2016 at 5:31 PM, Dennis Gilmore dennis@ausil.us wrote:
On Monday, April 18, 2016 2:59:18 PM CDT you wrote:
On 04/15/2016 05:28 PM, Joe Brockmeier wrote:
On 04/15/2016 10:38 AM, Dennis Gilmore wrote:
I would like us to demote them to secondary.
Why? We've already decided to drop. I'm not opposed, just curious why. IIRC we were hitting a major problem with kernel compat as well?
Pinging on this - I thought we'd reached a decision and wanted to publicize that sooner than later.
If there's a reason to prefer move to secondary, let's discuss.
Best,
jzb
I prefer to move it to secondary because people could be relying on it still, it gives us a way to move forward and not be blocked on 32 bit x86. If it does not work then it will not get shipped. Just dropping them on the floor does not give as smooth a transition, nor does it give people that want it still the chance to pick it up and continue to carry it forward.
Is the context Cloud, or in general? I think going from primary for all products to totally dropping it is a problem, even if install media is non-blocking. I have no stake in i686 at all, and I think Cloud and Server are less affected by totally dropping i686 than Workstation; but I think quitting i686 cold turkey needs reconsideration.
Anyway I think no one has done anything wrong here, but the warnings of the kernel team were maybe considered something like, "oh, we'll get by one more release or two by the skin of our teeth before it blows up" and yet it just turned out that it's blowing up already.
If the idea is we should block on i686 in general for upgrading, I'd agree, even though it's a pain.
For Cloud, maybe the way forward at worst is to support Cloud Atomic. And the images are i686 only? Of course that assumes any problems with binutil and kernel, or whatever else comes up, is sanely fixable with a best effort.
My stance is that it hold true for all products, spins etc. I think we should just make i686 a secondary arch completely in f25, in order to do it and keep i686 multilib we need to redefine secondary arches.
Dennis
On Tue, Apr 19, 2016 at 9:50 AM, Dennis Gilmore dennis@ausil.us wrote:
On Monday, April 18, 2016 9:34:35 PM CDT Chris Murphy wrote:
On Mon, Apr 18, 2016 at 5:31 PM, Dennis Gilmore dennis@ausil.us wrote:
On Monday, April 18, 2016 2:59:18 PM CDT you wrote:
On 04/15/2016 05:28 PM, Joe Brockmeier wrote:
On 04/15/2016 10:38 AM, Dennis Gilmore wrote:
I would like us to demote them to secondary.
Why? We've already decided to drop. I'm not opposed, just curious why. IIRC we were hitting a major problem with kernel compat as well?
Pinging on this - I thought we'd reached a decision and wanted to publicize that sooner than later.
If there's a reason to prefer move to secondary, let's discuss.
Best,
jzb
I prefer to move it to secondary because people could be relying on it still, it gives us a way to move forward and not be blocked on 32 bit x86. If it does not work then it will not get shipped. Just dropping them on the floor does not give as smooth a transition, nor does it give people that want it still the chance to pick it up and continue to carry it forward.
Is the context Cloud, or in general? I think going from primary for all products to totally dropping it is a problem, even if install media is non-blocking. I have no stake in i686 at all, and I think Cloud and Server are less affected by totally dropping i686 than Workstation; but I think quitting i686 cold turkey needs reconsideration.
Anyway I think no one has done anything wrong here, but the warnings of the kernel team were maybe considered something like, "oh, we'll get by one more release or two by the skin of our teeth before it blows up" and yet it just turned out that it's blowing up already.
Sort of, yes.
If the idea is we should block on i686 in general for upgrading, I'd agree, even though it's a pain.
For Cloud, maybe the way forward at worst is to support Cloud Atomic. And the images are i686 only? Of course that assumes any problems with binutil and kernel, or whatever else comes up, is sanely fixable with a best effort.
s/best/any.
My stance is that it hold true for all products, spins etc. I think we should just make i686 a secondary arch completely in f25, in order to do it and keep i686 multilib we need to redefine secondary arches.
I'd be agreeable to this. In a related vein, I plan on drafting a Change to drop one of the flavors of i686 kernel anyway (likely the non-PAE kernel). For something we don't expend time on and isn't a focus, building two variants just so we can keep really old hardware around is kind of silly. If it moves to a secondary arch, they secondary arch team can deal with it.
josh
> I would like us to demote them to secondary.
Why? We've already decided to drop. I'm not opposed, just curious why. IIRC we were hitting a major problem with kernel compat as well?
Pinging on this - I thought we'd reached a decision and wanted to publicize that sooner than later.
If there's a reason to prefer move to secondary, let's discuss.
Best,
jzb
I prefer to move it to secondary because people could be relying on it still, it gives us a way to move forward and not be blocked on 32 bit x86. If it does not work then it will not get shipped. Just dropping them on the floor does not give as smooth a transition, nor does it give people that want it still the chance to pick it up and continue to carry it forward.
Is the context Cloud, or in general? I think going from primary for all products to totally dropping it is a problem, even if install media is non-blocking. I have no stake in i686 at all, and I think Cloud and Server are less affected by totally dropping i686 than Workstation; but I think quitting i686 cold turkey needs reconsideration.
Anyway I think no one has done anything wrong here, but the warnings of the kernel team were maybe considered something like, "oh, we'll get by one more release or two by the skin of our teeth before it blows up" and yet it just turned out that it's blowing up already.
Sort of, yes.
If the idea is we should block on i686 in general for upgrading, I'd agree, even though it's a pain.
For Cloud, maybe the way forward at worst is to support Cloud Atomic. And the images are i686 only? Of course that assumes any problems with binutil and kernel, or whatever else comes up, is sanely fixable with a best effort.
s/best/any.
My stance is that it hold true for all products, spins etc. I think we should just make i686 a secondary arch completely in f25, in order to do it and keep i686 multilib we need to redefine secondary arches.
I'd be agreeable to this. In a related vein, I plan on drafting a Change to drop one of the flavors of i686 kernel anyway (likely the non-PAE kernel). For something we don't expend time on and isn't a focus, building two variants just so we can keep really old hardware around is kind of silly. If it moves to a secondary arch, they secondary arch team can deal with it.
Define "secondary arch team" as I don't technically deal with the kernel as part of the "secondary arch team" I deal with it for ARM as my own interest, I help others (Power64/s390x) when it's just pushing minor changes.
On Tue, Apr 19, 2016 at 10:17 AM, Peter Robinson pbrobinson@gmail.com wrote:
>> I would like us to demote them to secondary. > > Why? We've already decided to drop. I'm not opposed, just curious why. > IIRC we were hitting a major problem with kernel compat as well?
Pinging on this - I thought we'd reached a decision and wanted to publicize that sooner than later.
If there's a reason to prefer move to secondary, let's discuss.
Best,
jzb
I prefer to move it to secondary because people could be relying on it still, it gives us a way to move forward and not be blocked on 32 bit x86. If it does not work then it will not get shipped. Just dropping them on the floor does not give as smooth a transition, nor does it give people that want it still the chance to pick it up and continue to carry it forward.
Is the context Cloud, or in general? I think going from primary for all products to totally dropping it is a problem, even if install media is non-blocking. I have no stake in i686 at all, and I think Cloud and Server are less affected by totally dropping i686 than Workstation; but I think quitting i686 cold turkey needs reconsideration.
Anyway I think no one has done anything wrong here, but the warnings of the kernel team were maybe considered something like, "oh, we'll get by one more release or two by the skin of our teeth before it blows up" and yet it just turned out that it's blowing up already.
Sort of, yes.
If the idea is we should block on i686 in general for upgrading, I'd agree, even though it's a pain.
For Cloud, maybe the way forward at worst is to support Cloud Atomic. And the images are i686 only? Of course that assumes any problems with binutil and kernel, or whatever else comes up, is sanely fixable with a best effort.
s/best/any.
My stance is that it hold true for all products, spins etc. I think we should just make i686 a secondary arch completely in f25, in order to do it and keep i686 multilib we need to redefine secondary arches.
I'd be agreeable to this. In a related vein, I plan on drafting a Change to drop one of the flavors of i686 kernel anyway (likely the non-PAE kernel). For something we don't expend time on and isn't a focus, building two variants just so we can keep really old hardware around is kind of silly. If it moves to a secondary arch, they secondary arch team can deal with it.
Define "secondary arch team" as I don't technically deal with the kernel as part of the "secondary arch team" I deal with it for ARM as my own interest, I help others (Power64/s390x) when it's just pushing minor changes.
Your examples matches my expectations. To expand it formally, "the group of people that step up to maintain the architecture out of their own interest." Ideally we'd have that group lined up before demotion, or i686 will go the way of sparc and ia64.
PowerPC is the only architecture we've ever demoted, and there were people interested in plugging along with it as a secondary but initially it wasn't enough. Even it had to play catchup with a larger investment of people and machines. Machines shouldn't be an issue for i686 but we do need the people.
josh
On Mon, 2016-04-18 at 21:34 -0600, Chris Murphy wrote:
I prefer to move it to secondary because people could be relying on it still, it gives us a way to move forward and not be blocked on 32 bit x86. If it does not work then it will not get shipped. Just dropping them on the floor does not give as smooth a transition, nor does it give people that want it still the chance to pick it up and continue to carry it forward.
Is the context Cloud, or in general? I think going from primary for all products to totally dropping it is a problem, even if install media is non-blocking. I have no stake in i686 at all, and I think Cloud and Server are less affected by totally dropping i686 than Workstation; but I think quitting i686 cold turkey needs reconsideration.
I think it might be useful to try and be clear about terminology here. We have two established terminology pairs:
'primary' vs. 'secondary' 'blocking' vs. 'non-blocking'
these are entirely distinct from each other and relate to different things, at least as they've always been used before. 'primary' and 'secondary' refer specifically to *architectures* and their status in the build system. Package builds for 'primary' arches are done together, and if build fails on any arch, that package is considered failed for all primary arches and will not be tagged into any compose. Package builds for each 'secondary' arch are done in separate Koji instances and have no impact on whether the package build is considered 'good'. This is the main practical distinction between a 'primary arch' and a 'secondary arch'. It is implicit in this that 'primary arch' vs. 'secondary arch' status is *project-wide*, it cannot be decided at the level of a SIG or flavor or whatever.
'blocking' and 'non-blocking' are terms that refer to *images* (and possibly other release-related deliverables). A 'release blocking' deliverable is one to which we apply some or all release criteria and which must be present, and meet all defined requirements, for a release to be approved. A 'non release blocking' deliverable is the opposite: the release cannot be delayed for a 'non blocking' deliverable being broken or entirely absent, they are provided on a 'best effort' basis.
Whether a given deliverable is 'blocking' or not *can* be decided at the level of a flavor or WG, and I believe the current process is that the WG or SIG or whatever responsible for each deliverable can make this decision, with sign-off from FESCo.
The relationship between these two pairs of statuses is as follows: secondary arch deliverables cannot be release-blocking, primary arch deliverables may be blocking or non-blocking.
FESCo has already decided that no i686 image for Fedora 24 or later can be 'release blocking' by policy. There is (AIUI) no wiggle room in that decision: we cannot block releases on i686 images any more.
However, i686 is still a 'primary arch' within the meaning of that term: i686 builds are done in the main Koji along with x86_64 and armhfp builds, and if a build is submitted and fails on i686, that whole build is 'failed'.
If the idea is we should block on i686 in general for upgrading, I'd agree, even though it's a pain.
QA referred the question of whether upgrades from a release where i686 was 'release blocking' (<24) to releases where i686 is 'non blocking' (>23) should be considered 'release blocking' to FESCo. i.e. if there are violations of the release criteria in this upgrade path, should we treat that as blocking the Beta or Final releases. FESCo's decision was "no".
I really think it would help if we use these terms carefully and precisely, and if we're going to re-define them in any way, make that clear and explicit.
On Tue, Apr 19, 2016 at 1:11 PM, Adam Williamson adamwill@fedoraproject.org wrote:
QA referred the question of whether upgrades from a release where i686 was 'release blocking' (<24) to releases where i686 is 'non blocking' (>23) should be considered 'release blocking' to FESCo. i.e. if there are violations of the release criteria in this upgrade path, should we treat that as blocking the Beta or Final releases. FESCo's decision was "no".
So no matter what, all i686 images (qcow2, raw, ISOs) are non-blocking.
Any i686 package that fails to build means it's failed for all primary archs, because i686 is a primary arch. And a failed build means it won't be tagged for compose so depending on the package it could hold up composes.
But the current i686 problems aren't package build failures, rather it's a particular critical path package (or two) that are broadly or entirely non-functional when executed. So what's it called when a critical path package fails to function on a primary arch? And what's done about it?
From my limited perspective, such non-functional failure held up release when it violated a release criterion in effect because that non-functionality became coupled with image blocking, i.e. if kernel doesn't function, then image doesn't function/is DOA, DOA images are a release criteria violation, therefore block. Correct? Or is there some terminology nuance here that I'm still missing in the sequence?
I really think it would help if we use these terms carefully and precisely, and if we're going to re-define them in any way, make that clear and explicit.
It's best to assume I don't understand the terms well enough to use them precisely, rather than assume I'm trying to redefine them.
On Tue, 2016-04-19 at 13:48 -0600, Chris Murphy wrote:
On Tue, Apr 19, 2016 at 1:11 PM, Adam Williamson adamwill@fedoraproject.org wrote:
QA referred the question of whether upgrades from a release where i686 was 'release blocking' (<24) to releases where i686 is 'non blocking' (>23) should be considered 'release blocking' to FESCo. i.e. if there are violations of the release criteria in this upgrade path, should we treat that as blocking the Beta or Final releases. FESCo's decision was "no".
So no matter what, all i686 images (qcow2, raw, ISOs) are non-blocking.
Yes.
Any i686 package that fails to build means it's failed for all primary archs, because i686 is a primary arch. And a failed build means it won't be tagged for compose so depending on the package it could hold up composes.
True, though I hadn't actually mentioned that scenario. But indeed. Say we needed a fix to dracut, pronto, to make the x86_64 cloud base image boot, but the build with the fix failed on i686: that would have to be dealt with somehow. Good point.
But the current i686 problems aren't package build failures, rather it's a particular critical path package (or two) that are broadly or entirely non-functional when executed. So what's it called when a critical path package fails to function on a primary arch? And what's done about it?
A bug? :)
Basically AFAIK no particular 'special' status attaches to this situation. It's just...a bug.
From my limited perspective, such non-functional failure held up release when it violated a release criterion in effect because that non-functionality became coupled with image blocking, i.e. if kernel doesn't function, then image doesn't function/is DOA, DOA images are a release criteria violation, therefore block. Correct? Or is there some terminology nuance here that I'm still missing in the sequence?
No, even in this case there is no release blocking impact, because nothing release blocking is broken by the bug. The i686 images are not release blocking, end of story. Even if they are completely DOA, that does not block release.
This is exactly the situation that occurred with F24 Alpha, as it happens. All 32-bit F24 Alpha images fail to boot in at least a lot of cases, maybe all cases. We went ahead and shipped Alpha anyway. The only concession was to try and downplay the existence of the 32-bit images in the release notes and download pages.
I really think it would help if we use these terms carefully and precisely, and if we're going to re-define them in any way, make that clear and explicit.
It's best to assume I don't understand the terms well enough to use them precisely, rather than assume I'm trying to redefine them.
I was not actually thinking of you there (I just picked your post to reply to since it was at the top of the pile), more the vagueness in the thread in general.
On Tue, Apr 19, 2016 at 2:48 PM, Adam Williamson adamwill@fedoraproject.org wrote:
On Tue, 2016-04-19 at 13:48 -0600, Chris Murphy wrote:
From my limited perspective, such non-functional failure held up release when it violated a release criterion in effect because that non-functionality became coupled with image blocking, i.e. if kernel doesn't function, then image doesn't function/is DOA, DOA images are a release criteria violation, therefore block. Correct? Or is there some terminology nuance here that I'm still missing in the sequence?
No, even in this case there is no release blocking impact, because nothing release blocking is broken by the bug. The i686 images are not release blocking, end of story. Even if they are completely DOA, that does not block release.
Yes, I meant i686 in the past tense.
OK so I think I get it. i686 is officially primary, but in practice it's at best secondary. And that should be made official. TBD whether there's even enough people power and momentum to support it as secondary.
It's best to assume I don't understand the terms well enough to use them precisely, rather than assume I'm trying to redefine them.
I was not actually thinking of you there (I just picked your post to reply to since it was at the top of the pile), more the vagueness in the thread in general.
Got it.
On Tue, 2016-04-19 at 15:23 -0600, Chris Murphy wrote:
On Tue, Apr 19, 2016 at 2:48 PM, Adam Williamson adamwill@fedoraproject.org wrote:
On Tue, 2016-04-19 at 13:48 -0600, Chris Murphy wrote:
From my limited perspective, such non-functional failure held up release when it violated a release criterion in effect because that non-functionality became coupled with image blocking, i.e. if kernel doesn't function, then image doesn't function/is DOA, DOA images are a release criteria violation, therefore block. Correct? Or is there some terminology nuance here that I'm still missing in the sequence?
No, even in this case there is no release blocking impact, because nothing release blocking is broken by the bug. The i686 images are not release blocking, end of story. Even if they are completely DOA, that does not block release.
Yes, I meant i686 in the past tense.
OK so I think I get it. i686 is officially primary, but in practice it's at best secondary.
NONONONO SEE THAT'S WHAT I'M TALKING ABOUT
It is primary. That's all. It is a primary arch. In all the actual meanings of that term. It is not 'in practice' secondary. It is primary. "In practice secondary" is a bad way to describe what you're trying to say and just confuses things. Please don't. :)
And that should be made official. TBD whether there's even enough people power and momentum to support it as secondary.
If people want to lobby for i686 to be made a secondary arch, in the true meaning of that term, I'm completely fine with that. I just want to make sure anyone who says it is actually clear on what it means, and that it is, in fact, what they want.
On Tue, Apr 19, 2016 at 4:27 PM, Adam Williamson adamwill@fedoraproject.org wrote:
On Tue, 2016-04-19 at 15:23 -0600, Chris Murphy wrote:
On Tue, Apr 19, 2016 at 2:48 PM, Adam Williamson adamwill@fedoraproject.org wrote:
On Tue, 2016-04-19 at 13:48 -0600, Chris Murphy wrote:
From my limited perspective, such non-functional failure held up release when it violated a release criterion in effect because that non-functionality became coupled with image blocking, i.e. if kernel doesn't function, then image doesn't function/is DOA, DOA images are a release criteria violation, therefore block. Correct? Or is there some terminology nuance here that I'm still missing in the sequence?
No, even in this case there is no release blocking impact, because nothing release blocking is broken by the bug. The i686 images are not release blocking, end of story. Even if they are completely DOA, that does not block release.
Yes, I meant i686 in the past tense.
OK so I think I get it. i686 is officially primary, but in practice it's at best secondary.
NONONONO SEE THAT'S WHAT I'M TALKING ABOUT
It is primary. That's all. It is a primary arch. In all the actual meanings of that term. It is not 'in practice' secondary. It is primary. "In practice secondary" is a bad way to describe what you're trying to say and just confuses things. Please don't. :)
OK to me it seems like some kind of loophole has opened up here. Since the critical packages build OK, composes continue. If such a package didn't build, composes would stop. But the backend effect is the same either way, the binary execution on i686 is fatal.
Alpha has shipped without i686, and we've heard crickets and maybe a couple of zombie moans (if that). Do we ship beta and final without it working? It's maybe not yet ridiculous, but I think everyone would agree if Fedora 24 ships without any i686 media at all, considering i686 a primary architecture is kinda funny. You know, funny like the body out in the middle of the street, that each time someone brave enough goes to check on the pulse just comes back and shrugs because they don't really know so they just leave the body out there. "Hey maybe he'll stand up next week!"
https://fedoraproject.org/wiki/Architectures
When I read that, i686 sure doesn't seem like it's primary. But you're right, it's definitely not secondary.
And that should be made official. TBD whether there's even enough people power and momentum to support it as secondary.
If people want to lobby for i686 to be made a secondary arch, in the true meaning of that term, I'm completely fine with that. I just want to make sure anyone who says it is actually clear on what it means, and that it is, in fact, what they want.
Agreed.
On Tue, 2016-04-19 at 17:22 -0600, Chris Murphy wrote:
It is primary. That's all. It is a primary arch. In all the actual meanings of that term. It is not 'in practice' secondary. It is primary. "In practice secondary" is a bad way to describe what you're trying to say and just confuses things. Please don't. :)
OK to me it seems like some kind of loophole has opened up here. Since the critical packages build OK, composes continue. If such a package didn't build, composes would stop.
No they would not. This is the thing you're missing. The composes only break if whichever package build most recently succeeded and got pushed stable is now so broken it prevents the compose working.
If 'foo-1.0-1.fc25' is in Rawhide right now and you send a build of 'foo-1.0-2.fc25' and it fails on i686, 'foo-1.0-1.fc25' doesn't magically disappear. It stays there. The consequence is just that 'foo- 1.0-2.fc25' doesn't go out.
The 'loophole' you describe *does* exist, but it's not as severe as you think.
Alpha has shipped without i686, and we've heard crickets and maybe a couple of zombie moans (if that). Do we ship beta and final without it working?
The current agreement is that we would, yes. This is what has been explicitly decided.
It's maybe not yet ridiculous, but I think everyone would agree if Fedora 24 ships without any i686 media at all, considering i686 a primary architecture is kinda funny.
Sure, I agree. But that is how things are at present. That's how they work within the systems we've built. 'primary arch' has a definite and specific meaning that is tied to the release engineering systems that are in place, it's not just an arbitrary term. If you want it to mean something else we have to change what those systems actually *do*. If you want i686 not to be a 'primary arch' any more, the meaning of that is 'enforced' by that term's association with how the release engineering tools behave.
You know, funny like the body out in the middle of the street, that each time someone brave enough goes to check on the pulse just comes back and shrugs because they don't really know so they just leave the body out there. "Hey maybe he'll stand up next week!"
https://fedoraproject.org/wiki/Architectures
When I read that, i686 sure doesn't seem like it's primary. But you're right, it's definitely not secondary.
I would kinda quibble with that page. I would especially disagree with the text "To put it simply: These are the architectures for which Fedora will delay a release if they are not functional." That is *not* the actual definition of a 'primary arch', and I think whoever added it had an imperfect understanding.
I like wiki pages, but when they're wrong, they're wrong. =)
The dissonance here is really because in the past there *was* a perfect overlap between 'primary arches' and 'release blocking' deliverables: the exact same set of deliverables was 'release blocking' for each primary arch. But that was never really a cast-iron guarantee. It started to fray when ARM became a primary arch (because ARM's release- blocking deliverables are significantly different from the Intel arches') and broke entirely with the decision that no i686 images would be 'release blocking', *without* a decision that i686 was no longer 'primary'.
Now we *COULD* absolutely enact some kind of stronger relationship between primary and secondary arches, and state that by policy any arch with no release-blocking deliverables can't possibly be a 'primary arch' and thus i686 must be demoted. We could do lots of things! But that is *not* the current state of affairs.
On Tue, Apr 19, 2016 at 04:33:27PM -0700, Adam Williamson wrote:
https://fedoraproject.org/wiki/Architectures
When I read that, i686 sure doesn't seem like it's primary. But you're right, it's definitely not secondary.
I would kinda quibble with that page. I would especially disagree with the text "To put it simply: These are the architectures for which Fedora will delay a release if they are not functional." That is *not* the actual definition of a 'primary arch', and I think whoever added it had an imperfect understanding.
I like wiki pages, but when they're wrong, they're wrong. =)
Maybe it's better to say that the definition (as you are using it) has become more precise with time? The wiki history shows that phrasing as being there since the 2008 import from MoinMoin.
Now we *COULD* absolutely enact some kind of stronger relationship between primary and secondary arches, and state that by policy any arch with no release-blocking deliverables can't possibly be a 'primary arch' and thus i686 must be demoted. We could do lots of things! But that is *not* the current state of affairs.
Someone should fix the wiki. For now, I've simply removed the line about blocking the release, but it might be better to have more detail.
On Wed, Apr 20, 2016 at 04:35:45AM -0400, Matthew Miller wrote:
I would kinda quibble with that page. I would especially disagree with the text "To put it simply: These are the architectures for which Fedora will delay a release if they are not functional." That is *not* the actual definition of a 'primary arch', and I think whoever added it had an imperfect understanding.
I like wiki pages, but when they're wrong, they're wrong. =)
Maybe it's better to say that the definition (as you are using it) has become more precise with time? The wiki history shows that phrasing as being there since the 2008 import from MoinMoin.
Or, thinking about it another way: the terms are overloaded. There is the technical aspect of koji builds. There is the aspect of release blocking. And, there's the aspect of what we promote as a project and make user facing. Adam, I think you're arguing that we really shouldn't use "primary" and "secondary" for anything but the first. This is hard, because they're powerful words that _seem_ useful for describing main effort vs. other. I think that unless we come up with some other agreed-upon and equally powerful language, the less-technical sense is going to keep creeping back into use. Or, we could focus on the build system and use "koji-primary" and "koji-secondary" for that concept, making clear that it's technical jargon.
Maybe I'm overthinking, but this whole thread suggests that I'm not the only one. :)
On Wed, 2016-04-20 at 04:46 -0400, Matthew Miller wrote:
On Wed, Apr 20, 2016 at 04:35:45AM -0400, Matthew Miller wrote:
I would kinda quibble with that page. I would especially disagree with the text "To put it simply: These are the architectures for which Fedora will delay a release if they are not functional." That is *not* the actual definition of a 'primary arch', and I think whoever added it had an imperfect understanding.
I like wiki pages, but when they're wrong, they're wrong. =)
Maybe it's better to say that the definition (as you are using it) has become more precise with time? The wiki history shows that phrasing as being there since the 2008 import from MoinMoin.
Or, thinking about it another way: the terms are overloaded. There is the technical aspect of koji builds. There is the aspect of release blocking. And, there's the aspect of what we promote as a project and make user facing. Adam, I think you're arguing that we really shouldn't use "primary" and "secondary" for anything but the first. This is hard, because they're powerful words that _seem_ useful for describing main effort vs. other. I think that unless we come up with some other agreed-upon and equally powerful language, the less-technical sense is going to keep creeping back into use. Or, we could focus on the build system and use "koji-primary" and "koji-secondary" for that concept, making clear that it's technical jargon.
Maybe I'm overthinking, but this whole thread suggests that I'm not the only one. :)
I understand what you're saying, but the reason I'm so insistent on it is that this isn't just my usual linguistic nitpicking but an entirely practical issue. When you say 'i686 should be a secondary arch', and Dennis or Kevin reads it, what they understand by that is exactly what I've been saying, what you want to call 'koji-secondary'. If someone sends an 'i686 should be secondary' proposal to FESCo and it gets passed, what is going to happen is that releng is going to make it a secondary arch in the strict technical sense of the term I've been explaining. This is why I think it's rather important to use the term in the way that's understood by the people in charge of the things that make the bits. ;)
I think it'd be *extremely* confusing to try and draw some distinction between 'primary' and 'koji primary' and expect everyone to always keep that straight. If we want i686 to not actually be a 'secondary arch' but continue to live in the state it's in right now, it'd be a much better idea to come up with a better way of describing its current state, and also consider what we want to do about what cmurf calls the 'loophole'.
On Wed, Apr 20, 2016 at 04:30:39PM -0700, Adam Williamson wrote:
I understand what you're saying, but the reason I'm so insistent on it is that this isn't just my usual linguistic nitpicking but an entirely practical issue. When you say 'i686 should be a secondary arch', and Dennis or Kevin reads it, what they understand by that is exactly what I've been saying, what you want to call 'koji-secondary'. If someone sends an 'i686 should be secondary' proposal to FESCo and it gets passed, what is going to happen is that releng is going to make it a secondary arch in the strict technical sense of the term I've been explaining. This is why I think it's rather important to use the term in the way that's understood by the people in charge of the things that make the bits. ;)
That's fair, but it goes the other way too — the way the people making the bits are using the term isn't always obvious to everyone else. It might be easier to get a few technical users to change their jargon than it will be to continually explain to incoming contributors that the words have this precise non-obvious meaning.
On Tue, Apr 19, 2016 at 2:48 PM, Adam Williamson adamwill@fedoraproject.org wrote:
On Tue, 2016-04-19 at 13:48 -0600, Chris Murphy wrote:
Any i686 package that fails to build means it's failed for all primary archs, because i686 is a primary arch. And a failed build means it won't be tagged for compose so depending on the package it could hold up composes.
True, though I hadn't actually mentioned that scenario. But indeed. Say we needed a fix to dracut, pronto, to make the x86_64 cloud base image boot, but the build with the fix failed on i686: that would have to be dealt with somehow. Good point.
Oh and about terminology, it may be here where "block" gets reused as a term in a confusing way. If dracut build fails on i686, that "blocks" composes. But it's really a kind of claw back: zombie i686 is grabbing the leg of other primary archs, and that stops the workflow.
Making i686 secondary would prevent this?
On Tue, Apr 19, 2016 at 5:37 PM, Chris Murphy lists@colorremedies.com wrote:
On Tue, Apr 19, 2016 at 2:48 PM, Adam Williamson adamwill@fedoraproject.org wrote:
On Tue, 2016-04-19 at 13:48 -0600, Chris Murphy wrote:
Any i686 package that fails to build means it's failed for all primary archs, because i686 is a primary arch. And a failed build means it won't be tagged for compose so depending on the package it could hold up composes.
True, though I hadn't actually mentioned that scenario. But indeed. Say we needed a fix to dracut, pronto, to make the x86_64 cloud base image boot, but the build with the fix failed on i686: that would have to be dealt with somehow. Good point.
Oh and about terminology, it may be here where "block" gets reused as a term in a confusing way. If dracut build fails on i686, that "blocks" composes. But it's really a kind of claw back: zombie i686 is grabbing the leg of other primary archs, and that stops the workflow.
Making i686 secondary would prevent this?
Concretely, in today's existing infrastructure and world, yes. A secondary arch does builds on a separate koji instance and failing builds there don't impact the builds in primary.
However, a while ago Dennis proposed a different world (with related infrastructure changes) where that wouldn't necessarily be the case. I forget where we had the discussion about it. It might have been the rel-eng list. He hasn't pushed it much publicly though, and I don't want to speak for him.
josh
On Tue, 2016-04-19 at 17:41 -0400, Josh Boyer wrote:
Concretely, in today's existing infrastructure and world, yes. A secondary arch does builds on a separate koji instance and failing builds there don't impact the builds in primary.
However, a while ago Dennis proposed a different world (with related infrastructure changes) where that wouldn't necessarily be the case. I forget where we had the discussion about it. It might have been the rel-eng list. He hasn't pushed it much publicly though, and I don't want to speak for him.
I think my understanding of the proposal was that secondary arch builds would happen in the primary Koji instance, but still would not prevent the primary arch builds from being tagged. IMBW, though.
On Tue, 2016-04-19 at 15:37 -0600, Chris Murphy wrote:
On Tue, Apr 19, 2016 at 2:48 PM, Adam Williamson adamwill@fedoraproject.org wrote:
On Tue, 2016-04-19 at 13:48 -0600, Chris Murphy wrote:
Any i686 package that fails to build means it's failed for all primary archs, because i686 is a primary arch. And a failed build means it won't be tagged for compose so depending on the package it could hold up composes.
True, though I hadn't actually mentioned that scenario. But indeed. Say we needed a fix to dracut, pronto, to make the x86_64 cloud base image boot, but the build with the fix failed on i686: that would have to be dealt with somehow. Good point.
Oh and about terminology, it may be here where "block" gets reused as a term in a confusing way. If dracut build fails on i686, that "blocks" composes.
No, not really, it only 'blocks composes' if the existing stable dracut build can't be used in the compose for some reason.
But it is true to say that there are ways in which a primary arch that has no blocking images (as i686 currently is) can impact on the 'deliverability' of blocking images for other primary arches. Yes. And if we think that's a big problem, it is a very legitimate issue to use in support of the argument that i686 should be made a secondary arch.
But it's really a kind of claw back: zombie i686 is grabbing the leg of other primary archs, and that stops the workflow.
Making i686 secondary would prevent this?
Yes it would.
On Tuesday, April 19, 2016 3:37:05 PM CDT Chris Murphy wrote:
On Tue, Apr 19, 2016 at 2:48 PM, Adam Williamson
adamwill@fedoraproject.org wrote:
On Tue, 2016-04-19 at 13:48 -0600, Chris Murphy wrote:
Any i686 package that fails to build means it's failed for all primary archs, because i686 is a primary arch. And a failed build means it won't be tagged for compose so depending on the package it could hold up composes.
True, though I hadn't actually mentioned that scenario. But indeed. Say we needed a fix to dracut, pronto, to make the x86_64 cloud base image boot, but the build with the fix failed on i686: that would have to be dealt with somehow. Good point.
Oh and about terminology, it may be here where "block" gets reused as a term in a confusing way. If dracut build fails on i686, that "blocks" composes. But it's really a kind of claw back: zombie i686 is grabbing the leg of other primary archs, and that stops the workflow.
Making i686 secondary would prevent this?
if a dracut i686 build fails the dracut build fails and nothing changes, the compose is not blocked. your view here is not quite the reality of the world. moving i686 to secondary does not change that. particullary in the way releng is looking to redefine secondary arches. It is also the only way we have to not force us to drop i686 being multilib on x86_64
Dennis
On Wed, 2016-04-20 at 11:43 -0500, Dennis Gilmore wrote:
if a dracut i686 build fails the dracut build fails and nothing changes, the compose is not blocked. your view here is not quite the reality of the world. moving i686 to secondary does not change that.
well, wait, moving i686 to secondary *does* change that, doesn't it? If i686 is secondary, and a dracut i686 build fails, then the dracut x86_64 and armhfp builds will still get tagged, right? Just like right now, if dracut fails to build on aarch64 but succeeds on the primary arches, those primary arch builds get tagged.
particullary in the way releng is looking to redefine secondary arches.
I would like to read more about this 'redefinition'. Where can I?
On Thu, Apr 21, 2016 at 12:32 AM, Adam Williamson adamwill@fedoraproject.org wrote:
On Wed, 2016-04-20 at 11:43 -0500, Dennis Gilmore wrote:
if a dracut i686 build fails the dracut build fails and nothing changes, the compose is not blocked. your view here is not quite the reality of the world. moving i686 to secondary does not change that.
well, wait, moving i686 to secondary *does* change that, doesn't it? If i686 is secondary, and a dracut i686 build fails, then the dracut x86_64 and armhfp builds will still get tagged, right? Just like right now, if dracut fails to build on aarch64 but succeeds on the primary arches, those primary arch builds get tagged.
No, because we still need i686 in x86_64 multilib because of some proprietary "stuff" (no idea, I don't use it)
particullary in the way releng is looking to redefine secondary arches.
I would like to read more about this 'redefinition'. Where can I?
In Flock Rochester talks. It was discussed in Dennis's rel-eng talk, that should all be on video, and in the hallway quite widely where I discussed it with a lot of groups/individuals.
Basically for example it currently makes sense to say promote aarch64 for server but not workstation (although that's starting to actually move somewhere) and cloud so how do you define primary and secondary? Well basically you remove koji from that definition and define it on the product outputs. With i686 discussing/doing this for cloud/server that is already basically happening. This is already done internally for brew (internal brew for those that don't know) where all architectures are built in the one build system.
As a person that has dealt with builds across 6 secondary arches over the years the "failures" come into a few general categories: * archful builds that if they fail will fail on any architecture - this is 95% of the package set and the maintainers already deal with that * noarch builds that generally don't cause issue (and could currently get x86_64/armv7/ppc64/ppc64le builders now anyway) * core toolchain build issues (gcc/binutils/glibc and friends) - the maintainers already deal with secondary arches and end up doing multiple koji builds of their packages now * kernel - already has process that actively deals with secondary arches * a of arch specific HW - already dealt with in packages either by the maintainers or arch maintainers * software written for a specific arch - as with arch specific HW, but this may change in time (see docker stack as a good example here) again already explicitly dealt with and gets adjusted with time * packages with arch specific failures - bugs filed, arch specialists assist, already process in place, would need adjustment, the biggest one here is generally endian assumptions. These are generally a very small amount a cycle easily < 1% of the package set
So in your dracut example above I don't ever remember seeing it fail on other arches and not on x86_64 too.
Peter
On Thu, 2016-04-21 at 12:44 +0100, Peter Robinson wrote:
On Thu, Apr 21, 2016 at 12:32 AM, Adam Williamson adamwill@fedoraproject.org wrote:
On Wed, 2016-04-20 at 11:43 -0500, Dennis Gilmore wrote:
if a dracut i686 build fails the dracut build fails and nothing changes, the compose is not blocked. your view here is not quite the reality of the world. moving i686 to secondary does not change that.
well, wait, moving i686 to secondary *does* change that, doesn't it? If i686 is secondary, and a dracut i686 build fails, then the dracut x86_64 and armhfp builds will still get tagged, right? Just like right now, if dracut fails to build on aarch64 but succeeds on the primary arches, those primary arch builds get tagged.
No, because we still need i686 in x86_64 multilib because of some proprietary "stuff" (no idea, I don't use it)
Oh, right, so in theory it would, except multilib complicates things. Makes sense.
particullary in the way releng is looking to redefine secondary arches.
I would like to read more about this 'redefinition'. Where can I?
In Flock Rochester talks. It was discussed in Dennis's rel-eng talk, that should all be on video, and in the hallway quite widely where I discussed it with a lot of groups/individuals.
Basically for example it currently makes sense to say promote aarch64 for server but not workstation (although that's starting to actually move somewhere) and cloud so how do you define primary and secondary? Well basically you remove koji from that definition and define it on the product outputs. With i686 discussing/doing this for cloud/server that is already basically happening. This is already done internally for brew (internal brew for those that don't know) where all architectures are built in the one build system.
I'm fine with that, but if we're going to do that, I'd rather retire the 'primary' / 'secondary' terms as I think they'd only cause confusion.
On Tue, Apr 19, 2016 at 12:11:04PM -0700, Adam Williamson wrote:
I really think it would help if we use these terms carefully and precisely, and if we're going to re-define them in any way, make that clear and explicit.
Thanks Adam. I've been guilty of using them incorrectly (or at least sloppily) and will try to mend my ways.
This is probably mostly because "non-blocking" seems like it needs more explanation than "secondary" ("non-blocking" is generally a _good_ thing in software, right?). Also, it seems pretty tied to only what happens at release time, when really, what we want to say is "this is best-effort at release, and, if that release effort happens to go well, keeping it maintained is in a similar boat".
So, if we redefine anything, maybe "blocking" and "best-effort"? (I'm fine with "non-blocking" as a technical term... I'm just thinking about explaining it to people....)
On Tue, Apr 12, 2016 at 11:18:05AM -0400, Joe Brockmeier wrote:
On 04/07/2016 06:59 PM, Joe Brockmeier wrote:
On 04/07/2016 06:47 PM, Ryan Lerch wrote:
Will it just be a featured image? Or are there other graphics required?
Just the featured image. Thanks!
OK, looks like we have the featured image here:
https://pagure.io/fedoramagazine-images/issue/12
Is this ready to go? Is there a specific date to target or run it now, or...?
I can't make tonight's Magazine meeting, but I'm +1 to get this out ASAP. I've moved this article to Pending Review, but I already reviewed it and it looks fine. I added a couple additional links for SEO power.
Anything Justin OK's for schedule for this is fine by me, assume +1. :-) And I'll be happy to help with social media, just please leave me an email or IRC message to do so and I'll catch it in the morning.
On 04/21/2016 05:04 PM, Paul W. Frields wrote:
I can't make tonight's Magazine meeting, but I'm +1 to get this out ASAP. I've moved this article to Pending Review, but I already reviewed it and it looks fine. I added a couple additional links for SEO power.
Anything Justin OK's for schedule for this is fine by me, assume +1. :-) And I'll be happy to help with social media, just please leave me an email or IRC message to do so and I'll catch it in the morning.
I'd like to put this up too, but I don't think we've settled the question on 32-bit that Dennis raised. Will check again.
Hi all,
Somehow despite me sending this last night - this was published. How can we prevent communication breakdowns like this in the future?
This was sent prior to or during the magazine meeting, if I'm not mistaken.
If there's *any* uncertainty around something like this, it should not be published.
Best,
jzb
On 04/21/2016 05:10 PM, Joe Brockmeier wrote:
On 04/21/2016 05:04 PM, Paul W. Frields wrote:
I can't make tonight's Magazine meeting, but I'm +1 to get this out ASAP. I've moved this article to Pending Review, but I already reviewed it and it looks fine. I added a couple additional links for SEO power.
Anything Justin OK's for schedule for this is fine by me, assume +1. :-) And I'll be happy to help with social media, just please leave me an email or IRC message to do so and I'll catch it in the morning.
I'd like to put this up too, but I don't think we've settled the question on 32-bit that Dennis raised. Will check again.
On Fri, Apr 22, 2016 at 09:59:40AM -0400, Joe Brockmeier wrote:
Hi all,
Somehow despite me sending this last night - this was published. How can we prevent communication breakdowns like this in the future?
This was sent prior to or during the magazine meeting, if I'm not mistaken.
If there's *any* uncertainty around something like this, it should not be published.
Agreed, and it's incumbent on whoever schedules a post to confirm it's meant to go out. The bar is higher for articles about what we deliver, as opposed to helpful how-to articles. It's hard to see how a premature article like that would cause any issues, but clearly this one could.
Also, this was picked up by at least one venue that checks our feeds for reportable news: https://www.phoronix.com/scan.php?page=news_item&px=Fedora-Cloud-32-bit-...
Does someone have a contact with Phoronix so we could explain the snafu here?
-
Adam Williamson -
Chris Murphy -
Dennis Gilmore -
Joe Brockmeier -
Josh Boyer -
Matthew Miller -
Paul W. Frields -
Peter Robinson