Hi folks!
So we're now frozen for Alpha and we have an anaconda build with blocker fixes in updates-testing...I guess this means it's a good time to decide what to do about milestone builds with Pungi 4. :)
I'm assuming here that we're going to use more or less the same "TCs and RCs" approach for F24 and just try to adapt it to Pungi 4. It seems a bit late to make radical changes to the process itself at this point.
I have a couple of ideas. I'm gonna recap, briefly, what (as I understand it) Pungi 4 expects us to do for milestone builds:
We're expected to run 'pungi-koji --label foo', where 'foo' is a compose 'label' in Pungi's expected format. AIUI, this is the format Pungi expects for a compose 'label':
MILESTONE-N.R
where MILESTONE is the milestone (there is a list of valid milestones; for our purposes we're likely to use Alpha, Beta and RC), 'N' is the release number, and 'R' is a respin number. Both N and R have to be integers (though they can have multiple digits; Alpha-11.23 would be a valid label).
Fedora does not do numbered milestone releases; we just have Alpha, Beta and Final. RHEL does, which is why the label format has 'N'. In the RHEL process, as I understand it, they build "Alpha-1.1", then if it passes whatever requirements are needed for an "Alpha 1" release, they ship it as Alpha 1; otherwise they build "Alpha-1.2" and ship that if it passes, etc etc. The "respin" value is considered "private", and respins are used for about the same purpose we use TCs/RCs.
It *is* possible to do a 'production' compose with no label, I believe, by calling 'pungi-koji --no-label'. It would *also* be technically possible to add new label formats to Pungi. So we do have three options at the high level:
a. Work with Pungi's currently-expected label format b. Patch Pungi to support a more Fedora-y label format c. Don't apply a label to our TC/RC composes at all
I would suggest that a is the best practical choice, though, because we know this is how Pungi is kind of expected to be used. b is likely to take time to work out any kinks, and c seems unnecessarily extreme; we should be able to come up with a usable label format that gives us valuable information.
So assuming we go with 1) at the high level, I see a few possibilities for how to do this.
== 1. Just use one value ==
In this design, we'd set one of the numbers to 1 (or I guess 0) for every single Fedora compose, and just increment the other. I was thinking we'd lock down the release number (N) and just increment the respin (R) - so we'd do Alpha-1.1 then Alpha-1.2 then Alpha-1.3 and so on.
This seems like it'd work fine for Alpha and Beta but be a little odd for Final. 'Final' is not a valid milestone for Pungi 4, it expects you to use 'RC' as a milestone and build 'RC1.1', 'RC1.2', 'RC1.3', 'RC2.1' etc etc. It'd be a bit odd for us to have 'RC1.1' as 'Final TC1', 'RC1.2' as 'Final TC2', 'RC1.3' as 'Final RC1', etc. So we could instead lock down R and only increment N, so we'd do 'RC 1.1', 'RC 2.1', 'RC 3.1' and so on. The build called 'RC 1.1' would likely not really be a "real" RC (in the sense it'd happen while we were not frozen and not have all blocker bugs fixed), but that's not horrible. I suppose we could relatively easily add 'Final' to Pungi's list of milestones too, if we wanted (or 'TC', come to that).
The other possible problem with this is it makes it hard to keep the TC/RC distinction. We could build the composes as 'Alpha 1.1', 'Alpha 2.1', 'Alpha 3.1' and put 'Alpha 1.1' in a directory called 'Alpha_TC1', 'Alpha 2.1' in 'Alpha_TC2', and 'Alpha 3.1' in 'Alpha_RC1' if we wanted, but you couldn't easily map from one name to the other, because there's no way to know the cutoff where we switched over - if you only know you're dealing with 'Alpha 2.1', you can't tell if that's 'Alpha TC2' or 'Alpha RC1'.
The question then is, do we actually need the TC/RC distinction for anything? I'm not sure we really do. But if we want to keep it, we could try...
== 2. N indicates TC/RC, R indicates number ==
In this scheme, we'd build e.g. 'Alpha 1.1' (Alpha TC1), 'Alpha 1.2' (Alpha TC2), 'Alpha 2.1' (Alpha RC1), 'Alpha 2.2' (Alpha RC2).
This seems like the closest possible way to map to our current system. Again it's a bit weird at Final because there is no 'Final' milestone, only 'RC', so 'RC1.1' would be 'TC1' and 'RC2.1' would be 'RC1', which is kinda strange; again we could add a 'Final' milestone to Pungi, I guess.
I'm just not sure, as per 1), if we really *need* to maintain the TC/RC distinction at least in terms of how the composes are labelled and distributed.
== 3. Some kinda hybrid ==
We could combine things, so we do 1.n for Alpha and Beta, then n.1 for RC (so we have Alpha 1.1, Alpha 1.2, Alpha 1.3, Alpha 1.4, Beta 1.1, Beta 1.2, Beta 1.3, RC1.1, RC2.1, RC3.1 etc). This is viable, but I'm not really sure it's necessary. It seems simpler to pick some variant of 1 or 2.
There's a final question, which is whether we adopt the Pungi 'label' naming format for all purposes - so we put the composes in directories named Alpha-1.1, Alpha-1.2, Alpha-1.3 and so on, and we call the validation events something like 'Test Results:Fedora 24 Alpha 1.1' rather than 'Test Results:Fedora 24 Alpha TC1'. It's probably simpler to just go with the label names, but if people are really used to the TC/RC names we can stick with them if we choose option 2 or 3. It would be difficult to keep them with option 1. I'm not sure what Dennis' thoughts are on this.
As a final note, just so people are aware: all Pungi 4 composes have compose IDs, so a 'production' compose with a label has *both* a compose ID *and* a label. The compose ID for a production compose is, I believe, something like 'Fedora-24-20160309.0' - it looks quite a lot like a nightly compose ID, but without the '.n.' that denotes nightly. There is no letter identifier for 'production' composes, they just omit it entirely. This is mostly a thing for me to deal with in tooling, though... -- Adam WilliamsonFedora QA Community MonkeyIRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . nethttp://www.happyassassin.net
On Wed, Mar 9, 2016 at 11:16 AM, Adam Williamson adamwill@fedoraproject.org wrote:
Hi folks!
So we're now frozen for Alpha and we have an anaconda build with blocker fixes in updates-testing...I guess this means it's a good time to decide what to do about milestone builds with Pungi 4. :)
I'm assuming here that we're going to use more or less the same "TCs and RCs" approach for F24 and just try to adapt it to Pungi 4. It seems a bit late to make radical changes to the process itself at this point.
I have a couple of ideas. I'm gonna recap, briefly, what (as I understand it) Pungi 4 expects us to do for milestone builds:
We're expected to run 'pungi-koji --label foo', where 'foo' is a compose 'label' in Pungi's expected format. AIUI, this is the format Pungi expects for a compose 'label':
MILESTONE-N.R
where MILESTONE is the milestone (there is a list of valid milestones; for our purposes we're likely to use Alpha, Beta and RC), 'N' is the release number, and 'R' is a respin number. Both N and R have to be integers (though they can have multiple digits; Alpha-11.23 would be a valid label).
Fedora does not do numbered milestone releases; we just have Alpha, Beta and Final. RHEL does, which is why the label format has 'N'. In the RHEL process, as I understand it, they build "Alpha-1.1", then if it passes whatever requirements are needed for an "Alpha 1" release, they ship it as Alpha 1; otherwise they build "Alpha-1.2" and ship that if it passes, etc etc. The "respin" value is considered "private", and respins are used for about the same purpose we use TCs/RCs.
It *is* possible to do a 'production' compose with no label, I believe, by calling 'pungi-koji --no-label'. It would *also* be technically possible to add new label formats to Pungi. So we do have three options at the high level:
a. Work with Pungi's currently-expected label format
In this format option, MILESTONE-N.R would be MILESTONE= alpha, beta, RC N = arbitrary integer R = arbitrary integer
Can N and R be 0? Or must it be 1-9?
So we could instead lock down R and only increment N, so we'd do 'RC 1.1', 'RC 2.1', 'RC 3.1' and so on.
To reduce granularity, lock down either N or R. But if locked down, I suggest a value of 0 rather than 1 to indicate nullification.
The build called 'RC 1.1' would likely not really be a "real" RC (in the sense it'd happen while we were not frozen and not have all blocker bugs fixed), but that's not horrible. I suppose we could relatively easily add 'Final' to Pungi's list of milestones too, if we wanted (or 'TC', come to that).
At least from the test side, I'm not thinking there was ever a meaningful distinction between TC and RC for alpha and beta. For final, I vaguely think TCs and RCs did change behavior in some ways, maybe it was disabling u-t by default? Or maybe some installer labeling or warning goes away?
If you want to keep some granularity to distinguish between TC/RC you could use N=0 to indicate "TC" and N=1 to indicate "RC". :-P
TC's alpha-0.1 alpha-0.2 alpha-0.3 alpha-0.4 beta-0.1 beta-0.2 beta-0.3 RC-0.1 RC-0.2
legit RC's alpha-1.1 alpha-1.2 alpha-1.3 alpha-1.4 beta-1.1 beta-1.2 beta-1.3 RC-1.1 RC-1.2
Further, if there is no meaningful distinction between TC/RC during alpha and beta, then those would always be N=1. You'd only nullify "RC" with a trailing 0 for the final TC's.
On Wed, Mar 9, 2016 at 1:47 PM, Chris Murphy lists@colorremedies.com wrote:
On Wed, Mar 9, 2016 at 11:16 AM, Adam Williamson adamwill@fedoraproject.org wrote:
Hi folks!
So we're now frozen for Alpha and we have an anaconda build with blocker fixes in updates-testing...I guess this means it's a good time to decide what to do about milestone builds with Pungi 4. :)
I'm assuming here that we're going to use more or less the same "TCs and RCs" approach for F24 and just try to adapt it to Pungi 4. It seems a bit late to make radical changes to the process itself at this point.
I have a couple of ideas. I'm gonna recap, briefly, what (as I understand it) Pungi 4 expects us to do for milestone builds:
We're expected to run 'pungi-koji --label foo', where 'foo' is a compose 'label' in Pungi's expected format. AIUI, this is the format Pungi expects for a compose 'label':
MILESTONE-N.R
where MILESTONE is the milestone (there is a list of valid milestones; for our purposes we're likely to use Alpha, Beta and RC), 'N' is the release number, and 'R' is a respin number. Both N and R have to be integers (though they can have multiple digits; Alpha-11.23 would be a valid label).
Fedora does not do numbered milestone releases; we just have Alpha, Beta and Final. RHEL does, which is why the label format has 'N'. In the RHEL process, as I understand it, they build "Alpha-1.1", then if it passes whatever requirements are needed for an "Alpha 1" release, they ship it as Alpha 1; otherwise they build "Alpha-1.2" and ship that if it passes, etc etc. The "respin" value is considered "private", and respins are used for about the same purpose we use TCs/RCs.
It *is* possible to do a 'production' compose with no label, I believe, by calling 'pungi-koji --no-label'. It would *also* be technically possible to add new label formats to Pungi. So we do have three options at the high level:
a. Work with Pungi's currently-expected label format
In this format option, MILESTONE-N.R would be MILESTONE= alpha, beta, RC N = arbitrary integer R = arbitrary integer
Can N and R be 0? Or must it be 1-9?
So we could instead lock down R and only increment N, so we'd do 'RC 1.1', 'RC 2.1', 'RC 3.1' and so on.
To reduce granularity, lock down either N or R. But if locked down, I suggest a value of 0 rather than 1 to indicate nullification.
The build called 'RC 1.1' would likely not really be a "real" RC (in the sense it'd happen while we were not frozen and not have all blocker bugs fixed), but that's not horrible. I suppose we could relatively easily add 'Final' to Pungi's list of milestones too, if we wanted (or 'TC', come to that).
At least from the test side, I'm not thinking there was ever a meaningful distinction between TC and RC for alpha and beta. For final, I vaguely think TCs and RCs did change behavior in some ways, maybe it was disabling u-t by default? Or maybe some installer labeling or warning goes away?
If you want to keep some granularity to distinguish between TC/RC you could use N=0 to indicate "TC" and N=1 to indicate "RC". :-P
TC's alpha-0.1 alpha-0.2 alpha-0.3 alpha-0.4 beta-0.1 beta-0.2 beta-0.3 RC-0.1 RC-0.2
legit RC's alpha-1.1 alpha-1.2 alpha-1.3 alpha-1.4 beta-1.1 beta-1.2 beta-1.3 RC-1.1 RC-1.2
Further, if there is no meaningful distinction between TC/RC during alpha and beta, then those would always be N=1. You'd only nullify "RC" with a trailing 0 for the final TC's.
-- Chris Murphy
So if "RC-1.8" passes all tests and is a go, how is this renamed? Or is it not renamed?
I assume you don't want to do a production compose at this point because then it's a compose that isn't tested, while ostensibly identical the process has any number of non-deterministic aspects to it, right? If so, is there an foreseen consequence of just renaming the filename of the ISOs? Or does the final distributed ISO need to retain the "RC-1.8" add-on?
It's sort of a minor point. There are build artifacts now as a single digit since I think Fedora 20, that don't mean anything to the outside world (e.g. for workstation it's 21-5, 22-3, 23-10 for the past three releases). But I don't think they hurt anything really, so if this RC-1.8 add-on is in the distributed ISO, it's only very slightly crusty. But compare this to how Windows eval downloads are named:
9600.17050.WINBLUE_REFRESH.140317-1640_X64FRE_ENTERPRISE_EVAL_EN-US-IR3_CENA_X64FREE_EN-US_DV9.ISO 10586.0.151029-1700.TH2_RELEASE_CLIENTENTERPRISEEVAL_OEMRET_X64FRE_EN-US.ISO
So yea, that's pretty terrible. One of those is Windows 8 the other Windows 10. But the world continues to function.
On Wed, 2016-03-09 at 13:57 -0700, Chris Murphy wrote:
So if "RC-1.8" passes all tests and is a go, how is this renamed? Or is it not renamed?
I assume you don't want to do a production compose at this point because then it's a compose that isn't tested, while ostensibly identical the process has any number of non-deterministic aspects to it, right? If so, is there an foreseen consequence of just renaming the filename of the ISOs? Or does the final distributed ISO need to retain the "RC-1.8" add-on?
I believe Dennis' position is that the RC is *exactly what we ship* - we don't rebuild or rename the images. We just sync the entire tree exactly as is into the release location. So it wouldn't be a directory called '24-RC-1.8' or whatever any more, but the image filenames and volume labels would still contain that identifier, if they originally did.
It's sort of a minor point. There are build artifacts now as a single digit since I think Fedora 20, that don't mean anything to the outside world (e.g. for workstation it's 21-5, 22-3, 23-10 for the past three releases). But I don't think they hurt anything really, so if this RC-1.8 add-on is in the distributed ISO, it's only very slightly crusty.
Right, exactly. -- Adam WilliamsonFedora QA Community MonkeyIRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . nethttp://www.happyassassin.net
On Wed, 2016-03-09 at 13:47 -0700, Chris Murphy wrote:
a. Work with Pungi's currently-expected label format In this format option, MILESTONE-N.R would be MILESTONE= alpha, beta, RC N = arbitrary integer R = arbitrary integer
Can N and R be 0? Or must it be 1-9?
At least according to the regex that tests their validity, they can be 0, yes.
At least from the test side, I'm not thinking there was ever a meaningful distinction between TC and RC for alpha and beta.
Right, this is my take on it too. The *definition* is clear enough: it's only an RC if it happens after the freeze *and* there are no outstanding blockers *and* releng flipped any necessary switches for 'release' behaviour.
There's also a distinction in terms of the content of the compose: for an RC compose, the filenames and volume labels are supposed to be in the form of the 'final' release, whereas for TCs, the filenames and volume labels are supposed to indicate TC-ness.
But so far as the actual test process goes, I don't think there's any need to distinguish them. Of course when it comes to Go/No-Go time we need to be sure the thing we're going-or-not-going-with meets all the 'RC' requirements, but it doesn't have to be explicitly labelled an 'RC' for that purpose, I don't think.
For Pungi 4 I'm assuming the 'label' winds up in the image filenames and volume labels, though I haven't explicitly checked that. So I suppose if we went with a scheme that didn't distinguish between a TC and an RC, you couldn't tell just from looking at some random image's filename or volume label whether it was a TC or RC. But the value of this has always seemed quite small to me, since you also can't tell an earlier RC from the final release in this way; if you just have an image called 'Fedora 24' you don't know if it's actually the final release, or an earlier RC of the final release.
The TC/RC naming was pretty much done for human consumption, but I'm not sure it really does much for us.
For final, I vaguely think TCs and RCs did change behavior in some ways, maybe it was disabling u-t by default? Or maybe some installer labeling or warning goes away?
Aside from the image naming, the primary difference is that the 'betanag' warning - the Timbuktu screen in the installer - is disabled. The point where we disable updates-testing by default is actually independent of the compose process, I think.
If you want to keep some granularity to distinguish between TC/RC you could use N=0 to indicate "TC" and N=1 to indicate "RC". :-P
Hmm, yeah, I like that more than 1 and 2 if we're gonna go that path, now you suggest it. 0 makes clear that this is something a bit different, and it lets us use '1' as the number for the solitary actual milestone release, which is natural.
TC's alpha-0.1 alpha-0.2 alpha-0.3 alpha-0.4 beta-0.1 beta-0.2 beta-0.3 RC-0.1 RC-0.2
legit RC's alpha-1.1 alpha-1.2 alpha-1.3 alpha-1.4 beta-1.1 beta-1.2 beta-1.3 RC-1.1 RC-1.2
I think this still works a bit better if we add a 'Final' milestone rather than using the 'RC' milestone, but yeah, this isn't bad. Dennis, WDYT?
I know releng has kinda treated Final a bit differently in the past; they basically left out all milestone-y indicators in Final builds (so they went into directories like 23_TC1 or 23_RC1, for instance, not 23_Final_TC1 and 23_Final_RC1, and the string 'Final' does not appear in their filenames or volume IDs). But I don't think Pungi 4 really allows for that, unless I'm missing something, production builds are required to have some kind of milestone identifier, so the question is only do we use the 'RC' milestone or add a new one, I guess.
rel-eng@lists.fedoraproject.org