dustymabe reported a new issue against the project: `releng` that you are following: `` With the new [Fedora CI objective](https://fedoraproject.org/wiki/Objectives/Continuous_Integration_and_Deliver...) we will have better testing of the packages that comprise Fedora Atomic Host and eventually extend that out to packages that aren't included in Atomic Host (i.e. the broader Fedora).
It is almost guaranteed that we won't catch everything before some packages make it into stable (some long running tests or even possibly some failures that we don't yet have tests for). For that reason we have established that we will need the ability to revert packages in Atomic Host that fail tests (or are otherwise bad). This is mostly so that the testing of future packages doesn't get blocked by a host that is now in a bad state.
We had previously considered using modularity for this purpose, but we aren't quite ready to use modularity yet. In the interim we'd like to use a few koji tags and a rigorous definition of Atomic Host to achieve this goal.
Today:
- we have the `f27-atomic` tag. This is used to primarily "freeze" our install media for Atomic Host at the f27 "release day" package set. Occasionally we will need a new version of anaconda or ostree/rpm-ostree for the installer and we'll ask to have things tagged into this tag. - we build our ostree for Atomic Host from the release day repo (f27) as well as the updates repo that was created by Bodhi (f27-updates).
Proposed Future:
- we have an `f27-atomic-installer` tag that is used for the same purpose as the `f27-atomic` tag is today. Mainly to freeze the installer package set at f27 release day packages. This tag inherites from f27 (release day) tag. - we have an `f27-atomic-overrides` tag. This tag does not inherit from any other tag and only contains rpms that we have tagged in there for reverting to previous versions of rpms. - we have a dist-repo for `f27-atomic-overrides` so that whenever we tag something into that tag we can request a repo get created for it. - we update the ostree compose definition (in our pungi configs) to pull from this new dist-repo, as well as f27 and f27-updates. In our fedora-atomic tree manifest we will specify NVR for rpms (possibly even for all packages that make up atomic host). - we request a new "atomic" koji permission. This permission will have the appropriate policy to allow people with this permission to add/remove packages from these tags.
Thanks to @mikeb who helped me come up with this solution by providing technical guidance. ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
mohanboddu added a new comment to an issue you are following: `` @ausil We thought the major issues with this approach is signing and koji-gc. But @mikeb confirmed that any packages tagged into -overrides should already be signed, because they're older packages and koji-gc shouldn't gc packages that are "latest" in a tag and anything tagged into -overrides would be latest. So, he thinks that with current configuration, it just works. We need your opinion on this. ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
dustymabe added a new comment to an issue you are following: `` FYI from [todays meeting](https://meetbot.fedoraproject.org/fedora-meeting-2/2017-10-19/releng.2017-10...):
``` #info The plan seems okay so far, but we will get another review from dgilmore. ``` ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
dustymabe added a new comment to an issue you are following: `` Discussed in today's meeting. Mohan is going to try to get Dennis to review this. ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
maxamillion added a new comment to an issue you are following: `` +1 ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
dustymabe added a new comment to an issue you are following: `` from today's releng meeting:
``` 13:41:44 dustymabe | #info waiting another week for dgilmore's input. the third time is the charm ``` ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
ausil added a new comment to an issue you are following: `` There is a probability that the older updates have had the signed copies cleaned up on disk before hand, assuming it was the previous build.
Rather than using dist-repos we should use pungi to make the repo. A side effect of this change is that you are unlikely to get the result taht you want. as when anaconda sees the updates in the updates repo witha higher nvr it will choose that. you will need to manage the entire chain of packages in the pungi compose to get your intended behaviour. ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
dustymabe added a new comment to an issue you are following: ``
There is a probability that the older updates have had the signed copies cleaned up on disk before hand, assuming it was the previous build.
I believe @mikeb did not think this was a problem because if we tag them into `f27-atomic-overrides` they won't be gc by koji??
Rather than using dist-repos we should use pungi to make the repo.
dist repo was because we'd only need to change this repo few times and we can just run the create the dist repo command right after we tagged a package into the tag. The ostree creation (that runs during bodhi call to pungi) would pick up rpms from f27 release, the f27 updates run that was just done as part of the bodhi run, AND the dist-repo. The goal is to prevent having to create the atomic overrides repo as part of the bodhi run every day.
A side effect of this change is that you are unlikely to get the result that you want. as when anaconda sees the updates in the updates repo witha higher nvr it will choose that. you will need to manage the entire chain of packages in the pungi compose to get your intended behaviour.
The installer would come from the `f27-atomic-installer` tag which inherits from `f27` release tag. Similar to today, it wouldn't pull in updates. The ostree that gets embedded in the installer would be the one from the bodhi updates run and it would already have the `f27-atomic-overrides` packages baked into the ostree. Anaconda has no ability to pull newer updates, the ostree is already baked.
``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
ausil added a new comment to an issue you are following: `` The risk of koji-gc has nothing to do with the f27-atomic-overrides tag and everything to do with the signed copies being cleaned up before they are tagged into it. we can work around it by setting up some process to ensure they are signed and have signed copies written out to disk.
pungi builds the installer every time we run it to ensure that we have the correct bits inside of the installer. it has to gather the rpms and make the repos anyway as part of the compose process, you are adding additional overhead ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
dustymabe added a new comment to an issue you are following: `` so I talked with dgilmore (log below). The TL;DR is that he does not want to use a dist-repo but he is fine with the other parts of the proposal. I accept not using a dist-repo but building the `f27-atomic-overrides` repo every time we do an updates compose that includes an ostree creation. We should be able to move forward with this ticket.
``` 09:33:36 +dgilmore | dustymabe: the pungi run will make the ostree, but we also need to make it make the isntaler dvd and the | cloud images 09:33:50 dustymabe | dgilmore: which pungi run? 09:33:59 +dgilmore | dustymabe: the one taht makes teh ostree 09:34:05 dustymabe | dgilmore: yes, that's bodhi 09:34:11 dustymabe | bodhi->pungi 09:34:14 +dgilmore | dustymabe: the nightly cronjob that we have today needs to go 09:34:23 dustymabe | dgilmore: sure I don't disagree with that 09:34:41 +dgilmore | dustymabe: and the pungi run for making the ostree will also make the things taht today are done in the | cronjob 09:34:57 dustymabe | dgilmore: ok. but I still don't think that means we shouldn't do a dist-repo 09:35:09 +dgilmore | ideally witha agate that checks if any of the inputs has actually changed 09:35:10 dustymabe | since the content will rarely change 09:35:15 +dgilmore | so we only make it when things change 09:35:52 dustymabe | so we either create a dist-repo or we make the repo every time the bodhi->pungi run goes 09:35:57 dustymabe | i'm fine with either 09:36:06 dustymabe | was just trying to not complicate the bodhipungi run 09:38:19 <-- | claudioz (~darklight@unaffiliated/darklight/x-2853543) has quit (Quit: My MacBook has gone to sleep. | ZZZzzz…) 09:38:35 +dgilmore | dustymabe: we are making the repos anyway 09:39:49 dustymabe | dgilmore: so you prefer the 'non dist-repo' case ? 09:40:12 --> | vedranm (~vedranm@inf2.uniri.hr) has joined #fedora-releng 09:40:21 smooge | Reminder: Fedora Infrastructure will meet at 1800 UTC today. This will be 1 hour earlier on the clock | for many people. Please update gobby with any info as the meeting is rather bare 09:44:06 dustymabe | dgilmore: if so I'll update the ticket with our conversation 09:44:20 dustymabe | dgilmore: and I assume you have no other issues with the proposal 09:44:29 +dgilmore | dustymabe: I do 09:44:42 <-- | cqi (~cqi@125.33.67.165) has quit (Quit: Konversation terminated!) 09:44:48 +dgilmore | that is I do prefer the non dist-repo case 09:44:52 dustymabe | ok 09:45:14 dustymabe | and you are ok with everything else in the proposal? 09:45:23 +dgilmore | otherwise I am fine with it. though I am not afan of special casing the config to pull in older nvrs 09:45:51 dustymabe | dgilmore: indeed, I understand the concern 09:45:55 dustymabe | thanks for the chat I appreciate it
``` ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
dustymabe added a new comment to an issue you are following: `` from the meeting today:
``` 12:14:10 mboddu | #info Except for the dist-repo part, we are okay on the proposal. The tasks are divided as: releng has to | create tags and dustymabe will take care of config changes and infra will handle "atomic" koji permission ```
so we can move forward with these changes. The only other thing that came up in the meeting is, considering we will now ship updates for Atomic Workstation, should we namespace these tags. I think we should so that would mean we would create a tags like `f27-atomichost-installer` and `f27-atomichost-overrides` instead of what was proposed above.
``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
dustymabe added a new comment to an issue you are following: `` ok discussed in the meeting today we're going to go with `f27-atomic-host-installer` and `f27-atomic-host-overrides`. Please add those two new tags and permissions for @dustymabe @cgwalters and @jlebon. ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
dustymabe added a new comment to an issue you are following: `` mohan showed me these commands and had me review them. LGTM:
``` $ koji add-tag f27-atomic-host-installer --parent=f27 $ koji add-tag f27-atomic-host-overrides $ koji edit-tag f27-atomic-host-installer --perm=atomic $ koji edit-tag f27-atomic-host-overrides --perm=atomic $ koji grant-permission atomic dustymabe walters jlebon ``` ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
ausil added a new comment to an issue you are following: `` I am okay with that being run. We do however need to have a tracker issue in pagure where we keep track of overrides. When something is tagged into the overrides tag it needs to be liested in the tracker issue witha reference to the bug that is being solved. I would like to better understand how often we need to use the permission and why. at the veryleast to be able to point the CI team at to show them some needed test cases to be covered. ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
dustymabe added a new comment to an issue you are following: `` @ausil sure - we can create the issue in the atomic-wg pagure if you're ok with that
``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
mohanboddu added a new comment to an issue you are following: `` @dustymabe Can we just use releng repo? Its more a tracking for us on whats going on in the overrides. ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
dustymabe added a new comment to an issue you are following: `` sure: created this ticket: [https://pagure.io/releng/issue/7213%5D(https://pagure.io/releng/issue/7213) ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
mohanboddu added a new comment to an issue you are following: ``
$ koji add-tag f27-atomic-host-installer --parent=f27 $ koji add-tag f27-atomic-host-overrides $ koji grant-permission --new atomic mohanboddu $ koji edit-tag f27-atomic-host-installer --perm=atomic $ koji edit-tag f27-atomic-host-overrides --perm=atomic $ koji grant-permission atomic dustymabe walters jlebon
All the necessary tags are permissions are created. ``
To reply, visit the link below or just reply to this email https://pagure.io/releng/issue/7100
The status of the issue: `new koji tags for atomic host` of project: `releng` has been updated to: Closed as Fixed by mohanboddu.
rel-eng@lists.fedoraproject.org
-
Adam Miller -
Dennis Gilmore
-
Dusty Mabe
-
Mohan Boddu