# Bodhi Updates and AutoQA Discussion
# Date: 2010-04-14
# Time: 14:00 UTC (11:00 EDT, 15:00 CET)
# Participants: lmacken, kparal, jlaska, jkeating, wwoods
= Discussion Topics =
== Goals ==
* Do *not* add more steps for package maintainers (unless the autoqa
test fails)
== Background: depcheck ==
* triggered on post-bodhi-update hook
* inputs: update NVRs, target repo, [other pending packages]
** Why "other pending packages"?
*** Can't handle interdependent packages without it
== Current Bodhi ==
* What states/stages do updates go through?
** See workflow below
* How do we recognize them through the API?
** See workflow below
== Review bodhi update workflow ==
Package built by maintainer
| koji tag dist-fxx-updates-candidate
V
Bodhi update created by maintainer
| koji tag dist-fxx-updates-candidate
| bodhi (status: pending, request: None)
V
Bodhi update requested by maintainer (for stable or testing)
| koji tag dist-fxx-updates-candidate
| bodhi (status: pending, request: ['testing' or 'stable'])
| bodhi tags the package with the *-pending tag, sends message to bus
V
pending
| could be koji tag dist-fxx-updates(-testing)-pending
| koji uses newRepo to make repos
V
Wait for newRepo <---+
| |
| FAIL (remains in pending)
V |
Acceptance testing ---+
|
PASS - autoqa gives +1 to update (future work: autoqa sets 'test-ok'
tag/field)
|
V
Available for pushing (still in dist-fxx-updates(-testing)-pending)
|
V
Pushed by releng (package moved from -candidate/-testing to
-updates(-testing) and drops -pending tag)
(-testing updates can now go to "update requested" with request:
'stable')
== What does depcheck/autoqa need? ==
* Needs a repo of pending packages
* Package sanity requirement:
https://fedorahosted.org/autoqa/ticket/139
** Multiple tests are needed for complete package acceptance, the autoqa
+1 should happen after all have completed
* un-depsolved packages = the *-pending tags
* depsolved packages (eligible for push) = *-pending tag & +1 from
autoqa
= Open Questions =
Q. Are we going to need a lot more koji tags?
A. The tag could live on the bodhi side, just need a way to determine
which builds are eligible for pushing to requested repo (stable,
testing)
Q. How to note that AutoQA has passed acceptance tests?
A. Agreement that there should be AutoQA specific karma in bodhi. This
lets maintainers and other testers see that the response is from an
automated tool
Q. Re-test everything once a maintainer retracts an update request?
A. Yes - any time the -pending set changes, we need to re-run the
depcheck/acceptance test
Q. What happens when things change and an update which once passed
acceptance no longer does?
A.
Q. What about the tests beyond depcheck, individual package acceptance
tests
A. This process would only relate to package acceptance testing in
accordance with the criteria
(
https://fedoraproject.org/wiki/Package_update_acceptance_criteria).
Functional testing will occur once the packages/updates are moved to
updates-testing.
Q. How to handle packages that failed testing, and then a new NVR comes
in a new update request?
A.
Q. If a maintainer revokes a push request what now?
A. Need to remove the -pending tag
Q. What do we do when a firefox security update is headed out, but
breaks other packages?
A.
Q. What to do when acceptance test fails
A. Don't add negative karma, add karma 0 with a comment in the bodhi
request
Q. We have 4 planned automated acceptance tests, how to make sure all
run before approving the update
A.
= Action Items =
[ ] - Update the bodhi-1.0 watcher to detect builds (quick'n'dirty
method to watch for koji *-pending tags) (wwoods)
[ ] - Integrate use of mash into acceptance tests (jkeating && wwoods)
[ ] - Create new koji tags for *-pending (jkeating)
[ ] - Bodhi-1.0 changes? (lmacken)
[ ] - Add -pending tag when update is requested
[ ] - Magic handling for autoqa's +1 karma (only show autoqa-approved
stuff)
[ ] - Bodhi-2.0 roadmap changes?
- Tickets in bodhi trac, against 2.0 milestone
[ ] - Move package acceptance test plan out of Draft (kparal && jlaska)
= Next Meeting =
2010-04-28 - Another 'touch-base' meeting