#204: Adjust watchers to pass over all information at once (required for depcheck) --------------------+------------------------------------------------------- Reporter: wwoods | Owner: jskladan Type: task | Status: assigned Priority: major | Milestone: 0.4.4 Component: tests | Resolution: Keywords: | --------------------+------------------------------------------------------- Comment (by wwoods):
[It] seems like we would have a lot of duplicate testing going on if we
only provide feedback for the requested update.
Agreed. It seems to make sense for the test to provide feedback for ''all'' packages that pass, not just the package that triggered the test.
- If providing feedback against all -pending updates, shouldn't the
wrapper get a list of *untested* -pending updates, and only test that set? If no untested -pending updates ... no testing needed? Otherwise, again we'd have repeated testing.
No. Repeated testing is ''required'' for packages that haven't been accepted. We have to keep testing them until they get accepted, or they get pulled out of the 'pending' list (by being obsoleted or forcibly removed).
To illustrate the point: say you have an update for package A, which requires package B. The build for B isn't done yet, though, so A gets tested and fails. When the update for B arrives, we need to test A again - and then it will pass, and be accepted.
So the wrapper needs: 1. A way to get the pending package list (let's call it {{{get_pending()}}}) 1. A way to get the accepted package list (which is a subset of the pending list). Let's call it {{{get_accepted()}}}.
We can do {{{get_pending()}}} just by listing the contents of the -pending tag. {{{get_accepted()}}} is not as straightforward, but we can achieve that by checking our previous test results (as reported in Bodhi or wherever) to see which pending packages have previously passed. Something like: {{{ accepted = [] for p in get_pending(): if check_accepted(p): accepted.append(p) return accepted }}}
- Then there is the whole timing issue kparal and wwoods were
discussing regarding multiple depcheck tests running at the same time, each trying to move items out of the -pending set, and into the ACCEPTED -pending set.
Yes - this is ticket #248. And Kamil is correct: if the pending/accepted sets change during the test, our results become invalid. So when this happens, the test wrapper needs to retry the test with the new data.
We can optimize this though. If any new packages have been added to the 'pending' set, that means that a new test run has already been scheduled for those new packages - so we can just cancel our run and let the other one handle it.
See ticket #248 for details.