----- "Josef Skladanka" jskladan@redhat.com wrote:
Hi gang,
according to #228 https://fedorahosted.org/autoqa/ticket/228, and discussions with kparal, I've rewritten the koji watcher, so it can check also the -pending tags in koji.
For the -pendig tags, we're using quite a different querying model based on parsing the tagHistory (e.g. # koji list-tag-history --tag='dist-f14-updates-pending' ) The benefit of this solution is, that we can catch the situations like
- package Foo is built at date XYZ. It gains tag
dist-f14-updates-pending
That should read dist-f14-updates-candidate I guess.
- koji-watcher founds out "ha, new package, let's test it"
- tests are OK
- package Foo gains dist-f14-updates-testing-pending tag
- we'd like to run tests like depcheck on it, but because the 'built
at' date, which the actual watcher checks is not altered, we miss the change
But parsing the tag history, we only check for new 'events' in the tag history (i.e. 'package foo got the -pending tag few minutes ago'), independently on the build time -> win :)
The drawback is, that at the moment, querying for the tag history takes time because koji sends the whole tag history (a lot of data). I'm discussing a minor change which would allow us to specify "give me the history which is newer than date XYZ" (as is currently used for fetching new builds) with jkeating.
Because of this, the non-pending repos are still handled the 'old' way.
....
We also found us in need of 'batch' scheduling - e.g. we don't want to run depcheck for every package built, but we'd like just to inform autoqa "hey, there is new stuff in dist-f14-updates-pending tag, run stuff". So there is new watch-koji-builds-batch watcher.
The thing is, that it is not really a watcher as such, only the hook.py file. The reason to do this is, that querying koji is time-consuming operation, and there is no need to do it twice, to get the same data. So there are two different 'schedule jobs' methods in the watch-koji-build hook, and one schedules 'per package' jobs, and the second does it as a 'batch'.
(note that the batch one is not yet tested - I just wanted to post you the patch, so you can see the new 'concept'. The per-package part is tested, and considered to be working).
Comments are more than welcomed - looking forward to hearing from you!
(master)$ git apply koji_watcher.patch fatal: git apply: bad git-diff - expected /dev/null on line 4
Gosh, do I have git somehow broken today? (Or is it my hands?) Edit: Ah, CRLF characters confused git. Good to know. Dos2unix fixed that.
It's too much new code to check for correctness, but I tried to run that, and I have a few tracebacks for you :)
[root@aqd autoqa]# /usr/share/autoqa/post-koji-build/watch-koji-builds.py --dryrun File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 376 exit status = 0 ^ SyntaxError: invalid syntax
[root@aqd autoqa]# /usr/share/autoqa/post-koji-build/watch-koji-builds.py --dryrun ... Traceback (most recent call last): File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 459, in <module> exit_status = watcher.run() File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 434, in run exit_status_2.append(self.schedule_jobs_batch(new_builds)) NameError: global name 'exit_status_2' is not defined
[root@aqd autoqa]# /usr/share/autoqa/post-koji-build/watch-koji-builds.py --dryrun ... Traceback (most recent call last): File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 459, in <module> exit_status = watcher.run() File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 434, in run exit_status.append(self.schedule_jobs_batch(new_builds)) File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 404, in schedule_jobs_batch print " ".join(harnescall) TypeError: sequence item 4: expected string, list found
After correcting these problems, here is the sample output: http://fpaste.org/mOtd/
This approach comes from my and Josef's discussion and I think it's the best thing we can currently do to accommodate depcheck-style tests into our framework.
autoqa-devel@lists.fedorahosted.org