Hi, all,
Included are package for iso watcher, which use lftp to mirror
dl.fp.org/pub/alt/stage and kick off mediakit_sanity test,
package for post-iso-build event, and package for mediakit_sanity test
which test the iso's size, md5, sha256sum, repoclosure
and file conflicts.
During the first run of /iso/watcher.py, it will mirror all the isos
at
dl.fp.org/pub/alt/stage and kick off mekiakit_sanity test,
after that, it will only mirror the new isos, the tree compose and
deltaiso are not included in the mirror.
Hello, I haven't examined the code closely, but I have some overall architecture
thoughts:
1. We need to speak to RelEng team about
http://dl.fedoraproject.org/pub/alt/stage/
We can definitely monitor that URL in the worst case, but it would be so much better if
RelEng team could ensure the following for us:
a) Consistent file naming
I have a feeling that currently all the directories there are manually created (or at
least manually named). If there were some systematic approach that we can rely on, it
would help us greatly (e.g. in determining the latest release).
b) No irrelevant files there
Currently there are not just TC/RC composes, but also "deltaisos/" and
"f16-translation/" directories that we need to blacklist. I am afraid
"stage/" serves as a storage directory for anything needed and those
"clutter" directories will change over time. It would be great to know that no
other unwanted files appear there anytime.
c) Rsync possibility
Using rsync would simplify the mirroring process at least for our Red Hat offices. I
bet we could also use it in AutoQA if desired.
2. Mirroring everything seems overkill. Personally I don't have Internet bandwidth nor
hard disk space to run this test. Can we modify the watcher so that it detects just the
last compose and downloads that one?
3. Only now I realized that watcher downloading files doesn't help us. Because the
files will be accessible on the AutoQA server, but we need them on AutoQA clients. That
means that your code works as long as you run it with "--local", but won't
work with external clients. It also means we have to work on ticket #350 "Add support
for using file proxy" (I'll try to do that soon). Ideally we will set up the
proxy on AutoQA server and all clients will be using it. The first test will download the
ISO and the other tests (if there are some) will just use the cached version (assuming
there is a fast network between the server and the client, otherwise it is of no use).
And an extra code-related question at the end:
4. Do we need labels = ['virt_capable'] in control.autoqa? This test doesn't
seem to be requiring any form of virtualization capabilities.
To sum it up, I believe we will need to do some changes. But definitely thanks for your
work!
Kamil