On Thu, 2015-02-26 at 04:23 -0500, Kamil Paral wrote:
> Hey folks! Anyone paying attention to the Project Coconut box
logs
> might've noticed it failed when it tried to run against Alpha TC5.
> The problem is to do with the download URLs. I wrote fedfind to
> always use
https://download.fedoraproject.org/URLs for mirror HTTP
> downloads, but this turns out not to work great for TCs/RCs
> because of mirror lag, people (and Coconut) want the images
> immediately but mirrors don't pick them up for a few hours after
> the TC/RC is cut.
I think the problem is different. (Almost) nobody mirrors TCs/RCs.
There are about 3 mirrors in the whole world which follow stage/
directory:
http://mirrors.fedoraproject.org/mirrorlist?path=/pub/alt/stage
Also, you would need to ask infrastructure folks, but I don't think
download.fp.o can be used for this. I think it redirects to a random
close mirror without actually checking the path you are requesting.
If you wanted to be sure that the file is present, you would need to
make calls like these in advance:
http://mirrors.fedoraproject.org/mirrorlist?path=/
pub/alt/stage/22_Alpha_TC4/Workstation/i386/iso/Fedora-Workstation-
netinst-i386-22_Alpha_TC4.iso
Those calls are quite slow and probably expensive. And they change
in time, you can't use them immediately after TC/RC is published.
So, for anything under stage/, I believe using dl.fp.o is the best
Well, in the end it's academic, but download.fp.o is a *bit* more
sophisticated.
download.fp.o does not redirect just to any 'fedora mirror', it does
make an attempt to redirect to a 'valid' mirror, but the check is
limited.
What it does is make sure it redirects to a mirror which claims to
carry the relevant bundle. So for a /stage/ URL it will never redirect
you to a mirror that simply doesn't mirror stage at all.
However, apparently it can't do an actual path check for the exact
location you request, so it can't quite manage to be as accurate as
mirrormanager is when you make a 'path' request.
So the problem is, as I said, the mirror lag: the mirrors that *do*
carry /stage/ don't mirror it instantly. Some of them don't seem to
mirror it even hourly, or anything - some of them still didn't have
TC5 7 or 8 hours after it hit dl.fp.o. So download.fp.o gives you a
URL that *ought* to be OK, but in practice, the mirror likely hasn't
caught up yet.
So in the end we come to the same conclusion, for things in stage/,
using download.fp.o just isn't a good idea, and as I said, I've made
it so fedfind doesn't use download.fp.o for TC/RC images...only thing
is I had to do it in a bit of a hacky way because of how I designed
fedfind, doing it in a cleaner way isn't *entirely* straightforward.
I'll have to think of the best way to go about it for a permanent fix.
The hack should *work* fine for the foreseeable future, though, it's
just theoretically stupid code (it's really in the wrong place and
messes up the design).
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net