On Thu, Apr 30, 2020 at 05:27:18AM -0400, Stephen John Smoogen wrote:
On Wed, 29 Apr 2020 at 20:22, Chuck Anderson <cra(a)wpi.edu>
wrote:
>
> On Wed, Apr 29, 2020 at 10:50:00PM +0200, Adrian Reber wrote:
> > On Wed, Apr 29, 2020 at 04:44:08PM -0400, James Cassell wrote:
> > >
> > > On Wed, Apr 29, 2020, at 1:31 PM, Theo Morra wrote:
> > > > Heya,
> > > >
> > > > We're adding some more storage to our mirrors - how big is
buffet?
> > > >
> > >
> > > Someone said 3.8 TB the other day.
> >
> > Unfortunately I was off by one. It is 38TB. Knowing that, my proposal to
> > require mirroring fedora-buffet for new tier 1 mirrors sounds not so
> > good anymore as that is really a lot.
>
> It is a lot, but I think it should still be required. Tier1 should
> have everything any Tier2 might want to pull a subset of.
In the past I would agree, but we (aka Fedora) had to drop mirroring
archives from 2 of our systems (download-ib01/download-i2 and
download-cc-rdu01) because we didn't have the disk space on these
servers to do that much storage. I am not sure trying to shove every
use case for a top mirror into one box is going to work anymore.
At the moment I only see the following mirrors offering archives via
mirrormanager:
mirror.math.princeton.edu
pubmirror1.math.uh.edu
d2lzkl7pfhq30w.cloudfront.net # a fedora project caching server
dl.fedoraproject.org
I see a couple more from my location:
ftp-stud.hs-esslingen.de
fedora-archive.ip-connect.info
mirrors.dotsrc.org
fedora-archive.ip-connect.vn.ua
fedora-mirror02.rbc.ru
I asked MirrorManager for Fedora 26 based mirrors, so this does not mean
that all these mirrors mirror the complete archive.
I expect there are probably private ones which I am not seeing via
my
simple curl |& grep but the majority of users who are grabbing stuff
from archived data are only getting it from those 4 places. While it
would be good for more, I can also see that the majority of mirror
traffic is EPEL driven with only the latest 3 Fedora's taking up the
majority of users. Having those fed fast is probably more important
than Fedora 18.
Would a different tiering system work better in the same way as we
have different rsync targets?
What I would love to see in any tiering system we set up is full support
for quick-fedora-mirror. If quick-fedora-mirror could handle a partial
tree (not only via fedora-buffet) that would be perfect.
Adrian