On 4/2/24 12:35, Carsten Otto wrote:
I'm one of the admins of ftp.halifax.rwth-aachen.de, which offers fedora and fedora-epel among other distributions and projects. I've recently noticed that many Fedora users (including EPEL, CentOS, and BlackArch) frequently re-download "repodata" files that haven't been updated since the previous request.
I'm not sure if we're seing the same thing that Carsten is seing, but it certainly sounds similiar: We are seing a massively increased number of requests to our EPEL mirror (but _not_ our CentOS-old mirror, and we have no CentOS-stream mirror).
More precisely, starting March 27 (last Wednesday), the number of requests to the EPEL-mirror has more than quadrupled, from about 400k-ish to 1.5M-2.5M per day [1], as has the number of transferred bytes, from about 1 TB to 4-7 TB per day [2].
That certainly does look like something is wrong.
Taking a closer look at the 1.79M EPEL-requests yesterday, 1.57M of them reported an user-agent of "urlgrabber/3.10 yum/3.4.3", which I guess is CentOS7? But there has not been a recent update to yum in CentOS7, has there? Have there perhaps some EPEL mirrors been dropped in Germany recently, or is something wrong with the mirrorlist-load-balancing, that could explain the sudden surge in requests?
[1] https://ftp.fau.de/cgi-bin/show-ftp-stats.cgi?statstype=2&what=files&... [2] https://ftp.fau.de/cgi-bin/show-ftp-stats.cgi?statstype=2&what=bytes&...
Regards,
On Wed, Apr 03, 2024 at 11:06:08AM +0200, Michael Meier (FTP-Admin) wrote:
More precisely, starting March 27 (last Wednesday), the number of requests to the EPEL-mirror has more than quadrupled, from about 400k-ish to 1.5M-2.5M per day [1], as has the number of transferred bytes, from about 1 TB to 4-7 TB per day [2].
Similar here: https://ftp.halifax.rwth-aachen.de/~cotto/loganalyzer/fedora-epel_month.png https://ftp.halifax.rwth-aachen.de/~cotto/loganalyzer/fedora-epel_month-req....
On Wed, Apr 03, 2024 at 11:06:08AM +0200, Michael Meier (FTP-Admin) wrote:
Have there perhaps some EPEL mirrors been dropped in Germany recently, or is something wrong with the mirrorlist-load-balancing, that could explain the sudden surge in requests?
Not sure if mirrors have been dropped but the current list of mirrors for a country can be queried using:
$ curl "https://mirrors.fedoraproject.org/mirrorlist?repo=epel-7&arch=x86_64&..."
Which currently gives me 16 mirrors.
Worldwide I see 152 mirrors for EPEL.
$ curl "https://mirrors.fedoraproject.org/mirrorlist?repo=epel-7&arch=x86_64&..."
The load-balancing can be seen by looking at the output of the list. It should change each time you query it.
The normal mode of operation is without the "country" parameter. The result is different for each IP address doing the query, but I currently get:
$ curl "https://mirrors.fedoraproject.org/mirrorlist?repo=epel-7&arch=x86_64" # repo = epel-7 arch = x86_64 country = DE country = NO country = RU country = TR country = PL country = IT country = CH country = MK country = FI country = HU country = UA country = LV country = GB country = SE country = AT country = SI country = CZ country = NL country = SK country = ES country = MD country = BG country = DK country = RO country = IS country = EE country = PT country = FR
Over 70 mirrors for my location in Europe. The first line starting with "# repo" gives you an explanation which mirrors are included in the list. yum/dnf actually uses metalink, but the result is more or less the same. You can replace "mirrorlist" with "metalink" to see the difference.
Adrian
mirror-admin@lists.fedoraproject.org