On Tue, 2016-08-16 at 11:24 -0500, Jason L Tibbitts III wrote:
It would also help to have the following information. The mirrors
will
need to have this information in order to make informed
decisions. (I
will also have to make changes to quick-fedora-mirror to
accommodate.)
1) How much content will the mirrors need to store? How will this
amount change over time?
Hello Jason! I confess that I don't have good answers to your
questions, and I'm not sure who would. Many of these questions depend
on how popular Docker images become with Fedora packagers.
How many bytes of content we will be creating does depend on how many
applications get packaged as Docker images. I would guess the base
image to be a few hundred megabytes, but we can probably use some fancy
hardlinking to help reduce disk/network usage so that the base image is
only stored once. The rest of the storage is going to be the diffs
applied as layers on top of the base image that add whatever each
individual image needs. The sizes of these layers will vary greatly by
application, so this is also difficult to guess.
It's difficult to make informed guesses about this since I don't know
how many Docker images the fedora packagers will create (or at what
rate they will create them over time).
2) Do you have a plan for placing an upper bound on the total amount
of
data? (In Fedora things are moved to archive, though that has its
own problems and of course doesn't really place an upper bound on
anything.)
I don't have such a plan at this time. If anyone has suggestions about
this, that would be helpful. It's unclear whether Docker images would
live inside or outside of the traditional Fedora cycle (i.e.,
F24/F25/F26). It may have its own separate cycle, or we may just go
with the current Fedora cycle.
3) How much change do you expect per day? Churn is really
important,
and even now we can come close to the point where the master
mirrors
simply can't feed new content to the tier 1 mirrors fast enough
for
them to keep ahead of the changes we're making.
This again depends on how popular the Docker image offering becomes
with our packagers, so it is difficult for me to make an educated
guess. Popularity is difficult to predict.
4) How will this be organized on the master mirrors? It really
should
be in a separate rsync module, and the archive (if that happens)
should also be in a separate rsync module.
In my proposal e-mail I mentioned that it was important for mirror
manager to allow mirror admins to opt-in to hosting Docker content.
Since we don't know the answer to so many of these questions, I suggest
we opt mirrors out by default, and let admins opt themselves in as they
please. Our proposal didn't have an exact path for storing Docker
images, but it was planned to be separated from the RPM and ISO content
at a fairly high level in the tree.
I apologize for having so few answers. If anyone can shed more light on
Jason's questions, please reply.