On Fri, Apr 29, 2016 at 8:42 AM, <aweiteka(a)redhat.com> wrote:
> Hello all,
> I have been working on a proposal on how to handle Fedora's Docker
> Registry requirements to deliver our future "docker-ized" content to users
in
> a way that won't leave us with a bottleneck to the user community hitting a
> single download location. Below you will find that proposal and I would
> greatly appreciate feedback. The good news is I did a small PoC of the plan
> I would like to propose along with the workflow (simulating some steps in the
> process for brevity), and it all worked! The potentially bad news is that
> this introduces a reasonable amount of net-new services inside the Fedora
> Infrastructure. I wanted to sanity-check this proposal with the Rel-Eng crew
> and then send to the Infrastructure Team for feedback and advisement after
> I'm sure this makes sense to everyone from a rel-eng perspective. Thank you
> all in advance.
>
> Quick vocab background for Docker stuff:
>
> registry: a collection of docker image repositories
>
> repository: named after an image and is a collection of multiple tags of
> an that image
>
> tag: an arbitrary string assigned to a specific docker image
> (identified by the image's sha256 checksum)
>
> NOTE: The "latest" tag is special and is assumed if no tag
is
> provided. This is true also for a 'docker pull' operation
and
> an image tagged "latest" will be the default image pulled
by
> users.
>
> Proposal:
>
> Pulp[0] + Crane[1] + MirrorManager[2] + Docker Distribution[3]
>
This is a lot of different tech to maintain. You essentially have three different
registries: the one in OSBS (even if you're not using it--why?), crane,
docker/distribution upstream. OSBS and docker distribution track very closely; crane has
struggled to track upstream changes.
There is a reasonable amount of tech to maintain, I will agree.
However, OSBS is the buildsystem so we need it. We need a registry to
push to from the build system (much like koji drops rpms in
intermediate build tags which are mashed into repos), from there we
need a way to sync content out to the world in a way that Fedora can
reasonably maintain which is where crane + mirror manager come in.
> Pulp is a platform for managing repositories of content, such as software
> packages, and making it available to a large numbers of consumers. It is also
> capable of managing docker content.
>
> Crane is a stand-alone python flask wsgi application written by the Pulp team
> to serve as a API entry point for the docker client and will answer to an
> user's 'docker pull'. It does not however create content manifests or
provide
> hosting for docker image content, instead it depends on someone creating the
> manifest metadata themselves or having pulp publish it and serves 302
> redirects to the docker client so they can find where the docker images
> actually live.
>
> I'll assume everyone here knows their way around MirrorManager.
>
> Docker Distribution is the defacto standard open source implementation of the
> Docker Registry V2 API spec[4]. It provides many features but the ability to
> have it's back-end storage be provided by a "mirror network" much like
the
> one
> Fedora has at it's disposal is not one of them. The reason we need this in
Content mirroring is a very common requirement for registries. It would be ideal to
get the support you need into the storage driver. Have you looked very deep into the
blockers o possible alternatives? So far we're recommending cloud object storage or an
enterprise storage solution with geo replication.
It has been looked into, the big issue with all of those solutions is
they are designed around an idea that you have a CDN or enterprise
storage solution at your disposal, however Fedora survives off of our
amazing volunteer mirror network and the CDN-style backend is
something we can't afford (as mattdm noted).
> place is because the mechanism by which you could push a docker image directly
> to Pulp in Docker Registry v1 no longer exists in v2 so we must instead
> perform a "sync" operation between the two. (This is a common problem for
> all known "third party" v2 registry implementations).
>
>
> Workflow:
> OSBS will perform Builds, as these builds complete they will be pushed
> to the docker-distribution (v2) registry, these will be considered
> "candidate images". Pulp will sync and publish the candidate
repository.
Why would you want to sync and publish a candidate image that may be broken? Wouldn't
it be better to use the internal OSBS registry for candidate images then sync if test
passes?
I think Ralph covered this, I'm not sure I could explain it better
than he did but if there are more questions beyond what Ralph covered
I'd be happy to discuss it and/or answer any questions beyond that.
>
> Testing will occur using the "candidate images" (details of how we
want
> to handle that are outside the scope of this proposal).
This is pretty huge so it would be good to consider it. OSBS is OpenShift so couldn't
the testing happen in that integrated environment?
Again, a tip of the hat to Ralph's answer here.
>
> A "candidate image" will be marked stable once it's criteria have
been
> satisfied to do so. (This is vague because this is a topic of ongoing
> discussion and work to decide what criteria an image will need to abide
> by before being considered "stable" and promoted as such)
This image promotion workflow is pretty well defined in openshift, complete with CD
pipeline visualization, etc.
Once more, thanks to Ralph here but again, I'm happy to discuss if
there are any other aspect to discuss.
-AdamM
>>
>> Once stable, pulp will publish that repository's content to a directory,
>> we will split that content and sync the image layers along with their
>> metadata to Mirror Manager master mirror. We will also sync the repo
>> metadata published by Pulp to somewhere Crane can pick it up. (This could
>> and will likely be something that Bodhi triggers via the Pulp REST API)
>>
>> Mirror Manager will Pulp distribute to the mirrors the image layers and
>> their metadata.
>>
>> Crane will get the new repository metadata and will serve redirects to
>> the new content relative to
download.fedoraproject.org which will
>> perform another redirect (via MirrorManager) where the docker client
>> upon a "docker pull" will find it's content.
>>
>>
>> I have put together an ascii diagram in hopes that it will assist in seeing
>> the full picture.
>>
>>
https://maxamillion.fedorapeople.org/FedoraPulpDocker.txt
>>
>> (I couldn't get it to paste cleanly into my email client)
>>
>> If there are any questions, comments, or feedback please let me know.
>>
>> Thank you,
>> -AdamM
>>
>>
>> [0] -
http://www.pulpproject.org/
>> [1] -
https://github.com/pulp/crane
>> [2] -
https://github.com/fedora-infra/mirrormanager2/
>> [3] -
https://github.com/docker/distribution/
>> [4] -
https://docs.docker.com/registry/spec/api/
> _______________________________________________
> rel-eng mailing list
> rel-eng(a)lists.fedoraproject.org
>
http://lists.fedoraproject.org/admin/lists/rel-eng@lists.fedoraproject.org