jasonbrooks reported a new issue against the project: `atomic-wg` that you are following: `` We removed kubernetes, etcd, flannel, gluster and ceph from the atomic host for Fedora 25: https://pagure.io/fedora-atomic/c/219c9bb26426811a5f32188c59682ad70c3283e5?b...
The idea is that these pkgs can/should be run as docker containers or system containers. I wrote a post about running kube in containers on CentOS Atomic, but things work the same for Fedora Atomic: http://www.projectatomic.io/blog/2016/09/running-kubernetes-in-containers-on...
However, we don't currently have fedora containers for kubernetes. There's a year-old PR for this: https://github.com/fedora-cloud/Fedora-Dockerfiles/pull/112. I used this PR as the basis for kube containers for CentOS: https://github.com/CentOS/CentOS-Dockerfiles/tree/master/kubernetes.
Another option is using package layering to install the removed rpms if desired: http://www.projectatomic.io/blog/2016/07/hacking-and-extending-atomic-host/. This works, mostly, but the kube-apiserver that comes with kubernetes-master is granted CAP_NET_BIND_SERVICE so it can use port 443, and package layering won't support this: https://github.com/projectatomic/rpm-ostree/issues/462.
Whether we install kubernetes via package layering or containers, another issue is that the kube we have for fedora 25 is very old, 1.2, and current is 1.4.5. There's a kube 1.4.5 in koji for fc26 (http://koji.fedoraproject.org/koji/buildinfo?buildID=816481), and I've asked Jan about making it available for fedora 25. I built it for f25 and el7 in this copr: https://copr.fedorainfracloud.org/coprs/jasonbrooks/kubernetes/.
kubeadm (http://kubernetes.io/docs/getting-started-guides/kubeadm/) is a slick way to run an up-to-date kube cluster, and the kubernetes project offers rpms for kubeadm, but the kubernetes-cni package they provide doesn't work with atomic. I made a patched package in a copr for f25 and el7 (https://copr.fedorainfracloud.org/coprs/jasonbrooks/kube-release/) and wrote about it here: https://jebpages.com/2016/11/01/installing-kubernetes-on-centos-atomic-host-.... The rpm grabs the binaries from the upstream project, if fedora were to provide this package, we'd probably want to build the binaries ourselves. kubeadm runs kubernetes and etcd in containers, and these containers aren't fedora-based, and they come from upstream. They appear to be based on busybox. Also, kubeadm is considered alpha at this point.
We should:
* try to move to a more recent kubernetes version * get kubernetes containers in place for fedora * get fedora versions of the flannel and etcd system containers in place * document how to install kubernetes and how to use package layering on fedora * look further into kubeadm
``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
dustymabe added a new comment to an issue you are following: ``
There's a kube 1.4.5 in koji for fc26 (http://koji.fedoraproject.org/koji/buildinfo?buildID=816481), and I've asked Jan about making it available for fedora 25.
Can you keep us in the loop here on the progress of this one? ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
jasonbrooks added a new comment to an issue you are following: ``
There's a kube 1.4.5 in koji for fc26 (http://koji.fedoraproject.org/koji/buildinfo?buildID=816481), and I've asked Jan about making it available for fedora 25.
Can you keep us in the loop here on the progress of this one?
Will do ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
dwalsh added a new comment to an issue you are following: `` I have asked Giuseppe Scrivano gscrivan@redhat.com to move forward on system containers to implement kubernetes workflow on atomic host. He currently has most of services available as system containers and is moving them into github.com/projectatomic/atomic-system-containers
We need to get these containers built for Fedora 25.
He even has an experimental system container that runs docker in it. ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
gscrivano added a new comment to an issue you are following: `` There are already two system containers for etcd and flannel, already available on Docker hub as gscrivano/etcd and gscrivano/flannel. I'll move them to Fedora as soon as you tell me how to proceed for doing that. There is no system container for Kubernetes yet, so we will need to run it in Docker ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
dustymabe added a new comment to an issue you are following: `` as an alternative for now I have been working on getting the openshift-ansible installer to work against F25 atomic host. I have opened up several issuses:
https://github.com/openshift/openshift-ansible/issues/2808 https://github.com/openshift/openshift-ansible/issues/2809 https://github.com/openshift/openshift-ansible/issues/2811 ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
jberkus added a new comment to an issue you are following: `` Well, OpenShift is its own thing rather than an alternative to Kubernetes. Some people want Kube, some want OpenShift.
So I've been doing a bunch with Kubeadm on AH. Jason's version does actually work, with some caveats:
1) it requires package layering, which really pushes up the idea of having a way for users to build their own OStree servers.
2) it doesn't work with setenforce=1. Even relabelling a bunch of directories, I have to setenforce 0 to get kubernetes to work reliably.
If we can get over those humps, though, I 'd like to push forward with something based on Kubeadm. I think it's the way the kubernetes project is headed, and it means that we can track them for anyone who wants to use kube-latest on Atomic. Importantly, the kubernetes community will continue to add features to kubeadm (like let's encrypt support), which won't happen with solutions we devise.
One of the things I was wondering about is maybe installing the kubeadm packages as part of Atomic. My reasoning is this: kubeadm is an installer rather than kubernetes itself. You can, in fact, use it to install an older/stable version of Kubernetes (back to 1.4.0, though), so we could track upstream Kubeadm without breaking people's stuff. Also, if not enabled in systemd (which it wouldn't be, by default), it doesn't interfere with installing something else like OpenShift.
``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
jasonbrooks added a new comment to an issue you are following: ``
- it requires package layering, which really pushes up the idea of having a way for users to build their own OStree servers.
What do you mean by that? Package layering doesn't require someone to build their own ostree server, it does require installing the layered pkgs separately on each host, though, and rebooting in between, although Colin had thoughts on cutting out the reboot step, which would make it nicer.
We could also offer multiple trees, some with fewer things rolled in by default, some with more.
And we could run these bits as system containers, like the etcd and flannel ones. I don't understand exactly what's involved in making those, in particular the bits about creating the config.json.template. It seems that this project could help w/ that part: https://github.com/jessfraz/riddler but I haven't been able to figure out how to build it to use it. ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
jberkus added a new comment to an issue you are following: `` Well, the whole point of Atomic is to be immutable: "configure once, deploy many times". Having a key piece of infrastructure depend on per-server package layering kinda breaks that. It makes it hard to explain why people should use Atomic in the first place.
Offering multiple trees would definitely help, but do we have the ability to do that? I mean, we have issues with the one tree on a fairly regular basis.
Rebuilding kubeadm as system containers would be *awesome* if we could really do that. Weren't there technical issues with running Kubelet in a container though? ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
jasonbrooks added a new comment to an issue you are following: `` kube 1.4.5 for f25 in bodhi: https://bodhi.fedoraproject.org/updates/FEDORA-2016-fffea4b1c3
I've been working on getting it running in rpm-installed node pkgs / master pkgs in containers. Notes on that here: https://gist.github.com/jasonbrooks/bdbb0b142fc82bbd7b33759b3d934ba0 ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
jasonbrooks added a new comment to an issue you are following: `` Some updates:
* the issue of fedora having an aged kube is resolved, v1.4.5 is in f25 stable * the issue of not being able to just `rpm-ostree install kubernetes` and proceed as if we never removed kube from the image is open. This won't work because [rpm-ostree can't deal](: https://github.com/projectatomic/rpm-ostree/issues/462) w/ the kube-apiserver binary in kubernetes-master * the issue of running kube in containers on fedora atomic is in progress: * I've been working w/ these [kube containers](https://github.com/jasonbrooks/k8s-images/tree/f25), and these [etcd](https://github.com/projectatomic/atomic-system-containers/tree/master/etcd) and [flannel](https://github.com/projectatomic/atomic-system-containers/tree/master/flanne...) containers. They work (but need [this PR](https://github.com/projectatomic/atomic-system-containers/pulls)), and I'm adapting the [upstream ansible scripts](https://github.com/kubernetes/contrib/tree/master/ansible) to use them. * I need to get those kube, etcd and flannel containers into the new fedora build system, and could use help with that. * I need to get the ansible changes finished and upstreamed, and that ought to be straightforward, but I could use feedback on my PR once I get it together. ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
dustymabe added a new comment to an issue you are following: `` kubernetes rpms are in the install tree now: https://pagure.io/fedora-atomic/pull-request/39
the images produced tonight should have the content in there. Let's close this issue if testing is successful there. ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
dustymabe added a new comment to an issue you are following: `` new release is out with k8s rpms in it: https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/... ``
To reply, visit the link below or just reply to this email https://pagure.io/atomic-wg/issue/176
-
Pagure