I also posted a blog post with the same text. I was supposed to update
the list with the state of tests last week, but wanted to write that up
only after making all changes live.
Till Fedora 22 release we have tested our Cloud images only with manual help.
The amazing Fedora QA team organized test days, and also published [detailed
on the wiki about how to test the images. People tried to help as when
possible, as not having access to a Cloud was a problem for many. The images
are also big in size (than any random RPM), so that also meant only people with
enough bandwidth can help.
During Fedora 23 release cycle, we worked on a change for [Automated Two Week
. A part
of it was about having automated testing, which we enabled using
automatically tests every Cloud Base and Atomic qcow2 images, and libvirt, and
Virtualbox based Vagrant images.
This post will explain the state of [currently activated
for the same. These tests are written as Python3 unittest cases.
At first we run 5 non-gating tests, failing any of these will not do a release
blocking, but at the end of each run we get the summary for all of these
non-gating tests. These include bzip2, cpio, diffutils, Audit.
Then we go ahead in testing the basics test cases, things like journald
logging, package install (only on base image), SELinux should be enforcing, and
no service should fail during startup of the image.
The next test is about checking that /tmp should be world writable. We then
move into testing the mount status of the /tmp filesystem. If it is a link,
then also it should behave properly. Our next test checks that for Vagrant
images we are using predictable naming convention of network devices, basically
checking the existing of *eth0*.
We then disable *crond* service, after a reboot we make sure that the service
is still in disabled state, and do the rest of service manipulations. We also
check that the user journald log file exists on that reboot, this comes from a
regression test. Then we reboot the instance again.
After this second reboot, we test the status of the *crond* service once again,
and then we move to our special tests cases related to Atomic host. Means these
tests will run only on Atomic hosts (both cloud, and Vagrant ones).
In the first test we check if the package *docker* is at all installed or not.
Yes, if you guessed that this is a regression test, then you are correct. We
once had an image without the docker package inside :) Next, we test the docker
storage setup, that should be up, and in running state. We then run the busybox
image, and see that it should be able to run properly. The *atomic* command is
used next to start the same container. Pulling in the latest *Fedora* image,
and running it is the next test. As our next test, we try to mount */* as
*/host* in the container. The next test is about having */bin*, */sbin*, and
*/usr* mounted as read only in the Atomic host. In our final test, we check
that a privileged container should be able to talk to the host docker daemon.
We also have a [github wiki
explaining how to write any new test case. Feel free to ping if you want to
contribute, and make Fedora flying high in the Cloud.
Fedora Cloud Engineer
CPython Core Developer