New way to operate the test suite, PREPARE and VERIFY.

Marius Vollmer marius.vollmer at redhat.com
Mon Nov 11 10:05:59 UTC 2013


Hi,

I have changed the way that the test suite needs to be operated.  Right
now, it requires quite a bit of manual interaction and inside knowledge,
but as we gain experience, we will probably automate more of this, and
make it more fool-proof.

Here is how I (intend) to do it:

  $ export TEST_DATA=$HOME/cockpit-tarballs/

  # Get the magic base tarballs.  Do this only once, they wont change
  # without a loud announcement here.

  $ mkdir $TEST_DATA/base
  $ curl https://dl.dropboxusercontent.com/s/10b3q7ualqxew1d/fedora-18-x86_64.tar.gz \
    > $TEST_DATA/base/fedora-18-x86_64.tar.gz
  $ curl https://dl.dropboxusercontent.com/s/10b3q7ualqxew1d/fedora-20-x86_64.tar.gz \
    > $TEST_DATA/base/fedora-20-x86_64.tar.gz

  # Create the tarballs for the actual test runs.  Do this whenever the
  # OS repository or cockpit-deps have changed enough.  Once a day or
  # so.  This is slow and a bit unreliable since it runs VERIFY to test
  # the new tarballs.

  $ cd .../cockpit
  $ git checkout master
  $ TEST_OS=fedora-18 ./PREPARE
  $ mv ./test/fedora-18.tar.gz $TEST_DATA/
  $ TEST_OS=fedora-20 ./PREPARE
  $ mv ./test/fedora-20.tar.gz $TEST_DATA/

  # Run the actual tests.  This shouldn't go to the network much, only
  # mock still runs yum.

  $ cd .../cockpit
  $ git checkout wip/whatever
  $ TEST_OS=fedora-18 ./VERIFY
  $ TEST_OS=fedora-20 ./VERIFY

So the biggest change is that we are now isolated from changes in the OS
and in cockpit-deps when running tests (but not while building).  VERIFY
should also be a bit faster, but not dramatically so.

The idea is that the PREPARE step is done centrally, and we all download
fresh tarballs whenever necessary.  That comes soon, I hope, but for now
you all need to run PREPARE yourself.


More information about the cockpit-devel mailing list