Thank you guys, this is a good starter.
On 01/11/2016 02:06 PM, Stef Walter wrote:
On 11.01.2016 12:08, Marius Vollmer wrote:
> Jan Chaloupka <jchaloup(a)redhat.com> writes:
>
>> cockpit integration tests have proofed to be very useful for
>> discovering regression and bugs for kubernetes.
> I am happy to hear that!
>
>> [...]
>>
>> Would it be possible to run kubernetes related cockpit integration
>> tests on each new build of kubernetes in rawhide?
> We have pretty much given up on running our tests on Fedora Rawhide.
>
> - Testing "branched" seems much more relevant than testing
"rawhide".
>
> - Creating a VM image for rawhide seems to require a lot of luck.
>
> - At the time we didn't have the resource to test evey pull request
> against rawhide, and without that, we didn't seem to care enough
> about the results to actually do something about it in a timely
> manner.
That doesn't mean that you can't test rawhide kubernetes RPMs. You would
just test those RPMs against fedora-testing or fedora-23 instead.
Actually, testing rawhide build on f23 is better as I can see right away
if the build is a good candidate for f23 update.
> This might all have changed, but once Fedora 24 is branched, I
think
> we'll put some effort into testing against that for every PR.
>
> Do you really want to test against Rawhide, or are you happy to go along
> with our choice of OSes? (Now with Debian! Come while it's green! :)
Fedora is preferred. If Debian can help to discover additional issues,
why not use both of them.
>> Once I tried to run the tests and ended up with downloading a
lot of
>> images each over 1GB.
> Yes, the copmplete set of tests needs quite some images initially, such
> as complete FreeIPA and OpenShift installations. This can be reduced if
> you only want to run a subset of the tests, but it's only an initial
> setup cost so maybe it is better to just eat it once instead of spending
> a few days on trying to save one hour of downloading. :)
Definitely. The Jenkins script must respect that and does not delete all
images before every run.
>> How hard it would be to write down a simple script that would
expect
>> kubernetes build and distribution? As a result, it would set
>> everything up and run required tests (without user intervention). Or
>> if possible to use your system (possibly Jenkins?) and periodically
>> run integration tests (or with a suitable trigger).
> I think with the recent addition of "vm-customize" this should not be
> hard, not much harder than running the tests in the first place.
>
> Something like this:
>
> $ export TEST_OS=fedora-23
> $ ./testsuite-prepare
> $ ./vm-customize -i kubernetes-*.rpm
> $ ./check-kubernetes
>
> From there you should be able to quite easily add your own non-Cockpit
> tests.
This is better than what I was recommending earlier. But you still need
to remember to run '$ sudo ./vm-prep'.
It looks so easy :).
Stef
_______________________________________________
cockpit-devel mailing list
cockpit-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/admin/lists/cockpit-devel@lists.fedorahost...