On Thu, 2017-02-16 at 21:19 -0500, Colin Walters wrote:
On Thu, Feb 16, 2017, at 12:01 PM, Adam Williamson wrote:
> Glad to see you thinking about this. openQA also has desktop tests
> which we could certainly run on Atomic Workstation images;
This is somewhat of a tangent but I was just thinking that openQA
tests the installer *only*. One of the whole major goals of this is
that by using rpm-ostree to assemble content on the server side,
*that* is the base of the compose (plus any apps/containers to test) Which means that
it's
very valid to have a cached set of existing systems, and do in-place upgrades,
rather than doing fresh installs each time. Because of the image-based
nature of ostree, one can have a high degree of confidence that the result
of an update is exactly identical to a fresh install[1].
While you've done a lot of heroics maintaining needles for openCV, and
it does make sense to test the installer that way (particularly since
the installer is almost identical, we can share most of that across products),
I'd like to have a variety of systems that test the ostree commit via a mix of
inplace
updates and fresh installs, like we do for Atomic Host.
Well, sure, that would be great. But do you have a variety of systems
that can run such tests in place? If not, it doesn't seem like it'd do
any harm to run openQA's desktop tests for now. Then at least you'd
know if the terminal works. :p
Also, openQA isn't tied to installer testing; we could certainly have
it deploy a given Atomic Workstation image - via the installer or not -
then update to a later ostree and run its desktop tests. I'm currently
working on a mechanism to have openQA automatically run its post-
install tests on every critical path update that hits Bodhi, for
instance. Well, I'm working on that and ten other things, you know how
it goes. But it's not something openQA has any trouble doing.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net