On Feb 28, 2014, at 1:46 PM, Josh Boyer <jwboyer(a)gmail.com> wrote:
Can you elaborate on how that's eases the test matrix?
As I said in a conversation with Stephen yesterday, I don't think it's
the case that a common layout in partitions/fs is actually reducing
the test load. From a component standpoint, sure absolutely it is.
One filesystem to test is easier than 3. However, we don't do
_filesystem_ testing in the context of release testing. It's implicit
in the other tests.
Mostly true. Contra example from Fedora 20: LVM Thin Provisioning landed either right at
or just before branch as a Guided partitioning option. And it either didn't install or
the installation wasn't bootable, until just before beta, and then blew up before
release. So that's a lot of testing done for something that quite frankly not a lot of
users were interested in, yet because QA didn't rerun its entire matrix of tests,
include testing this one partition scheme option, it shipped broken. And that's kinda
annoying because a.) it's a prominently available feature by being in the guided
partitioning path, b.) because probably several hundred man hours went into it and because
1 more hour wasn't committed it blew up for shipping because we didn't know until
after committing to release that it was broken.
And that happened in large part because QA spends increasing amounts of time also on
testing the custom partitioning section compared to oldui.
So I'd say one of two things about custom partitioning must become true going forward.
- Custom partitioning is "best effort" in that it's entirely possible (maybe
even likely) something ships that doesn't work. And that's because no one (or too
few) are testing that particular combination of features.
- There needs to be a mandate to remove features from custom partitioning that quite
frankly don't make sense like rootfs on raid4, raid5 or raid6. OK maybe raid5. But not
raid 4 or raid 6. There are other examples I'm sure.
Or some combination of the two.
Chris Murphy