test September 2014

test@lists.fedoraproject.org

71 participants
261 discussions

[Fedora QA] #363: Require spins to go through smoke testing before being published
by fedora-badges 30 Jan '15

30 Jan '15

#363: Require spins to go through smoke testing before being published -------------------------+---------------------------------- Reporter: adamwill | Owner: Type: enhancement | Status: new Priority: minor | Milestone: Undetermined Future Component: Test cases | Version: Keywords: | Blocked By: Blocking: | -------------------------+---------------------------------- = problem = Sometimes, we publish spins that are completely DOA. = analysis = There is no mandatory testing of any Fedora images beyond netinst, DVD and the two release blocking desktop live spins, as things stand, but we publish rather more spins than that. In practice we test Xfce, LXDE, SoaS and AMI images to some extent, but others are rarely touched. = enhancement recommendation = viking-ice has suggested that we should produce some kind of validation matrix which has very basic, generic smoke tests: basically, does the image at least boot up sanely? We would require there to be a PASS for these mandatory smoke tests present in that matrix for a spin image to be published on the spins page. QA does not commit to doing the actual testing; we may do so if we have time, but ultimately, if a spin owner wants to have their spin published, it's their responsibility to get the testing done, either by poking QA, doing it themselves, or getting someone else to do it. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/363> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 1

[Fedora QA] #336: ARM as primary arch
by fedora-badges 30 Jan '15

30 Jan '15

#336: ARM as primary arch ------------------------------+------------------------ Reporter: jdulaney | Owner: Type: task | Status: new Priority: major | Milestone: Fedora 19 Component: Release criteria | Version: Keywords: | Blocked By: Blocking: | ------------------------------+------------------------ I'm opening this ticket to track integrating ARM into the current QA process. Paul Whalen is my contact from the ARM side; at FUDCon I gave him the 30,000' view of the current QA process. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/336> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 6

[Fedora QA] #327: test cases that require autopart shrink need to be updated
by fedora-badges 30 Jan '15

30 Jan '15

#327: test cases that require autopart shrink need to be updated ------------------------+------------------------ Reporter: kparal | Owner: Type: defect | Status: new Priority: major | Milestone: Fedora 18 Component: Test cases | Version: Keywords: | Blocked By: Blocking: | ------------------------+------------------------ Anaconda removed shrink option from guided part, now you need to do custom part to achieve it. The test cases must be adjusted to reflect that. See https://bugzilla.redhat.com/show_bug.cgi?id=891881 * https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_%28shrink%29_i… * https://fedoraproject.org/wiki/QA:Testcase_dualboot_with_windows -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/327> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 6

[Fedora QA] #307: Add multi-desktop and multi-arch images to validation process
by fedora-badges 30 Jan '15

30 Jan '15

#307: Add multi-desktop and multi-arch images to validation process ----------------------------------------+------------------------ Reporter: adamwill | Owner: adamwill Type: enhancement | Status: new Priority: critical | Milestone: Fedora 18 Component: Blocker/NTH review process | Version: Keywords: | Blocked By: Blocking: | ----------------------------------------+------------------------ For the last few releases, we've been releasing two dual-layer DVD images - one contains both the 32-bit and 64-bit DVD images, and the other contains the major live spins all together with a menu for picking which one to boot. We don't really encourage public downloads of these, but we _do_ print them on physical media and distribute them at conferences and via the ambassadors program. Apparently, for F17, at least one of the images turned out to be badly broken and we printed/gave out several thousand coasters. Whoops. So, it seems we can't rely on these compendium images to always be working properly, and QA should be validating them before they're used for anything. We'll need to co-ordinate with releng, FAmSCo and possibly FPL on this one, for timing issues. The compendium images are built _after_ the main set of final release images, and this is probably unavoidable, because we have to sign off definitely on the release images before building compilations of them. So we should probably define an official schedule and process for building and testing the compendium images right after the go/no-go, on a schedule that works for FAmSCo and regularly-scheduled events where these media are distributed. CCing Robyn (FPL) and cwickert for FAmSCo. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/307> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 5

[Fedora QA] #297: Review release criteria for Fedora 18
by fedora-badges 30 Jan '15

30 Jan '15

#297: Review release criteria for Fedora 18 -----------------------------------------+------------------------ Reporter: adamwill | Owner: pschindl Type: task | Status: new Priority: major | Milestone: Fedora 18 Component: Proventester Mentor Request | Version: Keywords: | Blocked By: Blocking: | -----------------------------------------+------------------------ Petr suggests on the F17 retrospective: "We should reviewed release criteria, dismiss all which are obsolete and maybe add some new (like we did for PXE). This step should be done in cooperation with other teams (like anaconda and FESCo)." This is kind of a follow-up to https://fedorahosted.org/fedora- qa/ticket/151 . -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/297> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 4

[Fedora QA] #277: Add ARM to release criteria / validation process somehow
by fedora-badges 30 Jan '15

30 Jan '15

#277: Add ARM to release criteria / validation process somehow ----------------------+------------------------ Reporter: adamwill | Owner: adamwill Type: task | Status: new Priority: critical | Milestone: Fedora 17 Component: Wiki | Version: Keywords: arm | Blocked By: Blocking: | ----------------------+------------------------ = phenomenon = ARM is becoming increasingly important to releases. Currently we have no kind of formal validation process for ARM images. Let's work with the ARM group to improve this, so we can test whatever is most important to them. = background analysis = ARM deployment is significantly different from x86: the standard way to do things is to provide a pre-built image to dump onto the system, not to 'install' via anaconda. So a lot of our test suite, which is based around anaconda, may not be entirely appropriate to ARM. Similarly, the desktop validation stuff isn't likely to be super-interesting to them, as desktop isn't currently a priority for ARM and things at that high a level ought to pretty much 'just work'. We should figure out with the ARM team what it would be most useful to test, and how to work testing into their release process. = implementation recommendation = We may need to write a few new tests, then I think we should design an ARM-specific validation matrix, which may be much smaller than the main ones. To start with we may also want to start separate ARM release criteria pages, to codify the things which are most important to ARM releases. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/277> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 9

[Fedora QA] #275: Create test plan for ARM
by fedora-badges 29 Jan '15

29 Jan '15

#275: Create test plan for ARM ----------------------+------------------ Reporter: jdulaney | Owner: Type: task | Status: new Priority: major | Milestone: Component: Trac | Version: Keywords: | Blocked By: Blocking: | ----------------------+------------------ I've decided to create this ticket to track this. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/275> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 3

[Fedora QA] #272: Add changelog to TC/RC announce mails
by fedora-badges 29 Jan '15

29 Jan '15

#272: Add changelog to TC/RC announce mails -------------------------+------------------------ Reporter: adamwill | Owner: robatino Type: enhancement | Status: new Priority: major | Milestone: Fedora 17 Component: Trac | Version: Keywords: | Blocked By: Blocking: | -------------------------+------------------------ = problem = cwickert pointed out in the F16 QA retrospective that the information on what has changed between TC / RC builds is not easy to find for those outside of QA. = enhancement recommendation = We should include a changelog in the TC / RC announce mails. The information can usually be found in the rel-eng trac ticket - e.g. https://fedorahosted.org/rel-eng/ticket/4967 for F16 final - although sometimes, also, packages are pushed to stable between TC / RC builds: these would not always be listed out in the trac ticket. After TC1, the TC / RC announce mails should list all the new builds included in the build compared to the previous build, and any other changes (e.g. kickstart or compose process change to fix some bug or other). -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/272> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 19

[Fedora QA] #237: tests to verify that torrents and mirrors contain signed checksum files
by fedora-badges 29 Jan '15

29 Jan '15

#237: tests to verify that torrents and mirrors contain signed checksum files ----------------------+----------------------------------------------------- Reporter: robatino | Owner: Type: task | Status: new Priority: major | Milestone: Component: Wiki | Version: Keywords: | ----------------------+----------------------------------------------------- In many of the last several releases (11, 13, 14, and now 16), at least some of the Alpha or Beta torrents contain only unsigned checksum files. This would be easy to prevent by examining the .torrent files, which contain file sizes (signing a checksum file adds about 1K to the size). Unfortunately, at present these are not made available for testing prior to being posted on http://torrent.fedoraproject.org , and when the problem is pointed out, no matter how quickly, one is told that the torrent can't be replaced since people are already downloading it. This makes it important to catch the problem in advance. Many (but not all) of the torrent files for the last several releases are still available at http://torrent.fedoraproject.org/torrents/ and http://torrent.fedoraproject.org/spins/ , and can be examined for example with gtorrentviewer. I have not checked any older than 11, and not all the ones after that are available, so the above list of affected releases is probably incomplete. A less serious issue is when the checksum files get signed more than once. For example, the checksum files for F15 Final install discs were signed twice, first for the torrents and again for the mirrors - see http://robatino.fedorapeople.org/checksums/15-Final/Fedora/ . The checksums are identical, and both signatures are valid, but still, it shouldn't happen. Looking at https://fedoraproject.org/wiki/Release_Engineering_Release_Tickets , it says that for Alpha and Beta, the torrents should be staged before the mirrors, but the reverse for Final. I've asked why on #fedora-releng but gotten no response yet. It says nothing about signing the checksum files, though the linked page https://fedoraproject.org/wiki/Stage_final_release_for_mirrors (under the section "Final") mentions it. This may explain why Alpha and Beta torrents are much less likely to have signed files. If possible, it would be nice for the order (torrents vs. mirrors) to be the same for all three, and in any case, the checksum files should be signed once and then used for both torrents and mirrors. None of this is currently documented. -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/237> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 16

[Fedora QA] #261: Revise upgrade test case set
by fedora-badges 29 Jan '15

29 Jan '15

#261: Revise upgrade test case set -----------------------------------------+---------------------------------- Reporter: adamwill | Owner: Type: task | Status: new Priority: major | Milestone: Fedora 17 Component: Proventester Mentor Request | Version: Keywords: | -----------------------------------------+---------------------------------- Our present upgrade test cases are probably a sub-optimal set. We exercise various rarely-used choices - bootloader action, and text upgrade path - but don't exercise more common variations, like installed package sets, install media, and disk layouts. While we can't commit to supporting absolutely any upgrade, we should cover at least some _more_ cases in order to catch major fails. Based on the experience in F16 and the recommendations in the retrospective, I'd suggest we could consider dropping or downgrading the importance of some tests, perhaps: * QA:Testcase_Anaconda_Upgrade_Skip_Bootloader * QA:Testcase_Anaconda_Upgrade_Skip_Bootloader_Text_Mode * QA:Testcase_Anaconda_Upgrade_Update_Bootloader_Text_Mode and add test cases for all or some of the following scenarios (please add any other suggestions as comments): * upgrade from image written to USB stick (livecd-iso-to-disk) * upgrade with KDE (and possibly Xfce and LXDE as 'optional' test cases) * upgrade an EFI install (see also https://fedorahosted.org/fedora- qa/ticket/256 for organizational issues here) * upgrade with bootloader on first partition (not MBR) * upgrade with separate /usr , /home and /var partitions * upgrade a RAID install This should be completed ideally by Fedora 17 Alpha phase, and definitely by Beta phase (when upgrade issues become blockers). -- Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/261> Fedora QA <http://fedorahosted.org/fedora-qa> Fedora Quality Assurance

1 13

← Newer
1
2
3
4
5
6
7
...
27
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

test September 2014