On Sun, Nov 18, 2018 at 9:49 PM Frantisek Zatloukal <fzatlouk@redhat.com> wrote:

We have encountered a bug[0] which seemingly “broke” offline updates after systems were upgraded from an older Fedora to Fedora 29 and had some multilib packages installed. After the discussion at last week's Release Retrospective meeting, I am proposing some changes to our blocking criterions in order to address similar issues in the future:


What we test now

We are currently blocking on upgrading from Fedora n-1 and n-2 releases only with default package set installed.


What we can test

We can alter our upgrade test cases to also verify that updates after an upgrade work. The test case might look like this:

  • Install Fedora n-1

  • Upgrade to Fedora n (updates-testing disabled)

  • Enable updates-testing

  • Update to latest packages

  • Verify that upgrade and update went fine and that the multilib packages installed before are still present


Apart from that, we can add (Final Blocking) test case upgrade_dnf_current_workstation_multilib / upgrade_dnf_current_minimal_multilib which will test upgrades (and then updates as described above) with at least one i686 package installed on x86_64 system. The other (less-generic, but closer to what users probably do have on their systems) approach would be to test upgrade with steam (or wine) installed.


What are our opinions about this?


[0] https://bugzilla.redhat.com/show_bug.cgi?id=1642796

I think Adam is right and this was not related to just upgraded systems. So the proposal can be to add a "multilib update must work" criterion (and optionally add a test case for it). The question is whether it is needed. If we knew that multilib updates were broken when we discussed that during the blocker bug meeting, would we have accepted it as a blocker (as affecting a large portion of our user base), or not? I think we would've accepted it as a blocker. Updates must work in general, not just for a default system installation. So if others agree with me on this, no new criterion might be needed. If others disagree, they'll probably also disagree with a dedicated criterion for exactly this use case :-) A new test case can of course be added, sure. Or perhaps suggest an optional step to the existing system update test case, if we don't want to have a fully dedicated test case for this.