Analysis of and Fix for Today's Phabricator Problems
by Tim Flink
After the update yesterday, we discovered several issues with the new
packages.
1) Upgrade of the package was not possible due to how the libphutil
replaced a dir with a symlink. In order to install, you had to
uninstall the old libphutil and reinstall arcanist
2) Arcanist didn't really work (diff and patch, at least) due to an
exception that was thrown about not being able to find a json linter.
Both of these problems can be traced back to an attempt to remove the
bundled php-jsonlint from the libphutil package. Unfortunately, the
library is not a straight bundle and has been modified to diverge from
upstream. Trying to use the packaged php-jsonlint causes problem 2.
To fix this, I've built and released a new package that uses the
bundled php-jsonlint. If you upgraded to the newer libphutil already,
you'll have to uninstall before upgrading for pretty much the same
reason that the last version had to be installed that way - rpm doesn't
like it when you swap dirs for symlinks in a transaction.
I apologize for the problems this caused - I'll be testing the next
upgrade more thoroughly so that this doesn't hit everyone again. I
think that everything is fixed now but let me know if you see any other
issues.
Tim
9 years, 2 months
2015-02-24 @ 21:00 UTC - Downtime for qadevel.cloud
by Tim Flink
The newer version of phabricator has been in staging for a while now
and it's time to update production.
I'm going to be bringing the instance down for upgrade today at 21:00
UTC and I don't expect the downtime to last longer than an hour.
I'll send out another email when the downtime is complete.
Tim
9 years, 2 months
openqa_fedora_tools patch: add 'all' mode
by Adam Williamson
This adds an 'all' mode which runs for the current validation event
compose if it hasn't already been done, then runs for the current
date's Rawhide and Branched nightlies, if they exist and aren't the
same as the current validation event. Has a --yesterday parameter to
run on the nightlies from a day earlier instead, if your timezone /
cron config don't hook up great with releng's. (In future we ought to
have a daemon that listens for compose events from fedmsg or
something). 'Today' and 'yesterday' are calculated in UTC.
Right now the non-'validation event' results are just going to sit in
OpenQA, but I have Grand Plans to get 'em out via fedmsg and/or
special wiki pages. For now only folks with VPN access or their own
Coconut instance will be able to see the results, sorry!
We *may* wind up running the tests for a nominated nightly compose
twice - once before it gets nominated, once after - but that doesn't
seem like a huge problem. Obviously we ought to build one Glorious
Unified Sausage Machine for nightlies which pre-flights 'em via OpenQA
then does the nomination if that passes, but for now this is fine, I
think.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
9 years, 2 months
Re: Fedora Beaker instance
by Dan Callaghan
Hi Honza,
Excerpts from Honza Horak's message of 2015-02-13 00:44 +10:00:
> Hi Dan,
>
> I missed your workshop at DevConf unfortunately, but as one of the
> members of Env & Stacks Working Group I'm wondering what is the current
> status of beaker instance [1], since the wiki page [2] seems to be quite
> short.
>
> Is it available for some testing already? Is there any doc how to use it
> as I'm ordinary fedora packager (I was not able to log in)?
Tim Flink and the Fedora QA team have been working on deploying Beaker
in Fedora, so they are best able to answer this.
Currently there is no way for regular Fedora contributors to log in,
it's just username+password accounts created by Tim by hand. The plan is
to use mod_auth_openid pointing as FASOpenID with some group
restrictions enforced, but that needed some support on the FASOpenID
side to expose Fedora group info. I guess that will be covered as part
of the Ipsilon upgrade coming sometime soon.
There is also an issue with the LC hostname in log URLs which we are
working on.
> I'm also not sure what is it's use case actually. Is it generally
> providing a virtual machine for any use case or just for some special
> type of work?
The initial use case was to run installer tests:
https://bitbucket.org/fedoraqa/fedora-beaker-tests
There has also been some interest in using Beaker to make non-x86 arches
available to packagers who might need them temporarily for porting
purposes. That's something we would look at once Beaker is fully up and
running.
I expect Tim will want to discourage the use of Beaker for "just give me
an x86 VM to play with" scenarios, because there are other tools for
that, and Fedora Beaker's hardware pool will be fairly limited.
> Btw. I'm also thinking about some fedora proposal for being able to
> store some arbitrary tests in Fedora (basically some best practices how
> to write tests above unit tests; written in anything, including
> beakerlib). Is this something you care about or is this something
> totally irrelevant for beaker instance?
Yes I think Tim is very interested in the idea of package integration
tests, which may include running jobs in Beaker. I'll let him elaborate
further.
--
Dan Callaghan <dcallagh(a)redhat.com>
Software Engineer, Hosted & Shared Services
Red Hat, Inc.
9 years, 2 months
Tested patches for openqa_fedora_tools to use fedfind and wikitcms, fix some result submissions
by Adam Williamson
OK, I completed my fedfind/wikitcms conversion patch for
openqa_fedora_tools. I also fixed up some of the test case submission
data bits - they broke with relval report-auto (instead of jskladan's
version of non-interactive result submission) because they didn't
quite grok the 'test instance' concept report-auto follows.
Note this needs the very latest git wikitcms. I don't think latest
relval or fedfind is strictly required, but may as well bump them
anyway. I've installed the current wikitcms on fedora-qa-01. I will
cut new releases tomorrow.
I have tested this on fedora-qa-01, it ran and submitted a full set of
results for 2015-02-07. I believe it should work for Branched
nightlies (if we wind up nominating any) and TC/RC builds also, which
is a significant win. I did test this a bit:
https://fedora-qa-01.lab.bos.redhat.com/tests/15 is a one-job run
against F21 Final RC5, you can see it successfully got the ISO and
started the test but the test fails because the 'Timbuktu' screen
isn't there.
I revised the openqa_trigger.py CLI to use argparse; it has the same
basic modes as before but the syntax is a bit different.
'openqa_trigger.py current' runs against the 'current' compose event
if it has not already done so (what just plain 'openqa_trigger.py' did
before). 'openqa_trigger.py event' takes the same 'version id'
arguments as 'relval report-auto' and runs tasks (but does not
automatically report results) against the specified compose (what you
used to get by passing args, approximately, but much more robust and
capable).
The basic approach is that openqa_trigger gets a ValidationEvent from
python-wikitcms - either the Wiki.current_event property for
'current', or the event specified, obtained via the newly-added
Wiki.get_validation_event(), for 'event'. For 'event' it then just
goes ahead and runs the jobs and prints the IDs. For 'current' it
checks the last run compose version for each arch and runs if needed,
as before. The ValidationEvent's 'sortname' property is the value
written out to PERSISTENT to track the 'last run' - this property is
intended to always sort compose events 'correctly', so we should
always run when appropriate even when going from Rawhide to Branched,
Branched to a TC, TC to RC, RC to (next milestone) TC.
On both paths it gets a fedfind.Release object via the ValidationEvent
- ValidationEvents have a ff_release property which is the
fedfind.Release object that matches that event. It then queries
fedfind for image locations using a query that tries to get just *one*
generic-ish network install image for each arch. It passes the
location to download_image(), which is just download_rawhide_iso()
renamed and does the same job, only it can be simpler now.
>From there it works pretty much as before, except we use the
ValidationEvent's 'version' property as the BUILD setting for OpenQA,
and report_job_results get_relval_commands() is tweaked slightly to
parse this properly to produce a correct report-auto command.
Probably the most likely bits to break here are the sortname thing
(see wikitcms helpers.py fedora_release_sort(), it's pretty stupid, I
should re-write it) and the image query, which might wind up getting
more than one image depending on how exactly the F22 Alpha composes
look. I'll keep a close eye on that. We can always take the list from
fedfind and further filter it so we have just one image per arch.
Image objects have a .arch attribute so this will be easy to do if
necessary. I *could* give the fedfind query code a 'I'm feeling lucky'-
ish mode to only return one image per (whatever), but not sure if that
would be too specialized, I'll think about it.
Please do ask any questions :)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
9 years, 2 months
First cut at openqa-fedora-tools patch for fedfind/wikitcms
by Adam Williamson
I haven't been able to test this one fully as I'm on my laptop and
haven't fully deployed the openqa bits here, but I at least tested it
just running openqa_trigger.py - it gets as far as
download_rawhide_iso() and fails because the expected dir doesn't exist.
It also doesn't handle the case where you pass in args, yet. But it
should at least give the general idea. Wanted to send it out before I
forgot.
(note: the all_boot_images property isn't available for non-nightly
composes likes TCs/TCs, so I went with a find_images() query. I will try
and keep that interface stable...and send patches if I don't :> unless
anyone thinks the query object design is bad and wants to suggest a
better way.)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net
9 years, 2 months
fedfind use of koji
by Adam Williamson
Hi, list. Josef asked me on IRC about this, but he logged off just
before I replied, so I figured I'd make the answer public for
posterity.
He asked "do you think it would be possible to either remove the
dependency, or make it into a conditional import? As the koji python
library is not really standalone, it makes it quite weird to deploy on
the OpenSuse"
Unfortunately the answer is it's not entirely straightforward. Koji is
used for a large part of fedfind, in fact the core of the whole thing -
finding the nightly compose tasks. Every image for a nightly, except
the boot.isos, is only available as a Koji task. The very first bit of
fedfind that got written was the code to poke Koji and figure out the
nightly images for a given date. I really only extended it out to
stable/milestone releases because it seemed like an easy win, the
nightly stuff is actually the core of fedfind.
I suppose in theory it *could* be made optional and the NightlyEvent
classes would simply be extremely incomplete in such a case - they'd
have to ifdef around the lack of koji and return only boot.iso images
whatever you asked them for, which I personally think would be kind of
horrible.
The other option I guess is to talk to the Koji XML-RPC API directly
instead of through the koji library; I can take a look at how
messy/tedious that would be to implement today.
Thanks!
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
9 years, 2 months