On Tue, 19 Jan 2016 07:44:32 -0500 (EST)
Kamil Paral <kparal(a)redhat.com> wrote:
> I'm preparing for an automation workshop at DevConf in
February and
> I'm trying to think of good example cases to show off what
> Taskotron can do.
>
> At the moment, I figure that I'll do at least 1 easy task and 1 not
> so easy but not crazy task.
>
> Rpmlint is usually my go-to for a trivial/example task but I was
> hoping to use something that's a bit more package specific. The
> things I've thought of so far are:
>
> - Put together some package specific stuff for a package that we
> have some control over (testcloud, yourls etc.)
>
> - Find some other guinea pig package that just works well for
> package-specific tasks since the triggering will be artificial
> for the demos
Do you have any ideas what those package specific checks could be?
Because all I can think of is running some test suite which can't be
used during build time in Koji, but we have nothing like that in our
own packages, we would have to find and use some other package (and
study and write a wrapper for that suite).
Yeah, I don't have too many brilliant ideas either. I figured that
writing one to run testcloud's test suite would work well enough - it's
trivial but it runs quickly and would get the idea across well enough.
The only thing I've thought of beyond the list of testcases you
listed below is the Docker test suite that we're likely to run before
long anyways.
>
> - See if there are any existing test cases which would automate
> easily.
Not sure if you mean release validation test cases, but we can most
probably rule them out, because they all require testing with that
specific TC/RC (or an installed system that matches those package
versions), which we can't offer at the moment - not without nested
virt or some heavy modifications in the environment setup process.
So I guess you mean package test cases (everything that starts with
"Package "):
https://fedoraproject.org/wiki/Category:Test_Cases
And this is really a great idea. I have went through all of those
test cases and the following seem simple enough and easily automated:
https://fedoraproject.org/wiki/QA:Testcase_ClamAV
https://fedoraproject.org/wiki/QA:Testcase_Django_smoketest
https://fedoraproject.org/wiki/QA:Testcase_HTTPd
https://fedoraproject.org/wiki/QA:Testcase_MySQL
https://fedoraproject.org/wiki/QA:Testcase_clone_distgit_repository
https://fedoraproject.org/wiki/QA:Testcase_download_upstream_sources
https://fedoraproject.org/wiki/QA:Testcase_build_from_distgit_with_mock
https://fedoraproject.org/wiki/QA:Testcase_Rkhunter
I think that picking one of them could be a nice demonstration of
what we are aiming for with Taskotron. The only downside is that we
would be showing them something they still can't do themselves (since
we're not still open to arbitrary third-party task submissions, nor
we support storing tasks in distgit-like system). But as a
demonstration of what is hopefully coming in a reasonably close
future, I think this would serve well.
Yeah, that'd be good to at least investigate.
Before DevConf, I want to at least have a scheme for supporting tasks
in git and it'd be great if that was in dev or stg before I do the
workshop. The other things that I want to have on our roadmap before
DevConf and be able to say more than "I don't know" about when asked
when they'll land are:
- Report Generation with jinja or something similar
(I thought we had a ticket for this, will write one up)
- Generic test runner
https://phab.qadevel.cloud.fedoraproject.org/T589
Hopefully we'll get disposable clients into prod this week and then get
back to a saner planning method than what we seem to have slipped back
into. Mostly my fault and I apologize for that :(
Tim