On 07/19/2017 07:32 AM, Stef Walter wrote:
On 18.07.2017 16:38, Merlin Mathesius wrote:
> On 07/17/2017 05:34 AM, Stef Walter wrote:
>> I've added documentation about launching test subjects using Ansible
>> Dynamic inventory scripts. Test subjects are the thing to be tested. For
>> example a QCow2, container image, or often a set of RPMs. These get
>> launched into a form that Ansible (and thus the test playbooks) can run
>> against them:
>>
>> The tutorial style documentation is here:
>>
>>
https://fedoraproject.org/wiki/CI/Tests
>>
>> More documentation will be landing here about writing tests, preparing a
>> dist-git repo for tests, and so on.
>>
>> The information about the spec is here:
>>
>>
https://fedoraproject.org/wiki/Changes/InvokingTests
>>
>> Note that older version of the spec used to have a test_local.yml file,
>> and test subject specific playbooks. These test subject specific
>> playbooks are no longer necessary with the inventory.
>>
>> In addition test_local.yml may be used in place of tests.yml in the
>> documentation. Once more people have looked over this, I'll open some
>> pull requests to rename test_local.yml to tests.yml.
>>
>> For the documentation to work, we do need to package the inventory
>> scripts that are already present in standard-test-roles. In addition,
>> I've opened a pull request to standard-test-roles for adding support for
>> launching docker containers as dynamic inventory:
>>
>>
https://pagure.io/standard-test-roles/pull-request/22
> Hi, Stef,
>
> Thank you for providing the documentation and examples at
>
https://fedoraproject.org/wiki/CI/Tests.
>
> PR#22 and several other fixe/improvement PRs have been merged.
>
> I was able to experiment with the inventory scripts a bit, and there's a
> few improvements I'd like to see:
>
> 1. The docker script is not compatible with Fedora 25 due to a
> particular improvement in docker (combining "--rm" and "--detach"
for
> "docker run ...") that's available on F26 that the script is taking
> advantage of. F25 could be accommodated (I think) by eliminating the
> "--rm" from "docker run ..." and doing a "docker rm -f
..." during cleanup.
Good catch thanks. Hope this fixes that:
https://pagure.io/standard-test-roles/pull-request/23
Stef,
Yes it does. It has been merged. Thanks.
> 2. The qcow2 script creates a log file in the current directory.
That
> should really be in the artifacts directory--or other directory of
> choice. What would you think of having the script honor a log directory
> environment variable? (eg., "LOGDIR")
Ah yes. It's such a shame that Ansible doesn't pass it's -e variables to
inventory scripts. For this reason the spec mandates also passing the
artifacts directory as an environment variable:
$TEST_ARTIFACTS
https://fedoraproject.org/wiki/Changes/InvokingTests#Invocation
I'll start to make use of it to accomplish this.
> 3. The docker script should probably create a log file too. ("docker
> logs ... > logfile")
Will do along with the above.
Thanks for
https://pagure.io/standard-test-roles/pull-request/24 which
accomplishes placing the inventory script logfiles in the
$TEST_ARTIFACTS directory. It has also been merged.
To further make use of the $TEST_ARTIFACTS environment variable, I have
raised
https://pagure.io/standard-test-roles/pull-request/25 so the
current roles will take their default artifacts directory from
$TEST_ARTIFACTS--using "./artifacts" if not specified. Thus, specifying
"-e artifacts=..." on the ansible-playbook command line is no longer
necessary--but can still be provided as an override if desired.
Would you please let me know what you think about PR#25, Stef?
> I'd be happy to take a shot at implementing the above improvements.
> Then, I think we have something ready to tag and package as v2.0.
Wonderful.
> I would also be interested in seeing the example "test.yml" you refer to
> on the wiki page. I couldn't find it checked in anywhere.
See the gzip git repo. But you can typically substitute test_local.yml.
Got it. I thought maybe you had come up with a new and interesting
standard format for a master test playbook. :)
> Also, do you have any thoughts on how to have a single test.yml playbook
> cleanly pick and choose amongst which tests to run in which
> environments? Some tests simply won't run in a container and/or cloud
> image, and so should be skipped when they can't run properly. I also
> imagine there could be tests that should be run ONLY in a container
> and/or cloud image. I attempted to implement something like this with
> the "selector" role
> (
https://pagure.io/standard-test-roles/blob/master/f/roles/standard-test-s...),
> but it feels rather clumsy to use (eg.,
>
https://upstreamfirst.fedorainfracloud.org/sed/blob/master/f/test_local.yml).
I believe we're brainstorming this today. We could factor this aspect in.
Stef
_______________________________________________
CI mailing list -- ci(a)lists.fedoraproject.org
To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org