why not using travis? https://travis-ci.org/

You would simply need to create you unit tests and define travis test commands (a simple and single .travis.yml file), bind your github repo on your travis account and travis will stage and run you test on a temporary instance and destroy it when tests are over.
It also enables you to automatically do the full testing and integretion testing on each push in a branch or pull request.

see this example file: https://github.com/savon-noir/python-libnmap/blob/master/.travis.yml


On Wed, Sep 3, 2014 at 5:19 PM, Trevor Vaughan <tvaughan@onyxpoint.com> wrote:
Seems like this might be a good case for a public OpenShift instance.

Shawn?

Trevor


On Wed, Sep 3, 2014 at 10:39 AM, Gabe Alford <redhatrises@gmail.com> wrote:
Right. That makes sense. Docker wouldn't be a bad idea. Could you use a single image and have a bunch of dockerfiles or run 'docker run' commands for test cases?


On Wed, Sep 3, 2014 at 7:32 AM, Martin Preisler <mpreisle@redhat.com> wrote:
----- Original Message -----
> From: "Gabe Alford" <redhatrises@gmail.com>
> To: "SCAP Security Guide" <scap-security-guide@lists.fedorahosted.org>
> Sent: Wednesday, September 3, 2014 3:13:44 PM
> Subject: Re: Test run of Jenkins (CI tool)
>
> It would be a lot of work, but we could write automated unit testing
> scripts using python unittest, python nose, or something else. Then, run
> those test with a command like 'run-tests'

The test harness is a non issue. There are tons of frameworks that would
work well.

The issue in my opinion is how we run the tests themselves. That's the part
that is hard to solve and we should focus on that first IMO. Unfortunately,
it's hard to evaluation or remediate the system itself without potentially
breaking it. This is simply unacceptable on the developer's workstation.
However we can break docker instances all we want because we just delete
them afterwards :-)

In the future we may have a test runner in upstream repo that would pull
docker images and run all the test cases. This way contributors can run
the tests themselves without having to depend on Jenkins. Because of the
size of the images we shall not keep them in the ssg repo but we could
keep the surrounding test harness, test cases and scripts in there.


--
SCAP Security Guide mailing list
scap-security-guide@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
https://github.com/OpenSCAP/scap-security-guide/



--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvaughan@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

--
SCAP Security Guide mailing list
scap-security-guide@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
https://github.com/OpenSCAP/scap-security-guide/