Test run of Jenkins (CI tool)

Gabe Alford redhatrises at gmail.com
Wed Sep 3 13:13:44 UTC 2014


It would be a lot of work, but we could write automated unit testing
scripts using python unittest, python nose, or something else. Then, run
those test with a command like 'run-tests'


On Wed, Sep 3, 2014 at 6:06 AM, Martin Preisler <mpreisle at redhat.com> wrote:

> ----- Original Message -----
> > From: "Shawn Wells" <shawn at redhat.com>
> > To: "scap-security-guide" <scap-security-guide at lists.fedorahosted.org>
> > Sent: Wednesday, September 3, 2014 6:31:00 AM
> > Subject: Test run of Jenkins (CI tool)
> >
> > Also in the vein of automation, went ahead and installed an instance of
> > Jenkins and connected it to SSG:
> >
> > http://jenkins.ssgproject.org:8080/
> >
> > Not meant to be permanent right now, but wanted to get something stood
> > up for us to play with. Martin stood one up for the OpenSCAP
> > interpreter, which was the source of thinking to stand up a sandbox for
> > SSG too. There really isn't much to see in the UI, but one thing it does
> > is integrate into GitHub. Whenever someone issues a pull request,
> > Jenkins will automagically detect that, apply the patch(s), and run
> > "make validate."
>
> I wanted to create a testing Jenkins instance for all things OpenSCAP.
> There are fixed costs in maintaining the master and all the slaves.
> You have to take care of the machines, keep them updated, etc... You
> have to keep updating Jenkins itself and be on top of plugin updates.
>
> In my opinion it makes no sense to have 2 Jenkins servers for projects
> so closely related. I would prefer to pool resources.
>
> Unfortunately we can't just host our Jenkins on OpenShift because we need
> multiple slaves for RHEL 5, 6, 7 and Fedora. And the storage space
> available
> is not enough for our use-case.
>
> > Check out https://github.com/OpenSCAP/scap-security-guide/pull/45 for an
> > example. Or for the lazy :-), see below:
>
> The integration is really cool. I planned to make the Jenkins instance
> public after a testing period and do exactly this.
>
> > Does anyone have experience with Jenkins,  thoughts on how we could
> > begin building out unit tests, or really any thoughts on usefulness
> > beyond "make validate" on pull requests?
>
> I was experimenting with docker for this purpose. The idea was to start
> with EL6 image, put ssg and a test script inside. Then you can call
> docker run and get the results. The docker instance is deleted after
> each run so there is no risk of side effects.
>
> The idea is to do the following:
> 1) pull ssg HEAD
>
> 2) make content
>
> 3) generate a profile with just one Rule selected - the tested rule
>    (a simple python script can easily do this)
>
> 4) evaluate that profile, compare the results with expected results on a
>    freshly installed RHEL6
>
> 5) anti-remediate - break the configuration (if applicable)
>
> 6) evaluate again, check that the result is "fail" (if applicable)
>
> 7) remediate - fix the configuration (if applicable)
>
> 8) evaluate for the third time, check that the rule passes
>
> *) as an added bonus we could run whatever we are testing, let it
>    parse the config files and do another sanity check (sshd for example)
>
> This should provide repeatable tests for all the rules.
>
> Unfortunately we can't do this on OpenShift (AFAIK). We need real
> slaves with docker-io installed on them. Another problem is that we can
> not test every single rule with this approach. "process test" comes to
> mind. Or even systemd tests. However we should be able to test most of
> the rules.
>
> --
> Martin Preisler
> --
> SCAP Security Guide mailing list
> scap-security-guide at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
> https://github.com/OpenSCAP/scap-security-guide/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fedorahosted.org/pipermail/scap-security-guide/attachments/20140903/5ca85e12/attachment-0001.html>


More information about the scap-security-guide mailing list