----- Original Message -----
From: "Gabe Alford" redhatrises@gmail.com To: "SCAP Security Guide" scap-security-guide@lists.fedorahosted.org Cc: "open-scap-list" open-scap-list@redhat.com Sent: Wednesday, July 27, 2016 3:00:34 PM Subject: Re: Latest OpenSCAP changes to speed up SSG builds
On Tue, Jul 26, 2016 at 3:24 PM, Martin Preisler < mpreisle@redhat.com > wrote:
I found pretty bad inefficiencies in some of our XSLTs.
Check out https://github.com/OpenSCAP/openscap/commit/a65bf27dec4a93e2b87cec8cbcd80bec... or https://github.com/OpenSCAP/openscap/commit/1dd5573b2c964b00af57215cadb7f13b...
Long story short I was able to cut SSG build time on my machine from 2m21.061s to 1m08.771s. In other words my machine now needs roughly half the time to build SSG. I was timing `make -j 4`.
It will take a while for these changes to make it into releases and into distributions but I am writing this email because the savings are significant and perhaps you want to enjoy them sooner. SCAP Security Guide contributors are doing a lot of content builds.
This will only work if you !!! have OPENSCAP 1.2.x !!!
If you want to speed up the build, replace
/usr/share/openscap/xsl/xccdf_1.1_to_1.2.xsl with https://raw.githubusercontent.com/OpenSCAP/openscap/maint-1.2/xsl/xccdf_1.1_...
/usr/share/openscap/xsl/xccdf-share.xsl with https://raw.githubusercontent.com/OpenSCAP/openscap/maint-1.2/xsl/xccdf-shar...
and
/usr/share/openscap/xsl/xccdf-guide-impl.xsl with https://github.com/OpenSCAP/openscap/blob/maint-1.2/xsl/xccdf-guide-impl.xsl
Perhaps it would be worth it to deploy this on our Jenkins slaves even before it hits releases? What do you think?
+1
Not like I would want to be the blocker for speedups / SSG Jenkins optimizations, but how would you like to deploy these changes?
Two scenarios come to mind: * just replace the affected XSLT transformations in recent RHEL openscap builds, * make a separate openscap RPM (applied on Jenkins slaves outside of RHEL release cycle process)
While the advantage might be more quicker completion of Jenkins SSG build jobs, there are two disadvantages / concerns related with this: * if modified oscap returns different result than latest oscap available in RHEL - which report should be considered the right one? Which one is valid? * the concern with maybe even more importance is we wouldn't test on default RHEL / Fedora systems like we do now. The environment would be modified a bit (and it might or might not return same results). So to ensure we "will continue working" on default RHEL / Fedora - should we create another Jenkins CI jobs for default RHEL / Fedora?
Maybe we could use "speed optimized" openscap on 'scap-security-guide-pull-requests' Jenkins CI job, and 'default one' on 'scap-security-guide' Jenkins CI job (the latter one being which actually tells if we don't regress across PRs).
But even this approach won't solve the issue when the result of 'speed optimized' openscap would differ from 'default' openscap (but maybe the result on the default one would be more authoritative / determinant in this case).
Just my 2cs.
Jan -- Jan iankko Lieskovsky / Red Hat Security Technologies Team
-- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org https://lists.fedorahosted.org/admin/lists/scap-security-guide@lists.fedorah... https://github.com/OpenSCAP/scap-security-guide/