Hi Gabe,

 

I probably should have been more specific as to the Makefile change, but essentially, it just involved changing the combinechecks.py line from this:

 

                $(SHARED)/$(TRANS)/combinechecks.py $(CONF) $(IN)/checks > $(OUT)/unlinked-$(PROD)-oval.xml

 

To this:

 

                $(SHARED)/$(TRANS)/combinechecks.py $(CONF) $(IN)/checks $(SHARED)/oval > $(OUT)/unlinked-$(PROD)-oval.xml

 

That part seems to work great with the code I provided. But the filtering of shared checks included is where the challenge lies.

 

Best regards,

 

Trey Henefield, CISSP

Senior IAVA Engineer

 

Ultra Electronics

Advanced Tactical Systems, Inc.

4101 Smith School Road

Building IV, Suite 100

Austin, TX 78744 USA

 

Trey.Henefield@ultra-ats.com

Tel: +1 512 327 6795 ext. 647

Fax: +1 512 327 8043

Mobile: +1 512 541 6450

 

www.ultra-ats.com

 

From: scap-security-guide-bounces@lists.fedorahosted.org [mailto:scap-security-guide-bounces@lists.fedorahosted.org] On Behalf Of Gabe Alford
Sent: Thursday, April 09, 2015 9:59 AM
To: SCAP Security Guide
Subject: Re: Shared Checks/Fixes ...

 

On Thu, Apr 9, 2015 at 8:04 AM, Trey Henefield <trey.henefield@ultra-ats.com> wrote:



Thank you everyone for the responses!

I completely agree with the benefits of the shared directory. That really wasn't so much my concern, but rather a better way of utilizing it.

As Jan pointed out, I am using a mixed Windows/Linux environment, making it challenging to maintain the symbolic links when traversing different filesystems (NTFS -> EXT).

So far, I have looked at some minor adjustments that can be made to shared/transforms/combinechecks.py and shared/transforms/combinefixes.py.

I have tried adding an additional parameter through the Makefile to point to the $(SHARED)/oval folder and adding the following to shared/transforms/combinechecks.py:

for filename in os.listdir(sys.argv[2]):
if filename.endswith(".xml"):
with open(sys.argv[2] + "/" + filename, 'r') as xml_file:
body = body + xml_file.read()
+
+ for filename in os.listdir(sys.argv[3]):
+ if filename.endswith(".xml"):
+ with open(sys.argv[3] + "/" + filename, 'r') as xml_file:
+ body = body + xml_file.read()

 

You could do it this way for combinechecks.py:

+ SHARED = os.environ.get('SHARED_DIR')


for filename in os.listdir(sys.argv[2]):
     if filename.endswith(".xml"):

+         local_path = sys.argv[2] + "/" + filename

+         if not os.path.islink(local_path):

+             xml_filename = local_path

+         else:

+             xml_filename = SHARED + "/oval/" + filename
-          with open(sys.argv[2] + "/" + filename, 'r') as xml_file:
+         with open(xml_filename, 'r') as xml_file:

 

Then add the following to the Makefile(s):

export SHARED_DIR=$(SHARED)

 

This has the proper effect of adding all project-specific checks first, then adding in all shared checks. If there is a shared check with the same name as a project-specific check, the shared check will be excluded in favor of the project-specific check. Although, I personally think the identifier of each shared check should be prefaced with 'shared_' to help identify it as a shared check. This could help to prevent duplicate IDs in any case.

The downside of the above code though is that it adds all oval checks in the shared folder to the oval output of a specific project. It builds fine. But 'make validate' complains about oval checks not being included in the XCCDF profile. There should be a better way to filter xml file content added from the shared folder to just those specific to the project's supported platform.

As Shawn has hinted, this could perhaps be resolved in some XSLT magic. Are there any XSLT magicians out there? ;) This is a language I have yet to conquer.

Best regards,


Trey Henefield, CISSP
Senior IAVA Engineer

Ultra Electronics
Advanced Tactical Systems, Inc.
4101 Smith School Road
Building IV, Suite 100
Austin, TX 78744 USA

Trey.Henefield@ultra-ats.com
Tel: +1 512 327 6795 ext. 647
Fax: +1 512 327 8043
Mobile: +1 512 541 6450

www.ultra-ats.com

-----Original Message-----
From: scap-security-guide-bounces@lists.fedorahosted.org [mailto:scap-security-guide-bounces@lists.fedorahosted.org] On Behalf Of Jan Lieskovsky
Sent: Thursday, April 09, 2015 7:31 AM
To: SCAP Security Guide
Subject: Re: Shared Checks/Fixes ...

Hello Trey,

----- Original Message -----
> From: "Trey Henefield" <trey.henefield@ultra-ats.com>
> To: "scap-security-guide@lists.fedorahosted.org"
> <scap-security-guide@lists.fedorahosted.org>
> Sent: Wednesday, April 8, 2015 11:13:48 PM
> Subject: Shared Checks/Fixes ...
>
> Greetings,
>
>
>
> I wanted to propose a change to the current structure in place for
> shared checks (shared/oval) and fixes (shared/fixes/bash). I was
> curious to get everyone’s opinion before committing.

Thank you for opening this topic.

>
>
>
> So the problem I see is with the symbolic linking of checks and fixes
> to the shared folder. I have found it problematic when working between
> different operating systems, file systems, and version control systems.

Right from the beginning we knew the concept of symbolic links will introduce issues when dealing with SSG content across multiple OS versions.
But so far there hasn't been urgent need to fix this (to be read as so far there haven't been much reports current behaviour causing issues for developers).
But as the count of the SCAP content products raises it's time to fix this.

>
>
>
> Rather than creating symbolic links to certain oval checks in the
> shared/oval folder, we could choose to just process all oval checks in
> both the project’s checks folder and the shared/oval folder.

The original idea behind introducing /shared directory was the following - when trying to identify corresponding OVAL check for particular XCCDF rule the following scheme should be applied:
* when there's is OVAL check for that product present in input/checks
directory for that product, this OVAL check should be used with priority,
* when there isn't, use the corresponding (equally named) OVAL check from
shared/ directory.

During the time we advanced the idea / concept even more in the sense that shared/ version of OVAL checks doesn't necessarily need to be working for all of the products having these checks (that's the case below mentioning there are shared/ OVAL checks used just for RHEL-7 & Fedora systems, while
RHEL-6 version is different / RHEL-6 product specific).

>
>
>
> However, not all checks in the current ‘shared/oval’ folder are shared
> by all OS. For example, there are some that only apply to RHEL7 and
> Fedora, and not RHEL6.

See the explanation above why that's the case.

>
>
>
> Any thoughts?

If I should attempt to summary:
* the concept of shared/ checks has proven to be beneficial (in the sense it's
less work to keep just one concrete copy updated than try to keep updated multiple
identical copies),
* the mechanism of symbolic links is not the way to go (since it's known not to work
e.g. on Microsoft Windows OS systems),

So IMHO we want to keep the shared/ OVAL checks / fixes concept, but implement it different way. Maybe as already suggested by modificating the build process to look not just into CWD for OVAL check / remediation fix, but also to look into particular subdirectory of shared/ directory (OVAL checks vs fixes) simultaneously with preserving priority (if there's OVAL check / fix present for that product, use that one. If there isn't and there's is symlink, use the /shared version. If there isn't even symlink, return no check / no fix available leading to 'notchecked' state).

HTH

Regards, Jan.
--
Jan iankko Lieskovsky / Red Hat Security Technologies Team

P.S.: As the count of products we produce SCAP content for is only going to
raise in the future, IMHO concept of /shared directory for OVAL checks &
fixes can only save us a lot of duplicate work in the future (when compared
to the scenario of managing multiple copies of the same file on per product
basis).

>
>
>
> Thanks!
>
>
>
> Best regards,
>
>
>
>
>
> Trey Henefield, CISSP
>
> Senior IAVA Engineer
>
>
>
> Ultra Electronics
>
> Advanced Tactical Systems, Inc.
>
> 4101 Smith School Road
>
> Building IV, Suite 100
>
> Austin, TX 78744 USA
>
>
>
> Trey.Henefield@ultra-ats.com
>
> Tel: +1 512 327 6795 ext. 647
>
> Fax: +1 512 327 8043
>
> Mobile: +1 512 541 6450
>
>
>
> www.ultra-ats.com
>
>
>
>
>
>
> Disclaimer
> The information contained in this communication from
> trey.henefield@ultra-ats.com sent at 2015-04-08 17:13:55 is private
> and may be legally privileged or export controlled. It is intended
> solely for use by scap-security-guide@lists.fedorahosted.org and
> others authorized to receive it. If you are not
> scap-security-guide@lists.fedorahosted.org you are hereby notified
> that any disclosure, copying, distribution or taking action in
> reliance of the contents of this information is strictly prohibited and may be unlawful.
>
>
> --
> SCAP Security Guide mailing list
> scap-security-guide@lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
> https://github.com/OpenSCAP/scap-security-guide/
--
SCAP Security Guide mailing list
scap-security-guide@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
https://github.com/OpenSCAP/scap-security-guide/

Disclaimer
The information contained in this communication from trey.henefield@ultra-ats.com sent at 2015-04-09 10:04:14 is private and may be legally privileged or export controlled. It is intended solely for use by scap-security-guide@lists.fedorahosted.org and others authorized to receive it. If you are not scap-security-guide@lists.fedorahosted.org you are hereby notified that any disclosure, copying, distribution or taking action in reliance of the contents of this information is strictly prohibited and may be unlawful.


--
SCAP Security Guide mailing list
scap-security-guide@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
https://github.com/OpenSCAP/scap-security-guide/