On the purpose of cpe_generate.py script (& mainly cpe_variables section within it)

Shawn Wells shawn at redhat.com
Tue Aug 26 18:01:26 UTC 2014


On 8/26/14, 6:05 AM, Jan Lieskovsky wrote:
> Hello folks,
>
>   within effort to rewrite umask checks (more about the motivation for this
> later within its dedicated email thread) encountered the following 
> RHEL/6/transforms/cpe_generate.py script related issue.
>
>   Use case: Let's consider RHEL/6/input/checks/accounts_umask_etc_profile.xml
> OVAL check would have the form as attached (it's not the final form of the
> check. The final one will be slightly more complex yet. In any case the attached
> form is syntactically correct one & sufficient to demonstrate the issue I would
> like to speak below about), then running 'make' in scap-security-guide/RHEL/6
> subdirectory returns the following error when trying to generate the datastream
> output:
>
> oscap ds sds-compose output/ssg-rhel6-xccdf-1.2.xml output/ssg-rhel6-ds.xml
> oscap ds sds-add output/ssg-rhel6-cpe-dictionary.xml output/ssg-rhel6-ds.xml
> File 'output/ssg-rhel6-ds.xml' line 32236: Element '{http://oval.mitre.org/XMLSchema/oval-definitions-5}object_component': No match found for key-sequence ['oval:ssg:obj:111'] of keyref '{http://oval.mitre.org/XMLSchema/oval-definitions-5}objectKeyRef'.
> Invalid SCAP Source Datastream content(1.2) in output/ssg-rhel6-ds.xml.
> make: *** [content] Error 1

Tried to replicate this to no avail... building on RHEL 6:
> xsltproc /usr/share/openscap/xsl/xccdf_1.1_remove_dangling_sub.xsl
> output/ssg-rhel6-xccdf.xml \
>         > output/ssg-rhel6-xccdf-nodangles.xml
> xsltproc --stringparam reverse_DNS org.ssgproject.content
> /usr/share/openscap/xsl/xccdf_1.1_to_1.2.xsl \
>         output/ssg-rhel6-xccdf-nodangles.xml >
> output/ssg-rhel6-xccdf-1.2.xml
> sed -i '/idref="dangling reference to /d' output/ssg-rhel6-xccdf-1.2.xml
> oscap ds sds-compose output/ssg-rhel6-xccdf-1.2.xml
> output/ssg-rhel6-ds.xml
> oscap ds sds-add output/ssg-rhel6-cpe-dictionary.xml
> output/ssg-rhel6-ds.xml
> oscap ds sds-add output/ssg-rhel6-oval.xml output/ssg-rhel6-ds.xml



>
> Investigated the problem further, and noticed the problem is content of RHEL/6/output/ssg-rhel6-cpe-oval.xml
> file. Namely it contains the content as follows:
>
> ...
>     </states><variables><local_variable id="oval:ssg:var:110" datatype="int" comment="x" version="1">
>       <substring substring_start="1" substring_length="1">
>         <object_component item_field="subexpression" object_ref="oval:ssg:obj:111"/>
>       </substring>
>     </local_variable>
>     </variables></oval_definitions>
>
> which after more detailed look explains why 'oval:ssg:obj:111' wouldn't be defined in output/ssg-rhel6-ds.xml
> file (the oval:ssg:var:110 local variable references oval:ssg:obj:111, but this object isn't defined
> within that ssg-rhel6-cpe-oval.xml file [only in different part of ssg-rhel6-ds.xml file]).
>
> So investigated further to see, which part of the RHEL/6's Makefile generates / includes this variable,
> when creating ssg-rhel6-cpe-oval.xml file, and noticed the responsible script being RHEL/6/transforms/cpe_generate.py.
>
> The questions:
> 1) Can someone more familiar with the implementation of this script clarify it's purpose?
>    It requires two input files (OVAL file & CPE file), and one identifier (ssg), and generates
>    'ssg-rhel6-cpe-oval.xml' file.
>
> 2) Can someone more familiar with the implementation of this script clarify the need for the
>    following construct within it?:
>
>     82         variables = ovaltree.find("./{%s}variables" % oval_ns)
>     83         cpe_variables = extract_referred_nodes(ovaltree, variables, "var_ref")
>     84         if cpe_variables:
>     85                 variables.clear()
>     86                 [variables.append(cpe_variable) for cpe_variable in cpe_variables]
>     87         else:
>     88                 ovaltree.remove(variables)
>
>     Is it there to (when generating ssg.ini object name to object id file) to incorporate
>     also OVAL local_variables that might reference another variables via the "var_ref"
>     attribute?
>
>     The reason why I am asking being that have noticed, when I comment out the following part
>     of this file (IOW when the cpe_generate.py script has the following form around the
>     critical lines):
>
>     82         variables = ovaltree.find("./{%s}variables" % oval_ns)
>     83         #cpe_variables = extract_referred_nodes(ovaltree, variables, "var_ref")
>     84         #if cpe_variables:
>     85         #       variables.clear()
>     86         #       [variables.append(cpe_variable) for cpe_variable in cpe_variables]
>     87         #else:
>     88         ovaltree.remove(variables)
>
>     the aforementioned error message:
>
>     File 'output/ssg-rhel6-ds.xml' line 32236: Element '{http://oval.mitre.org/XMLSchema/oval-definitions-5}object_component': No match found for key-sequence ['oval:ssg:obj:111'] of keyref '{http://oval.mitre.org/XMLSchema/oval-definitions-5}objectKeyRef'.
>
>     when generating ssg-rhel6-ds.xml isn't reported anymore & the content builds successfully.
>
> Since I wouldn't like to break something, the purpose of this email is to ask someone more familiar
> with the implementation of cpe_generate.py script, why the cpe_variables section (lines from #83 up
> to #87 need to be present there)?
>
> Is there a chance this behaviour is somehow related with the following commit:
>   [1] https://github.com/OpenSCAP/scap-security-guide/commit/59d650ffe140e4ae742dc7b4893df278bf1b3334 ?
>
> For example in the sense the original meaning of 'cpe_variables' part of cpe_generate.py was to include
> also variables, that reference other variables via 'var_ref' attribute into final ssg.ini file
> (and assign IDs in the form of "oval:ssg:var:..." to them too)? But the change [1] is causing
> variable objects having 'var_ref' attribute is now included within relabelids.py script & therefore
> the 'cpe_variables' section of the cpe_generate.py script is not needed anymore & can be safely remove
> as suggested above (or in the also attached RHEL_6_transforms_cpe_generate_py.diff patch)?
>
> Any hints you can provide wrt to observed cpe_generate.py behaviour & how to possibly fix it
> are appreciated.
>
> Thank you && Regards, Jan.
> --
> Jan iankko Lieskovsky / Red Hat Security Technologies Team
>
> P.S.: If wondering why this problem didn't show earlier, assuming there isn't such construct
>       (local_variable referencing another local_variable referencing textfilecontent54_object)
>       used in the current content yet. Tried also different approach (encapsulate those local_variables
>       into variable_objects & reference those objects instead), but it didn't help (in the sense
>       datastream output would be successfully generated).
>
>


-- 
Shawn Wells
Director, Innovation Programs
shawn at redhat.com | 443.534.0130
@shawndwells

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fedorahosted.org/pipermail/scap-security-guide/attachments/20140826/85b3f5a3/attachment.html>


More information about the scap-security-guide mailing list