A tool for support of SCAP content development

Jan Lieskovsky jlieskov at redhat.com
Fri Sep 11 09:39:40 UTC 2015


 Hello Jan,

----- Original Message -----
> From: "Jan Cerny" <jcerny at redhat.com>
> To: "SCAP Security Guide" <scap-security-guide at lists.fedorahosted.org>
> Sent: Thursday, June 4, 2015 9:15:42 AM
> Subject: A tool for support of SCAP content development
> 
> Hello,
> 
> Nowadays, there is almost no support for OVAL content development.

Agree in this point.

> The developers have to edit the XML files manually, there is no
> analysis tool, no debugger or any other tool that could make
> the content development faster and easier.
> I have started working on a new utility that will be able to
> debug OVAL checks, show step-by-step how they are evaluated,
> browse collected objects and system characteristics, communicate
> with probes, etc.

Two questions:
* is this new tool intended to become part of the openscap-* package,
  or rather the plan being it to be a separate tool?,
* above you speak about debugger -- is the plan to add also profiler?

> I'm working on design of the utility now. I'm considering many
> alternatives, so I would like to ask you a few questions.
> * Can you describe most common problems that you have to face
>   when you create the OVAL content?

Absence of ability to debug a particular OVAL check. Right now
it's possible to use 'oscap oval eval / oscap oval collect' 
for particular rule to determine what got actually collected on
the system. This approach is working for simple rules. But it does
not scale properly (if the underlying OVAL object consist of another
OVAL variable, that variable using some OVAL functions on other
variables and so on, it's hard to determine which OVAL object failed
to be collected at the beginning -- which OVAL object is responsible
for the whole rule failure).

Next step is XCCDF evaluation - while the corresponding OVAL check might
work properly, it's possible (due to combination of "check", "check_existence",
"entity_check" attributes, order and / or logical operations between the
different OVAL criterions) the final XCCDF rules is still failing even
in the case "some" OVAL objects have been collected on the system.

Right now oscap provides 'oscap xccdf eval .. --oval-results' combination
alllowing to inspect, why the rule as a whole did not pass (e.g. wrong
"check" / "check_existence" attributes etc.).

But it would be good if the new tool would support XCCDF debugging too
(something like 'oscap xccdf eval ... --oval-results' is allowing now,
but enhanced).


> * How do you usually solve these problems?

See above. By separating the rule intended to be debugged. Then running:
* oscap oval eval --id 
* oscap oval collect --id
* oscap xccdf eval .. --oval-results

combination of commands (depending on the level [e.g. if OVAL itself,
or at XCCDF level] the OVAL check is failing).

> * Can you tell me an example when did you run into issues with OVAL?

Basically any a bit more complex OVAL check needs to be designed in
stages (from top to bottom). Consider OVAL criterion requiring presence
just some OVAL object, this object to consist of multiple stages, e.g.:

* you first collect something on the system into object, save the
  subexpression of it into <local_variable>,
* apply couple of functions to that variable, finally create new final
  object from the last variable. This final object is then used in the
  OVAL test.

Right now it needs to be done from bottom to the top (at each level, at
each <local_variable> you mostly need to verify it returns appropriate
results). Other issue is OVAL standard reporting functionality. AFAICT
the standard requires the rule to return 'error' also in the case the
particular OVAL rule wasn't able to obtain child results (results for
the child objects compositing the final object). Having seen this 'error'
result for the first time, it's not clear if:
* OVAL check truly has a syntax error,
* or OVAL check is correct, but the engine wasn't able to determine value
  for some of the OVAL variables above in the evaluation tree.

Another issue / RFE in 'oscap oval eval / collect' functionality is the
'--id' parameter. IIRC as of right now it supports specification of
an ID just for OVAL objects (therefore having OVAL variable manipulating
some previously collected OVAL object via some OVAL function, it's 
impossible to see the retrieved values of that variable directly.

You need to see the values of the original OVAL object, then create
another OVAL object from that OVAL variable using the OVAL function,
to be able to see the true values, that OVAL variable is holding).

Not to speak to abstract. Consider the following:

<ind:textfilecontent54_object id="my_object" ..>
  <ind:filepath>..</ind:filepath>
  <ind:pattern ..>..</ind:pattern>
  <ind:instance ..>..</ind:instance>
</ind:textfilecontent54_object>

<local_variable id="my_variable>
  <some_oval_function_here, e.g. "split", "concat", whatever>
    <object_component object_ref="my_object" />
    ..
  </some_oval_function_here>
</local_variable>

Right now it's not possible to perform the following:

  oscap oval collect --id "my_variable" --syschar /tmp/out

because "my_variable" doesn't constitute an OVAL object (therefore
to bypass it, it's necessary to define new artificial OVAL object
from that "my_variable" local variable, and try to collect it).

Therefore it would be very handy if 'collect' could be enhanced
to support variables too.

> * Imagine an ideal tool for an OVAL developer. What should it be
>   able to do? What would be its features?

Maybe above being a sketch. From the above it's not clear if the
new tool should be just debugger, or profiler too. If the debugger,
basically it would be able to:
* report values of particular OVAL objects, variables at any time
  of OVAL evaluation,
* report the branch OVAL check has been evaluated (like OVAL check
  went through criterion #1, the rest of the criteria weren't taken
  into account, since criterion #1 already succeeded),
* + the requests as formulated above.

If we would talk also about profiling capabilities - it would be
able to report things like CPU usage in %, virtual memory usage,
and or count of system files processed when evaluating particular rule.

This would be handy to optimize current form in many rules.

> I would appreciate any suggestions, remarks or other inputs.
> Thank you very much for your reply.

Should you need any other clarification wrt to some of the points
above, just ask.

Thank you && Regards, Jan
--
Jan iankko Lieskovsky / Red Hat Security Technologies Team

> 
> Regards
> 
> Jan Černý
> Security Technologies | Red Hat, Inc.
> --
> SCAP Security Guide mailing list
> scap-security-guide at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
> https://github.com/OpenSCAP/scap-security-guide/


More information about the scap-security-guide mailing list