Hello Jan,
----- Original Message -----
From: "Jan Cerny" jcerny@redhat.com To: "SCAP Security Guide" scap-security-guide@lists.fedorahosted.org Sent: Thursday, June 4, 2015 9:15:42 AM Subject: A tool for support of SCAP content development
Hello,
Nowadays, there is almost no support for OVAL content development.
Agree in this point.
The developers have to edit the XML files manually, there is no analysis tool, no debugger or any other tool that could make the content development faster and easier. I have started working on a new utility that will be able to debug OVAL checks, show step-by-step how they are evaluated, browse collected objects and system characteristics, communicate with probes, etc.
Two questions: * is this new tool intended to become part of the openscap-* package, or rather the plan being it to be a separate tool?, * above you speak about debugger -- is the plan to add also profiler?
I'm working on design of the utility now. I'm considering many alternatives, so I would like to ask you a few questions.
- Can you describe most common problems that you have to face when you create the OVAL content?
Absence of ability to debug a particular OVAL check. Right now it's possible to use 'oscap oval eval / oscap oval collect' for particular rule to determine what got actually collected on the system. This approach is working for simple rules. But it does not scale properly (if the underlying OVAL object consist of another OVAL variable, that variable using some OVAL functions on other variables and so on, it's hard to determine which OVAL object failed to be collected at the beginning -- which OVAL object is responsible for the whole rule failure).
Next step is XCCDF evaluation - while the corresponding OVAL check might work properly, it's possible (due to combination of "check", "check_existence", "entity_check" attributes, order and / or logical operations between the different OVAL criterions) the final XCCDF rules is still failing even in the case "some" OVAL objects have been collected on the system.
Right now oscap provides 'oscap xccdf eval .. --oval-results' combination alllowing to inspect, why the rule as a whole did not pass (e.g. wrong "check" / "check_existence" attributes etc.).
But it would be good if the new tool would support XCCDF debugging too (something like 'oscap xccdf eval ... --oval-results' is allowing now, but enhanced).
- How do you usually solve these problems?
See above. By separating the rule intended to be debugged. Then running: * oscap oval eval --id * oscap oval collect --id * oscap xccdf eval .. --oval-results
combination of commands (depending on the level [e.g. if OVAL itself, or at XCCDF level] the OVAL check is failing).
- Can you tell me an example when did you run into issues with OVAL?
Basically any a bit more complex OVAL check needs to be designed in stages (from top to bottom). Consider OVAL criterion requiring presence just some OVAL object, this object to consist of multiple stages, e.g.:
* you first collect something on the system into object, save the subexpression of it into <local_variable>, * apply couple of functions to that variable, finally create new final object from the last variable. This final object is then used in the OVAL test.
Right now it needs to be done from bottom to the top (at each level, at each <local_variable> you mostly need to verify it returns appropriate results). Other issue is OVAL standard reporting functionality. AFAICT the standard requires the rule to return 'error' also in the case the particular OVAL rule wasn't able to obtain child results (results for the child objects compositing the final object). Having seen this 'error' result for the first time, it's not clear if: * OVAL check truly has a syntax error, * or OVAL check is correct, but the engine wasn't able to determine value for some of the OVAL variables above in the evaluation tree.
Another issue / RFE in 'oscap oval eval / collect' functionality is the '--id' parameter. IIRC as of right now it supports specification of an ID just for OVAL objects (therefore having OVAL variable manipulating some previously collected OVAL object via some OVAL function, it's impossible to see the retrieved values of that variable directly.
You need to see the values of the original OVAL object, then create another OVAL object from that OVAL variable using the OVAL function, to be able to see the true values, that OVAL variable is holding).
Not to speak to abstract. Consider the following:
<ind:textfilecontent54_object id="my_object" ..> ind:filepath..</ind:filepath> <ind:pattern ..>..</ind:pattern> <ind:instance ..>..</ind:instance> </ind:textfilecontent54_object>
<local_variable id="my_variable> <some_oval_function_here, e.g. "split", "concat", whatever> <object_component object_ref="my_object" /> .. </some_oval_function_here> </local_variable>
Right now it's not possible to perform the following:
oscap oval collect --id "my_variable" --syschar /tmp/out
because "my_variable" doesn't constitute an OVAL object (therefore to bypass it, it's necessary to define new artificial OVAL object from that "my_variable" local variable, and try to collect it).
Therefore it would be very handy if 'collect' could be enhanced to support variables too.
- Imagine an ideal tool for an OVAL developer. What should it be able to do? What would be its features?
Maybe above being a sketch. From the above it's not clear if the new tool should be just debugger, or profiler too. If the debugger, basically it would be able to: * report values of particular OVAL objects, variables at any time of OVAL evaluation, * report the branch OVAL check has been evaluated (like OVAL check went through criterion #1, the rest of the criteria weren't taken into account, since criterion #1 already succeeded), * + the requests as formulated above.
If we would talk also about profiling capabilities - it would be able to report things like CPU usage in %, virtual memory usage, and or count of system files processed when evaluating particular rule.
This would be handy to optimize current form in many rules.
I would appreciate any suggestions, remarks or other inputs. Thank you very much for your reply.
Should you need any other clarification wrt to some of the points above, just ask.
Thank you && Regards, Jan -- Jan iankko Lieskovsky / Red Hat Security Technologies Team
Regards
Jan Černý Security Technologies | Red Hat, Inc. -- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/