<div dir="ltr"><div class="gmail_default" style="font-size:small">+1 on work to support OVAL authoring.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Greg Elin</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 14, 2015 at 3:17 AM, Jan Cerny <span dir="ltr"><<a href="mailto:jcerny@redhat.com" target="_blank">jcerny@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello,<br>
<div><div class="h5"><br>
----- Original Message -----<br>
> From: "Jan Lieskovsky" <<a href="mailto:jlieskov@redhat.com">jlieskov@redhat.com</a>><br>
> To: "Jan Cerny" <<a href="mailto:jcerny@redhat.com">jcerny@redhat.com</a>><br>
> Cc: "SCAP Security Guide" <<a href="mailto:scap-security-guide@lists.fedorahosted.org">scap-security-guide@lists.fedorahosted.org</a>><br>
> Sent: Friday, September 11, 2015 11:39:40 AM<br>
> Subject: Re: A tool for support of SCAP content development<br>
><br>
><br>
> Hello Jan,<br>
><br>
> ----- Original Message -----<br>
> > From: "Jan Cerny" <<a href="mailto:jcerny@redhat.com">jcerny@redhat.com</a>><br>
> > To: "SCAP Security Guide" <<a href="mailto:scap-security-guide@lists.fedorahosted.org">scap-security-guide@lists.fedorahosted.org</a>><br>
> > Sent: Thursday, June 4, 2015 9:15:42 AM<br>
> > Subject: A tool for support of SCAP content development<br>
> ><br>
> > Hello,<br>
> ><br>
> > Nowadays, there is almost no support for OVAL content development.<br>
><br>
> Agree in this point.<br>
><br>
> > The developers have to edit the XML files manually, there is no<br>
> > analysis tool, no debugger or any other tool that could make<br>
> > the content development faster and easier.<br>
> > I have started working on a new utility that will be able to<br>
> > debug OVAL checks, show step-by-step how they are evaluated,<br>
> > browse collected objects and system characteristics, communicate<br>
> > with probes, etc.<br>
><br>
> Two questions:<br>
> * is this new tool intended to become part of the openscap-* package,<br>
> or rather the plan being it to be a separate tool?,<br>
<br>
</div></div>It was decided to be as part of the openscap.<br>
Official instructions approved by university says "implement this in the OpenSCAP project<br>
by form of editing existing source code."<br>
For university instructions, see Description in Czech on this page:<br>
<a href="https://diplomky.redhat.com/topic/show/335/a-tool-for-development-of-oval-definitions-within-openscap-project" rel="noreferrer" target="_blank">https://diplomky.redhat.com/topic/show/335/a-tool-for-development-of-oval-definitions-within-openscap-project</a><br>
I should definitely put the link on the wiki.<br>
<span class=""><br>
> * above you speak about debugger -- is the plan to add also profiler?<br>
<br>
</span>It is not in my plan now, but thank your very much for your idea!<br>
<div><div class="h5"><br>
><br>
> > I'm working on design of the utility now. I'm considering many<br>
> > alternatives, so I would like to ask you a few questions.<br>
> > * Can you describe most common problems that you have to face<br>
> > when you create the OVAL content?<br>
><br>
> Absence of ability to debug a particular OVAL check. Right now<br>
> it's possible to use 'oscap oval eval / oscap oval collect'<br>
> for particular rule to determine what got actually collected on<br>
> the system. This approach is working for simple rules. But it does<br>
> not scale properly (if the underlying OVAL object consist of another<br>
> OVAL variable, that variable using some OVAL functions on other<br>
> variables and so on, it's hard to determine which OVAL object failed<br>
> to be collected at the beginning -- which OVAL object is responsible<br>
> for the whole rule failure).<br>
><br>
> Next step is XCCDF evaluation - while the corresponding OVAL check might<br>
> work properly, it's possible (due to combination of "check",<br>
> "check_existence",<br>
> "entity_check" attributes, order and / or logical operations between the<br>
> different OVAL criterions) the final XCCDF rules is still failing even<br>
> in the case "some" OVAL objects have been collected on the system.<br>
><br>
> Right now oscap provides 'oscap xccdf eval .. --oval-results' combination<br>
> alllowing to inspect, why the rule as a whole did not pass (e.g. wrong<br>
> "check" / "check_existence" attributes etc.).<br>
><br>
> But it would be good if the new tool would support XCCDF debugging too<br>
> (something like 'oscap xccdf eval ... --oval-results' is allowing now,<br>
> but enhanced).<br>
<br>
</div></div>I am working now on enhancing the debug infos and introduce "--verbose" option<br>
in oscap tool. I think it is a good first step.<br>
<div><div class="h5"><br>
><br>
><br>
> > * How do you usually solve these problems?<br>
><br>
> See above. By separating the rule intended to be debugged. Then running:<br>
> * oscap oval eval --id<br>
> * oscap oval collect --id<br>
> * oscap xccdf eval .. --oval-results<br>
><br>
> combination of commands (depending on the level [e.g. if OVAL itself,<br>
> or at XCCDF level] the OVAL check is failing).<br>
><br>
> > * Can you tell me an example when did you run into issues with OVAL?<br>
><br>
> Basically any a bit more complex OVAL check needs to be designed in<br>
> stages (from top to bottom). Consider OVAL criterion requiring presence<br>
> just some OVAL object, this object to consist of multiple stages, e.g.:<br>
><br>
> * you first collect something on the system into object, save the<br>
> subexpression of it into <local_variable>,<br>
> * apply couple of functions to that variable, finally create new final<br>
> object from the last variable. This final object is then used in the<br>
> OVAL test.<br>
><br>
> Right now it needs to be done from bottom to the top (at each level, at<br>
> each <local_variable> you mostly need to verify it returns appropriate<br>
> results). Other issue is OVAL standard reporting functionality. AFAICT<br>
> the standard requires the rule to return 'error' also in the case the<br>
> particular OVAL rule wasn't able to obtain child results (results for<br>
> the child objects compositing the final object). Having seen this 'error'<br>
> result for the first time, it's not clear if:<br>
> * OVAL check truly has a syntax error,<br>
> * or OVAL check is correct, but the engine wasn't able to determine value<br>
> for some of the OVAL variables above in the evaluation tree.<br>
><br>
> Another issue / RFE in 'oscap oval eval / collect' functionality is the<br>
> '--id' parameter. IIRC as of right now it supports specification of<br>
> an ID just for OVAL objects (therefore having OVAL variable manipulating<br>
> some previously collected OVAL object via some OVAL function, it's<br>
> impossible to see the retrieved values of that variable directly.<br>
><br>
> You need to see the values of the original OVAL object, then create<br>
> another OVAL object from that OVAL variable using the OVAL function,<br>
> to be able to see the true values, that OVAL variable is holding).<br>
><br>
> Not to speak to abstract. Consider the following:<br>
><br>
> <ind:textfilecontent54_object id="my_object" ..><br>
> <ind:filepath>..</ind:filepath><br>
> <ind:pattern ..>..</ind:pattern><br>
> <ind:instance ..>..</ind:instance><br>
> </ind:textfilecontent54_object><br>
><br>
> <local_variable id="my_variable><br>
> <some_oval_function_here, e.g. "split", "concat", whatever><br>
> <object_component object_ref="my_object" /><br>
> ..<br>
> </some_oval_function_here><br>
> </local_variable><br>
><br>
> Right now it's not possible to perform the following:<br>
><br>
> oscap oval collect --id "my_variable" --syschar /tmp/out<br>
><br>
> because "my_variable" doesn't constitute an OVAL object (therefore<br>
> to bypass it, it's necessary to define new artificial OVAL object<br>
> from that "my_variable" local variable, and try to collect it).<br>
><br>
> Therefore it would be very handy if 'collect' could be enhanced<br>
> to support variables too.<br>
<br>
</div></div>This is a nice use-case.<br>
<span class=""><br>
><br>
> > * Imagine an ideal tool for an OVAL developer. What should it be<br>
> > able to do? What would be its features?<br>
><br>
> Maybe above being a sketch. From the above it's not clear if the<br>
> new tool should be just debugger, or profiler too. If the debugger,<br>
> basically it would be able to:<br>
> * report values of particular OVAL objects, variables at any time<br>
> of OVAL evaluation,<br>
> * report the branch OVAL check has been evaluated (like OVAL check<br>
> went through criterion #1, the rest of the criteria weren't taken<br>
> into account, since criterion #1 already succeeded),<br>
<br>
</span>I am going to achieve this by collecting all the possible information<br>
during evaluating the content and processing those information to have<br>
a meaningful output.<br>
<span class=""><br>
> * + the requests as formulated above.<br>
<br>
><br>
> If we would talk also about profiling capabilities - it would be<br>
> able to report things like CPU usage in %, virtual memory usage,<br>
> and or count of system files processed when evaluating particular rule.<br>
<br>
</span>I would consider this, it can be useful feature to avoid bugs with<br>
excessive memory consumption caused by badly written content.<br>
<span class=""><br>
><br>
> This would be handy to optimize current form in many rules.<br>
><br>
> > I would appreciate any suggestions, remarks or other inputs.<br>
> > Thank you very much for your reply.<br>
><br>
> Should you need any other clarification wrt to some of the points<br>
> above, just ask.<br>
<br>
</span>Thank you for your input.<br>
<br>
Jan Černý<br>
<div class="HOEnZb"><div class="h5"><br>
><br>
> Thank you && Regards, Jan<br>
> --<br>
> Jan iankko Lieskovsky / Red Hat Security Technologies Team<br>
><br>
> ><br>
> > Regards<br>
> ><br>
> > Jan Černý<br>
> > Security Technologies | Red Hat, Inc.<br>
> > --<br>
> > SCAP Security Guide mailing list<br>
> > <a href="mailto:scap-security-guide@lists.fedorahosted.org">scap-security-guide@lists.fedorahosted.org</a><br>
> > <a href="https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide" rel="noreferrer" target="_blank">https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide</a><br>
> > <a href="https://github.com/OpenSCAP/scap-security-guide/" rel="noreferrer" target="_blank">https://github.com/OpenSCAP/scap-security-guide/</a><br>
><br>
--<br>
SCAP Security Guide mailing list<br>
<a href="mailto:scap-security-guide@lists.fedorahosted.org">scap-security-guide@lists.fedorahosted.org</a><br>
<a href="https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide" rel="noreferrer" target="_blank">https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide</a><br>
<a href="https://github.com/OpenSCAP/scap-security-guide/" rel="noreferrer" target="_blank">https://github.com/OpenSCAP/scap-security-guide/</a></div></div></blockquote></div><br></div>