New report and guide in openscap 1.1.0

Martin Preisler mpreisle at redhat.com
Tue Sep 2 16:22:13 UTC 2014


----- Original Message -----
> From: "Greg Elin" <gregelin at gitmachines.com>
> To: "SCAP Security Guide" <scap-security-guide at lists.fedorahosted.org>
> Sent: Tuesday, September 2, 2014 5:21:23 PM
> Subject: Re: New report and guide in openscap 1.1.0
> 
> On Mon, Sep 1, 2014 at 5:32 AM, Martin Preisler <mpreisle at redhat.com> wrote:
> 
> > ----- Original Message -----
> > > From: "Greg Elin" <gregelin at gitmachines.com>
> > > To: "SCAP Security Guide" <scap-security-guide at lists.fedorahosted.org>
> > > Sent: Friday, August 29, 2014 11:18:37 PM
> > > Subject: Re: New report and guide in openscap 1.1.0
> > >
> > > I agree with Trey. The details would be more useful if they contained
> > more
> > > information about how something failed. I would expect that in a detail.
> >
> > As I said in my previous email, we show these details. We always have!
> >
> > They were not present in one of the previous ! draft ! versions of
> > XCCDF report. When I released that draft I explicitly said that check
> > details
> > are not there but will be there in the final version. Are you sure you are
> > looking at the new report?
> >
> 
> Thanks for the prodding. I looked more closely last night. At least two of
> us had some confusion with this.

Many more people are confused by this AFAIK :-)

> AFAIK, Oval results only show up if the `--oval-results` is added to `oscap
> xccdf eval` command. It seems to me specific errors should be shared by
> default and flag used to hide them. Yes, it is in the man page and of
> course we all read the man page... (But I guess that is a comment for the
> OpenSCAP List?)

Yes, that is the case. We are planning to fix this with Result DataStreams
at some point in the future but it's not in git yet.

> Oval details show up on only some failed tests in the sample report (
> https://mpreisle.fedorapeople.org/openscap/1.1.0_xslt/report.html)
> "package_aide_installed" for example does not show any oval details though
> it fails while "service_atd_disabled" does show oval results.

Our OVAL details implementation supports a subset of all possible OVAL
objects. I think we support most of the use-cases where you really need to
know which item is the culprit.

We may support more at some point in the future. Patches are always welcome.

> [snip]
> 
> It seems OVAL details show up when there are specific "items violating."
> That is _incredibly_ helpful for a test like "rpm_verify_permissions". It's
> awesome.
> 
> It seems OVAL details do not show up if the control just fails. That is
> confusing. Didn't an OVAL criterion fail? Which one? Maybe there is more
> than one criterion for a given test? The fact sometimes OVAL details show
> up and sometimes they don't seems like yet another secret, undocumented
> detail that is obvious to those in the know, but confusing to those that
> aren't.  At the very least, there should always be an explanation of the
> OVAL detail even if it is "OVAL failed without any detail. Here is the
> criterion that failed: "

When we don't show anything in OVAL details it most likely means we don't
support details for that particular OVAL object.

> > > I'd be OK with default report ONLY showing details for the severity=high
> > > items that I failed and all other details went either into a second
> > report
> >
> > We show check system details for all failed rules, always. This works for
> > both OVAL and SCE checks.
> >
> 
> What I was trying to get at is that I think fails that are high severity
> should be treated differently in the report than those of medium or high.
> For example, they could their own section, or -- as I was musing -- be the
> only fails included in the report.

I wanted to implement some sort of a "remediation priority list" that the
report would generate for users on demand. You could print it and go through
it in order. However I didn't have enough time to implement this in the end.

If there is customer demand it may happen in the near future.

> [snip]
>
> > Are you sure you want detailed reports of vulnerabilities on your
> > infrastructure to end up on the "WEB"? One of our goals is to keep the
> > reports and guides self sufficient. We do not want to rely on remote tools.
> > That's why we bundle even all the JavaScript and CSS.
> >
> >
> No, I do not want results of what controls *my* system passed or failed
> posted on the web.
> 
> I am speaking about the generic content that is a part of each control
> being on the web:
>  rationale, links, related identifiers, remediation scripts, etc.
> 
> The most important information in the results of my scan, are the details
> about my system and my scan results. Generic information is _generic_. I
> only want that generic information co-located with details when it
> specifically helps me take an action.

This is a general feature/issue with XCCDF and/or Source/Result DataStreams.
Sorry but we can't fix this in report without making proprietary extensions.

> Maybe what I am driving at as a general theme here is the difference
> between an "Action/Working Report" optimized to help me get to work
> resolving issues and "Complete Report" that represents an artifact
> capturing a state that is less transitory.
> 
> I'll try to mockup what I mean.

Maybe consider patching the XSLT directly. I have restructured it and it's
much easier to follow how it works now.

> [snip]
> 
> By static details I am referring to information in the SSG that does not
> change from system to system or from scan result to scan result.
> 
> I feel I should say a bit more about this because the above may seem
> contradictory.
> 
> It might seem I am one hand suggesting more information be added to the
> report to make the report easier for beginners (e.g., include OVAL detail
> on all failed tests).
> 
> On the other hand, it might seem I am suggesting less information be
> included in the report (e.g., removing generic static details, or putting
> medium and low control fails details in a second report).
> 
> I am arguing for clarity and less cognitively taxing presentation that does
> way better than most other scan reports out there.
> 
> Only some fails having an OVAL detail section seems ambiguous. Meanwhile
> including all explanatory detail for every item often seems extraneous.
> 
> If I knew the perfect algorithm, I would share it. I do know what I want to
> read most of all in the report is :  "The control ____ failed because
> __________" and be able to follow a link to get more detail if I happen to
> need it.

OK, this clarification helped a lot. The only superfluous detail I know about
is rule description. And we only show this in the modal dialog. Titles, IDs,
references, identifiers, remediation fixes, severity, ... were all
feature requests.

> I think I really want a simple report that lists what failed and why, and
> then a second report (or hidden second half of the report) that has way
> more details.

Having multiple reports adds maintenance costs. But maybe we can hide this
additional information under a [+] button or something like that.

-- 
Martin Preisler


More information about the scap-security-guide mailing list