Martin,

Thank you for the work on cleaner, modern, interactive report.

The three feature improvements that are most valuable are:
   1. Including the priority of a control in the summary listing b/c it saves a lot of time looking things up. I needed this.
   2. The interactive filtering b/c it also saves a lot of time b/c it allows me to use the report in planning and to have a conversation. I needed this.
   3. Modern look and pop-up details. Also saves me time b/c it keeps my place while I look deeper (less cumbersome). I will use this.

Least needed features:
   1. I'm less sure if I currently need the collapsing groups feature. I've been playing with it. On the plus side, the grouping has utility and it's particularly nice to collapse things to less than a handful of summary groups. It is also a big nerd knob whose presence is felt in everything I do in the detail section. The filters provide me with optional controls; the collapsing sections I have to actively manage. Even fully expanded, I am cognitively managing the groups presence as I scan down and look at items. I have to coordinate the filters and the grouping collapse/expansion. It's a powerful feature, and I might like it at times, but I also might find it cumbersome and bothersome in the future, too. I'd almost rather try the report a while with just the filters and other improvements first. (Or maybe make it optional...)
  2. I keep wondering if the detail section should even be part of the report, or should be online by default. Does it make sense to include by default static information? I suppose running the report on a system on segregated network would be one reason -- but is that the most common scenario?  Why have the detail information in the same report vs a second-related report?

Features I still want:
  1. I still want an even higher level executive summary that begins to tie the details of the scan to actual system and organizational risk. I want a report that creates shared awareness between Dev, Ops, Sec, and Mission. 
  2. Diff'ing: Comparing result with previous report. If I am using the repot to lock down a system, I want to easily see what's changed since the previous scan.
  3. Inclusion of comments and waivers. If I am consciously ignoring a test that is in the profile, it would be helpful to see and share that in the report (and see the result therefore as 'passed*' or 'pass waiver". 
  4. Time estimates and prioritization. How long will it take me to fix different items? Which ones have automated fixes? 
  

Greg Elin
http://govready.org - Making FISMA compliance easier for innovators

email: gregelin@gitmachines.com
phone: 917-304-3488



On Fri, Aug 29, 2014 at 9:28 AM, Gabe Alford <redhatrises@gmail.com> wrote:
On Fri, Aug 29, 2014 at 3:37 AM, Martin Preisler <mpreisle@redhat.com> wrote:
----- Original Message -----
> From: "Andrew Gilmore" <agilmore2@gmail.com>
> To: "SCAP Security Guide" <scap-security-guide@lists.fedorahosted.org>
> Sent: Thursday, August 28, 2014 8:29:48 PM
> Subject: Re: New report and guide in openscap 1.1.0
>
> I like the new look and functionality.
>
> Two first blush comments:
> 1) On the report document, I can imagine my security officials freaking out
> over the in-your-face "*The system is not compliant!*" text. What is the
> recommended course to ensure this text does not appear if you're running
> the scan on a webserver, for example? Is it as simple as creating a custom
> profile derived from the STIG profile? Does anyone directly use the STIG
> profile, have a completely compliant system, and have a server that
> actually does anything useful?
> Up to now, I've left tests in that I have waivers for, and then pointed at
> the waivers to justify the test failures. Perhaps I will need to change
> that practice.

Isn't that a good thing? They should freak out, their system is not compliant!
The recommended course is to tailor the profile, leaving out rules that make
no sense on your system. Then you fix the remaining rules using remediation.
In the end the machine will be compliant.

I would maybe add or modify the message here to be something along the lines:

- "The system is not compliant! Please review rule results, site/network security requirements, and consider applying remediation."

--- or ---

 - "The system may not be compliant! Please review rule results, site/network security requirements, and consider applying remediation."

I personally would prefer the last one as it says, "Hey. Check your system as well as check your security requirements to see if what you are seeing from the scan matches with those security requirements."


The job of openscap is to check your machines for compliance over and over.
When the machines are suddenly not compliant you really want to know that!

> 2) On the guide document, the text beginning "Providing system
> administrators" occurs twice.

Looks like an issue with SSG but I will look more into it.


--
SCAP Security Guide mailing list
scap-security-guide@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
https://github.com/OpenSCAP/scap-security-guide/