Configuration testing vs Forensic testing

Jan Lieskovsky jlieskov at redhat.com
Tue Dec 16 15:18:32 UTC 2014


Hello Shawn, Steve, folks,

  I am going to reopen this thread since (I think)
the previous discussion didn't went to conclusion:
  [1] https://github.com/OpenSCAP/scap-security-guide/issues/320

(agreement on the design concept how this change should be implemented
isn't known yet).

----- Original Message -----
> From: "Shawn Wells" <shawn at redhat.com>
> To: scap-security-guide at lists.fedorahosted.org
> Sent: Friday, October 31, 2014 3:47:37 PM
> Subject: Re: Configuration testing vs Forensic testing
> 
> On 10/31/14, 10:31 AM, Steve Grubb wrote:
> > Hello,
> >
> > I think there is a problem in the SSG content. I think that the current
> > content is intended to check the system configuration. This would be done
> > by
> > examining the files on disk to warn about changes or thing that are
> > misconfigured. There is also another category of testing that is forensics
> > which checks the ephemeral / current values being enforced. Both are
> > necessary
> > and useful, but they should not be mixed.
> >
> > Some examples to illustrate the point:
> >
> > Forensic                      Configuration
> > -----------------------------------------------------------------
> > auditctl -l    vs       cat /etc/audit/audit.rules
> > mount         vs       cat /etc/fstab
> > sysctl -a      vs       cat/etc/sysctl.conf
> > service ip6tables status     vs     chkconfig ip6tables --list
> >
> > All these need to be changed in the prose to better express what the SCAP
> > tool
> > is actually checking. IOW, you can get different results by hand than the
> > tool
> > itself would report.
> 
> Emphatically agree there needs to be better separation. Thanks for
> starting the discussion.
> 
> The auditctl usage in OCIL vs the regex on audit.rules in OVAL is a
> perfect example of this (which was patched earlier this week btw).
> 
> I've opened tickets to track mount vs fstab, sysctl, and service vs
> chkconfig:
> 
> "persistent vs runtime in OVAL+OCIL for mount checks"
> https://github.com/OpenSCAP/scap-security-guide/issues/320
> 
> "persistent vs runtime in OVAL+OCIL for sysctl checks"
> https://github.com/OpenSCAP/scap-security-guide/issues/321
> 
> "persistent vs runtime in OVAL+OCIL for service/chkconfig"
> https://github.com/OpenSCAP/scap-security-guide/issues/322
> 
> As/if you identify additional sections which need better separation,
> please bring them to our attention!

First let me summary that:
* it's great we agreed on the need to separate configuration vs
  runtime checks,
* we identified the areas which needs fixing.

But obvious question being what level of separation is required:
* 1) IOW should each existing rule be turned into a new group, consisting
  of two rules - one for configuration testing, one for runtime testing.
  Then the description of the group would be more generalized form of
  the check, where each of the two new rules would be described according
  to the way they perform the check - IOW the configuration one would mention
  checking configuration files, while the runtime one would focus on system
  actions checking runtime state,
* 2) or is it sufficient to mention in the (HTML version of the guide) that the
  current implementation checks just configuration status (AND the runtime state where
  appropriate) and basically do no changes in current XCCDF / OVAL rules implementation,
* 3) another options / possibility (as pointed out by Simon Lukasik - thanks for it!)
  is the following - modify the current rules implementation in the way to keep
  the configuration tests the default ones (IOW when they don't pass the check would
  fail) and simultaneously make the runtime checks the optional ones. The content
  user would be able via e.g. an OVAL variable to instruct the scanner what kind
  of testing should be performed.

  Example:

       "Check system property" rule
       if ($runtime_check) not set
       then
         check just configuration settings
       else
         check configuration settings
         check runtime settings
       fi

  And analogous approach (same global OVAL variable) customizable by the SCAP content
user would be used for all rules.

* 4) another option (but maybe just enhancement of case 1)) is to follow the way
to have two dedicated profiles for each of the existing ones (e.g. USGCB-configuration and
USGCB-runtime) each of them containing rules from particular category.

* 5) another option is to use "runtime / configuration" (or both of them) as suffix in the
     rule title - so for example:
     -- existing "Install Aide" rule would become "Install Aide (runtime)"

     meaning here just runtime check would be performed, while e.g.

     -- existing "Disable the Automounter" would become "Disable the Automounter (configuration, runtime)"

     meaning in this case both configuration & runtime checks were performed.

In my opinion we first need to agree on the way how the separation should be
performed in order to:
* this separation to be sufficiently clear enough for the content consumers
* we don't need to change the approach during its implementation (during
  updating actual state to reflect the expectations)

Should I vote for some of the aforementioned approaches to select one I
prefer the global OVAL variable approach. E.g. the following:

* Update existing XCCDF rules description to mention / describe only configuration
  checks,
* Update OVAL checks to perform just configuration testing by default,
* Add new global OVAL variable ($runtime_check for example) to instruct the
  content, that in case this variable is defined, the optional runtime checks
  should be performed too.

Yet another possibility - taking the variable approach even to yet more /
wider extent, there could be two of them:
* $configuration
* $runtime

and the content would perform checks (and maybe also remediations / corrections)
based on the actual user setting / preference. E.g. in case $configuration
would be defined, it would perform just system configuration checking & configuration
corrections. With $runtime defined it would check & correct (just) runtime state.

The default setting then would be to use (check & correct) the $configuration setting(s).

Comments, opinions, feedback welcome.

Thank you && Regards, Jan.
--
Jan iankko Lieskovsky / Red Hat Security Technologies Team

> 
> > This really needs to be addressed before anyone else uses SSG as the basis
> > of
> > their own recommendations. Again, forensic checking is useful and I would
> > say
> > content should be specifically designed with that in mind. But it is not
> > what
> > should be in a baseline.
> That's a bit strong of language. SSG represents a catalog of controls,
> from which agencies make selections for formal baselines that we turn
> into profiles. Often (e.g. with the STIG) the agency wishes to include
> capabilities for static/persistent configuration (e.g. sysctl.conf)
> *and* ephemeral system state (sysctl -a).
> --
> SCAP Security Guide mailing list
> scap-security-guide at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
> https://github.com/OpenSCAP/scap-security-guide/


More information about the scap-security-guide mailing list