Hello Trey,
we are talking about two different things, and those are not mutually exclusive.
Some subjects see the projects as a gigantic jigsaw puzzle, where individual pieces are rules, and they can use those to compose custom profiles - I understood that this is your case as well. For those, everything else than rules and associated content doesn't really matter, and improvements to how rules are defined may become quite disruptive.
However, a huge number of subjects would like to use the high-level aspect of the project - profiles in datastreams that you can feed to the scanner right away, and that will scan your system. We get a huge number of requests for profiles, and we try to make things easier to profile authors, so they can start contributing to ComplianceAsCode, and we can focus on rules and on the testing infrastructure.
I disagree that we introduce new features just for the sake of creativity, and again, the announcement that started this discussion was about a backwards-compatible improvement that profile authors are on board with. In other words, this won't break anything, so you can relax about this.
I am really curious though what is it that broke your builds. Could you please share some details? We may be able to help, or to at least do a better job in this regard next time.
Best regards, Matej
On 08. 06. 20 16:07, Trey Henefield wrote:
Hi Matej,
I see what you’re saying.
It does improve readability.
Ultimately, automation is king. I had some code in the project once upon a time that would improve visibility of coverage of the requirements. Pulling the data from the source requirements (i.e. STIG), then aligning that data with the STIG overlay to identify total coverage of the STIG, then aligning that data with the mapped OVAL rules to identify overall coverage of requirements by the checks and remediations included. This probably could have been extended to compare the rules in the STIG profile with the rules in the STIG overlay to identify rules in the overlay not captured in the profile and rules in the profile that don’t align to a requirement in the STIG overlay.
I still respectfully disagree with you.
The goal of this project is to provide content that facilitates helping organizations meet requirements. From the consumer side of this, this project is pretty pointless if it only gets you a fraction of the way there. To make this project useful, it needs to be more complete and accurate. Having it build better or include a better way showing what few requirements are addressed doesn’t provide any value add if it still doesn’t increase the percentage of coverage of requirements to be validated and mitigated.
You can change the frame of a picture, but it doesn’t change the picture.
I just want to give you the perspective from a user of this content. I can’t use the content as it’s provided. I have to go in and address all the issues and add code to get compliant for our systems. But for the many other users out there that don’t have the luxury of understanding how to deal with updating the code, they end up with a solution that gets them compliant after days\hours of manual changes to fix the issues it causes and make up for the lack of coverage where it doesn’t.
If the goal of this project is to be a pet project for developers to try new things and be creative, then I can understand your current direction.
If the goal of this project is to help users in meeting industry standards with applicable software instaled, then the direction should be to provide complete and accurate checks and remedations. This is what end users really need and would value most from this project.
Best regards,
*Trey Henefield, CISSP*
Cyber Security Manager
UltraIntelligence & Communications
4101 Smith School Road
Building IV, Suite 100
Austin, TX 78744 USA T:+1 512 327 6795 ext. 647
M:+1 512 541 6450 ultra.group http://www.ultra.group
*From:* Matej Tyc matyc@redhat.com *Sent:* Monday, June 8, 2020 4:52 AM *To:* scap-security-guide@lists.fedorahosted.org *Subject:* Re: Policy source data format proposal is ready for comments
Hello Trey,
the change that is discussed here is a backwards-compatible one, so it wouldn't break anything for you.
I understand your pain, but as the repository increases in size both in rule count and profile count, actions need to be taken so the project retains its level of maintainability. For example, the recent redesign of templates was quite disruptive, but it also fixed tens of rules that nobody had capacity to fix before.
This change aims to introduce possibility to reuse chunks of rules between profiles, and to make profile definitions easier to read and to maintain.
How would you rate maintainability of https://github.com/ComplianceAsCode/content/blob/master/rhel7/profiles/ncp.p... and of https://github.com/ComplianceAsCode/content/blob/master/rhel8/profiles/ospp.... ? Both profiles select over hundred of rules, and I am sure that the RHEL7 profile would make it very difficult for anybody to determine whether a rule is missing, or the profile is complete. The RHEL8 profile has comments that make things more clear, and the proposal we are discussing in this thread basically moves things one step further, from optional comments to a schema. It will still be possible to do things the old way though.
I am sure that all contributors feel disappointed when contributions are not accepted, and trust me, it happens to everybody. I would recommend you to have a fork with all rules that were rejected upstream added.
You say that the immediate focus should be content completeness and accuracy. I think that this should be the long-term focus. When you have incorrect copy-pasted code, the immediate thing to do is not fixing it again by more copy-pasting, but one should rather remove the copy-pasting, if it is possible. Unfortunately, this is what breaks the backwards compatibility, but it is always for a good reason.
If you have a particular backward-compatibility problem, please tell us, and we may write a blog post about it, so porting your old code will be fast and efficient: https://complianceascode.github.io https://complianceascode.github.io
Best regards, Matej Tyc
On 05. 06. 20 20:25, Trey Henefield wrote:
This won't solve the problems with your content. The problem is that there has been pushback to accept community suggestions due to personal perferences. There is much more focus on continuously changing the structure of the project, than the content of the project. The ultimate goal for this project should be to provide checks and remediations that accurately reflect (no more and no less) what is required by a particular regulation. Regaurdless of what default behaviors are present in the RedHat operating system, if a configuration is required to be explicitly configured, it should be configured. So that the intent of what the requirement is explicitly requiring is addressed. Saying that this does not apply because it does this already, does not hold well when this content is used and this discrepency has to be explained to validators. Our organization has basically gave up contributing and maintain our own personal branch to ensure we provide solutions that meet the needed requirements. To greatly improve this project, the immediate focus needs to be on content completeness and accuracy. This would provide the best value this project could offer to the community. The structure of this project and how to better improve it should be a secondary focus, with these structural changes better thought out and implemented, before being integrated into the project. These major changes should be introduced less frequently (1 or 2 times a year) to allow contributors time to complete their changes. Just about every two months when a new release is pushed, we have to redo allot of stuff we did to get our changes working in the new release. This is very time consuming and difficult to maintain. I truly hope someone there at RedHat is finally listening to this and takes our advice. Best regards, Trey Henefield, CISSP Cyber Security Manager Ultra Intelligence & Communications 4101 Smith School Road Building IV, Suite 100 Austin, TX 78744 USA T: +1 512 327 6795 ext. 647 M: +1 512 541 6450 ultra.group <http://ultra.group> -----Original Message----- From: Jan Cerny <jcerny@redhat.com> <mailto:jcerny@redhat.com> Sent: Friday, June 5, 2020 9:36 AM To: SCAP Security Guide <scap-security-guide@lists.fedorahosted.org> <mailto:scap-security-guide@lists.fedorahosted.org> Subject: Policy source data format proposal is ready for comments Hi, The policy source data format proposal is available and ready for comments. The text has been submitted as a pull request on GitHub to make the discussion easier using comments and reviews. See https://github.com/ComplianceAsCode/content/pull/5817 <https://github.com/ComplianceAsCode/content/pull/5817> We are looking forward to seeing your feedback on GitHub. What is it about? We will use the policy source data format to improve development of our profiles. It will allow us to store security controls and requirements in the repository and then define profiles by using their IDs instead of separate rules. This is done in order to solve the problem that there is no easy way to demonstrate to profile stakeholders the status of their profile. Intended workflow: * SME identifies security controls the policy consists of. Those controls serve as direct input for our profiles. * SME goes through controls, and makes sure that they are sufficiently covered by rules. * SME fine-tunes the profile by overriding a couple of individual rules in the profile file. Once the format is accepted we can start developing tools that support this new workflow. In future, we can also use it for further refactoring, for example streamlining the generation of HTML tables. Best regards -- Jan Černý Security Technologies | Red Hat, Inc. _______________________________________________ scap-security-guide mailing list -- scap-security-guide@lists.fedorahosted.org <mailto:scap-security-guide@lists.fedorahosted.org> To unsubscribe send an email to scap-security-guide-leave@lists.fedorahosted.org <mailto:scap-security-guide-leave@lists.fedorahosted.org> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines <https://fedoraproject.org/wiki/Mailing_list_guidelines> List Archives: https://lists.fedorahosted.org/archives/list/scap-security-guide@lists.fedorahosted.org *Disclaimer* The information contained in this communication from *trey.henefield@ultra-ats.com <mailto:trey.henefield@ultra-ats.com> *sent at 2020-06-05 14:25:25 is private and may be legally privileged or export controlled. It is intended solely for use by *scap-security-guide@lists.fedorahosted.org <mailto:scap-security-guide@lists.fedorahosted.org> *and others authorized to receive it. If you are not *scap-security-guide@lists.fedorahosted.org <mailto:scap-security-guide@lists.fedorahosted.org> *you are hereby notified that any disclosure, copying, distribution or taking action in reliance of the contents of this information is strictly prohibited and may be unlawful. _______________________________________________ scap-security-guide mailing list --scap-security-guide@lists.fedorahosted.org <mailto:scap-security-guide@lists.fedorahosted.org> To unsubscribe send an email toscap-security-guide-leave@lists.fedorahosted.org <mailto:scap-security-guide-leave@lists.fedorahosted.org> Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedorahosted.org/archives/list/scap-security-guide@lists.fedorahosted.org
scap-security-guide mailing list -- scap-security-guide@lists.fedorahosted.org To unsubscribe send an email to scap-security-guide-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/scap-security-guide@lists.fedor...
Hi Matej,
So to put things in perspective, I am still on the 0.1.48 release. I just now got it to a point to where I have addressed as many requirements as my system will allow. I have roughly just under 400 checkins.
This is how far behind the baseline is. I am scared to even look at 0.1.50 as I will have to do this all over again with any new challenges that release brings me. Keeping in mind, this is just one small aspect of many other things I am involved with.
So to give some examples.
The implementation of jinja was interesting. I spent hours trying to figure out why some remediations and checks would not run even though the syntax looked correct. Only to discover that some files had the jinja syntax defined differently causing certain lines not being processed. It would have been nice if this code had been properly tested for correctness before being merged into the baseline.
The audit rules have consistent inaccuracies.
It seems like someone thought it was a good idea to replace every instance of ‘auid!=4294967295’ with ‘auid!=unset’. While in theory, this may accomplish the same goal. But in practice, the STIG and its associated benchmark explicitly look for a specific value, which it no longer finds and is considered a finding. This results in allot of findings.
I also noticed that in support of the Suse Enterprise Linux STIG, all audit rules for privileged commands include ‘-perm x’, which creates a finding for the RHEL7 STIG because it looks for the rules to exist without defining the execution bit as a filter.
Java and Firefox are a bit outdated.
Jinja bash macro for firefox is pointing to the wrong file locations and has a syntax error.
SELinux syntax is incorrect for the remediations.
SSH ciphers and MACs OVAL check and remediation are wrong.
Screenlock keybinding OVAL check is incorrect.
RHEL6 was not included as a platform for account_umask_etc_profile.
set_password_hashing_algorithm_systemauth does not also check password-auth and also does not check for non-compliant values.
No password complexity checks/remediations exist.
The checks and remediations for disable_ctrlaltdel_reboot do not align with the RHEL6/7 STIGs.
Login banner text is not defined correctly.
The list goes on and on. I don’t have enough time to list everything. But luckily I have fixes for all of these things, among other things.
Does anyone on this project do any validation of the content against the requirements to ensure accurate implementation of the required configurations? Because if they did, they would see there is allot of work to get this content into a usable form.
I would highly advise someone there at Red Hat apply this content to a RHEL6 or RHEL7 system. Then run a scan using the latest official DISA benchmark. Then perform manual checks on the system using the latest version of the DISA STIG and the STG Viewer. You will then get an understanding of what I am seeing along with every other Red Hat customer using this content in an effort to try and be compliant.
Again, I would love to contribute my changes to help get this project back on track, but with the release of major changes to the structure and process every two months, its just impossible.
Best regards,
Trey Henefield, CISSP Cyber Security Manager
Ultra Intelligence & Communications 4101 Smith School Road Building IV, Suite 100 Austin, TX 78744 USA T: +1 512 327 6795 ext. 647 M: +1 512 541 6450 ultra.grouphttp://www.ultra.group
From: Matej Tyc matyc@redhat.com Sent: Wednesday, June 10, 2020 4:50 AM To: scap-security-guide@lists.fedorahosted.org Subject: Re: Policy source data format proposal is ready for comments
Hello Trey,
we are talking about two different things, and those are not mutually exclusive.
Some subjects see the projects as a gigantic jigsaw puzzle, where individual pieces are rules, and they can use those to compose custom profiles - I understood that this is your case as well. For those, everything else than rules and associated content doesn't really matter, and improvements to how rules are defined may become quite disruptive.
However, a huge number of subjects would like to use the high-level aspect of the project - profiles in datastreams that you can feed to the scanner right away, and that will scan your system. We get a huge number of requests for profiles, and we try to make things easier to profile authors, so they can start contributing to ComplianceAsCode, and we can focus on rules and on the testing infrastructure.
I disagree that we introduce new features just for the sake of creativity, and again, the announcement that started this discussion was about a backwards-compatible improvement that profile authors are on board with. In other words, this won't break anything, so you can relax about this.
I am really curious though what is it that broke your builds. Could you please share some details? We may be able to help, or to at least do a better job in this regard next time.
Best regards, Matej On 08. 06. 20 16:07, Trey Henefield wrote: Hi Matej,
I see what you’re saying.
It does improve readability.
Ultimately, automation is king. I had some code in the project once upon a time that would improve visibility of coverage of the requirements. Pulling the data from the source requirements (i.e. STIG), then aligning that data with the STIG overlay to identify total coverage of the STIG, then aligning that data with the mapped OVAL rules to identify overall coverage of requirements by the checks and remediations included. This probably could have been extended to compare the rules in the STIG profile with the rules in the STIG overlay to identify rules in the overlay not captured in the profile and rules in the profile that don’t align to a requirement in the STIG overlay.
I still respectfully disagree with you.
The goal of this project is to provide content that facilitates helping organizations meet requirements. From the consumer side of this, this project is pretty pointless if it only gets you a fraction of the way there. To make this project useful, it needs to be more complete and accurate. Having it build better or include a better way showing what few requirements are addressed doesn’t provide any value add if it still doesn’t increase the percentage of coverage of requirements to be validated and mitigated.
You can change the frame of a picture, but it doesn’t change the picture.
I just want to give you the perspective from a user of this content. I can’t use the content as it’s provided. I have to go in and address all the issues and add code to get compliant for our systems. But for the many other users out there that don’t have the luxury of understanding how to deal with updating the code, they end up with a solution that gets them compliant after days\hours of manual changes to fix the issues it causes and make up for the lack of coverage where it doesn’t.
If the goal of this project is to be a pet project for developers to try new things and be creative, then I can understand your current direction.
If the goal of this project is to help users in meeting industry standards with applicable software instaled, then the direction should be to provide complete and accurate checks and remedations. This is what end users really need and would value most from this project.
Best regards,
Trey Henefield, CISSP Cyber Security Manager
Ultra Intelligence & Communications 4101 Smith School Road Building IV, Suite 100 Austin, TX 78744 USA T: +1 512 327 6795 ext. 647 M: +1 512 541 6450 ultra.grouphttp://www.ultra.group
From: Matej Tyc matyc@redhat.commailto:matyc@redhat.com Sent: Monday, June 8, 2020 4:52 AM To: scap-security-guide@lists.fedorahosted.orgmailto:scap-security-guide@lists.fedorahosted.org Subject: Re: Policy source data format proposal is ready for comments
Hello Trey,
the change that is discussed here is a backwards-compatible one, so it wouldn't break anything for you.
I understand your pain, but as the repository increases in size both in rule count and profile count, actions need to be taken so the project retains its level of maintainability. For example, the recent redesign of templates was quite disruptive, but it also fixed tens of rules that nobody had capacity to fix before.
This change aims to introduce possibility to reuse chunks of rules between profiles, and to make profile definitions easier to read and to maintain.
How would you rate maintainability of https://github.com/ComplianceAsCode/content/blob/master/rhel7/profiles/ncp.p...https://github.com/ComplianceAsCode/content/blob/master/rhel7/profiles/ncp.profile and of https://github.com/ComplianceAsCode/content/blob/master/rhel8/profiles/ospp....https://github.com/ComplianceAsCode/content/blob/master/rhel8/profiles/ospp.profile ? Both profiles select over hundred of rules, and I am sure that the RHEL7 profile would make it very difficult for anybody to determine whether a rule is missing, or the profile is complete. The RHEL8 profile has comments that make things more clear, and the proposal we are discussing in this thread basically moves things one step further, from optional comments to a schema. It will still be possible to do things the old way though.
I am sure that all contributors feel disappointed when contributions are not accepted, and trust me, it happens to everybody. I would recommend you to have a fork with all rules that were rejected upstream added.
You say that the immediate focus should be content completeness and accuracy. I think that this should be the long-term focus. When you have incorrect copy-pasted code, the immediate thing to do is not fixing it again by more copy-pasting, but one should rather remove the copy-pasting, if it is possible. Unfortunately, this is what breaks the backwards compatibility, but it is always for a good reason.
If you have a particular backward-compatibility problem, please tell us, and we may write a blog post about it, so porting your old code will be fast and efficient: https://complianceascode.github.iohttps://complianceascode.github.io
Best regards, Matej Tyc On 05. 06. 20 20:25, Trey Henefield wrote:
This won't solve the problems with your content.
The problem is that there has been pushback to accept community suggestions due to personal perferences. There is much more focus on continuously changing the structure of the project, than the content of the project.
The ultimate goal for this project should be to provide checks and remediations that accurately reflect (no more and no less) what is required by a particular regulation.
Regaurdless of what default behaviors are present in the RedHat operating system, if a configuration is required to be explicitly configured, it should be configured. So that the intent of what the requirement is explicitly requiring is addressed. Saying that this does not apply because it does this already, does not hold well when this content is used and this discrepency has to be explained to validators.
Our organization has basically gave up contributing and maintain our own personal branch to ensure we provide solutions that meet the needed requirements.
To greatly improve this project, the immediate focus needs to be on content completeness and accuracy. This would provide the best value this project could offer to the community.
The structure of this project and how to better improve it should be a secondary focus, with these structural changes better thought out and implemented, before being integrated into the project. These major changes should be introduced less frequently (1 or 2 times a year) to allow contributors time to complete their changes. Just about every two months when a new release is pushed, we have to redo allot of stuff we did to get our changes working in the new release. This is very time consuming and difficult to maintain.
I truly hope someone there at RedHat is finally listening to this and takes our advice.
Best regards,
Trey Henefield, CISSP Cyber Security Manager Ultra Intelligence & Communications 4101 Smith School Road Building IV, Suite 100 Austin, TX 78744 USA T: +1 512 327 6795 ext. 647 M: +1 512 541 6450 ultra.grouphttp://ultra.group
-----Original Message----- From: Jan Cerny jcerny@redhat.commailto:jcerny@redhat.com Sent: Friday, June 5, 2020 9:36 AM To: SCAP Security Guide scap-security-guide@lists.fedorahosted.orgmailto:scap-security-guide@lists.fedorahosted.org Subject: Policy source data format proposal is ready for comments
Hi,
The policy source data format proposal is available and ready for comments. The text has been submitted as a pull request on GitHub to make the discussion easier using comments and reviews. See https://github.com/ComplianceAsCode/content/pull/5817https://github.com/ComplianceAsCode/content/pull/5817 We are looking forward to seeing your feedback on GitHub.
What is it about? We will use the policy source data format to improve development of our profiles. It will allow us to store security controls and requirements in the repository and then define profiles by using their IDs instead of separate rules.
This is done in order to solve the problem that there is no easy way to demonstrate to profile stakeholders the status of their profile.
Intended workflow:
* SME identifies security controls the policy consists of. Those controls serve as direct input for our profiles. * SME goes through controls, and makes sure that they are sufficiently covered by rules. * SME fine-tunes the profile by overriding a couple of individual rules in the profile file.
Once the format is accepted we can start developing tools that support this new workflow.
In future, we can also use it for further refactoring, for example streamlining the generation of HTML tables.
Best regards
-- Jan Černý Security Technologies | Red Hat, Inc. _______________________________________________ scap-security-guide mailing list -- scap-security-guide@lists.fedorahosted.orgmailto:scap-security-guide@lists.fedorahosted.org To unsubscribe send an email to scap-security-guide-leave@lists.fedorahosted.orgmailto:scap-security-guide-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conducthttps://docs.fedoraproject.org/en-US/project/code-of-conduct List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelineshttps://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/scap-security-guide@lists.fedor...https://lists.fedorahosted.org/archives/list/scap-security-guide@lists.fedorahosted.org
Disclaimer The information contained in this communication from trey.henefield@ultra-ats.commailto:trey.henefield@ultra-ats.com sent at 2020-06-05 14:25:25 is private and may be legally privileged or export controlled. It is intended solely for use by scap-security-guide@lists.fedorahosted.orgmailto:scap-security-guide@lists.fedorahosted.org and others authorized to receive it. If you are not scap-security-guide@lists.fedorahosted.orgmailto:scap-security-guide@lists.fedorahosted.org you are hereby notified that any disclosure, copying, distribution or taking action in reliance of the contents of this information is strictly prohibited and may be unlawful.
_______________________________________________
scap-security-guide mailing list -- scap-security-guide@lists.fedorahosted.orgmailto:scap-security-guide@lists.fedorahosted.org
To unsubscribe send an email to scap-security-guide-leave@lists.fedorahosted.orgmailto:scap-security-guide-leave@lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelineshttps://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorahosted.org/archives/list/scap-security-guide@lists.fedor...https://lists.fedorahosted.org/archives/list/scap-security-guide@lists.fedorahosted.org
_______________________________________________
scap-security-guide mailing list -- scap-security-guide@lists.fedorahosted.orgmailto:scap-security-guide@lists.fedorahosted.org
To unsubscribe send an email to scap-security-guide-leave@lists.fedorahosted.orgmailto:scap-security-guide-leave@lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelineshttps://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorahosted.org/archives/list/scap-security-guide@lists.fedor...https://lists.fedorahosted.org/archives/list/scap-security-guide@lists.fedorahosted.org
scap-security-guide@lists.fedorahosted.org
-
Matej Tyc -
Trey Henefield