Ok! Now we're getting somewhere good, and thanks for continuing the discussion, I'm finding it to be very valuable.

On Tue, Nov 14, 2017 at 4:52 PM, Steve Grubb <sgrubb@redhat.com> wrote:
On Tuesday, November 14, 2017 4:01:50 PM EST Trevor Vaughan wrote:
> Hi Steve,
>
> Your statement "The ability to modify the repo metadata is the same
> capability as creating the package in the first place." is actually the core
> of the issue that I have.

And there simply is no getting around it. The metadata is a proxy for the rpm
information that allows yum to decide what is in scope to download. Trusting
TLS gives less assurance that the package resolution occurred using
trustworthy data.

But, does that matter if the packages themselves will not install if not signed by a trustworthy key? This is what I'm getting at.
 


> I push up packages with a *package signing key* that should not be the same
> as the *repo signing key*.

Why? You can do more harm with the rpm info than the repo metadata.


Yes! I want *less* trust on the repo metadata. Heck, I don't want to have to trust the repo metadata at all because I've restricted the GPG keys for my packages to only be the specific vendor keys that I want. Not being able to split the trust between these two doesn't let me isolate my trust effectively.
 


> They should be different keys since one is the repo provider's domain and
> one is the package provider's domain.

They should be considered one in the same since the repo is a proxy for the
rpm info. If you let a repo provider sign metadata, then the package provider
has no way to let the end user know they received the right dependencies based
on the package signed by the package provider. The repo provider in the yum
model is untrusted. Its like the two generals problem. A message has to get
from one general to the other but the message has to go through hostile
territory.

Maybe there is a mismatch in the trust model where you want the repo provider
to be ultimately trusted and the packager a second class citizen? If that is
the case, then you can sign the repo metadata with your own key, but you'll
have to distribute the public portion to end users. But I don't know what's
been gained since the package developer has to assume nothing nefarious is
happening in the repo provider's scripting. And the end user now has to worry
that the repo provider won't try adding some packages to the repo with his/her
key.

This is the issue, but in reverse! I trust the *packager* because they are the ones with ultimate power on my system. I may, or may not, fully trust the repo provider. This means that I can keep my package GPG key list as restricted as plausible and I want to only trust the repo provider for the repo metadata.
 

But even so, using yum-plugin-priorities you can be certain this 3rd party
repo is last so that they cannot provide packages shipped by other repos.

Or, with the suggested feature, I can validate that the repo is OK, and still validate that the packages come from a vendor that I trust to install on my system.
 


> Again, I'm not saying that it shouldn't be done, I'm just saying that
> having them be the same key is just too dangerous since the repo key will
> be trusted to install packages and should not be.

Hmm. The repo creation and package creation should be done in a secure system.
Possibly with the key in a HSM so that its not possible to access it. At the
time a package is merged with a repo, its signed and updated metadata
generated. As the developer, all is good. You are certain of your package and
the metadata. There is only one key and you can reason about it getting to the
end user correctly. This gets pushed out to mirrors that stage your content.
The idea is to ensure every possible angle is covered to directly update the
end user.

There should be no key laying around for a rogue developer to use. The build
system should be tightly controlled. There should be git logs collected and
all sources tightly controlled so that any commit requires a kerberos ticket,
gpg key, two factor auth, whatever makes you comfortable. I think this is the
core of the issue.


My main issue is that I trust packagers and possibly not repo providers. For a concrete example, let's assume that Varnish has their own package signing keys. Now, let's go download Varnish from https://packagecloud.io/varnishcache/varnish5/install#manual-rpm. If you follow their instructions, you're allowing PackageCloud to be *fully trusted* as an installation provider. I would rather just trust the Varnish keys and not trust PackageCloud to be able to install random other RPMs. Does this make more sense?

I complete agree that these steps should be followed, and it seems like that would be another great security guide that should be written and tied to this requirement. This option should only be enabled for repositories that follow the procedure that you have described *and no others*. Which means that it should not be enabled by default and vendors should provide a statement regarding their package and repo signing practices in some way that the remote system could process in an automated fashion for validation.
 

After this, it's all transmission to end user with no injections possible.

-Steve

> On Tue, Nov 14, 2017 at 2:48 PM, Steve Grubb <sgrubb@redhat.com> wrote:
> > Hello Trevor,
> >
> > On Tuesday, November 14, 2017 10:24:35 AM EST Trevor Vaughan wrote:
> > > I get the bigger value of the GPG validation check but I think that the
> > > current implementation is severely flawed.
> > >
> > > If there were a separate setting for GPG keys used for repo validation,
> > > such as repo_gpgkey, I would be more than happy to use it and flip it
> > > on.
> >
> > The ability to modify the repo metadata is the same capability as creating
> > the
> > package in the first place. That is why it uses the same key. For example,
> > someone having direct access to repo metadata can possibly modify
> > dependency
> > information to force a new but vulnerable package onto your system. This
> > is
> > the same as directly modifying the rpm to require a new dependency. So,
> > how do
> > you detect this threat without repo gpg signature checking?
> >
> > (All packages resolved and downloaded still have to pass gpg key
> > verification.
> > So, its not like they are forcing some random, unsigned package onto your
> > system.)
> >
> > The reason that yum developed several of the defenses to protect the
> > integrity
> > of the system came from this study way back in 2008:
> >
> > https://www2.cs.arizona.edu/stork/packagemanagersecurity/
> > otherattacks.html#extradep
> >
> > > However, currently, these are the two potential threat avenues:
> > >    1. Accept GPG Keys for Repos
> > >
> > >       - Allows *repository maintainer* (Nexus, PackageCloud, random
> > >       directory on a webserver) to transparently add or replace
> > >
> > > arbitrary vendor packages with those of their choosing targeted for
> > > thousands of systems without the downstream user knowledge
> >
> > This is also true in trust TLS option. They can add dependencies which
> > installs new software. I'm not sure about the replace an arbitrary vendor
> > supplied package yet. You can fix this with a yum plugin. See my last
> > comment
> > below.
> >
> > >          - Mitigation: Manually validate all package signatures on your
> > >          system after installing them (Horrible)
> > >          - Mitigation: Require internal YUM mirrors of all upstream
> > >          packages to a trusted repository via the SSG (I'm kind of OK
> > >
> > > with this one, but how do you check it? Also, the fundamental issue
> > > still
> > > holds unless you're resigning the repodata and, if this is automated, a
> > > system compromise is just as bad, if not worse, than the TLS case
> > > since arbitrary things could be signed)
> >
> > The way it starts out is by finding a time stamp of latest signing. It
> > then
> > uses this to check the time stamp of mirrors. If they pass then it
> > proceeds to
> > download and checks the gpg signature. So, the way that it works _is_
> > trustworthy with no need of mitigation except to enable the repo_gpgcheck.
> >
> > However, if you wanted to take this on yourself, then information can be
> > found
> > here:
> >
> > https://blog.packagecloud.io/eng/2015/07/20/yum-repository-internals/
> >
> > They show how to verify things by hand. This can of course all be
> > scripted.
> >
> > >          2. Trust TLS
> > >
> > >       - In the event of a repository system compromise, bypassing
> > >       SELinux
> > >       restrictions and DAC permissions (kernel level exploit?) someone
> > >
> > > can remove flawed package and regenerate the metadata
> >
> > But how do you even detect the repo has been compromised to know it need
> > regenerating?
> >
> > >          - Mitigation: Run the vendor OVAL for checking insecure
> > >          packages
> > >          (which we're all doing anyway, right?!)
> >
> > It is true that every async RHSA errata gets an entry in the OVAL content.
> > But not every reason to do an update/install is correlated to a security
> > advisory. Perhaps a functionality upgrade now pulls in some new packages?
> >
> > > Unless I'm missing something, I know which one I'm much more comfortable
> > > with latter as something that is better for the user and easy to
> > > mitigate
> > > using currently mandated best practice.
> > >
> > > Again, once something like repo_gpgkey exists and is fully integrated,
> >
> > I'd
> >
> > > be more than comfortable with this.
> >
> > The only issue I see is how does yum handle collisions on packages between
> > repos. I think the answer may be to use yum-plugin-priorities. Using that
> > you
> > can assign the repo you trust least a higher number. That would make it
> > use
> > redhat repos first, and then down to the one you are suspicious of.
> >
> > Description  : This plugin allows repositories to have different
> > priorities.
> >
> >                    : Packages in a repository with a lower priority can't
> >
> > be
> > overridden by packages from a repository with a higher priority even if
> > repo
> > has a later version.
> >
> > An example of its use is here:
> > https://access.redhat.com/documentation/en-US/
> > Red_Hat_Enterprise_Linux_OpenStack_Platform/2/html/
> > Getting_Started_Guide/sect-
> > Configuring_Software_Repositories.html
> >
> > -Steve
> >
> > > On Tue, Nov 14, 2017 at 9:47 AM, Steve Grubb <sgrubb@redhat.com> wrote:
> > > > On Tuesday, November 14, 2017 9:37:18 AM EST Arnold, Paul C CTR USARMY
> >
> > PEO
> >
> > > > STRI (US) wrote:
> > > > > On 11/13/2017 06:59 PM, Steve Grubb wrote:
> > > > > > ...the current rev of OSPP
> > > > > > calls out for auditing of software update integrity checks. It
> >
> > calls
> >
> > > > out
> > > >
> > > > > > for integrity checks and for them to be enabled. It calls out for
> >
> > the
> >
> > > > > > vendor to supply SCAP content for the evaluated configuration. So
> >
> > that
> >
> > > > > > means we shouldn't be turning it off.
> > > > >
> > > > > What are we gaining by enabling repo_gpgcheck in addition to
> >
> > gpgcheck?
> >
> > > > It's for checking that the metadata hasn't been tampered with since
> > > > signing.
> > > > For example, suppose you need some packages out of EPEL. EPEL has a
> > > > distributed mirror list that volunteers contribute bandwidth for
> > > > everyone's
> > > > benefit. However, what if their server became compromised and an
> >
> > attacker
> >
> > > > removed the entry for a critical package update for a network facing
> > > > daemon?
> > > > The intent being to keep people from patching to allow more
> >
> > compromises.
> >
> > > > This setting would check the metadata to ensure that the signature
> > > > verification shows the metadata is untampered with. TLS protects
> >
> > against
> >
> > > > modifying an in-transit package or metadata. But it doesn't tell you
> >
> > that
> >
> > > > your
> > > > package resolution is using trustworthy data.
> > > >
> > > > -Steve





--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699 x788

-- This account not approved for unencrypted proprietary information --