Network setup of builders

seth vidal skvidal at fedoraproject.org
Wed Jun 26 20:03:01 UTC 2013


On Wed, 26 Jun 2013 21:25:05 +0200
Miroslav Suchy <msuchy at redhat.com> wrote:

> On 06/26/2013 04:53 PM, seth vidal wrote:
> > 1. remember that the user can add random repos to their mock
> > configs in addition to the urls for their own srpms. Those repos
> > can be from anywhere - not just from coprs. This is on purpose - so
> > we can support a wide variety of systems and packages - w/
> > buildreqs from all over.
> 
> Why we have such requirement? None build service (including very open 
> OBS) allows this.
> For example OBS allow to build to very wide scale of repositories:
> Arch, Debian, Suse, Fedora, RHEL, Mandriva, Ubuntu and many others.
> But they always have to be added by build service administrator first.

Which is the point. That really only lets you build from the blessed
locations. It's an extra bit of red tape to cut through. Why would we
want to restrict our users in that way?


> 
> > 2. uploading a 400-1GB pkg to cloud-fe will murder it. Fetching the
> > file and putting limits on the fetch are MUCH easier
> 
> Why? We will have to have work with big packages anyway. For example 
> package 0ad-data have more than 1GB. I do not see difference if this 
> file is downloaded by builder or uploaded to cloud-fe. OK there is 
> subtle difference, we will not need to store src.rpm. But it will be 
> max. 1/2 of stored data. If you build for more targets (which most
> user will) then it will be e.g 1/8 of data. So I do not see problem
> here.


Have you uploaded a 1GB file to a website from a home network
connection? It's AWFUL. Not to mention we'd need to store this one the
-fe system and the whole goal was to have a separation between -fe and
-be so they didn't need to communicate in anyway but, ultimately REST


Here's how I would use this:

1. construct my spec file(s) locally or over ssh.
2. build srpms on a system <out_in_the_world>
3. put srpms in a webaccessible location
4. paste the srpm urls into the box
5. wait for builds
6. win


> BTW: We will need some storage with terabytes of free space for sure.
> Do we have some? Or I have to start looking around?

Our existing private cloud infrastructure has some space - but not
infinite space. The answer I was given was - if this is useful we'll be
able to manifest space for it by adding space onto the private cloud
and more importantly if it is very popular then running out of space is
a good problem to have.

 
> > 3. the whole point of evil in the builder is that we don't care -
> > the builders are destroyed once they are used and they do not
> > contain ANY sensitive information. Furthermore, they are timed out
> > if they are hanging out for too long.
> 
> But the timeout is few hours. And how many builders we will have? OBS 
> have 400. We will start will smaller number.

I have no idea what OBS has to do with this.

Could you explain what you're talking about here?


> I (as attacker) would care less about sensitive information (there is 
> only one, hard to get, require a lot of work). I would rather welcome 
> the possibility to get bunch of machines for few hours for free. And 
> once they timeout I can get them again in few minutes. Ideal for some 
> botnet or as source of DOS attack.


Your concern here is someone using these systems as a botnet? I think
there are an array of ways to forestall such a thing from happening w/o
restricting all access out from the system.


Off the top of my head I can think of a couple of trivial ways to
verify that the places we're pulling from are repos and not
random websites - whitelisting them through.

What's the REAL risk of this as a botnet? I understand scary
theoretical risk. At the moment the plan was to open up use of this
only to people who are contributors to fedora. The same level of access
required to get access to fedorapeople.org and a fedora alias. That set
of users is far more narrow.

Also - why are you so adamant about this? Maybe I'm misunderstanding
your tone.

If we want to change what the builders are allowed access to
it's not terribly difficult. We just change the security group the
instance is created in to reflect the set of hosts we want it to be
able to access. It's.... well... trivial.
 
> > Remember coprs is not koji and koji is not coprs.  Coprs is not a
> > koji replacement. Coprs is for the space that Koji does not want to
> > and should not occupy - The untrusted build.
> >
> > Koji is about creating trusted builds from trusted sources and
> > trusted contributors.
> >
> > Coprs is about building pkgs from untrusted sources and
> > potentially untrusted contributors. This is why we don't touch the
> > rpm on any system AS an rpm. We only deal with it as files.
> 
> Therefore our whole environment should be more secure than
> environment of Koji. Including builders.

They are MORE secure. We are not threatened by the builders. The
builders are disposable. If you're concerned that these systems would
be dangerous to others, that's not an entirely unreasonable concern. 
However, I don't think that your 'ALL CLOSED' mechanism is a good
way to address that.


> 
> > So if we were to implement your suggestions we would eliminate:
> >   1. external repositories of any kind
> fine with me.
> >   2. pkgs above a certain size
> why? I see no problem with uploading file with GB size.
> >   3. pkgs requiring interesting network access to build.

> Such packages exist? Can you give me example. This is given as case
> of bad packing on every packaging workshop.

The whole point of this is to allow steaming pile of crap packages to
be built, if people so choose.

COPRs exists b/c the packaging policies of various bodies are too
strict and are a barrier to entry. The Powers-That-Be wanted a place
where people could build whatever package they wanted to build and not
be hampered by bundling or bad build processes or anything.

As to your question of an example, the most recent one I encountered
was a perl module that wanted to talk out to test before it was built.
In fedora the test was disabled, iirc. I'm sure members of the Fedora
Package Committee could comment on some of the nutbar things they've
seen.

-sv






More information about the copr-devel mailing list