On Tue, 20 Mar 2012 21:38:13 -0500
Dennis Gilmore <dennis(a)ausil.us> wrote:
Today there is not a way to do an anaconda install on any arm
system.
though hopefully we will have that for deployment.
I would hope so. :)
probably we would be adding 100-300 systems. not only do we need to
consider overloading of puppet, but also logging and monitoring. I
guess its more how do we scale our infrastructure from at a guess ~100
nodes today to 3 to 4 times that
Centrally logging the builders is probably unnecessary. Especially if
we're bouncing them all the time.
honestly we could do this instead of the monthly updates. just
rebuild
them instead
Sure - but I'm thinking of the emergency "oh look at that nightmare"
updates.
im ok with that, im pretty sure fas will scale to the extra boxes.
do
we drop monitoring of the builders? what about collectd etc.
Collectd - off. We're not gaining much by having that punish the syslog
server.
We can monitor the builders w/o needing all of the copious info that
collectd provides.
fas I'm not very worried about - though I suspect a couple of things
will change w/how we get the dbs onto the hosts.
main issue is that today we are not 100% sure of how we will install
arm boxes. how do we deal with all the non puppet related systems?
I think, if the playbooks are working well, we can use ansible to do
this.
also need to look into how we can better scale koji itself. when we
go from 20 to 200+ builders we need to make sure that load doesn't
cause koji to fall over.
okay - but I think that's more something for the kojidevs than fedora
infra?
all the arm boxes will have management consoles. but today im not
100%
sure how access to that would be. we would also need to deploy fedora
for any arm based systems. things we need to reconsider also is
networking today the storage network and the builder networks
are /24's so we could use 253 nodes. i suspect we will go over that
on the build network. we could not have the storage network on arm
builders. it is really only needed for createrepo. but we may need to
look at expanding kojipkgs to more nodes. or increase its network
throughput with multiple bonded gig network ports. think mass rebuild
and 100 or 200 buildroots initialising at once. it will stress our
resources on all levels. but the flexibility of so many nodes could
allow us to deploy solid solutions to scale and show that fedora is
still the leader in open infrastructure and sets industry best
practices.
So one thing I'm not sure I understand - why would we need so many arm
builders? Is it b/c there are so many more arm archs so there will need
to be more pkgs built?
-sv