On Thu, 2017-11-02 at 12:41 -0400, Stephen John Smoogen wrote:
I realized I had only informally gone over some of the items which
will be changing in December when hardware is moved. However the date
is finally arriving so it needs to be laid out more formally :).
In December, Infrastructure will be moving to new racks with better
power monitoring, networking, and other tools. One of the side goals
of this is that any old hardware which has been meant to be recycled
over the years is to be done so. Which brings us to a set of qa
systems which were listed to be replaced a couple of years ago.
qa01, qa02, qa03, qa04, qa05, qa07, qa08 are all IBM systems which
have reached the end of the road and are planned not to be moved over
to the new racks. [They have been scheduled to be removed for 2-3
years now but it has finally come to that date.] What processes are
currently running on these systems and what is needed to replace them?
qa05 and qa07 are the 'small' worker hosts for openQA (prod and stg,
respectively). Each hosts 4 of the total 14 x86_64 worker instances. So
losing them will cut the capacity of each openQA instance by about 30%.
What I'd need to replace them *most easily* is...another couple of
hosts in infra (technically, 'able to mount the NFS shares from the
openQA servers and connect back to the openQA servers via http and
possibly websockets', I think) capable of running four or more x86_64
VMs simultaneously, basically. Up till now openQA worker hosts have
always been dedicated to that task; there's no fundamental theoretical
reason why openQA workers couldn't run on a box which also does some
other stuff, though we'd have to check if there were any conflicts in
requirements as to how the box has to be set up (check the ansible
openQA has some newer capabilities that are basically about making
'more remote' workers possible, but I have not investigated those yet.
It *may* be possible to use those to run workers in some kind of cloud,
but that'd involve much more research and configuration and probably
debugging than just doing straight-up replacements in infra.
Another possibility I guess is having a single more powerful box
hosting workers for both prod and staging. Again I believe it may be
possible to get openQA to do this but it's different from what we do
now and I'd need to look into it.
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net