On Tue, 22 Jul 2014 12:12:36 -0600
Kevin Fenzi <kevin(a)scrye.com> wrote:
On Tue, 22 Jul 2014 10:56:12 -0600
Tim Flink <tflink(a)redhat.com> wrote:
> Now that Taskotron staging is up and running, I'm not sure what's
> generally required for services before moving to production.
> I'm still working out a few kinks in the initial production systems
> but I think that the only big thing left is the proxy settings,
> which I'm leaving alone for the moment.
> Is there any review process or checklist for moving things from stg
> to prod?
Thanks, I don't think that I would have looked in that doc.
I'd like monitoring added. What should we monitor to make sure
working as expected?
It depends on how complicated we want to get, I suppose. The most basic
things would be to check to make sure resultsdb, resultsdb_frontend and
buildbot are responding to http but there are json apis for
resultsdb and buildbot which would give more details. I'd like to
monitor free disk space if that's not already done by default.
Are most other services monitored with a GET to a url to see if they're
Also, do we need any SOP's written up? Anything special in
stopping/starting things? updates order or anything?
Yeah, the triggered jobs need to be buffered during downtime for
playback after everything comes up. I've always used my local system
for that but I'm not sure what the best choice is for infra.
Also, I need to make sure we have backups going on the db server at
least. Anything else that needs backing up?
/home/buildmaster/master on the taskotron server (taskotron01.qa for
production) should be backed up as that's where all the logs are
stored. Everything else that needs to be backed up is in the databases.