Well, here's the simple way I imagine:
- new machines increment number.
- we setup CNAME aliases.
- CNAMEs change when new machine is placed in service.
We have a log01. We make a new log02, get it all setup, working, etc.
We have a 'collectd-server' cname that points to log01
We have a 'syslog-server' cname that points to log01
We have (optionally) a 'log-server' cname that points to log01.
When changing things, we just change the cnames. Admins learn to login
to the cname and it always goes to the live one. Although that doesn't
help for puppet.
Alternately, we could reuse. So, after the above, say we want to move
log02 on to a new os or something, we make a new log01 (since thats no
longer in use) and point it back to that. (that would save us from
having a log8679305).
Or if there is a more sane way to make a 'log01tmp' and rename it, but
still have a easy way to go back to the old one (including state in
puppet), that would be ok too.
I think names (I use evil wizards here at home) are fun, but can be
challenging to come up with sometimes.