Hi,
Further to a question I asked on the Puppet mailing list, I wonder if anyone here has had any success managing Cobbler with Puppet which they could share?
http://groups.google.com/group/puppet-users/browse_thread/thread/31040119df2...
As I said there, what I'd like to do is:
- write a Puppet node definition manifest for a new machine - include DHCP, DNS, hostname and network declarations - commit this to Git - deploy via Puppet - have the Cobber server pick the config up and set up PXE, DHCP, and DNS accordingly (Cobbler being managed via Puppet) - shortly after, boot a new machine, and during the PXE boot either - have its MAC address recognised, or - select the system from a menu - have it provisioned with the Puppet package and an appropriate hostname - have Puppet configure it in the pre-assigned role
Ideally, I'd be able to re-deploy a new Cobbler server in exactly the same way as another node.
Puppet modules which allow automating setting up repos, profiles, systems etc. don't seem to exist on the public internet.
One way might be to write a Puppet type based on the Ruby library for controlling Cobbler 2.0 (2.x?) I gather exists here:
http://www.google.com/url?sa=D&q=https://github.com/duritong/ruby-cobble...
However, more recently I've been experimenting with Augeas (from Puppet) to manage certain config files and I notice there are Augeas lenses (parsers) for cobbler's settings and modules.conf files. Plus there is a generic JSON lens, which presumably could handle everything in /var/lib/cobbler/config.
So another approach might be to use that - although this would would be bypassing the API, of course, which comes with its own set of problems.
Cheers,
Nick
This already exists, but you're kinda trying to do it backwards :)
Provisioning happens well before configuration, and Puppet is very much about things that already exist -- so it makes sense for Cobbler to lead at first and then get out of the way.
While, in provisioning, we talk about profiles that haven't been installed yet, and we can actually decide what the hostnames for given MAC addresses are going to be. You may very well want to say when AA:BB:CC:DD:EE:FF mac address comes up, install base CentOS 6 and give it Puppet classes A and B, and variables x=2 and y=3. We can do that. We can also do very clever things about saying that all RHEL distros get additional classes and variables, and different profiles also assign certain classes and variables. See the kickstart snippets which we have here for trivial bootstrapping:
https://github.com/cobbler/cobbler/blob/master/snippets/puppet_install_if_en...
https://github.com/cobbler/cobbler/blob/master/snippets/puppet_register_if_e...
It probably works best with the config tool in auto sign mode if you want the install to complete in the first pass.
Information about using Cobbler as an external nodes resource is also here, where you can say, "when this system comes online, give them these Puppet classes":
https://github.com/cobbler/cobbler/wiki/Using%20Cobbler%20With%20A%20Configu...
This works pretty darn well, and ensures that the two are well connected. I'd say it's pretty widely used among the set of users that use both tools.
You could also backup your /var/lib/cobbler configuration files in git if you so desired. There's actually a cobbler trigger for this already that will auto commit changes:
https://github.com/cobbler/cobbler/blob/master/cobbler/modules/scm_track.py
Hope this helps.
--Michael
On Tuesday, January 31, 2012 at 6:34 AM, Nick wrote:
Hi,
Further to a question I asked on the Puppet mailing list, I wonder if anyone here has had any success managing Cobbler with Puppet which they could share?
http://groups.google.com/group/puppet-users/browse_thread/thread/31040119df2...
As I said there, what I'd like to do is:
- write a Puppet node definition manifest for a new machine
- include DHCP, DNS, hostname and network declarations
- commit this to Git
- deploy via Puppet
- have the Cobber server pick the config up and set up PXE, DHCP, and
DNS accordingly (Cobbler being managed via Puppet)
- shortly after, boot a new machine, and during the PXE boot either
- have its MAC address recognised, or
- select the system from a menu
- have it provisioned with the Puppet package and an appropriate hostname
- have Puppet configure it in the pre-assigned role
Ideally, I'd be able to re-deploy a new Cobbler server in exactly the same way as another node.
Puppet modules which allow automating setting up repos, profiles, systems etc. don't seem to exist on the public internet.
One way might be to write a Puppet type based on the Ruby library for controlling Cobbler 2.0 (2.x?) I gather exists here:
http://www.google.com/url?sa=D&q=https://github.com/duritong/ruby-cobble...
However, more recently I've been experimenting with Augeas (from Puppet) to manage certain config files and I notice there are Augeas lenses (parsers) for cobbler's settings and modules.conf files. Plus there is a generic JSON lens, which presumably could handle everything in /var/lib/cobbler/config.
So another approach might be to use that - although this would would be bypassing the API, of course, which comes with its own set of problems.
Cheers,
Nick _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.org (mailto:cobbler@lists.fedorahosted.org) https://fedorahosted.org/mailman/listinfo/cobbler
On 31/01/12 13:06, Michael DeHaan wrote:
This already exists, but you're kinda trying to do it backwards :)
Hm, I'm starting to think I must have contrarian tendancies...
Provisioning happens well before configuration, and Puppet is very much about things that already exist -- so it makes sense for Cobbler to lead at first and then get out of the way.
I think I get that, but from my point of view, I don't want have two tools (Cobbler and Puppet) managed independently.
Instead, I was hoping to have just *one* definition of how the whole site is set up, in version control. Puppet seems potentially capable of defining Cobbler's configuration in entirety, but Cobbler's scope is more limited: the data-structures describing profiles, systems, etc. are inaccessible to Puppet, and Cobbler can only pass a list of class names to Puppet via an external node classifier.
So it seems to make sense to have Puppet configure Cobbler rather than vice versa. Then there is just one point of control: the central Puppet repository in a Git repo. Unified, with no duplication (I hope).
This is also the main issue I have with Foreman: I can't put all my config in one version controlled repository.
While, in provisioning, we talk about profiles that haven't been installed yet, and we can actually decide what the hostnames for given MAC addresses are going to be. You may very well want to say when AA:BB:CC:DD:EE:FF mac address comes up, install base CentOS 6 and give it Puppet classes A and B, and variables x=2 and y=3. We can do that.
Yep, and I think I read enough of the docs to see how to do that. It's neat, yes.
As I said, what I don't quite like about it this way is that part of the site-wide configuration is then embedded in Cobbler's internal configs, and divorced from the rest of it in Puppet.
I know the Cobbler configuration can be snapshotted into git using a hook - I'm currently doing it - but it does not integrate easily into my Puppet repository. It wants to be in its own branch/own repository, and commit changes using the rather meaningless description "API update" whenever I do a "cobbler sync".
I also looked at Cobbler's node classifier, but yet again, to use it assumes Cobbler is driving things, and then Cobbler's (perfectly reasonable) limitations force a configuration split across Puppet and Cobbler.
Not to mention that as I am using a "masterless" mode, I need more than just a map of node names to classes, I need a way to emulate external resource definitions. (I'm using a variation of that described here [1], which needs exported declarations to go outside the nodes.)
We can also do very clever things about saying that all RHEL distros get additional classes and variables, and different profiles also assign certain classes and variables. See the kickstart snippets which we have here for trivial bootstrapping:
https://github.com/cobbler/cobbler/blob/master/snippets/puppet_install_if_en...
https://github.com/cobbler/cobbler/blob/master/snippets/puppet_register_if_e...
Yes, I've seen those. Those snippets assume there's a Puppet master. But that's ok, I've adapted them for my own purposes. Well, good enough for now, anyway.
Information about using Cobbler as an external nodes resource is also here, where you can say, "when this system comes online, give them these Puppet classes":
https://github.com/cobbler/cobbler/wiki/Using%20Cobbler%20With%20A%20Configu...
You could also backup your /var/lib/cobbler configuration files in git if you so desired. There's actually a cobbler trigger for this already that will auto commit changes:
https://github.com/cobbler/cobbler/blob/master/cobbler/modules/scm_track.py
Yes, I've also studied those, see above.
It's certainly useful and interesting to learn that I seem to be doing something different to everyone else. Maybe that's enough to make me reverse direction.
However, arguably I could say doing it "backward" is the way forward, if it means I can define my systems/profiles in one place, have that information available to Puppet, and configure my provisioning server with Puppet too. No need for a hook then, and my changes to Cobbler can be sensibly documented with the rest of my site-wide changes.
Cheers,
N
1. Puppet Without Masters: In the section "Backup and Nagios monitoring without storeconfigs"
http://current.workingdirectory.net/posts/2011/puppet-without-masters/
On Tuesday, January 31, 2012 at 9:39 AM, Nick wrote:
On 31/01/12 13:06, Michael DeHaan wrote:
This already exists, but you're kinda trying to do it backwards :)
Hm, I'm starting to think I must have contrarian tendancies...
Provisioning happens well before configuration, and Puppet is very much about things that already exist -- so it makes sense for Cobbler to lead at first and then get out of the way.
I think I get that, but from my point of view, I don't want have two tools (Cobbler and Puppet) managed independently.
Proverbially, I don't think you want to replace all of your tools with a hammer.
I'm far beyond the point of caring about which tools people use, it just makes it really *weird* do to so.
A Puppet define for each, in particularly, would be horribly slow.
JSON files for each ignore cobbler's own validation and side effects, which were put there for a reason.
Lots of reasons to use separate tools, you would be shooting yourself in the foot.
Instead, I was hoping to have just *one* definition of how the whole site is set up, in version control. Puppet seems potentially capable of defining Cobbler's configuration in entirety, but Cobbler's scope is more limited: the data-structures describing profiles, systems, etc. are inaccessible to Puppet, and Cobbler can only pass a list of class names to Puppet via an external node classifier.
It's not that simple. There's a reason we didn't implement this when I worked at Puppet Labs :)
So it seems to make sense to have Puppet configure Cobbler rather than vice versa. Then there is just one point of control: the central Puppet repository in a Git repo. Unified, with no duplication (I hope).
This is also the main issue I have with Foreman: I can't put all my config in one version controlled repository.
The main issue I have with Foreman is the parasitic way users were recruited from this project and the aggressive self promotion :)
Would rather not discuss it's name here.
While, in provisioning, we talk about profiles that haven't been installed yet, and we can actually decide what the hostnames for given MAC addresses are going to be. You may very well want to say when AA:BB:CC:DD:EE:FF mac address comes up, install base CentOS 6 and give it Puppet classes A and B, and variables x=2 and y=3. We can do that.
Yep, and I think I read enough of the docs to see how to do that. It's neat, yes.
As I said, what I don't quite like about it this way is that part of the site-wide configuration is then embedded in Cobbler's internal configs, and divorced from the rest of it in Puppet.
Using an external-nodes-tool in pretty a much a best practice from the other side of the house. Though I can understand the desire to see the changes in a git RSS feed or something, you would still see that if you used the scm_track module.
I know the Cobbler configuration can be snapshotted into git using a hook - I'm currently doing it - but it does not integrate easily into my Puppet repository. It wants to be in its own branch/own repository, and commit changes using the rather meaningless description "API update" whenever I do a "cobbler sync".
True, you'd have to look at the diffs to understand what was done.
The main point was rollback.
I also looked at Cobbler's node classifier, but yet again, to use it assumes Cobbler is driving things, and then Cobbler's (perfectly reasonable) limitations force a configuration split across Puppet and Cobbler.
Right, deploy the OS and your configuration separately.
Not to mention that as I am using a "masterless" mode, I need more than just a map of node names to classes, I need a way to emulate external resource definitions. (I'm using a variation of that described here [1], which needs exported declarations to go outside the nodes.)
Point, ext-nodes don't work for you. I wish it did work that way, actually.
Ah well.
We can also do very clever things about saying that all RHEL distros get additional classes and variables, and different profiles also assign certain classes and variables. See the kickstart snippets which we have here for trivial bootstrapping:
https://github.com/cobbler/cobbler/blob/master/snippets/puppet_install_if_en...
https://github.com/cobbler/cobbler/blob/master/snippets/puppet_register_if_e...
Yes, I've seen those. Those snippets assume there's a Puppet master. But that's ok, I've adapted them for my own purposes. Well, good enough for now, anyway.
Information about using Cobbler as an external nodes resource is also here, where you can say, "when this system comes online, give them these Puppet classes":
https://github.com/cobbler/cobbler/wiki/Using%20Cobbler%20With%20A%20Configu...
You could also backup your /var/lib/cobbler configuration files in git if you so desired. There's actually a cobbler trigger for this already that will auto commit changes:
https://github.com/cobbler/cobbler/blob/master/cobbler/modules/scm_track.py
Yes, I've also studied those, see above.
It's certainly useful and interesting to learn that I seem to be doing something different to everyone else. Maybe that's enough to make me reverse direction.
However, arguably I could say doing it "backward" is the way forward, if it means I can define my systems/profiles in one place, have that information available to Puppet, and configure my provisioning server with Puppet too. No need for a hook then, and my changes to Cobbler can be sensibly documented with the rest of my site-wide changes.
If you did it, you should write set of Ruby types/provider.
The provider must speak Cobbler XMLRPC, not JSON.
The provider will need to notify a cobbler sync if anything changes.
Make sure you don't leak the cobbler username/password to your nodes.
Still, I think that's a lot of work and a confusing way to do it. There are basically four pieces to systems management
- provisioning - package management / updates - config - monitoring
You pick what tools you want for each. I think having config drive provisioning makes about as much sense as having monitoring drive package updates.
But that's just me.
Cheers,
N
- Puppet Without Masters:
In the section "Backup and Nagios monitoring without storeconfigs"
http://current.workingdirectory.net/posts/2011/puppet-without-masters/ _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.org (mailto:cobbler@lists.fedorahosted.org) https://fedorahosted.org/mailman/listinfo/cobbler
Thanks for this, noted.
For what it's worth, compared the - *ahem* - alternatives, I think Cobbler fulfils its role rather well and is easier to set up and understand.
Anyway, I'm not about to embark on writing any new tools if I can get away with what's on the shelf. Writing a Cobbler provider for Puppet is more of a mission than I want to take on at this moment in time. (I have other missions which may kill me yet.)
Cheers,
N
Hi I have not read all of the posts in minute detail but I have seen some similarities with issues we are experiencing and thought I would share our view on the world.
We have all our server config in yaml and use a custom script using xmlrpc to connect to cobbler and populate it. To get a working cobbler server we run puppet on a box which will pull down the repositories and copy all the nessecary configs over. Some of the config files is using environment variables to set things like dhcp. The idea we are having is that the yaml files becomes a cmdb in a way which can be read by our script that populates cobbler and can be read by puppet as well to get data using via hiera. Its not brilliant but it works. I am also likely to be lambasted for the yaml thing since it could be viewed as a duplication of the json files in cobbler... especially because I have templated my yaml files so I have a basic level of inheritance in the config files. (I am in a way duplicating cobbler functionality) The script is very rudementary at this stage but I would happy to share it with anyone interested. Having the yaml files does allow us to keep system config in svn and have control over it and you could easily adjust the script to repopulate cobbler everytime someone commits a change to the config.
Regards
On 31 January 2012 14:54, Michael DeHaan michael.dehaan@gmail.com wrote:
On Tuesday, January 31, 2012 at 9:39 AM, Nick wrote:
On 31/01/12 13:06, Michael DeHaan wrote:
This already exists, but you're kinda trying to do it backwards :)
Hm, I'm starting to think I must have contrarian tendancies...
Provisioning happens well before configuration, and Puppet is very much about things that already exist -- so it makes sense for Cobbler to lead at first and then get out of the way.
I think I get that, but from my point of view, I don't want have two tools (Cobbler and Puppet) managed independently.
Proverbially, I don't think you want to replace all of your tools with a hammer.
I'm far beyond the point of caring about which tools people use, it just makes it really *weird* do to so.
A Puppet define for each, in particularly, would be horribly slow.
JSON files for each ignore cobbler's own validation and side effects, which were put there for a reason.
Lots of reasons to use separate tools, you would be shooting yourself in the foot.
Instead, I was hoping to have just *one* definition of how the whole site is set up, in version control. Puppet seems potentially capable of defining Cobbler's configuration in entirety, but Cobbler's scope is more limited: the data-structures describing profiles, systems, etc. are inaccessible to Puppet, and Cobbler can only pass a list of class names to Puppet via an external node classifier.
It's not that simple. There's a reason we didn't implement this when I worked at Puppet Labs :)
So it seems to make sense to have Puppet configure Cobbler rather than vice versa. Then there is just one point of control: the central Puppet repository in a Git repo. Unified, with no duplication (I hope).
This is also the main issue I have with Foreman: I can't put all my config in one version controlled repository.
The main issue I have with Foreman is the parasitic way users were recruited from this project and the aggressive self promotion :)
Would rather not discuss it's name here.
While, in provisioning, we talk about profiles that haven't been installed yet, and we can actually decide what the hostnames for given MAC addresses are going to be. You may very well want to say when AA:BB:CC:DD:EE:FF mac address comes up, install base CentOS 6 and give it Puppet classes A and B, and variables x=2 and y=3. We can do that.
Yep, and I think I read enough of the docs to see how to do that. It's neat, yes.
As I said, what I don't quite like about it this way is that part of the site-wide configuration is then embedded in Cobbler's internal configs, and divorced from the rest of it in Puppet.
Using an external-nodes-tool in pretty a much a best practice from the other side of the house. Though I can understand the desire to see the changes in a git RSS feed or something, you would still see that if you used the scm_track module.
I know the Cobbler configuration can be snapshotted into git using a hook
- I'm
currently doing it - but it does not integrate easily into my Puppet repository. It wants to be in its own branch/own repository, and commit changes using the rather meaningless description "API update" whenever I do a "cobbler sync".
True, you'd have to look at the diffs to understand what was done.
The main point was rollback.
I also looked at Cobbler's node classifier, but yet again, to use it assumes Cobbler is driving things, and then Cobbler's (perfectly reasonable) limitations force a configuration split across Puppet and Cobbler.
Right, deploy the OS and your configuration separately.
Not to mention that as I am using a "masterless" mode, I need more than just a map of node names to classes, I need a way to emulate external resource definitions. (I'm using a variation of that described here [1], which needs exported declarations to go outside the nodes.)
Point, ext-nodes don't work for you. I wish it did work that way, actually.
Ah well.
We can also do very clever things about saying that all RHEL distros get additional classes and variables, and different profiles also assign certain classes and variables. See the kickstart snippets which we have here for trivial bootstrapping:
https://github.com/cobbler/cobbler/blob/master/snippets/puppet_install_if_en...
https://github.com/cobbler/cobbler/blob/master/snippets/puppet_register_if_e...
Yes, I've seen those. Those snippets assume there's a Puppet master. But that's ok, I've adapted them for my own purposes. Well, good enough for now, anyway.
Information about using Cobbler as an external nodes resource is also here, where you can say, "when this system comes online, give them these Puppet classes":
https://github.com/cobbler/cobbler/wiki/Using%20Cobbler%20With%20A%20Configu...
You could also backup your /var/lib/cobbler configuration files in git if you so desired. There's actually a cobbler trigger for this already that will auto commit changes:
https://github.com/cobbler/cobbler/blob/master/cobbler/modules/scm_track.py
Yes, I've also studied those, see above.
It's certainly useful and interesting to learn that I seem to be doing something different to everyone else. Maybe that's enough to make me reverse direction.
However, arguably I could say doing it "backward" is the way forward, if it means I can define my systems/profiles in one place, have that information available to Puppet, and configure my provisioning server with Puppet too. No need for a hook then, and my changes to Cobbler can be sensibly documented with the rest of my site-wide changes.
If you did it, you should write set of Ruby types/provider.
The provider must speak Cobbler XMLRPC, not JSON.
The provider will need to notify a cobbler sync if anything changes.
Make sure you don't leak the cobbler username/password to your nodes.
Still, I think that's a lot of work and a confusing way to do it. There are basically four pieces to systems management
- provisioning
- package management / updates
- config
- monitoring
You pick what tools you want for each. I think having config drive provisioning makes about as much sense as having monitoring drive package updates.
But that's just me.
Cheers,
N
- Puppet Without Masters:
In the section "Backup and Nagios monitoring without storeconfigs"
http://current.workingdirectory.net/posts/2011/puppet-without-masters/ _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
cobbler mailing list cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
On 31/01/12 16:11, Gerhardus Geldenhuis wrote:
We have all our server config in yaml and use a custom script using xmlrpc to connect to cobbler and populate it.
This sounds almost exactly what I meant when I wrote "another approach might be to write a Python script..."
Could post it somewhere public, for reference later?
Thanks,
N
Hi Its on git in a private repo so I will polish it up a bit and republish it for general consumption.
Regards
On 31 January 2012 16:17, Nick oinksocket@letterboxes.org wrote:
On 31/01/12 16:11, Gerhardus Geldenhuis wrote:
We have all our server config in yaml and use a custom script using
xmlrpc to
connect to cobbler and populate it.
This sounds almost exactly what I meant when I wrote "another approach might be to write a Python script..."
Could post it somewhere public, for reference later?
Thanks,
N _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
Yeah, that could work, but it's kind of ironic too -- Many people consider Cobbler to be a lightweight CMDB.
It used to have a YAML data store, but we moved away from that due to YAML being annoying for humans to edit (and also for some performance reasons), with all of the line endings and making it easy to get an error in it. JSON is more or less the same thing.
While you don't have all of your nodes in one file, you have them in multiple files, which is more or less the same thing.
On Tuesday, January 31, 2012 at 11:11 AM, Gerhardus Geldenhuis wrote:
Hi I have not read all of the posts in minute detail but I have seen some similarities with issues we are experiencing and thought I would share our view on the world.
We have all our server config in yaml and use a custom script using xmlrpc to connect to cobbler and populate it. To get a working cobbler server we run puppet on a box which will pull down the repositories and copy all the nessecary configs over. Some of the config files is using environment variables to set things like dhcp. The idea we are having is that the yaml files becomes a cmdb in a way which can be read by our script that populates cobbler and can be read by puppet as well to get data using via hiera. Its not brilliant but it works. I am also likely to be lambasted for the yaml thing since it could be viewed as a duplication of the json files in cobbler... especially because I have templated my yaml files so I have a basic level of inheritance in the config files. (I am in a way duplicating cobbler functionality) The script is very rudementary at this stage but I would happy to share it with anyone interested. Having the yaml files does allow us to keep system config in s
vn and have control over it and you could easily adjust the script to repopulate cobbler everytime someone commits a change to the config.
Regards
On 31 January 2012 14:54, Michael DeHaan <michael.dehaan@gmail.com (mailto:michael.dehaan@gmail.com)> wrote:
On Tuesday, January 31, 2012 at 9:39 AM, Nick wrote:
On 31/01/12 13:06, Michael DeHaan wrote:
This already exists, but you're kinda trying to do it backwards :)
Hm, I'm starting to think I must have contrarian tendancies...
Provisioning happens well before configuration, and Puppet is very much about things that already exist -- so it makes sense for Cobbler to lead at first and then get out of the way.
I think I get that, but from my point of view, I don't want have two tools (Cobbler and Puppet) managed independently.
Proverbially, I don't think you want to replace all of your tools with a hammer.
I'm far beyond the point of caring about which tools people use, it just makes it really *weird* do to so.
A Puppet define for each, in particularly, would be horribly slow.
JSON files for each ignore cobbler's own validation and side effects, which were put there for a reason.
Lots of reasons to use separate tools, you would be shooting yourself in the foot.
Instead, I was hoping to have just *one* definition of how the whole site is set up, in version control. Puppet seems potentially capable of defining Cobbler's configuration in entirety, but Cobbler's scope is more limited: the data-structures describing profiles, systems, etc. are inaccessible to Puppet, and Cobbler can only pass a list of class names to Puppet via an external node classifier.
It's not that simple. There's a reason we didn't implement this when I worked at Puppet Labs :)
So it seems to make sense to have Puppet configure Cobbler rather than vice versa. Then there is just one point of control: the central Puppet repository in a Git repo. Unified, with no duplication (I hope).
This is also the main issue I have with Foreman: I can't put all my config in one version controlled repository.
The main issue I have with Foreman is the parasitic way users were recruited from this project and the aggressive self promotion :)
Would rather not discuss it's name here.
While, in provisioning, we talk about profiles that haven't been installed yet, and we can actually decide what the hostnames for given MAC addresses are going to be. You may very well want to say when AA:BB:CC:DD:EE:FF mac address comes up, install base CentOS 6 and give it Puppet classes A and B, and variables x=2 and y=3. We can do that.
Yep, and I think I read enough of the docs to see how to do that. It's neat, yes.
As I said, what I don't quite like about it this way is that part of the site-wide configuration is then embedded in Cobbler's internal configs, and divorced from the rest of it in Puppet.
Using an external-nodes-tool in pretty a much a best practice from the other side of the house. Though I can understand the desire to see the changes in a git RSS feed or something, you would still see that if you used the scm_track module.
I know the Cobbler configuration can be snapshotted into git using a hook - I'm currently doing it - but it does not integrate easily into my Puppet repository. It wants to be in its own branch/own repository, and commit changes using the rather meaningless description "API update" whenever I do a "cobbler sync".
True, you'd have to look at the diffs to understand what was done.
The main point was rollback.
I also looked at Cobbler's node classifier, but yet again, to use it assumes Cobbler is driving things, and then Cobbler's (perfectly reasonable) limitations force a configuration split across Puppet and Cobbler.
Right, deploy the OS and your configuration separately.
Not to mention that as I am using a "masterless" mode, I need more than just a map of node names to classes, I need a way to emulate external resource definitions. (I'm using a variation of that described here [1], which needs exported declarations to go outside the nodes.)
Point, ext-nodes don't work for you. I wish it did work that way, actually.
Ah well.
We can also do very clever things about saying that all RHEL distros get additional classes and variables, and different profiles also assign certain classes and variables. See the kickstart snippets which we have here for trivial bootstrapping:
https://github.com/cobbler/cobbler/blob/master/snippets/puppet_install_if_en...
https://github.com/cobbler/cobbler/blob/master/snippets/puppet_register_if_e...
Yes, I've seen those. Those snippets assume there's a Puppet master. But that's ok, I've adapted them for my own purposes. Well, good enough for now, anyway.
Information about using Cobbler as an external nodes resource is also here, where you can say, "when this system comes online, give them these Puppet classes":
https://github.com/cobbler/cobbler/wiki/Using%20Cobbler%20With%20A%20Configu...
You could also backup your /var/lib/cobbler configuration files in git if you so desired. There's actually a cobbler trigger for this already that will auto commit changes:
https://github.com/cobbler/cobbler/blob/master/cobbler/modules/scm_track.py
Yes, I've also studied those, see above.
It's certainly useful and interesting to learn that I seem to be doing something different to everyone else. Maybe that's enough to make me reverse direction.
However, arguably I could say doing it "backward" is the way forward, if it means I can define my systems/profiles in one place, have that information available to Puppet, and configure my provisioning server with Puppet too. No need for a hook then, and my changes to Cobbler can be sensibly documented with the rest of my site-wide changes.
If you did it, you should write set of Ruby types/provider.
The provider must speak Cobbler XMLRPC, not JSON.
The provider will need to notify a cobbler sync if anything changes.
Make sure you don't leak the cobbler username/password to your nodes.
Still, I think that's a lot of work and a confusing way to do it. There are basically four pieces to systems management
- provisioning
- package management / updates
- config
- monitoring
You pick what tools you want for each. I think having config drive provisioning makes about as much sense as having monitoring drive package updates.
But that's just me.
Cheers,
N
- Puppet Without Masters:
In the section "Backup and Nagios monitoring without storeconfigs"
http://current.workingdirectory.net/posts/2011/puppet-without-masters/ _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.org (mailto:cobbler@lists.fedorahosted.org) https://fedorahosted.org/mailman/listinfo/cobbler
cobbler mailing list cobbler@lists.fedorahosted.org (mailto:cobbler@lists.fedorahosted.org) https://fedorahosted.org/mailman/listinfo/cobbler
-- Gerhardus Geldenhuis _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.org (mailto:cobbler@lists.fedorahosted.org) https://fedorahosted.org/mailman/listinfo/cobbler
Hi I agree, and why I was expecting to be shown the error of my ways. :)
I am still in two minds about where cobbler stops and where puppet starts and where config should be living... the idea/view I have is that cobbler should facilitate building of an OS and puppet should facilitate configuration of an OS but that the data should be a third entity but that is very abstract view of things. Ironically I decided on yaml because I felt it would be easier to read and even more ironic is that I could not get the initial yaml file to work so wrote a json file and then used ruby to convert the json into yaml... another longer term idea was to write a validator that in addition to testing normal parsing would also validate the config file for things like sane ip addresses, duplicate entries etc.
Regards
On 31 January 2012 16:20, Michael DeHaan michael.dehaan@gmail.com wrote:
Yeah, that could work, but it's kind of ironic too -- Many people consider Cobbler to be a lightweight CMDB.
It used to have a YAML data store, but we moved away from that due to YAML being annoying for humans to edit (and also for some performance reasons), with all of the line endings and making it easy to get an error in it. JSON is more or less the same thing.
While you don't have all of your nodes in one file, you have them in multiple files, which is more or less the same thing.
My view is basically that you have three entities:
OS setup -> Config Setup -> Monitoring
The thing that owns OS setup also owns the way that control is transferred to the next, similar to the way that many configuration tools can also help you set up monitoring. They don't own monitoring, but they own the setup of the next tool in the chain
( cobbler: OS setup -> ) (config tool: Config Setup -> ) Monitoring
Each tool owns the arrow that sets up the next.
But to do the thing with each tool, you go to tweak the tool that can best tweak itself, rather than working through a little tiny porthole and making it harder.
Bad analogy -- suppose you have a car. You can either drive the car or you can tell someone else how to drive the car, but it will never be as immediate as driving the actual car. My first analogy would have been with analog synthesizer knobs vs MIDI commands in a Digital Audio Workstation but that might not work for everyone :)
On Tuesday, January 31, 2012 at 11:37 AM, Gerhardus Geldenhuis wrote:
Hi I agree, and why I was expecting to be shown the error of my ways. :)
I am still in two minds about where cobbler stops and where puppet starts and where config should be living... the idea/view I have is that cobbler should facilitate building of an OS and puppet should facilitate configuration of an OS but that the data should be a third entity but that is very abstract view of things. Ironically I decided on yaml because I felt it would be easier to read and even more ironic is that I could not get the initial yaml file to work so wrote a json file and then used ruby to convert the json into yaml... another longer term idea was to write a validator that in addition to testing normal parsing would also validate the config file for things like sane ip addresses, duplicate entries etc.
Regards
On 31 January 2012 16:20, Michael DeHaan <michael.dehaan@gmail.com (mailto:michael.dehaan@gmail.com)> wrote:
Yeah, that could work, but it's kind of ironic too -- Many people consider Cobbler to be a lightweight CMDB.
It used to have a YAML data store, but we moved away from that due to YAML being annoying for humans to edit (and also for some performance reasons), with all of the line endings and making it easy to get an error in it. JSON is more or less the same thing.
While you don't have all of your nodes in one file, you have them in multiple files, which is more or less the same thing.
cobbler mailing list cobbler@lists.fedorahosted.org (mailto:cobbler@lists.fedorahosted.org) https://fedorahosted.org/mailman/listinfo/cobbler
My view is basically that you have three entities:
OS setup -> Config Setup -> Monitoring
The thing that owns OS setup also owns the way that control is transferred to the next, similar to the way that many configuration tools can also help you set up monitoring. They don't own monitoring, but they own the setup of the next tool in the chain
Well and good, but now imagine a tornado comes and wipes all three away. Now you need to reconstitute your network, and you need to start with your Cobbler server, because you're about to kickstart a whole bunch of machines. You have to install the OS manually on that server, but then installing and configuring Cobbler the way you run it at your site is an item of "config setup," which you want Puppet to know about and do.
(In order to reconstruct a working Cobbler server, Puppet may not need to know each thing in Cobbler's database, but merely where a copy of Cobbler's VCS repo is, so it can put it in place before starting Cobbler up.)
The scenario I have described may not be what Nick nor Gerhardus are trying to do, but it's a big part of what I'm trying to use Puppet for.
Well and good, but now imagine a tornado comes and wipes all three away. Now you need to reconstitute your network, and you need to start with your Cobbler server, because you're about to kickstart a whole bunch of machines. You have to install the OS manually on that server, but then installing and configuring Cobbler the way you run it at your site is an item of "config setup," which you want Puppet to know about and do.
Cobbler can help you control your DHCP and DNS and get all of those machines and hostnames back, so it's a lot more concerned with low level datacenter bits. I would want those services running first, before configuration management. I would use Cobbler to install the OS that the config box runs on, just like I'd use it to install the OS that my package mirroring comes from. Cobbler probably manages the mirror of the puppet repo itself, in many cases. This is not hard to back up.
I would use cobbler replicate for disaster recovery purposes or include /var/lib/cobbler and /etc/cobbler for backup purposes to quickly get your config back. Nightly backups should be sufficient, or you could do the git thing.
----- Michael DeHaan michael.dehaan@gmail.com wrote:
Well and good, but now imagine a tornado comes and wipes all three away. Now you need to reconstitute your network, and you need to start with your Cobbler server, because you're about to kickstart a whole bunch of machines. You have to install the OS manually on that server, but then installing and configuring Cobbler the way you run it at your site is an item of "config setup," which you want Puppet to know about and do.
Cobbler can help you control your DHCP and DNS and get all of those machines and hostnames back, so it's a lot more concerned with low level datacenter bits. I would want those services running first, before configuration management. I would use Cobbler to install the OS that the config box runs on, just like I'd use it to install the OS that my package mirroring comes from. Cobbler probably manages the mirror of the puppet repo itself, in many cases. This is not hard to back up.
I would use cobbler replicate for disaster recovery purposes or include /var/lib/cobbler and /etc/cobbler for backup purposes to quickly get your config back. Nightly backups should be sufficient, or you could do the git thing.
I would like to add my voice to the opinion that managing Cobbler with Puppet is a bass-ackwards way of approaching the problem.
I started from this URL: http://consultancy.edvoncken.net/index.php/HOWTO_Set_up_a_Subversion_repository_for_provisioning
I set up both Cobbler and Puppet in Subversion and using that, I could stand up the framework for a duplicate Cobbler server in less than an hour. The time consuming part then would be (downloading if necessary and) importing all the install iso's and creation of any local repositories and local repository mirrors. This replication could be shortened by having good backups of not only /var/lib/cobbler and /etc/cobbler but of any local distributions and repositories.
All of the distro, profile, repo, and system definitions in Cobbler "live" in /var/lib/cobbler/config as JSON files. Go look.
Once a Cobbler Server is set up, the only changes, as I see the setup, is adding new systems. It seems to me that the distros, profiles, and repos change infrequently compared to the systems.
Also, to document how I set up the Cobbler server, I did all of my initial setup steps on the command line instead of through the web interface and saved all of my commands. Using that list of command lines, I could build a duplicate server from scratch almost as quickly as using the Subversion copy.
Trying to use Puppet to manage Cobbler seems to me to be one of those "Killing a house fly with a howitzer" things.
“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.” Bill Waterson (Calvin & Hobbes)
Hi Earlier in the thread I promised to send a link for the cobbler code I wrote... so here it is.
https://github.com/gjngeldenhuis/stuff
please ignore the very imaginative git repository name... :D
It should be lightly documented.
Regards
On 31 January 2012 16:46, Michael DeHaan michael.dehaan@gmail.com wrote:
My view is basically that you have three entities:
OS setup -> Config Setup -> Monitoring
The thing that owns OS setup also owns the way that control is transferred to the next, similar to the way that many configuration tools can also help you set up monitoring. They don't own monitoring, but they own the setup of the next tool in the chain
( cobbler: OS setup -> ) (config tool: Config Setup -> ) Monitoring
Each tool owns the arrow that sets up the next.
But to do the thing with each tool, you go to tweak the tool that can best tweak itself, rather than working through a little tiny porthole and making it harder.
Bad analogy -- suppose you have a car. You can either drive the car or you can tell someone else how to drive the car, but it will never be as immediate as driving the actual car. My first analogy would have been with analog synthesizer knobs vs MIDI commands in a Digital Audio Workstation but that might not work for everyone :)
--
Gerhardus Geldenhuis
On Tuesday 31 January 2012 05:04 PM, Nick wrote:
Ideally, I'd be able to re-deploy a new Cobbler server in exactly the same way as another node.
Puppet modules which allow automating setting up repos, profiles, systems etc. don't seem to exist on the public internet.
I've been working on this exact thing (minus the systems because i don't need them replicated everywhere) in the recent past. It's a bit uncooked atm so probably not worth sharing as is. Though i can share the general approach i've been using. For examples, for repos:
define cobbler::repo($mirror, $local = "N") {
$test_command = "/usr/bin/cobbler repo report --name=${name} > /dev/null" $params = "--name=${name} --arch=x86_64 --breed=yum --mirror=$mirror --mirror-locally=$local"
Exec { require => Cobbler::Distro[$cobbler_distros_list], }
exec { "/usr/bin/cobbler repo add $params": unless => $test_command, }
exec { "/usr/bin/cobbler repo edit $params": onlyif => $test_command, } }
... And, an actual instance:
cobbler::repo { "EPEL-x86_64": mirror => "http://download.fedora.redhat.com/pub/epel/$lsbmajdistrelease/$architecture" }
----
Similarly, for profiles:
define cobbler::profile($distro = "centos5.6-x86_64", $kickstart, $repos = "",$kopts = "") {
$ksdir = "/var/lib/cobbler/kickstarts" $test_command = "/usr/bin/cobbler profile report --name=${name}
/dev/null"
$params = " --name=${name} --distro=$distro --repos="$repos" --kickstart=$ksdir/$kickstart --kopts="$kopts""
Exec { require => Cobbler::Repo[$cobbler_repos_list], }
exec { "/usr/bin/cobbler profile add $params": unless => $test_command, }
exec { "/usr/bin/cobbler profile edit $params": onlyif => $test_command, }
} ... example usage:
cobbler::profile { "Dell-OMSA-livecd" : distro => "Dell-OMSA-livecd", kickstart => "default.ks", }
cobbler::profile { "linux-rescue" : kickstart => "default.ks", kopts => "text rescue", }
----
It's not really well tested, but in what little testing i did it seemed to work without noticeable trouble, but YMMV. I'll try to publish the whole module once it's properly done; in the meanwhile you can probably use this as a starting point. And do let me know if something doesn't work for you. And ideas for making it better are welcome, of course.
-- cheers, siddharth.
Ugh, Exec tasks. Not good, and quite limited.
If you're going to do it, you at *minimum* should write a provider so that you can express all the attributes you would want to express, and then, at that level, you have to use the XMLRPC API.
Even then though, I think Puppet is completely the wrong tool for the job -- you need Cobbler running in your infrastructure first to bring up that boot server.
If you have more than one node, you use Cobbler replicate, not Puppet, to keep configurations managed centrally.
Basically it's like requesting that you manipulate all your systems management apps with chopsticks. It just doesn't make sense. You are exposing only a few knobs and losing a lot of information, and require configuration management to come up before provisioning.
Bad.
On Tuesday, January 31, 2012 at 9:33 AM, Siddharth Deshpande wrote:
On Tuesday 31 January 2012 05:04 PM, Nick wrote:
Ideally, I'd be able to re-deploy a new Cobbler server in exactly the same way as another node.
Puppet modules which allow automating setting up repos, profiles, systems etc. don't seem to exist on the public internet.
I've been working on this exact thing (minus the systems because i don't need them replicated everywhere) in the recent past. It's a bit uncooked atm so probably not worth sharing as is. Though i can share the general approach i've been using. For examples, for repos:
define cobbler::repo($mirror, $local = "N") {
$test_command = "/usr/bin/cobbler repo report --name=${name} > /dev/null" $params = "--name=${name} --arch=x86_64 --breed=yum --mirror=$mirror --mirror-locally=$local"
Exec { require => Cobbler::Distro[$cobbler_distros_list], }
exec { "/usr/bin/cobbler repo add $params": unless => $test_command, }
exec { "/usr/bin/cobbler repo edit $params": onlyif => $test_command, } }
... And, an actual instance:
cobbler::repo { "EPEL-x86_64": mirror => "http://download.fedora.redhat.com/pub/epel/$lsbmajdistrelease/$architecture" }
Similarly, for profiles:
define cobbler::profile($distro = "centos5.6-x86_64", $kickstart, $repos = "",$kopts = "") {
$ksdir = "/var/lib/cobbler/kickstarts" $test_command = "/usr/bin/cobbler profile report --name=${name}
/dev/null"
$params = " --name=${name} --distro=$distro --repos="$repos" --kickstart=$ksdir/$kickstart --kopts="$kopts""
Exec { require => Cobbler::Repo[$cobbler_repos_list], }
exec { "/usr/bin/cobbler profile add $params": unless => $test_command, }
exec { "/usr/bin/cobbler profile edit $params": onlyif => $test_command, }
} ... example usage:
cobbler::profile { "Dell-OMSA-livecd" : distro => "Dell-OMSA-livecd", kickstart => "default.ks", }
cobbler::profile { "linux-rescue" : kickstart => "default.ks", kopts => "text rescue", }
It's not really well tested, but in what little testing i did it seemed to work without noticeable trouble, but YMMV. I'll try to publish the whole module once it's properly done; in the meanwhile you can probably use this as a starting point. And do let me know if something doesn't work for you. And ideas for making it better are welcome, of course.
-- cheers, siddharth. _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.org (mailto:cobbler@lists.fedorahosted.org) https://fedorahosted.org/mailman/listinfo/cobbler
On 31/01/12 14:33, Siddharth Deshpande wrote:
I've been working on this exact thing (minus the systems because i don't need them replicated everywhere) in the recent past. It's a bit uncooked atm so probably not worth sharing as is. Though i can share the general approach i've been using.
Thanks. I will bear this solution in mind. To be honest, I'm currently deferring tackling automating anything to do with the boot server config until other things shape up. For now I've whacked it all into a git repo.
As Michael says, doing it with execs seems likely to be fragile, although arguably easier than writing a Puppet type/provider. Another approach might be to write a Python script which sets cobbler up according to some config file and invoking that via Puppet.
Cheers,
N
cobbler@lists.fedorahosted.org