Hello all -
Hopefully you all can help me out. I am fairly new to Cobbler/PXE booting in general. I had an interesting issue show up this morning while doing some testing of various kickstart files that I am trying to wrap my head around.
For the past week or so I have been testing various kickstarts on two different systems by editing the kickstart, taking the system out of cobbler, and re-adding it so it is like new and it rebuilds the system via PXE. It has been working like a charm.
Until this morning.. I was working with one system (let's call it sysA) which is in a subnet 172.19.10.x. For some reason when this system boots up now it gets a DHCP address but then tftp times out and eventually boots from local disk instead of reinstalling.
I have another system (let's call it sysB) which is in subnet 172.19.34.x. It is still working fine. Both of these systems are being served from the same cobbler server which has two NICs, one on each subnet.
There were no changes to the cobbler server (besides changing kickstart templates slightly and snippets) or the network recently which is really making me confused as to why this has happened?
Any ideas on where I can look?
I have tested tftp commands from sysA (after booting from local disk) to the cobbler server which work just fine.
The only things I see in logs that are concerning are in the httpd error_logs:
[Wed Feb 22 08:55:07 2012] [error] possible cheetah template error [Wed Feb 22 08:58:32 2012] [error] possible cheetah template error [Wed Feb 22 14:34:03 2012] [error] /usr/lib64/python2.6/site-packages/mod_python/importer.py:32: DeprecationWarning: the md5 module is deprecated; use hashlib instead [Wed Feb 22 14:34:03 2012] [error] import md5
However I have not been able to find much in terms of solutions to these errors.
[root@cobbler ~]# rpm -qa|grep cobbler cobbler-web-2.0.11-2.el6.noarch cobbler-2.0.11-2.el6.noarch [root@cobbler ~]# cat /etc/redhat-release CentOS release 6.2 (Final) [root@cobbler ~]#
Any direction and any outputs needed please let me know!
Thanks in advance.
Daniel
On Wednesday 22 February 2012 10:04 PM, Daniel Lollman wrote:
Until this morning.. I was working with one system (let’s call it sysA) which is in a subnet 172.19.10.x. For some reason when this system boots up now it gets a DHCP address but then tftp times out and eventually boots from local disk instead of reinstalling.
At the very least, from sysA (or another machine on the same layer-2 network, you should be able to use tftp on the command line and get the pxe boot file from the PXE server. Just in case you haven't tried that already.
-- siddharth.
Hello Siddharth –
I tried that and it seems to be working ok!
[root@sysA ~]# tftp 172.19.10.200 tftp> get /pxelinux.0 tftp> get /pxelinux.cfg/01-00-00-00-00-00-00 tftp> get //images/CentOS6-x86_64/vmlinuz tftp> quit [root@sysA ~]#
Not sure if that helps anything or if I should try to get more.
[root@sysA ~]# ls -lah -rw-r--r-- 1 root root 15K Feb 22 12:15 pxelinux.0 -rw-r--r-- 1 root root 264 Feb 22 12:16 01-00-00-00-00-00-00 [root@sysA ~]# ls -lah|grep vml -rw-r--r-- 1 root root 3.8M Feb 22 12:16 vmlinuz [root@sysA ~]#
Thanks again.
Daniel
From: cobbler-bounces@lists.fedorahosted.org [mailto:cobbler-bounces@lists.fedorahosted.org] On Behalf Of Siddharth Deshpande Sent: Wednesday, February 22, 2012 12:06 PM To: cobbler mailing list Subject: Re: [cobbler] Testing cobbler install - pxe tftp open timeout
On Wednesday 22 February 2012 10:04 PM, Daniel Lollman wrote: Until this morning.. I was working with one system (let’s call it sysA) which is in a subnet 172.19.10.x. For some reason when this system boots up now it gets a DHCP address but then tftp times out and eventually boots from local disk instead of reinstalling.
At the very least, from sysA (or another machine on the same layer-2 network, you should be able to use tftp on the command line and get the pxe boot file from the PXE server. Just in case you haven't tried that already.
-- siddharth.
Google "tftp timeout" and you'll get a lot of hits… there's nothing cobbler specific here, most likely, just network related, and things can be intermittent.
Particularly read the timeout info here: http://fedoraproject.org/wiki/Anaconda/NetworkIssues
The portfast issue, in particular, came up a lot.
On Wednesday, February 22, 2012 at 12:24 PM, Daniel Lollman wrote:
Hello Siddharth –
I tried that and it seems to be working ok!
[root@sysA ~]# tftp 172.19.10.200
tftp> get /pxelinux.0
tftp> get /pxelinux.cfg/01-00-00-00-00-00-00
tftp> get //images/CentOS6-x86_64/vmlinuz
tftp> quit
[root@sysA ~]#
Not sure if that helps anything or if I should try to get more.
[root@sysA ~]# ls -lah
-rw-r--r-- 1 root root 15K Feb 22 12:15 pxelinux.0
-rw-r--r-- 1 root root 264 Feb 22 12:16 01-00-00-00-00-00-00
[root@sysA ~]# ls -lah|grep vml
-rw-r--r-- 1 root root 3.8M Feb 22 12:16 vmlinuz
[root@sysA ~]#
Thanks again.
Daniel
From: cobbler-bounces@lists.fedorahosted.org [mailto:cobbler-bounces@lists.fedorahosted.org] On Behalf Of Siddharth Deshpande Sent: Wednesday, February 22, 2012 12:06 PM To: cobbler mailing list Subject: Re: [cobbler] Testing cobbler install - pxe tftp open timeout
On Wednesday 22 February 2012 10:04 PM, Daniel Lollman wrote:
Until this morning.. I was working with one system (let’s call it sysA) which is in a subnet 172.19.10.x. For some reason when this system boots up now it gets a DHCP address but then tftp times out and eventually boots from local disk instead of reinstalling.
At the very least, from sysA (or another machine on the same layer-2 network, you should be able to use tftp on the command line and get the pxe boot file from the PXE server. Just in case you haven't tried that already.
-- siddharth.
cobbler mailing list cobbler@lists.fedorahosted.org (mailto:cobbler@lists.fedorahosted.org) https://fedorahosted.org/mailman/listinfo/cobbler
Do you get these cheetah template errors from sysB also ???
On Wed, Feb 22, 2012 at 6:57 PM, Michael DeHaan michael.dehaan@gmail.comwrote:
Google "tftp timeout" and you'll get a lot of hits… there's nothing cobbler specific here, most likely, just network related, and things can be intermittent.
Particularly read the timeout info here: http://fedoraproject.org/wiki/Anaconda/NetworkIssues
The portfast issue, in particular, came up a lot.
On Wednesday, February 22, 2012 at 12:24 PM, Daniel Lollman wrote:
Hello Siddharth –****
I tried that and it seems to be working ok!****
[root@sysA ~]# tftp 172.19.10.200****
tftp> get /pxelinux.0****
tftp> get /pxelinux.cfg/01-00-00-00-00-00-00****
tftp> get //images/CentOS6-x86_64/vmlinuz****
tftp> quit****
[root@sysA ~]#****
Not sure if that helps anything or if I should try to get more.****
[root@sysA ~]# ls -lah****
-rw-r--r-- 1 root root 15K Feb 22 12:15 pxelinux.0****
-rw-r--r-- 1 root root 264 Feb 22 12:16 01-00-00-00-00-00-00****
[root@sysA ~]# ls -lah|grep vml****
-rw-r--r-- 1 root root 3.8M Feb 22 12:16 vmlinuz****
[root@sysA ~]#****
Thanks again.****
Daniel****
*From:* cobbler-bounces@lists.fedorahosted.org [ mailto:cobbler-bounces@lists.fedorahosted.orgcobbler-bounces@lists.fedorahosted.org] *On Behalf Of *Siddharth Deshpande *Sent:* Wednesday, February 22, 2012 12:06 PM *To:* cobbler mailing list *Subject:* Re: [cobbler] Testing cobbler install - pxe tftp open timeout** **
On Wednesday 22 February 2012 10:04 PM, Daniel Lollman wrote: ****
Until this morning.. I was working with one system (let’s call it sysA) which is in a subnet 172.19.10.x. For some reason when this system boots up now it gets a DHCP address but then tftp times out and eventually boots from local disk instead of reinstalling.****
At the very least, from sysA (or another machine on the same layer-2 network, you should be able to use tftp on the command line and get the pxe boot file from the PXE server. Just in case you haven't tried that already.
-- siddharth.**** _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
cobbler mailing list cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
Hi -
Jorgen:
Yes, I do get this on sysB as well but it goes through the install (or did until I hit this latest issue below).
All:
Well, I am making some progress. Got through the network issue, for whatever reason it appears that commenting out the next_server option in the DHCP template allows it to continue just fine which is good.
Now I seem to be running into a permission problem with downloading the kickstart file, and I have not been able to gather a good enough Google search to figure out where to track this down.
[root@sys1 ~]# wget http://172.19.10.200/cblr/svc/op/ks/system/sys-dev --2012-02-22 14:22:13-- http://172.19.10.200/cblr/svc/op/ks/system/sys-dev Connecting to 172.19.10.200:80... connected. HTTP request sent, awaiting response... 500 SERVER ERROR 2012-02-22 14:22:13 ERROR 500: SERVER ERROR.
This is from a server on the same subnet.
Some troubleshooting steps I have done that do not seem to have worked:
I looked at the template files to see if anything permission-wise was different. Using one of the standard kickstarts that comes with Cobbler from what I remember work just fine (changed sysB to use a simple profile/default template). I matched up those permissions (in /var/lib/cobbler/kickstarts/) but to no avail.
From Apache:
172.19.10.10 - - [22/Feb/2012:19:22:13 +0000] "GET /cblr/svc/op/ks/system/sys-dev HTTP/1.0" 500 1595 "-" "Wget/1.12 (linux-gnu)"
Apache is running as apache, and all of the files in /var/www/cobbler* are owned by apache:apache. The /var/lib/cobbler* files are owned by root:root.
I am still trying to track down what is causing this but I cannot seem to find it so far.
All templates/snippets/profiles have only been created/edited in the web UI, never by shell or otherwise.
Daniel
From: cobbler-bounces@lists.fedorahosted.org [mailto:cobbler-bounces@lists.fedorahosted.org] On Behalf Of Jörgen Maas Sent: Wednesday, February 22, 2012 1:49 PM To: cobbler mailing list Subject: Re: [cobbler] Testing cobbler install - pxe tftp open timeout
Do you get these cheetah template errors from sysB also ??? On Wed, Feb 22, 2012 at 6:57 PM, Michael DeHaan <michael.dehaan@gmail.commailto:michael.dehaan@gmail.com> wrote: Google "tftp timeout" and you'll get a lot of hits... there's nothing cobbler specific here, most likely, just network related, and things can be intermittent.
Particularly read the timeout info here: http://fedoraproject.org/wiki/Anaconda/NetworkIssues
The portfast issue, in particular, came up a lot.
On Wednesday, February 22, 2012 at 12:24 PM, Daniel Lollman wrote:
Hello Siddharth -
I tried that and it seems to be working ok!
[root@sysA ~]# tftp 172.19.10.200
tftp> get /pxelinux.0
tftp> get /pxelinux.cfg/01-00-00-00-00-00-00
tftp> get //images/CentOS6-x86_64/vmlinuz
tftp> quit
[root@sysA ~]#
Not sure if that helps anything or if I should try to get more.
[root@sysA ~]# ls -lah
-rw-r--r-- 1 root root 15K Feb 22 12:15 pxelinux.0
-rw-r--r-- 1 root root 264 Feb 22 12:16 01-00-00-00-00-00-00
[root@sysA ~]# ls -lah|grep vml
-rw-r--r-- 1 root root 3.8M Feb 22 12:16 vmlinuz
[root@sysA ~]#
Thanks again.
Daniel
From: cobbler-bounces@lists.fedorahosted.orgmailto:cobbler-bounces@lists.fedorahosted.org [mailto:cobbler-bounces@lists.fedorahosted.org] On Behalf Of Siddharth Deshpande Sent: Wednesday, February 22, 2012 12:06 PM To: cobbler mailing list Subject: Re: [cobbler] Testing cobbler install - pxe tftp open timeout
On Wednesday 22 February 2012 10:04 PM, Daniel Lollman wrote:
Until this morning.. I was working with one system (let's call it sysA) which is in a subnet 172.19.10.x. For some reason when this system boots up now it gets a DHCP address but then tftp times out and eventually boots from local disk instead of reinstalling.
At the very least, from sysA (or another machine on the same layer-2 network, you should be able to use tftp on the command line and get the pxe boot file from the PXE server. Just in case you haven't tried that already.
-- siddharth. _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.orgmailto:cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
_______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.orgmailto:cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
-- Grtz, Jörgen Maas
Try cobbler system getks --name=foo
If it's failing to render a kickstart that would explain your Internal Server Error.
--Michael
On Wednesday, February 22, 2012 at 2:32 PM, Daniel Lollman wrote:
Hi –
Jorgen:
Yes, I do get this on sysB as well but it goes through the install (or did until I hit this latest issue below).
All:
Well, I am making some progress. Got through the network issue, for whatever reason it appears that commenting out the next_server option in the DHCP template allows it to continue just fine which is good.
Now I seem to be running into a permission problem with downloading the kickstart file, and I have not been able to gather a good enough Google search to figure out where to track this down.
[root@sys1 ~]# wget http://172.19.10.200/cblr/svc/op/ks/system/sys-dev
--2012-02-22 14:22:13-- http://172.19.10.200/cblr/svc/op/ks/system/sys-dev
Connecting to 172.19.10.200:80... connected.
HTTP request sent, awaiting response... 500 SERVER ERROR
2012-02-22 14:22:13 ERROR 500: SERVER ERROR.
This is from a server on the same subnet.
Some troubleshooting steps I have done that do not seem to have worked:
I looked at the template files to see if anything permission-wise was different. Using one of the standard kickstarts that comes with Cobbler from what I remember work just fine (changed sysB to use a simple profile/default template). I matched up those permissions (in /var/lib/cobbler/kickstarts/) but to no avail.
From Apache:
172.19.10.10 - - [22/Feb/2012:19:22:13 +0000] "GET /cblr/svc/op/ks/system/sys-dev HTTP/1.0" 500 1595 "-" "Wget/1.12 (linux-gnu)"
Apache is running as apache, and all of the files in /var/www/cobbler* are owned by apache:apache. The /var/lib/cobbler* files are owned by root:root.
I am still trying to track down what is causing this but I cannot seem to find it so far.
All templates/snippets/profiles have only been created/edited in the web UI, never by shell or otherwise.
Daniel
From: cobbler-bounces@lists.fedorahosted.org [mailto:cobbler-bounces@lists.fedorahosted.org] On Behalf Of Jörgen Maas Sent: Wednesday, February 22, 2012 1:49 PM To: cobbler mailing list Subject: Re: [cobbler] Testing cobbler install - pxe tftp open timeout
Do you get these cheetah template errors from sysB also ???
On Wed, Feb 22, 2012 at 6:57 PM, Michael DeHaan <michael.dehaan@gmail.com (mailto:michael.dehaan@gmail.com)> wrote:
Google "tftp timeout" and you'll get a lot of hits… there's nothing cobbler specific here, most likely, just network related, and things can be intermittent.
Particularly read the timeout info here: http://fedoraproject.org/wiki/Anaconda/NetworkIssues
The portfast issue, in particular, came up a lot.
On Wednesday, February 22, 2012 at 12:24 PM, Daniel Lollman wrote:
Hello Siddharth –
I tried that and it seems to be working ok!
[root@sysA ~]# tftp 172.19.10.200
tftp> get /pxelinux.0
tftp> get /pxelinux.cfg/01-00-00-00-00-00-00
tftp> get //images/CentOS6-x86_64/vmlinuz
tftp> quit
[root@sysA ~]#
Not sure if that helps anything or if I should try to get more.
[root@sysA ~]# ls -lah
-rw-r--r-- 1 root root 15K Feb 22 12:15 pxelinux.0
-rw-r--r-- 1 root root 264 Feb 22 12:16 01-00-00-00-00-00-00
[root@sysA ~]# ls -lah|grep vml
-rw-r--r-- 1 root root 3.8M Feb 22 12:16 vmlinuz
[root@sysA ~]#
Thanks again.
Daniel
From: cobbler-bounces@lists.fedorahosted.org (mailto:cobbler-bounces@lists.fedorahosted.org) [mailto:cobbler-bounces@lists.fedorahosted.org] On Behalf Of Siddharth Deshpande Sent: Wednesday, February 22, 2012 12:06 PM To: cobbler mailing list Subject: Re: [cobbler] Testing cobbler install - pxe tftp open timeout
On Wednesday 22 February 2012 10:04 PM, Daniel Lollman wrote:
Until this morning.. I was working with one system (let’s call it sysA) which is in a subnet 172.19.10.x. For some reason when this system boots up now it gets a DHCP address but then tftp times out and eventually boots from local disk instead of reinstalling.
At the very least, from sysA (or another machine on the same layer-2 network, you should be able to use tftp on the command line and get the pxe boot file from the PXE server. Just in case you haven't tried that already.
-- siddharth.
cobbler mailing list
cobbler@lists.fedorahosted.org (mailto:cobbler@lists.fedorahosted.org)
cobbler mailing list cobbler@lists.fedorahosted.org (mailto:cobbler@lists.fedorahosted.org) https://fedorahosted.org/mailman/listinfo/cobbler
-- Grtz, Jörgen Maas
cobbler mailing list cobbler@lists.fedorahosted.org (mailto:cobbler@lists.fedorahosted.org) https://fedorahosted.org/mailman/listinfo/cobbler
Hi Michael –
That helped tremendously.
Easily helped me track it down to one line in one of my Snippets causing the issue…. And here I thought it was something on the permission level!
All is working fine now, thanks for all the help as simple as it was.
Daniel
From: cobbler-bounces@lists.fedorahosted.org [mailto:cobbler-bounces@lists.fedorahosted.org] On Behalf Of Michael DeHaan Sent: Wednesday, February 22, 2012 2:34 PM To: cobbler mailing list Subject: Re: [cobbler] Testing cobbler install - pxe tftp open timeout
Try cobbler system getks --name=foo
If it's failing to render a kickstart that would explain your Internal Server Error.
--Michael
On Wednesday, February 22, 2012 at 2:32 PM, Daniel Lollman wrote:
Hi –
Jorgen:
Yes, I do get this on sysB as well but it goes through the install (or did until I hit this latest issue below).
All:
Well, I am making some progress. Got through the network issue, for whatever reason it appears that commenting out the next_server option in the DHCP template allows it to continue just fine which is good.
Now I seem to be running into a permission problem with downloading the kickstart file, and I have not been able to gather a good enough Google search to figure out where to track this down.
[root@sys1 ~]# wget http://172.19.10.200/cblr/svc/op/ks/system/sys-dev
--2012-02-22 14:22:13-- http://172.19.10.200/cblr/svc/op/ks/system/sys-dev
Connecting to 172.19.10.200:80... connected.
HTTP request sent, awaiting response... 500 SERVER ERROR
2012-02-22 14:22:13 ERROR 500: SERVER ERROR.
This is from a server on the same subnet.
Some troubleshooting steps I have done that do not seem to have worked:
I looked at the template files to see if anything permission-wise was different. Using one of the standard kickstarts that comes with Cobbler from what I remember work just fine (changed sysB to use a simple profile/default template). I matched up those permissions (in /var/lib/cobbler/kickstarts/) but to no avail.
From Apache:
172.19.10.10 - - [22/Feb/2012:19:22:13 +0000] "GET /cblr/svc/op/ks/system/sys-dev HTTP/1.0" 500 1595 "-" "Wget/1.12 (linux-gnu)"
Apache is running as apache, and all of the files in /var/www/cobbler* are owned by apache:apache. The /var/lib/cobbler* files are owned by root:root.
I am still trying to track down what is causing this but I cannot seem to find it so far.
All templates/snippets/profiles have only been created/edited in the web UI, never by shell or otherwise.
Daniel
From: cobbler-bounces@lists.fedorahosted.org [mailto:cobbler-bounces@lists.fedorahosted.org] On Behalf Of Jörgen Maas Sent: Wednesday, February 22, 2012 1:49 PM To: cobbler mailing list Subject: Re: [cobbler] Testing cobbler install - pxe tftp open timeout
Do you get these cheetah template errors from sysB also ???
On Wed, Feb 22, 2012 at 6:57 PM, Michael DeHaan <michael.dehaan@gmail.commailto:michael.dehaan@gmail.com> wrote:
Google "tftp timeout" and you'll get a lot of hits… there's nothing cobbler specific here, most likely, just network related, and things can be intermittent.
Particularly read the timeout info here: http://fedoraproject.org/wiki/Anaconda/NetworkIssues
The portfast issue, in particular, came up a lot.
On Wednesday, February 22, 2012 at 12:24 PM, Daniel Lollman wrote:
Hello Siddharth –
I tried that and it seems to be working ok!
[root@sysA ~]# tftp 172.19.10.200
tftp> get /pxelinux.0
tftp> get /pxelinux.cfg/01-00-00-00-00-00-00
tftp> get //images/CentOS6-x86_64/vmlinuz
tftp> quit
[root@sysA ~]#
Not sure if that helps anything or if I should try to get more.
[root@sysA ~]# ls -lah
-rw-r--r-- 1 root root 15K Feb 22 12:15 pxelinux.0
-rw-r--r-- 1 root root 264 Feb 22 12:16 01-00-00-00-00-00-00
[root@sysA ~]# ls -lah|grep vml
-rw-r--r-- 1 root root 3.8M Feb 22 12:16 vmlinuz
[root@sysA ~]#
Thanks again.
Daniel
From: cobbler-bounces@lists.fedorahosted.orgmailto:cobbler-bounces@lists.fedorahosted.org [mailto:cobbler-bounces@lists.fedorahosted.org] On Behalf Of Siddharth Deshpande Sent: Wednesday, February 22, 2012 12:06 PM To: cobbler mailing list Subject: Re: [cobbler] Testing cobbler install - pxe tftp open timeout
On Wednesday 22 February 2012 10:04 PM, Daniel Lollman wrote:
Until this morning.. I was working with one system (let’s call it sysA) which is in a subnet 172.19.10.x. For some reason when this system boots up now it gets a DHCP address but then tftp times out and eventually boots from local disk instead of reinstalling.
At the very least, from sysA (or another machine on the same layer-2 network, you should be able to use tftp on the command line and get the pxe boot file from the PXE server. Just in case you haven't tried that already.
-- siddharth.
_______________________________________________
cobbler mailing list
cobbler@lists.fedorahosted.orgmailto:cobbler@lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/cobbler
_______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.orgmailto:cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
-- Grtz, Jörgen Maas _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.orgmailto:cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
cobbler@lists.fedorahosted.org