[cobbler] How to debug...

Michael Tiernan mtiernan at MIT.EDU
Mon Aug 4 15:28:53 UTC 2014


On 8/2/14 5:57 AM, André Gemünd wrote:
> thats a bit strange, as you should at least get some error message.
> Was that a full cobbler sync output you posted? There should e.g. at
> least be a 'generating PXE configuration files' output. Which
> version of Cobbler are you using? cobbler distro list, cobbler
> profile list & cobbler system list all output the correct items and
> give you no error message? Nothing in the logs? The code you are
> looking for is in pxegen.py btw.

Yeah, a bit strange should be my middle name these days.

I've attached a number of outputs to this email so you can see all the
strangeness for yourself. It has gotten to the point that I'm using a
really limited configuration right now to get to the bottom of this
problem. For lots of reasons (internal) I've had to reload the machine
multiple times so this is a very clean installation. I've been using
simple scripts to populate the key components such as profiles, and
systems so not a lot has been changed.

The thing is, the DHCP part works fine. The tftp service is responding
properly. Everything seems to be working fine *except* that when the
sync is called, it doesn't seem to recognize that it has a machine
enabled for netboot and therefore has to generate the PXE files needed
to make it happen.

Opinions are welcome and thank you to all for your help.

Here's that output:
::::::::::::::
sync-out.txt
::::::::::::::
task started: 2014-08-04_094050_sync
task started (id=Sync, time=Mon Aug  4 09:40:50 2014)
running pre-sync triggers
cleaning trees
removing: /var/www/cobbler/images/scilin_6.4-x86_64
removing: /var/lib/tftpboot/pxelinux.cfg/default
removing: /var/lib/tftpboot/grub/images
removing: /var/lib/tftpboot/grub/efidefault
removing: /var/lib/tftpboot/grub/grub-x86.efi
removing: /var/lib/tftpboot/grub/grub-x86_64.efi
removing: /var/lib/tftpboot/s390x/profile_list
copying bootloaders
copying: /var/lib/cobbler/loaders/pxelinux.0 -> /var/lib/tftpboot/pxelinux.0
copying: /var/lib/cobbler/loaders/menu.c32 -> /var/lib/tftpboot/menu.c32
copying: /var/lib/cobbler/loaders/yaboot -> /var/lib/tftpboot/yaboot
copying: /var/lib/cobbler/loaders/grub-x86_64.efi ->
/var/lib/tftpboot/grub/grub-x86_64.efi
copying: /var/lib/cobbler/loaders/grub-x86.efi ->
/var/lib/tftpboot/grub/grub-x86.efi
copying files for distro: scilin_6.4-x86_64
trying hardlink
/var/www/cobbler/ks_mirror/scilin_6.4/images/pxeboot/vmlinuz ->
/var/www/cobbler/images/scilin_6.4-x86_64/vmlinuz
trying hardlink
/var/www/cobbler/ks_mirror/scilin_6.4/images/pxeboot/initrd.img ->
/var/www/cobbler/images/scilin_6.4-x86_64/initrd.img
Writing template files for scilin_6.4-x86_64
rendering DHCP files
generating /etc/dhcp/dhcpd.conf
rendering TFTPD files
generating /etc/xinetd.d/tftp
cleaning link caches
running post-sync triggers
running python triggers from /var/lib/cobbler/triggers/sync/post/*
running python trigger cobbler.modules.sync_post_restart_services
running: dhcpd -t -q
received on stdout:
received on stderr:
running: service dhcpd restart
received on stdout: Shutting down dhcpd: [  OK  ]
Starting dhcpd: [  OK  ]

received on stderr:
running shell triggers from /var/lib/cobbler/triggers/sync/post/*
running python triggers from /var/lib/cobbler/triggers/change/*
running python trigger cobbler.modules.scm_track
running shell triggers from /var/lib/cobbler/triggers/change/*
*** TASK COMPLETE ***

::::::::::::::
/etc/cobbler/settings
::::::::::::::
allow_duplicate_hostnames: 0
allow_duplicate_ips: 0
allow_duplicate_macs: 0
allow_dynamic_settings: 0
always_write_dhcp_entries: 1
anamon_enabled: 0
auth_token_expiration: 3600
authn_pam_service: login
bind_chroot_path: ''
bind_master: 127.0.0.1
build_reporting_email: [root at localhost]
build_reporting_enabled: 0
build_reporting_ignorelist: ['']
build_reporting_sender: ''
build_reporting_smtp_server: localhost
build_reporting_subject: ''
build_reporting_to_address: ''
buildisodir: /var/cache/cobbler/buildiso
cheetah_import_whitelist: [random, re, time]
client_use_https: 0
client_use_localhost: 0
cobbler_master: ''
consoles: /var/consoles
createrepo_flags: -c cache -s sha
default_deployment_method: ssh
default_kickstart: /var/lib/cobbler/kickstarts/default.ks
default_name_servers: []
default_name_servers_search: []
default_ownership: [admin]
default_password_crypted: xxxxxxxxxxxxxxxxxxxxxxxxxxxx
default_template_type: cheetah
default_virt_bridge: xenbr0
default_virt_disk_driver: raw
default_virt_file_size: 5
default_virt_ram: 512
default_virt_type: xenpv
enable_gpxe: 0
enable_menu: 1
func_auto_setup: 0
func_master: overlord.example.org
http_port: 80
isc_set_host_name: 0
iso_template_dir: /etc/cobbler/iso
kerberos_realm: EXAMPLE.COM
kernel_options: {ksdevice: bootif, lang: ' ', text: null}
kernel_options_s390x: {RUNKS: 1, ip: false, ramdisk_size: 40000, ro:
null, root: /dev/ram0,
  vnc: null}
ldap_anonymous_bind: 1
ldap_base_dn: DC=example,DC=com
ldap_management_default_type: authconfig
ldap_port: 389
ldap_search_bind_dn: ''
ldap_search_passwd: ''
ldap_search_prefix: uid=
ldap_server: ldap.example.com
ldap_tls: 1
ldap_tls_cacertfile: ''
ldap_tls_certfile: ''
ldap_tls_keyfile: ''
manage_dhcp: 1
manage_dns: 0
manage_forward_zones: []
manage_reverse_zones: []
manage_rsync: 0
manage_tftp: 1
manage_tftpd: 1
mgmt_classes: []
mgmt_parameters: {from_cobbler: 1}
next_server: 10.13.15.10
power_management_default_type: ipmitool
power_template_dir: /etc/cobbler/power
puppet_auto_setup: 0
puppet_parameterized_classes: 1
puppet_server: puppet
puppet_version: 2
puppetca_path: /usr/bin/puppet
pxe_just_once: 1
pxe_template_dir: /etc/cobbler/pxe
redhat_management_key: ''
redhat_management_permissive: 0
redhat_management_server: xmlrpc.rhn.redhat.com
redhat_management_type: 'off'
register_new_installs: 0
remove_old_puppet_certs_automatically: 0
replicate_repo_rsync_options: -avzH
replicate_rsync_options: -avzH
reposync_flags: -l -n -d
restart_dhcp: 1
restart_dns: 1
restart_xinetd: 1
run_install_triggers: 1
scm_track_enabled: 0
scm_track_mode: git
serializer_pretty_json: 0
server: 10.13.15.10
sign_puppet_certs_automatically: 0
signature_path: /var/lib/cobbler/distro_signatures.json
signature_url: http://www.cobblerd.org/signatures/latest.json
snippetsdir: /var/lib/cobbler/snippets
template_remote_kickstarts: 0
virt_auto_boot: 1
webdir: /var/www/cobbler
xmlrpc_port: 25151
yum_distro_priority: 1
yum_post_install_mirror: 1
yumdownloader_flags: --resolve

::::::::::::::
/etc/xinetd.d/tftp
::::::::::::::
# default: off
# description: The tftp server serves files using the trivial file
transfer \
#       protocol.  The tftp protocol is often used to boot diskless \
#       workstations, download configuration files to network-aware
printers, \
#       and to start the installation process for some operating systems.
# server_args = -B 1380 -v -s -v
service tftp
{
        disable                 = no
        socket_type             = dgram
        protocol                = udp
        wait                    = yes
        user                    = nobody
        server                  = /usr/sbin/tftpd.py
        server_args             = -vv -v
        per_source              = 11
        cps                     = 100 2
        flags                   = IPv4
}

::::::::::::::
/etc/dhcp/dhcpd.conf
::::::::::::::
# ******************************************************************
# Cobbler managed dhcpd.conf file
# generated from cobbler dhcp.conf template (Mon Aug  4 13:40:51 2014)
# Do NOT make changes to /etc/dhcpd.conf. Instead, make your changes
# in /etc/cobbler/dhcp.template, as /etc/dhcpd.conf will be
# overwritten.
# ******************************************************************

ddns-update-style interim;

allow booting;
allow bootp;

ignore client-updates;
set vendorclass = option vendor-class-identifier;

option pxe-system-type code 93 = unsigned integer 16;

#==============================
# NETMASK=255.252.0.0
# PREFIX=14
# BROADCAST=10.15.255.255
# NETWORK=10.12.0.0
#==============================

subnet 10.12.0.0 netmask 255.252.0.0 {
#	option routers             192.168.1.5;
#	option domain-name-servers 192.168.1.1;
#	option subnet-mask         255.255.255.0;
#	range dynamic-bootp        10.14.0.1 10.14.1.255;
	range                      10.15.128.1 10.15.255.254;
	default-lease-time         30;
	max-lease-time             7200;
	next-server                10.13.15.10;

     class "pxeclients" {
          match if substring (option vendor-class-identifier, 0, 9) =
"PXEClient";
          if option pxe-system-type = 00:02 {
                  filename "ia64/elilo.efi";
          } else if option pxe-system-type = 00:06 {
                  filename "grub/grub-x86.efi";
          } else if option pxe-system-type = 00:07 {
                  filename "grub/grub-x86_64.efi";
          } else {
                  filename "pxelinux.0";
          }
     }

}

# group for Cobbler DHCP tag: default
group {
    host generic2 {
        hardware ethernet F4:6D:04:37:26:03;
        fixed-address 10.13.15.102;
        option host-name "hprcf102.cmsaf.mit.edu";
        option subnet-mask 255.252.0.0;
        option routers 18.12.0.1;
        filename "/pxelinux.0";
        next-server 10.13.15.10;
    }
    host generic1 {
        hardware ethernet F4:6D:04:37:26:91;
        fixed-address 18.12.15.102;
        option host-name "hprcf102.cmsaf.mit.edu";
        option subnet-mask 255.255.240.0;
        option routers 18.12.0.1;
        filename "/pxelinux.0";
        next-server 10.13.15.10;
    }
}

group {
	host ipmi_F46D043898CD {
		hardware ethernet F4:6D:04:38:98:CD;
		fixed-address 10.12.15.102;
	}
}

::::::::::::::
/var/log/cobbler/cobbler.log
::::::::::::::
Mon Aug  4 09:39:00 2014 - DEBUG | REMOTE CLI Authorized; user(?)
Mon Aug  4 09:39:00 2014 - INFO | REMOTE start_task(Sync);
event_id(2014-08-04_093900_sync); user(?)
Mon Aug  4 09:39:01 2014 - INFO | sync
Mon Aug  4 09:40:26 2014 - DEBUG | REMOTE CLI Authorized; user(?)
Mon Aug  4 09:40:26 2014 - INFO | REMOTE start_task(Sync);
event_id(2014-08-04_094026_sync); user(?)
Mon Aug  4 09:40:27 2014 - INFO | sync
Mon Aug  4 09:40:50 2014 - DEBUG | REMOTE CLI Authorized; user(?)
Mon Aug  4 09:40:50 2014 - INFO | REMOTE start_task(Sync);
event_id(2014-08-04_094050_sync); user(?)
Mon Aug  4 09:40:51 2014 - INFO | sync

::::::::::::::
/bin/ls -AlR /var/lib/tftpboot/
::::::::::::::
/var/lib/tftpboot/:
total 328
drwxr-xr-x 3 root root   4096 Aug  4 09:40 boot
drwxr-xr-x 2 root root   4096 Jul 25 14:40 etc
drwxr-xr-x 2 root root   4096 Aug  4 09:40 grub
drwxr-xr-x 2 root root   4096 Jul 18 01:57 images
-rw-r--r-- 2 root root  26268 Jun  7  2013 memdisk
-rw-r--r-- 1 root root  54964 Aug  4 09:40 menu.c32
drwxr-xr-x 2 root root   4096 Jul 25 14:40 ppc
-rw-r--r-- 1 root root  16794 Aug  4 09:40 pxelinux.0
drwxr-xr-x 2 root root   4096 Aug  4 09:40 pxelinux.cfg
drwxr-xr-x 2 root root   4096 Aug  4 09:40 s390x
-rw-r--r-- 1 root root 198192 Aug  4 09:40 yaboot

/var/lib/tftpboot/boot:
total 4
drwxr-xr-x 2 root root 4096 Aug  4 09:40 grub

/var/lib/tftpboot/boot/grub:
total 4
-rw-r--r-- 1 root root 23 Aug  4 09:40 menu.lst

/var/lib/tftpboot/etc:
total 0

/var/lib/tftpboot/grub:
total 476
-rw-r--r-- 1 root root    695 Aug  4 09:40 efidefault
-rw-r--r-- 1 root root 243679 Aug  4 09:40 grub-x86_64.efi
-rw-r--r-- 1 root root 237224 Aug  4 09:40 grub-x86.efi
lrwxrwxrwx 1 root root      9 Jul 25 14:40 images -> ../images

/var/lib/tftpboot/images:
total 0

/var/lib/tftpboot/ppc:
total 0

/var/lib/tftpboot/pxelinux.cfg:
total 4
-rw-r--r-- 1 root root 1033 Aug  4 09:40 default

/var/lib/tftpboot/s390x:
total 0
-rw-r--r-- 1 root root 0 Aug  4 09:40 profile_list

::::::::::::::
cobbler report
::::::::::::::
distros:
==========
Name                           : scilin_6.4-x86_64
Architecture                   : x86_64
TFTP Boot Files                : {}
Breed                          : redhat
Comment                        : Scientific Linux V6.4 x86_64
Fetchable Files                : {}
Initrd                         :
/var/www/cobbler/ks_mirror/scilin_6.4/images/pxeboot/initrd.img
Kernel                         :
/var/www/cobbler/ks_mirror/scilin_6.4/images/pxeboot/vmlinuz
Kernel Options                 : {}
Kernel Options (Post Install)  : {}
Kickstart Metadata             : {'osver_major': '6', 'tree':
'http://@@http_server@@/cblr/links/scilin_6.4-x86_64', 'osver_minor': '4'}
Management Classes             : []
OS Version                     : rhel6
Owners                         : ['admin']
Red Hat Management Key         : <<inherit>>
Red Hat Management Server      : <<inherit>>
Template Files                 : {}


profiles:
==========
Name                           : scilin_6.4-x86_64
TFTP Boot Files                : {}
Comment                        :
DHCP Tag                       : default
Distribution                   : scilin_6.4-x86_64
Enable gPXE?                   : 0
Enable PXE Menu?               : 1
Fetchable Files                : {}
Kernel Options                 : {}
Kernel Options (Post Install)  : {}
Kickstart                      : /var/lib/cobbler/kickstarts/sample_end.ks
Kickstart Metadata             : {}
Management Classes             : []
Management Parameters          : <<inherit>>
Name Servers                   : []
Name Servers Search Path       : []
Owners                         : ['admin']
Parent Profile                 :
Proxy                          :
Red Hat Management Key         : <<inherit>>
Red Hat Management Server      : <<inherit>>
Repos                          : []
Server Override                : <<inherit>>
Template Files                 : {}
Virt Auto Boot                 : 1
Virt Bridge                    : xenbr0
Virt CPUs                      : 1
Virt Disk Driver Type          : raw
Virt File Size(GB)             : 5
Virt Path                      :
Virt RAM (MB)                  : 512
Virt Type                      : kvm

Name                           : T2_Worker
TFTP Boot Files                : {}
Comment                        :
DHCP Tag                       : default
Distribution                   : scilin_6.4-x86_64
Enable gPXE?                   : 0
Enable PXE Menu?               : True
Fetchable Files                : {}
Kernel Options                 : {}
Kernel Options (Post Install)  : {}
Kickstart                      : /var/lib/cobbler/kickstarts/T2_Worker.cfg
Kickstart Metadata             : {'osrel_minor': '4', 'osrel_major': '6'}
Management Classes             : []
Management Parameters          : <<inherit>>
Name Servers                   : ['18.70.0.160', '18.71.0.151', '18.72.0.3']
Name Servers Search Path       : ['cmsaf.mit.edu.', 'lns.mit.edu.',
'mit.edu.']
Owners                         : ['admin']
Parent Profile                 :
Proxy                          :
Red Hat Management Key         : <<inherit>>
Red Hat Management Server      : <<inherit>>
Repos                          : ['sl-mit', 'sl-mitcms']
Server Override                : <<inherit>>
Template Files                 : {}
Virt Auto Boot                 : 1
Virt Bridge                    : xenbr0
Virt CPUs                      : 1
Virt Disk Driver Type          : raw
Virt File Size(GB)             : 5
Virt Path                      :
Virt RAM (MB)                  : 512
Virt Type                      : xenpv


systems:
==========
Name                           : hprcf102
TFTP Boot Files                : {}
Comment                        :
Enable gPXE?                   : 0
Fetchable Files                : {}
Gateway                        : 18.12.0.1
Hostname                       : hprcf102.cmsaf.mit.edu
Image                          :
IPv6 Autoconfiguration         : False
IPv6 Default Device            :
Kernel Options                 : {}
Kernel Options (Post Install)  : {}
Kickstart                      : <<inherit>>
Kickstart Metadata             : {}
LDAP Enabled                   : False
LDAP Management Type           : authconfig
Management Classes             : <<inherit>>
Management Parameters          : <<inherit>>
Monit Enabled                  : False
Name Servers                   : []
Name Servers Search Path       : []
Netboot Enabled                : True
Owners                         : ['admin']
Power Management Address       :
Power Management ID            :
Power Management Password      :
Power Management Type          : ipmitool
Power Management Username      :
Profile                        : T2_Worker
Proxy                          : <<inherit>>
Red Hat Management Key         : <<inherit>>
Red Hat Management Server      : <<inherit>>
Repos Enabled                  : True
Server Override                : <<inherit>>
Status                         : development
Template Files                 : {}
Virt Auto Boot                 : <<inherit>>
Virt CPUs                      : <<inherit>>
Virt Disk Driver Type          : <<inherit>>
Virt File Size(GB)             : <<inherit>>
Virt Path                      : <<inherit>>
Virt PXE Boot                  : 0
Virt RAM (MB)                  : <<inherit>>
Virt Type                      : <<inherit>>
Interface =====                : eth1
Bonding Opts                   :
Bridge Opts                    :
CNAMES                         : []
DHCP Tag                       :
DNS Name                       :
Per-Interface Gateway          :
Master Interface               :
Interface Type                 :
IP Address                     : 18.12.15.102
IPv6 Address                   :
IPv6 Default Gateway           :
IPv6 MTU                       :
IPv6 Prefix                    :
IPv6 Secondaries               : []
IPv6 Static Routes             : []
MAC Address                    : F4:6D:04:37:26:91
Management Interface           : False
MTU                            :
Subnet Mask                    : 255.255.240.0
Static                         : True
Static Routes                  : []
Virt Bridge                    :
Interface =====                : eth0
Bonding Opts                   :
Bridge Opts                    :
CNAMES                         : []
DHCP Tag                       :
DNS Name                       :
Per-Interface Gateway          :
Master Interface               :
Interface Type                 :
IP Address                     : 10.13.15.102
IPv6 Address                   :
IPv6 Default Gateway           :
IPv6 MTU                       :
IPv6 Prefix                    :
IPv6 Secondaries               : []
IPv6 Static Routes             : []
MAC Address                    : F4:6D:04:37:26:03
Management Interface           : True
MTU                            :
Subnet Mask                    : 255.252.0.0
Static                         : True
Static Routes                  : []
Virt Bridge                    :


repos:
==========

Name                           : sl
Apt Components (apt only)      :
Apt Dist Names (apt only)      :
Arch                           : x86_64
Breed                          : yum
Comment                        : Scientific Linux 6.4 - x86_64 - Base
Createrepo Flags               : <<inherit>>
Environment Variables          : {}
Keep Updated                   : True
Mirror                         :
http://ftp.scientificlinux.org/linux/scientific/6.4/x86_64/os/
Mirror locally                 : True
Owners                         : ['admin']
Priority                       : 899
RPM List                       : []
Yum Options                    : {}

Name                           : sl-mit
Apt Components (apt only)      : []
Apt Dist Names (apt only)      : []
Arch                           : x86_64
Breed                          : yum
Comment                        : MIT SciLinux 6.4 - x86_64 - Base
Createrepo Flags               : <<inherit>>
Environment Variables          : {}
Keep Updated                   : True
Mirror                         :
http://kickstart.cmsaf.mit.edu/sl-mitcms/6.4/x86_64/os
Mirror locally                 : True
Owners                         : ['admin']
Priority                       : 99
RPM List                       : []
Yum Options                    : {}

images:
==========

mgmtclasses:
==========

packages:
==========

files:
==========


-- 
  << MCT >>   Michael C Tiernan xmpp:mtiernan at mit.edu +1 (617) 324-9173
  MIT - Laboratory for Nuclear Science - http://www.lns.mit.edu
  High Perf Research Computing Facility at The Bates Linear Accelerator
    Please avoid sending me MS-Word or MS-PowerPoint attachments.
    See http://www.gnu.org/philosophy/no-word-attachments.html


More information about the cobbler mailing list