Using Fedora-x86_64-19-20130627-sda (ami-10cce555) on us-west-1b, I updated to the recent kernel. Now it won't boot. It seems initrd is fried. Anyway to get in to make it to boot from a previous kernel?
Or is this another argument for making more frequent snapshots?
BTW, this kernel worked with another instance in asia. Perhaps it's an aws issue in us-west?
sean
syslog:
[J Booting 'Fedora (3.11.2-201.fc19.x86_64) 19 (Schrödinger’s Cat)'
root (hd0)
Filesystem type is ext2fs, using whole disk
kernel /boot/vmlinuz-3.11.2-201.fc19.x86_64 ro root=UUID=e3577648-5703-459d-b55
7-b3947684f8ab serial=tty0 console=ttyS0,115200n8 console=hvc0 console=tty1 LAN
G=en_US.UTF-8
initrd /boot/initramfs-3.11.2-201.fc19.x86_64.img
block error -1 for op 0
Error 3: Bad or corrupt data while decompressing file
Press any key to continue...Xen Minimal OS! start_info: 0xac4000(VA) nr_pages: 0x26700 shared_inf: 0x8c534000(MA) pt_base: 0xac7000(VA) nr_pt_frames: 0x9 mfn_list: 0x990000(VA) mod_start: 0x0(VA) mod_len: 0 flags: 0x0 cmd_line: root=/dev/sda1 ro 4 stack: 0x94f860-0x96f860 MM: Init _text: 0x0(VA) _etext: 0x5ff6d(VA) _erodata: 0x78000(VA) _edata: 0x80b00(VA) stack start: 0x94f860(VA) _end: 0x98fe68(VA) start_pfn: ad3 max_pfn: 26700 Mapping memory range 0xc00000 - 0x26700000 setting 0x0-0x78000 readonly skipped 0x1000 MM: Initialise page allocator for c01000(c01000)-26700000(26700000) MM: done Demand map pfns at 26701000-2026701000. Heap resides at 2026702000-4026702000. Initialising timer interface Initialising console ... done. gnttab_table mapped at 0x26701000. Initialising scheduler Thread "Idle": pointer: 0x2026702010, stack: 0x26640000 Initialising xenbus Thread "xenstore": pointer: 0x20267027c0, stack: 0x26650000 Dummy main: start_info=0x96f960 Thread "main": pointer: 0x2026702f70, stack: 0x26660000 "main" "root=/dev/sda1" "ro" "4" vbd 2049 is hd0 ******************* BLKFRONT for device/vbd/2049 **********
backend at /local/domain/0/backend/vbd/544/2049 Failed to read /local/domain/0/backend/vbd/544/2049/feature-barrier. Failed to read /local/domain/0/backend/vbd/544/2049/feature-flush-cache. 4194304 sectors of 512 bytes ************************** Thread "kbdfront": pointer: 0x2026830010, stack: 0x26680000 ******************* FBFRONT for device/vfb/0 **********
******************* KBDFRONT for device/vkbd/0 **********
Failed to read device/vfb/0/backend-id. Failed to read device/vkbd/0/backend-id. Error ENOENT when reading the backend path device/vkbd/0/backend Thread "kbdfront" exited. Error ENOENT when reading the backend path device/vfb/0/backend
On Tue, Oct 01, 2013 at 01:56:01PM -0400, Sean Darcy wrote:
Using Fedora-x86_64-19-20130627-sda (ami-10cce555) on us-west-1b, I updated to the recent kernel. Now it won't boot. It seems initrd is fried. Anyway to get in to make it to boot from a previous kernel?
So, here's what to do in ten actually-pretty-easy steps:
1. Stop (but do not terminate) the broken instance, 2. Launch a new temporary instance in the same zone. (Or use an existing one that's working.) 3. Detach the root volume from the broken instance 4. Attach it to the working instance as /dev/sdf (or whatever) 5. In the working instance, mount the filesystem 6. Find and fix the problem (possibly edit the /boot/grub/menu.lst?) 7. Unmount the filesystem 8. Detach from the working instance 9. Reattach to the hopefully-fixed original instance 10. Boot
As step four and a half, you may actually want to run fsck on the volume.
In general, you should store your data on a separate volume so that if this happens you just replace the machine and go on without worrying about it.
On Tue, Oct 1, 2013 at 12:08 PM, Matthew Miller mattdm@fedoraproject.org wrote:
happens you just replace the machine and go on without worrying about it.
Another step, after that that may be helpful to you is an md5sum check of the initrd
David Duncan http://about.me/davdunc Registered Linux User #279425 http://counter.li.org
On 10/01/2013 03:08 PM, Matthew Miller wrote:
On Tue, Oct 01, 2013 at 01:56:01PM -0400, Sean Darcy wrote:
Using Fedora-x86_64-19-20130627-sda (ami-10cce555) on us-west-1b, I updated to the recent kernel. Now it won't boot. It seems initrd is fried. Anyway to get in to make it to boot from a previous kernel?
So, here's what to do in ten actually-pretty-easy steps:
- Stop (but do not terminate) the broken instance,
- Launch a new temporary instance in the same zone. (Or use an existing one that's working.)
- Detach the root volume from the broken instance
- Attach it to the working instance as /dev/sdf (or whatever)
- In the working instance, mount the filesystem
- Find and fix the problem (possibly edit the /boot/grub/menu.lst?)
- Unmount the filesystem
- Detach from the working instance
- Reattach to the hopefully-fixed original instance
- Boot
As step four and a half, you may actually want to run fsck on the volume.
In general, you should store your data on a separate volume so that if this happens you just replace the machine and go on without worrying about it.
Finally got around to try to fix this. Did steps 1-4, using an F19 cloud image. But then:
fdisk -l /dev/xvdf
Disk /dev/xvdf: 2147 MB, 2147483648 bytes, 4194304 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x20ac7dda
This doesn't look like a partition table Probably you selected the wrong device.
Device Boot Start End Blocks Id System /dev/xvdf1 ? 3224498923 3657370039 216435558+ 7 HPFS/NTFS/exFAT /dev/xvdf2 ? 3272020941 5225480974 976730017 16 Hidden FAT16 /dev/xvdf3 ? 0 0 0 6f Unknown /dev/xvdf4 50200576 974536369 462167897 0 Empty
Partition table entries are not in disk order
fsck /dev/xvdf1 fsck from util-linux 2.23.2 e2fsck 1.42.7 (21-Jan-2013) fsck.ext2: No such file or directory while trying to open /dev/xvdf1 Possibly non-existent device?
mount /dev/xvdf1 /mnt/xvdf1 mount: special device /dev/xvdf1 does not exist
Really odd.
sean
fdisk /dev/xvdf Welcome to fdisk (util-linux 2.23.1).
Changes will remain in memory only, until you decide to write them. Be careful before using the write command.
Command (m for help): p
Disk /dev/xvdf: 2147 MB, 2147483648 bytes, 4194304 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x20ac7dda
This doesn't look like a partition table Probably you selected the wrong device.
Device Boot Start End Blocks Id System /dev/xvdf1 ? 3224498923 3657370039 216435558+ 7 HPFS/NTFS/exFAT /dev/xvdf2 ? 3272020941 5225480974 976730017 16 Hidden FAT16 /dev/xvdf3 ? 0 0 0 6f Unknown /dev/xvdf4 50200576 974536369 462167897 0 Empty
fsck /dev/xvdf1 fsck from util-linux 2.23.1 e2fsck 1.42.7 (21-Jan-2013) fsck.ext2: No such file or directory while trying to open /dev/xvdf1 Possibly non-existent device?
mount /dev/xvdf1 xvdf1 mount: special device /dev/xvdf1 does not exist
On Sun, Oct 13, 2013 at 01:54:37PM -0400, Sean Darcy wrote:
$ fdisk -l /dev/xvdf Disk /dev/xvdf: 2147 MB, 2147483648 bytes, 4194304 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x20ac7dda This doesn't look like a partition table
Try just mounting it, not as partitioned but as a "partition" itself.
mount /dev/xvdf1 /mnt/xvdf1 mount: special device /dev/xvdf1 does not exist
mount /dev/xvdf /mnt