Hi,
I have a problem with an old rack unit I use in a cluster. We had a problem with the HD and had to change it and reinstall. It was done by someone new to the job (I am locked and doing everything remotely) and he had a problem with the configuration as it seems that this unit HD HAS to be configured as RAID and F33 didn't see it at first.
I am not sure what he did try, only thing he said is that it stopped at boot and he had to comment a line in fstab, but the result is that the unit starts in weird way.
The situation is this (skipping some lines): # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 931.5G 0 disk └─sda2 8:2 0 930.5G 0 part ├─fedora_node06-root 253:0 0 50G 0 lvm / ├─fedora_node06-swap 253:1 0 4G 0 lvm [SWAP] └─fedora_node06-home 253:2 0 876.5G 0 lvm /home
#fdisk -l /dev/sda Disk /dev/sda: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors Disk identifier: 0x6ccdd7fd Device Boot Start End Sectors Size Id Type /dev/sda1 * 2048 2099199 2097152 1G 83 Linux /dev/sda2 2099200 1953523711 1951424512 930.5G 8e Linux LVM
#ls -l /dev/sd* brw-rw----. 1 root disk 8, 0 Feb 2 01:00 /dev/sda brw-rw----. 1 root disk 8, 2 Feb 2 01:00 /dev/sda2
# cat /etc/fstab
/dev/mapper/fedora_node06-root / xfs defaults 0 0 #UUID=f365a320-f3d7-4c07-8bfb-f0164b9ce8c0 /boot xfs defaults 0 0 /dev/mapper/fedora_node06-home /home xfs defaults 0 0 /dev/mapper/fedora_node06-swap none swap defaults 0 0
# ls -l /boot total 0
So as I understand it sda1 is bootable and contains the kernel, linux start from there and then finds (how?) and mounts / and /home but not /boot which is empty. I would have changed the line in fstab with /dev/sda1 but it doesn't exist (!).
dmesg has this:
[ 4.369576] sd 3:0:0:0: Attached scsi generic sg1 type 0 [ 4.384096] sda: sda1 sda2
so it is there and seen but why the block device is not created? Copying the content of (sda1)boot into /boot and reinstalling grub could probably work but how?
Any advice?
Thanks,
GiP
On 3/2/21 5:04 PM, GianPiero Puccioni wrote:
so it is there and seen but why the block device is not created? Copying the content of (sda1)boot into /boot and reinstalling grub could probably work but how?
Hi GianPiero,
This is strange. I don't know why there's no device file for the first partition (/dev/sda1). Can you try "partprobe /dev/sda" to see if it creates the file? Are you sure /dev/sda1 was used for /boot ? Can you show the output of "cat /proc/cmdline"?
On 3/2/21 10:24 PM, Jorge Fábregas wrote:
On 3/2/21 5:04 PM, GianPiero Puccioni wrote:
so it is there and seen but why the block device is not created? Copying the content of (sda1)boot into /boot and reinstalling grub could probably work but how?
Hi GianPiero,
This is strange. I don't know why there's no device file for the first partition (/dev/sda1). Can you try "partprobe /dev/sda" to see if it creates the file?
Thanks all for the help
I didn't know this command (never neded it..) but yes, it did create the file now I have # ls /dev/sd* /dev/sda /dev/sda1 /dev/sda2
and "blkid" has /dev/sda1: UUID="f365a320-f3d7-4c07-8bfb-f0164b9ce8c0" BLOCK_SIZE="4096" TYPE="xfs" PARTUUID="6ccdd7fd-01" that wasn't there before.
Are you sure /dev/sda1 was used for /boot ?
I think so(it was the partion marked as bootable), and now that I can mount it I can see that all the usal /boot stuff is there. Can you
show the output of "cat /proc/cmdline"?
# cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.10.19-200.fc33.x86_64 root=/dev/mapper/fedora_node06-root ro resume=/dev/mapper/fedora_node06-swap rd.lvm.lv=fedora_node06/root rd.lvm.lv=fedora_node06/swap rhgb quiet
As Roger said the problem seems to be multipath but if there is no fix to the fact that sda1 disappeared (any idea?) I suppose the solution now could be to copy all the stuff from sda1 in /boot and recreate grub in sda ignoring sda1. Unless it needs a separate /boot, I think UEFI is disabled but I am not sure, or for the weird RAID stuff.
As I said I am doing all this remotely so I have to be careful not to break the boot.
Thanks again.
GiP
Blacklist the wwwid in multipath.conf and/or blacklist the disk type in multipath.
And/or whitelist the disks you want multipath to manage.
If all of your disks have 2 paths and are expected to have 2 paths then you can set file_multipaths to only manage devices with 2 paths, but to make multipath work right in the one path case the disks need to be listed in the bindings file.
The short term is add ,nofail to the options on the mount then it will always boot up but may not mount /boot. But make sure /boot is mounted when doing the things mentioned below.
The bios finds the data on /boot and puts what is needed to boot into memory, once that is done you really only need /boot on the machine if you are updating kernels/changing grub options/rebuilding initrds.
On Wed, Mar 3, 2021 at 4:19 AM GianPiero Puccioni gianpiero.puccioni@isc.cnr.it wrote:
On 3/2/21 10:24 PM, Jorge Fábregas wrote:
On 3/2/21 5:04 PM, GianPiero Puccioni wrote:
so it is there and seen but why the block device is not created? Copying the content of (sda1)boot into /boot and reinstalling grub could probably work but how?
Hi GianPiero,
This is strange. I don't know why there's no device file for the first partition (/dev/sda1). Can you try "partprobe /dev/sda" to see if it creates the file?
Thanks all for the help
I didn't know this command (never neded it..) but yes, it did create the file now I have # ls /dev/sd* /dev/sda /dev/sda1 /dev/sda2
and "blkid" has /dev/sda1: UUID="f365a320-f3d7-4c07-8bfb-f0164b9ce8c0" BLOCK_SIZE="4096" TYPE="xfs" PARTUUID="6ccdd7fd-01" that wasn't there before.
Are you sure /dev/sda1 was used for /boot ?
I think so(it was the partion marked as bootable), and now that I can mount it I can see that all the usal /boot stuff is there. Can you
show the output of "cat /proc/cmdline"?
# cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.10.19-200.fc33.x86_64 root=/dev/mapper/fedora_node06-root ro resume=/dev/mapper/fedora_node06-swap rd.lvm.lv=fedora_node06/root rd.lvm.lv=fedora_node06/swap rhgb quiet
As Roger said the problem seems to be multipath but if there is no fix to the fact that sda1 disappeared (any idea?) I suppose the solution now could be to copy all the stuff from sda1 in /boot and recreate grub in sda ignoring sda1. Unless it needs a separate /boot, I think UEFI is disabled but I am not sure, or for the weird RAID stuff.
As I said I am doing all this remotely so I have to be careful not to break the boot.
Thanks again.
GiP _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On 3/3/21 12:39 PM, Roger Heflin wrote:
Blacklist the wwwid in multipath.conf and/or blacklist the disk type in multipath.
And/or whitelist the disks you want multipath to manage.
If all of your disks have 2 paths and are expected to have 2 paths then you can set file_multipaths to only manage devices with 2 paths, but to make multipath work right in the one path case the disks need to be listed in the bindings file.
I am not sure I understand any of this, as shown before I have only 1 disk with sda 8:0 0 931.5G 0 disk ├─sda1 8:1 0 1G 0 part └─sda2 8:2 0 930.5G 0 part ├─fedora_node06-root 253:0 0 50G 0 lvm / ├─fedora_node06-swap 253:1 0 4G 0 lvm [SWAP] └─fedora_node06-home 253:2 0 876.5G 0 lvm /home
So I think that sda2 is managed by multipath but sda1 is not. I do not have a multipath.conf and "multipath -t" writes A LOT of stuff, while "multipath show config" gives: Mar 03 13:29:38 | /etc/multipath.conf does not exist, blacklisting all devices. Mar 03 13:29:38 | You can run "/sbin/mpathconf --enable" to create Mar 03 13:29:38 | /etc/multipath.conf. See man mpathconf(8) for more details Mar 03 13:29:38 | DM multipath kernel driver not loaded
Not sure what this means or what is the proper way to do what you said.
The short term is add ,nofail to the options on the mount then it will always boot up but may not mount /boot. But make sure /boot is mounted when doing the things mentioned below.
The bios finds the data on /boot and puts what is needed to boot into memory, once that is done you really only need /boot on the machine if you are updating kernels/changing grub options/rebuilding initrds.
Yes that was why I did not things stay the way they are, after all it works. One possibility could be to use an rc.local (the systemd equivalent or a proper service) to run "partprobe" and "mount" at start, would this work?
GiP
On Wed, Mar 3, 2021 at 4:19 AM GianPiero Puccioni gianpiero.puccioni@isc.cnr.it wrote:
On 3/2/21 10:24 PM, Jorge Fábregas wrote:
On 3/2/21 5:04 PM, GianPiero Puccioni wrote:
so it is there and seen but why the block device is not created? Copying the content of (sda1)boot into /boot and reinstalling grub could probably work but how?
Hi GianPiero,
This is strange. I don't know why there's no device file for the first partition (/dev/sda1). Can you try "partprobe /dev/sda" to see if it creates the file?
Thanks all for the help
I didn't know this command (never neded it..) but yes, it did create the file now I have # ls /dev/sd* /dev/sda /dev/sda1 /dev/sda2
and "blkid" has /dev/sda1: UUID="f365a320-f3d7-4c07-8bfb-f0164b9ce8c0" BLOCK_SIZE="4096" TYPE="xfs" PARTUUID="6ccdd7fd-01" that wasn't there before.
Are you sure /dev/sda1 was used for /boot ?
I think so(it was the partion marked as bootable), and now that I can mount it I can see that all the usal /boot stuff is there. Can you
show the output of "cat /proc/cmdline"?
# cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.10.19-200.fc33.x86_64 root=/dev/mapper/fedora_node06-root ro resume=/dev/mapper/fedora_node06-swap rd.lvm.lv=fedora_node06/root rd.lvm.lv=fedora_node06/swap rhgb quiet
As Roger said the problem seems to be multipath but if there is no fix to the fact that sda1 disappeared (any idea?) I suppose the solution now could be to copy all the stuff from sda1 in /boot and recreate grub in sda ignoring sda1. Unless it needs a separate /boot, I think UEFI is disabled but I am not sure, or for the weird RAID stuff.
As I said I am doing all this remotely so I have to be careful not to break the boot.
Thanks again.
GiP _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
That looks like multipath is blacklisted.
you might to a lsinitrd | grep -i multipath and make sure it is not in the initrd with a config file.
multipath is the only service I have seen that actually deletes partition mappings, but it is possible that some of the other dm* stuff might (dm-raid maybe?)..
On Wed, Mar 3, 2021 at 6:35 AM GianPiero Puccioni gianpiero.puccioni@isc.cnr.it wrote:
On 3/3/21 12:39 PM, Roger Heflin wrote:
Blacklist the wwwid in multipath.conf and/or blacklist the disk type in multipath.
And/or whitelist the disks you want multipath to manage.
If all of your disks have 2 paths and are expected to have 2 paths then you can set file_multipaths to only manage devices with 2 paths, but to make multipath work right in the one path case the disks need to be listed in the bindings file.
I am not sure I understand any of this, as shown before I have only 1 disk with sda 8:0 0 931.5G 0 disk ├─sda1 8:1 0 1G 0 part └─sda2 8:2 0 930.5G 0 part ├─fedora_node06-root 253:0 0 50G 0 lvm / ├─fedora_node06-swap 253:1 0 4G 0 lvm [SWAP] └─fedora_node06-home 253:2 0 876.5G 0 lvm /home
So I think that sda2 is managed by multipath but sda1 is not. I do not have a multipath.conf and "multipath -t" writes A LOT of stuff, while "multipath show config" gives: Mar 03 13:29:38 | /etc/multipath.conf does not exist, blacklisting all devices. Mar 03 13:29:38 | You can run "/sbin/mpathconf --enable" to create Mar 03 13:29:38 | /etc/multipath.conf. See man mpathconf(8) for more details Mar 03 13:29:38 | DM multipath kernel driver not loaded
Not sure what this means or what is the proper way to do what you said.
The short term is add ,nofail to the options on the mount then it will always boot up but may not mount /boot. But make sure /boot is mounted when doing the things mentioned below.
The bios finds the data on /boot and puts what is needed to boot into memory, once that is done you really only need /boot on the machine if you are updating kernels/changing grub options/rebuilding initrds.
Yes that was why I did not things stay the way they are, after all it works. One possibility could be to use an rc.local (the systemd equivalent or a proper service) to run "partprobe" and "mount" at start, would this work?
GiP
On Wed, Mar 3, 2021 at 4:19 AM GianPiero Puccioni gianpiero.puccioni@isc.cnr.it wrote:
On 3/2/21 10:24 PM, Jorge Fábregas wrote:
On 3/2/21 5:04 PM, GianPiero Puccioni wrote:
so it is there and seen but why the block device is not created? Copying the content of (sda1)boot into /boot and reinstalling grub could probably work but how?
Hi GianPiero,
This is strange. I don't know why there's no device file for the first partition (/dev/sda1). Can you try "partprobe /dev/sda" to see if it creates the file?
Thanks all for the help
I didn't know this command (never neded it..) but yes, it did create the file now I have # ls /dev/sd* /dev/sda /dev/sda1 /dev/sda2
and "blkid" has /dev/sda1: UUID="f365a320-f3d7-4c07-8bfb-f0164b9ce8c0" BLOCK_SIZE="4096" TYPE="xfs" PARTUUID="6ccdd7fd-01" that wasn't there before.
Are you sure /dev/sda1 was used for /boot ?
I think so(it was the partion marked as bootable), and now that I can mount it I can see that all the usal /boot stuff is there. Can you
show the output of "cat /proc/cmdline"?
# cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.10.19-200.fc33.x86_64 root=/dev/mapper/fedora_node06-root ro resume=/dev/mapper/fedora_node06-swap rd.lvm.lv=fedora_node06/root rd.lvm.lv=fedora_node06/swap rhgb quiet
As Roger said the problem seems to be multipath but if there is no fix to the fact that sda1 disappeared (any idea?) I suppose the solution now could be to copy all the stuff from sda1 in /boot and recreate grub in sda ignoring sda1. Unless it needs a separate /boot, I think UEFI is disabled but I am not sure, or for the weird RAID stuff.
As I said I am doing all this remotely so I have to be careful not to break the boot.
Thanks again.
GiP _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- GianPiero Puccioni |Istituto dei Sistemi Complessi-CNR gianpiero.puccioni@isc.cnr.it |Via Madonna del Piano, 10 T:+39 0555226682 |50019 Sesto F. (Firenze) ITALY _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
and the multipath delete code is in udev so if you search udev then the code doing it should be in there. And I would think anyone else's partition mapping deletes would be in there.
On Wed, Mar 3, 2021 at 11:22 AM Roger Heflin rogerheflin@gmail.com wrote:
That looks like multipath is blacklisted.
you might to a lsinitrd | grep -i multipath and make sure it is not in the initrd with a config file.
multipath is the only service I have seen that actually deletes partition mappings, but it is possible that some of the other dm* stuff might (dm-raid maybe?)..
On Wed, Mar 3, 2021 at 6:35 AM GianPiero Puccioni gianpiero.puccioni@isc.cnr.it wrote:
On 3/3/21 12:39 PM, Roger Heflin wrote:
Blacklist the wwwid in multipath.conf and/or blacklist the disk type in multipath.
And/or whitelist the disks you want multipath to manage.
If all of your disks have 2 paths and are expected to have 2 paths then you can set file_multipaths to only manage devices with 2 paths, but to make multipath work right in the one path case the disks need to be listed in the bindings file.
I am not sure I understand any of this, as shown before I have only 1 disk with sda 8:0 0 931.5G 0 disk ├─sda1 8:1 0 1G 0 part └─sda2 8:2 0 930.5G 0 part ├─fedora_node06-root 253:0 0 50G 0 lvm / ├─fedora_node06-swap 253:1 0 4G 0 lvm [SWAP] └─fedora_node06-home 253:2 0 876.5G 0 lvm /home
So I think that sda2 is managed by multipath but sda1 is not. I do not have a multipath.conf and "multipath -t" writes A LOT of stuff, while "multipath show config" gives: Mar 03 13:29:38 | /etc/multipath.conf does not exist, blacklisting all devices. Mar 03 13:29:38 | You can run "/sbin/mpathconf --enable" to create Mar 03 13:29:38 | /etc/multipath.conf. See man mpathconf(8) for more details Mar 03 13:29:38 | DM multipath kernel driver not loaded
Not sure what this means or what is the proper way to do what you said.
The short term is add ,nofail to the options on the mount then it will always boot up but may not mount /boot. But make sure /boot is mounted when doing the things mentioned below.
The bios finds the data on /boot and puts what is needed to boot into memory, once that is done you really only need /boot on the machine if you are updating kernels/changing grub options/rebuilding initrds.
Yes that was why I did not things stay the way they are, after all it works. One possibility could be to use an rc.local (the systemd equivalent or a proper service) to run "partprobe" and "mount" at start, would this work?
GiP
On Wed, Mar 3, 2021 at 4:19 AM GianPiero Puccioni gianpiero.puccioni@isc.cnr.it wrote:
On 3/2/21 10:24 PM, Jorge Fábregas wrote:
On 3/2/21 5:04 PM, GianPiero Puccioni wrote:
so it is there and seen but why the block device is not created? Copying the content of (sda1)boot into /boot and reinstalling grub could probably work but how?
Hi GianPiero,
This is strange. I don't know why there's no device file for the first partition (/dev/sda1). Can you try "partprobe /dev/sda" to see if it creates the file?
Thanks all for the help
I didn't know this command (never neded it..) but yes, it did create the file now I have # ls /dev/sd* /dev/sda /dev/sda1 /dev/sda2
and "blkid" has /dev/sda1: UUID="f365a320-f3d7-4c07-8bfb-f0164b9ce8c0" BLOCK_SIZE="4096" TYPE="xfs" PARTUUID="6ccdd7fd-01" that wasn't there before.
Are you sure /dev/sda1 was used for /boot ?
I think so(it was the partion marked as bootable), and now that I can mount it I can see that all the usal /boot stuff is there. Can you
show the output of "cat /proc/cmdline"?
# cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.10.19-200.fc33.x86_64 root=/dev/mapper/fedora_node06-root ro resume=/dev/mapper/fedora_node06-swap rd.lvm.lv=fedora_node06/root rd.lvm.lv=fedora_node06/swap rhgb quiet
As Roger said the problem seems to be multipath but if there is no fix to the fact that sda1 disappeared (any idea?) I suppose the solution now could be to copy all the stuff from sda1 in /boot and recreate grub in sda ignoring sda1. Unless it needs a separate /boot, I think UEFI is disabled but I am not sure, or for the weird RAID stuff.
As I said I am doing all this remotely so I have to be careful not to break the boot.
Thanks again.
GiP _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- GianPiero Puccioni |Istituto dei Sistemi Complessi-CNR gianpiero.puccioni@isc.cnr.it |Via Madonna del Piano, 10 T:+39 0555226682 |50019 Sesto F. (Firenze) ITALY _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On 3/3/21 6:22 PM, Roger Heflin wrote:
That looks like multipath is blacklisted.
you might to a lsinitrd | grep -i multipath and make sure it is not in the initrd with a config file.
multipath is the only service I have seen that actually deletes partition mappings, but it is possible that some of the other dm* stuff might (dm-raid maybe?)..
Yes there is something
# lsinitrd | grep -i multipath -rw-r--r-- 1 root root 15120 Oct 23 17:30 usr/lib/modules/5.10.19-200.fc33.x86_64/kernel/drivers/md/dm-multipath.ko.xz
I have looked into udev and there is this: /usr/lib/udev/rules.d/62-multipath.rules but I don't understand what is in it.
GiP
the dm-multipath should be ok, so long as multipathd is not there.
grep -i del_part_nodes /lib/udev/rules.d/*
shows the multipath delete rule. I am on 32 and don't see anything else using it.
if you cat the rule with the del_part_nodes code you could see if anything else on 33 uses it.
On Wed, Mar 3, 2021 at 12:17 PM GianPiero Puccioni gianpiero.puccioni@isc.cnr.it wrote:
On 3/3/21 6:22 PM, Roger Heflin wrote:
That looks like multipath is blacklisted.
you might to a lsinitrd | grep -i multipath and make sure it is not in the initrd with a config file.
multipath is the only service I have seen that actually deletes partition mappings, but it is possible that some of the other dm* stuff might (dm-raid maybe?)..
Yes there is something
# lsinitrd | grep -i multipath -rw-r--r-- 1 root root 15120 Oct 23 17:30 usr/lib/modules/5.10.19-200.fc33.x86_64/kernel/drivers/md/dm-multipath.ko.xz
I have looked into udev and there is this: /usr/lib/udev/rules.d/62-multipath.rules but I don't understand what is in it.
GiP _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On 3/3/21 11:50 PM, Roger Heflin wrote:
the dm-multipath should be ok, so long as multipathd is not there.
grep -i del_part_nodes /lib/udev/rules.d/*
shows the multipath delete rule. I am on 32 and don't see anything else using it.
if you cat the rule with the del_part_nodes code you could see if anything else on 33 uses it.
del_part_nodes appears only in /lib/udev/rules.d/68-del-part-nodes.rules which has at the start:
# To suppress this, use the kernel parameter "dont_del_part_nodes", # or create an udev rule file that sets ENV{DONT_DEL_PART_NODES}="1".
As I cannot change the kernel par, I added a (found online) /lib/udev/rules.d/00-dont-del-part-nodes.rules with
SUBSYSTEM=="block", KERNEL=="sd*|dasd*|rbd*", ACTION=="add|change", \ ENV{DONT_DEL_PART_NODES}="1"
and rebooted but nothing changed.
GiP
The only command that I know of that can delete a partition mapping like this is partx. On mine partx is not in the initrd so it should not be happening there.
So: cd /usr/sbin mv partx partx.exe then create a file:
#!/bin/bash logger "$(/usr/bin/date) kartx run with options $*" /usr/sbin/partx.exe $*
Test with running "partx --help" and make sure a message shows up in messages file from the logger command.
If that finds something doing the delete then add a "ps -elf > somefile" to the script before you execute partx.exe
I have had to use this trick a number of times to figure out what code path was doing something odd.
You might also run "dmsetup table" and see what is referencing device 8:X (sda is :0, sda1: 1 and so on).
On Thu, Mar 4, 2021 at 5:16 AM GianPiero Puccioni gianpiero.puccioni@isc.cnr.it wrote:
On 3/3/21 11:50 PM, Roger Heflin wrote:
the dm-multipath should be ok, so long as multipathd is not there.
grep -i del_part_nodes /lib/udev/rules.d/*
shows the multipath delete rule. I am on 32 and don't see anything else using it.
if you cat the rule with the del_part_nodes code you could see if anything else on 33 uses it.
del_part_nodes appears only in /lib/udev/rules.d/68-del-part-nodes.rules which has at the start:
# To suppress this, use the kernel parameter "dont_del_part_nodes", # or create an udev rule file that sets ENV{DONT_DEL_PART_NODES}="1".
As I cannot change the kernel par, I added a (found online) /lib/udev/rules.d/00-dont-del-part-nodes.rules with
SUBSYSTEM=="block", KERNEL=="sd*|dasd*|rbd*", ACTION=="add|change", \ ENV{DONT_DEL_PART_NODES}="1"
and rebooted but nothing changed.
GiP _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On 3/4/21 1:56 PM, Roger Heflin wrote:
The only command that I know of that can delete a partition mapping like this is partx. On mine partx is not in the initrd so it should not be happening there.
So: cd /usr/sbin mv partx partx.exe then create a file:
#!/bin/bash logger "$(/usr/bin/date) kartx run with options $*" /usr/sbin/partx.exe $*
I imagine this file is to be named partx and chmod +x
Test with running "partx --help" and make sure a message shows up in messages file from the logger command.
Yes I got Mar 4 14:06:55 node06 root[1339]: Thu Mar 4 02:06:55 PM CET 2021 kartx run with options --help
If that finds something doing the delete then add a "ps -elf > somefile" to the script before you execute partx.exe
No, unfortunately a reboot didn't leave any trace.
I have had to use this trick a number of times to figure out what code path was doing something odd.
It's a nice trick, I'll have to remember that...
You might also run "dmsetup table" and see what is referencing device 8:X (sda is :0, sda1:# dmsetup table 1 and so on).
fedora_node06-swap: 0 8388608 linear 8:2 2048 fedora_node06-root: 0 104857600 linear 8:2 8390656 fedora_node06-home: 0 1838170112 linear 8:2 113248256 still a mistery...
GiP
On 03/03/2021 05:04, GianPiero Puccioni wrote:
Hi,
I have a problem with an old rack unit I use in a cluster. We had a problem with the HD and had to change it and reinstall. It was done by someone new to the job (I am locked and doing everything remotely) and he had a problem with the configuration as it seems that this unit HD HAS to be configured as RAID and F33 didn't see it at first.
I am not sure what he did try, only thing he said is that it stopped at boot and he had to comment a line in fstab, but the result is that the unit starts in weird way.
The situation is this (skipping some lines): # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 931.5G 0 disk └─sda2 8:2 0 930.5G 0 part ├─fedora_node06-root 253:0 0 50G 0 lvm / ├─fedora_node06-swap 253:1 0 4G 0 lvm [SWAP] └─fedora_node06-home 253:2 0 876.5G 0 lvm /home
#fdisk -l /dev/sda Disk /dev/sda: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors Disk identifier: 0x6ccdd7fd Device Boot Start End Sectors Size Id Type /dev/sda1 * 2048 2099199 2097152 1G 83 Linux /dev/sda2 2099200 1953523711 1951424512 930.5G 8e Linux LVM
#ls -l /dev/sd* brw-rw----. 1 root disk 8, 0 Feb 2 01:00 /dev/sda brw-rw----. 1 root disk 8, 2 Feb 2 01:00 /dev/sda2
# cat /etc/fstab
/dev/mapper/fedora_node06-root / xfs defaults 0 0 #UUID=f365a320-f3d7-4c07-8bfb-f0164b9ce8c0 /boot xfs defaults 0 0 /dev/mapper/fedora_node06-home /home xfs defaults 0 0 /dev/mapper/fedora_node06-swap none swap defaults 0 0
# ls -l /boot total 0
So as I understand it sda1 is bootable and contains the kernel, linux start from there and then finds (how?) and mounts / and /home but not /boot which is empty. I would have changed the line in fstab with /dev/sda1 but it doesn't exist (!).
dmesg has this:
[ 4.369576] sd 3:0:0:0: Attached scsi generic sg1 type 0 [ 4.384096] sda: sda1 sda2
so it is there and seen but why the block device is not created? Copying the content of (sda1)boot into /boot and reinstalling grub could probably work but how?
Any advice?
man fstab
Notice there are multiple ways to identify a disk partition. Including the device name as well as LABEL=, PARTUUID= and PARTLABEL= and UUID=
Run.... "blkid" and you'd probably see....
/dev/sda1: UUID="f365a320-f3d7-4c07-8bfb-f0164b9ce8c0" BLOCK_SIZE= etc.
So, uncomment the line
#UUID=f365a320-f3d7-4c07-8bfb-f0164b9ce8c0 /boot xfs defaults 0 0
in your fstab and then
mount /boot
if you are running multipath then here is what happened, multipath started up and attempted to manage sda and so it deleted sda1, then once it gets to sda2 it figures out the vg is live and online and it figures out it cannot manage it, so stops. No code exists apparently to go back and put back sda1 in some versions.
partprobe will work, but without partprobe sda1 won't exist, and the uuid links will be pointing to something that does not exist.
On Tue, Mar 2, 2021 at 4:02 PM Ed Greshko ed.greshko@greshko.com wrote:
On 03/03/2021 05:04, GianPiero Puccioni wrote:
Hi,
I have a problem with an old rack unit I use in a cluster. We had a problem with the HD and had to change it and reinstall. It was done by someone new to the job (I am locked and doing everything remotely) and he had a problem with the configuration as it seems that this unit HD HAS to be configured as RAID and F33 didn't see it at first.
I am not sure what he did try, only thing he said is that it stopped at boot and he had to comment a line in fstab, but the result is that the unit starts in weird way.
The situation is this (skipping some lines): # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 931.5G 0 disk └─sda2 8:2 0 930.5G 0 part ├─fedora_node06-root 253:0 0 50G 0 lvm / ├─fedora_node06-swap 253:1 0 4G 0 lvm [SWAP] └─fedora_node06-home 253:2 0 876.5G 0 lvm /home
#fdisk -l /dev/sda Disk /dev/sda: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors Disk identifier: 0x6ccdd7fd Device Boot Start End Sectors Size Id Type /dev/sda1 * 2048 2099199 2097152 1G 83 Linux /dev/sda2 2099200 1953523711 1951424512 930.5G 8e Linux LVM
#ls -l /dev/sd* brw-rw----. 1 root disk 8, 0 Feb 2 01:00 /dev/sda brw-rw----. 1 root disk 8, 2 Feb 2 01:00 /dev/sda2
# cat /etc/fstab
/dev/mapper/fedora_node06-root / xfs defaults 0 0 #UUID=f365a320-f3d7-4c07-8bfb-f0164b9ce8c0 /boot xfs defaults 0 0 /dev/mapper/fedora_node06-home /home xfs defaults 0 0 /dev/mapper/fedora_node06-swap none swap defaults 0 0
# ls -l /boot total 0
So as I understand it sda1 is bootable and contains the kernel, linux start from there and then finds (how?) and mounts / and /home but not /boot which is empty. I would have changed the line in fstab with /dev/sda1 but it doesn't exist (!).
dmesg has this:
[ 4.369576] sd 3:0:0:0: Attached scsi generic sg1 type 0 [ 4.384096] sda: sda1 sda2
so it is there and seen but why the block device is not created? Copying the content of (sda1)boot into /boot and reinstalling grub could probably work but how?
Any advice?
man fstab
Notice there are multiple ways to identify a disk partition. Including the device name as well as LABEL=, PARTUUID= and PARTLABEL= and UUID=
Run.... "blkid" and you'd probably see....
/dev/sda1: UUID="f365a320-f3d7-4c07-8bfb-f0164b9ce8c0" BLOCK_SIZE= etc.
So, uncomment the line
#UUID=f365a320-f3d7-4c07-8bfb-f0164b9ce8c0 /boot xfs defaults 0 0
in your fstab and then
mount /boot
-- People who believe they don't make mistakes have already made one.
users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-
Ed Greshko -
GianPiero Puccioni -
Jorge Fábregas -
Roger Heflin