I just upgraded my machine from F29 to F30. Now, whenever I install a new kernel, the new kernel does not automatically become the default. In other words, when I reboot, the previous kernel is still chosen by grub2.
I can manually choose the new kernel in the grub2 menu, at which point it _does_ become the new default. I don't wind up at the "grub>" prompt, so I think grub2 itself is fine. It is just that the grubenv is not updated when the new kernel is installed.
The machine has UEFI, but the system boots using the legacy BIOS compatibility layer. I know that the boot mechanism changed a bit for F30, but I'm not sure where to look to identify the cause of this problem. It doesn't seem to be the same issue as described in BZ 1652806.
Steve
On Sun, May 5, 2019 at 10:22 AM Steven A. Falco stevenfalco@gmail.com wrote:
I just upgraded my machine from F29 to F30. Now, whenever I install a new kernel, the new kernel does not automatically become the default. In other words, when I reboot, the previous kernel is still chosen by grub2.
I can manually choose the new kernel in the grub2 menu, at which point it _does_ become the new default. I don't wind up at the "grub>" prompt, so I think grub2 itself is fine. It is just that the grubenv is not updated when the new kernel is installed.
The machine has UEFI, but the system boots using the legacy BIOS compatibility layer. I know that the boot mechanism changed a bit for F30, but I'm not sure where to look to identify the cause of this problem. It doesn't seem to be the same issue as described in BZ 1652806.
Steve
It can be tricky for the grubby scripts to deduce which kernel was *really* the last one, especially if you've been hand-editing kernel entries or adding them manually, well, there might be debris.
/boog/grub/ has gotten *fascinating* Perhaps you can back up the system, and then delete *all* the extraneous kernels, to reset any confused default tracking mechanism in place?.
On 5/5/19 2:04 PM, Nico Kadel-Garcia wrote:
On Sun, May 5, 2019 at 10:22 AM Steven A. Falco stevenfalco@gmail.com wrote:
I just upgraded my machine from F29 to F30. Now, whenever I install a new kernel, the new kernel does not automatically become the default. In other words, when I reboot, the previous kernel is still chosen by grub2.
I can manually choose the new kernel in the grub2 menu, at which point it _does_ become the new default. I don't wind up at the "grub>" prompt, so I think grub2 itself is fine. It is just that the grubenv is not updated when the new kernel is installed.
The machine has UEFI, but the system boots using the legacy BIOS compatibility layer. I know that the boot mechanism changed a bit for F30, but I'm not sure where to look to identify the cause of this problem. It doesn't seem to be the same issue as described in BZ 1652806.
SteveIt can be tricky for the grubby scripts to deduce which kernel was *really* the last one, especially if you've been hand-editing kernel entries or adding them manually, well, there might be debris.
/boog/grub/ has gotten *fascinating* Perhaps you can back up the system, and then delete *all* the extraneous kernels, to reset any confused default tracking mechanism in place?.
"Fascinating" is a good word for it. I tried reading through the various scripts. They are a twisty little maze of passages. :-)
The problem has suddenly gone away. I just installed kernel 5.0.11, and this time it correctly became the default. I'm not sure what fixed my issue, but I can think of two possibilities.
1) Prior to installing 5.0.11, I did run the grub2-install command, as mentioned in the Common_F30_bugs wiki page. Perhaps that helped, even though my symptoms were very different.
2) Also, installing 5.0.11 obsoleted the last of the F29 kernels on my machine. Now there are only F30 kernels in /boot.
I'll keep an eye on it, but for now I'm good.
Steve
On Sun, May 5, 2019 at 8:22 AM Steven A. Falco stevenfalco@gmail.com wrote:
I just upgraded my machine from F29 to F30. Now, whenever I install a new kernel, the new kernel does not automatically become the default. In other words, when I reboot, the previous kernel is still chosen by grub2.
I can manually choose the new kernel in the grub2 menu, at which point it _does_ become the new default. I don't wind up at the "grub>" prompt, so I think grub2 itself is fine. It is just that the grubenv is not updated when the new kernel is installed.
The machine has UEFI, but the system boots using the legacy BIOS compatibility layer. I know that the boot mechanism changed a bit for F30, but I'm not sure where to look to identify the cause of this problem. It doesn't seem to be the same issue as described in BZ 1652806.
Post your /etc/default/grub file
I'm willing to bet there's a line
GRUB_SAVEDEFAULT=true
If so, delete that line or comment it out and then run the usual grub2-mkconfig and directing the output to the proper grub.cfg path for your firmware type.
The default that should be honored is found in the grubenv file, which (curiously) is found at the same path no matter your firmware type: /boot/efi/EFI/fedora/grubenv
You can list its contents
# grub2-editenv list
And you can change it with
#grub2-set-default <title of kernel>
The title of the kernel is found in the /boot/loader/entries/*conf files - there is one file for each kernel.
On 5/5/19 6:29 PM, Chris Murphy wrote:
On Sun, May 5, 2019 at 8:22 AM Steven A. Falco stevenfalco@gmail.com wrote:
I just upgraded my machine from F29 to F30. Now, whenever I install a new kernel, the new kernel does not automatically become the default. In other words, when I reboot, the previous kernel is still chosen by grub2.
I can manually choose the new kernel in the grub2 menu, at which point it _does_ become the new default. I don't wind up at the "grub>" prompt, so I think grub2 itself is fine. It is just that the grubenv is not updated when the new kernel is installed.
The machine has UEFI, but the system boots using the legacy BIOS compatibility layer. I know that the boot mechanism changed a bit for F30, but I'm not sure where to look to identify the cause of this problem. It doesn't seem to be the same issue as described in BZ 1652806.
Post your /etc/default/grub file
I'm willing to bet there's a line
GRUB_SAVEDEFAULT=true
If so, delete that line or comment it out and then run the usual grub2-mkconfig and directing the output to the proper grub.cfg path for your firmware type.
The default that should be honored is found in the grubenv file, which (curiously) is found at the same path no matter your firmware type: /boot/efi/EFI/fedora/grubenv
You can list its contents
# grub2-editenv list
And you can change it with
#grub2-set-default <title of kernel>
The title of the kernel is found in the /boot/loader/entries/*conf files - there is one file for each kernel.
Thanks for the explanation. Here are the contents of /etc/default/grub. As you suspected, there is a GRUB_DEFAULT=saved line in there.
GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="console" GRUB_CMDLINE_LINUX="resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.md.uuid=77ae1678:58a79067:c0ad29e6:bd1862f8 rd.md.uuid=bac1fa34:2d7a26e5:969d63ac:33ff4572 rd.lvm.lv=fedora/swap" GRUB_DISABLE_RECOVERY="true" GRUB_ENABLE_BLSCFG=true
I looked for grubenv, and the only one I found is at /boot/grub2/grubenv. There is nothing in /boot/efi/EFI/fedora. This machine was set up on 2018-11-24, so it started life as a Fedora 29 machine.
Is there a command I should run to move grubenv to /boot/efi/EFI/fedora? I think I would also have to create a symlink from /boot/efi/EFI/fedora/grubenv to /etc/default/grub. I could of course do it manually, but if there is a better procedure, like re-installing some package(s), that would be preferable.
Steve
On Mon, May 6, 2019 at 7:39 AM Steven A. Falco stevenfalco@gmail.com wrote:
Thanks for the explanation. Here are the contents of /etc/default/grub. As you suspected, there is a GRUB_DEFAULT=saved line in there.
GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="console" GRUB_CMDLINE_LINUX="resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.md.uuid=77ae1678:58a79067:c0ad29e6:bd1862f8 rd.md.uuid=bac1fa34:2d7a26e5:969d63ac:33ff4572 rd.lvm.lv=fedora/swap" GRUB_DISABLE_RECOVERY="true" GRUB_ENABLE_BLSCFG=true
This is a stock /etc/default/grub - there's nothing custom about it.
I looked for grubenv, and the only one I found is at /boot/grub2/grubenv. There is nothing in /boot/efi/EFI/fedora. This machine was set up on 2018-11-24, so it started life as a Fedora 29 machine.
Ahh not sure. Might be new in Fedora 30 and doesn't get changed on upgrades?
Is there a command I should run to move grubenv to /boot/efi/EFI/fedora? I think I would also have to create a symlink from /boot/efi/EFI/fedora/grubenv to /etc/default/grub. I could of course do it manually, but if there is a better procedure, like re-installing some package(s), that would be preferable.
The purpose of the symlink is so that grubenv commands and usage just work without respect to firmware type. You don't have to fix this. But I'm still curious about the contents of that grubenv, so what do you get fro
# grub2-editenv list
On 5/6/19 2:53 PM, Chris Murphy wrote:
On Mon, May 6, 2019 at 7:39 AM Steven A. Falco stevenfalco@gmail.com wrote:
Thanks for the explanation. Here are the contents of /etc/default/grub. As you suspected, there is a GRUB_DEFAULT=saved line in there.
GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="console" GRUB_CMDLINE_LINUX="resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.md.uuid=77ae1678:58a79067:c0ad29e6:bd1862f8 rd.md.uuid=bac1fa34:2d7a26e5:969d63ac:33ff4572 rd.lvm.lv=fedora/swap" GRUB_DISABLE_RECOVERY="true" GRUB_ENABLE_BLSCFG=true
This is a stock /etc/default/grub - there's nothing custom about it.
I looked for grubenv, and the only one I found is at /boot/grub2/grubenv. There is nothing in /boot/efi/EFI/fedora. This machine was set up on 2018-11-24, so it started life as a Fedora 29 machine.
Ahh not sure. Might be new in Fedora 30 and doesn't get changed on upgrades?
Is there a command I should run to move grubenv to /boot/efi/EFI/fedora? I think I would also have to create a symlink from /boot/efi/EFI/fedora/grubenv to /etc/default/grub. I could of course do it manually, but if there is a better procedure, like re-installing some package(s), that would be preferable.
The purpose of the symlink is so that grubenv commands and usage just work without respect to firmware type. You don't have to fix this. But I'm still curious about the contents of that grubenv, so what do you get fro
# grub2-editenv list
Here is the command output:
saved_entry=2aa6409d5c354eea9cc2e4630c4efda0-5.0.11-300.fc30.x86_64 boot_success=1 boot_indeterminate=1 kernelopts=root=/dev/mapper/fedora-root ro resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.md.uuid=77ae1678:58a79067:c0ad29e6:bd1862f8 rd.md.uuid=bac1fa34:2d7a26e5:969d63ac:33ff4572 rd.lvm.lv=fedora/swap
I'm reading through the various scripts trying to understand the impact of GRUB_DEFAULT. It seems like having GRUB_DEFAULT=saved is not currently hurting me. The last upgrade, to 5.0.11-300, properly made that kernel the new default.
If GRUB_DEFAULT is commented out, then I think grub will always choose the first item in its menu, which would be fine, because the newest kernel always appears first in the grub menu. Is that why you recommended commenting it out?
Steve
On Mon, May 6, 2019 at 1:04 PM Steven A. Falco stevenfalco@gmail.com wrote:
# grub2-editenv list
Here is the command output:
saved_entry=2aa6409d5c354eea9cc2e4630c4efda0-5.0.11-300.fc30.x86_64 boot_success=1 boot_indeterminate=1 kernelopts=root=/dev/mapper/fedora-root ro resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.md.uuid=77ae1678:58a79067:c0ad29e6:bd1862f8 rd.md.uuid=bac1fa34:2d7a26e5:969d63ac:33ff4572 rd.lvm.lv=fedora/swap
Looks normal.
Also, about the /boot/grub2/grubenv symlink to /boot/efi/EFI/fedora/grubenv - I'm only seeing this on clean installs from Fedora-Workstation-Live-x86_64-30-1.2.iso so it might be related to how the lives are assembled and rsync'd over. It doesn't happen with a minimal or server installation on a BIOS VM.
At the moment, I think whatever problem there was has been cleared and it's now behaving normally.
I'm reading through the various scripts trying to understand the impact of GRUB_DEFAULT. It seems like having GRUB_DEFAULT=saved is not currently hurting me. The last upgrade, to 5.0.11-300, properly made that kernel the new default.
If GRUB_DEFAULT is commented out, then I think grub will always choose the first item in its menu, which would be fine, because the newest kernel always appears first in the grub menu. Is that why you recommended commenting it out?
Nope, sorry, you're confused. I referred to GRUB_SAVEDEFAULT - thinking maybe you had a customized /etc/default/grub. GRUB_DEFAULT and GRUB_SAVEDEFAULT are two different things, but the latter depends on the former.
I suggest keeping things as is, with saved_entry set in the grubenv. And that's because GRUB and the grub-boot-success.service are able to do an automatic fallback to the previous working kernel if boot fails following a kernel upgrade.
On 5/6/19 4:47 PM, Chris Murphy wrote:
On Mon, May 6, 2019 at 1:04 PM Steven A. Falco stevenfalco@gmail.com wrote:
# grub2-editenv list
Here is the command output:
saved_entry=2aa6409d5c354eea9cc2e4630c4efda0-5.0.11-300.fc30.x86_64 boot_success=1 boot_indeterminate=1 kernelopts=root=/dev/mapper/fedora-root ro resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.md.uuid=77ae1678:58a79067:c0ad29e6:bd1862f8 rd.md.uuid=bac1fa34:2d7a26e5:969d63ac:33ff4572 rd.lvm.lv=fedora/swap
Looks normal.
Also, about the /boot/grub2/grubenv symlink to /boot/efi/EFI/fedora/grubenv - I'm only seeing this on clean installs from Fedora-Workstation-Live-x86_64-30-1.2.iso so it might be related to how the lives are assembled and rsync'd over. It doesn't happen with a minimal or server installation on a BIOS VM.
The install originated as a "server edition", so that is consistent.
At the moment, I think whatever problem there was has been cleared and it's now behaving normally.
Agreed.
I'm reading through the various scripts trying to understand the impact of GRUB_DEFAULT. It seems like having GRUB_DEFAULT=saved is not currently hurting me. The last upgrade, to 5.0.11-300, properly made that kernel the new default.
If GRUB_DEFAULT is commented out, then I think grub will always choose the first item in its menu, which would be fine, because the newest kernel always appears first in the grub menu. Is that why you recommended commenting it out?
Nope, sorry, you're confused. I referred to GRUB_SAVEDEFAULT - thinking maybe you had a customized /etc/default/grub. GRUB_DEFAULT and GRUB_SAVEDEFAULT are two different things, but the latter depends on the former.
It wouldn't be the first time I was confused. :-)
I suggest keeping things as is, with saved_entry set in the grubenv. And that's because GRUB and the grub-boot-success.service are able to do an automatic fallback to the previous working kernel if boot fails following a kernel upgrade.
I will leave it alone, as you recommend.
As I was reading through the documentation, I came across a statement that grubenv is unavailable on RAID - please see the second to last sentence here:
https://www.gnu.org/software/grub/manual/grub/html_node/Environment-block.ht...
My machine is set up with /boot on SW RAID-1 (and everything else on SW RAID-5 / LVM). That said, grubenv appears to update properly. I don't know if the manual is not quite current, or if there is some other explanation - perhaps any updates always occur under Linux, while the RAID-1 is assembled.
Regardless, everything is good now, so I'll stop obsessing about it. :-)
And thanks for all your help!
Steve
On Mon, May 6, 2019 at 3:15 PM Steven A. Falco stevenfalco@gmail.com wrote:
As I was reading through the documentation, I came across a statement that grubenv is unavailable on RAID - please see the second to last sentence here:
https://www.gnu.org/software/grub/manual/grub/html_node/Environment-block.ht...
My machine is set up with /boot on SW RAID-1 (and everything else on SW RAID-5 / LVM). That said, grubenv appears to update properly. I don't know if the manual is not quite current, or if there is some other explanation - perhaps any updates always occur under Linux, while the RAID-1 is assembled.
Regardless, everything is good now, so I'll stop obsessing about it. :-)
Yep. https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
GRUB pre-boot environment can read grubenv from anything GRUB supports reading, which is practically anything including mdadm RAID. Your grubenv can be read, it just can be changed by GRUB in the pre-boot environment. That means it can't change boot_success to 0. It'll always be a 1, meaning GRUB will think your boots are always successful.
When a new kernel is installed, the package scripts cause grubenv saved_entry to get updated. This is happening through kernel file system and md code, so it's a completely supported change, and that's how GRUB in the pre-boot environment can see the updated saved_entry and know which kernel version to boot by default.
But it's worth keeping an eye on anomalies. There is the potential for goofy things happening. Unrelated to this particular feature, rather it was grub.cfg being updated, in cases where that update happened very quickly followed by an immediate reboot, GRUB only saw the previous grub.cfg. On journaled file systems, the new file gets written out, and indicated only in the journal, and a particular set of circumstances preventing the root fs from being cleanly unmounted resulted in a hidden new grub.cfg that only became revealed after journal replay by the kernel code. The GRUB code can't read file system journals, so it was seeing the stale file as a result of reading stale file system metadata without the benefit of reading the journal. :P
-- Chris Murphy
On Mon, May 6, 2019 at 3:51 PM Chris Murphy lists@colorremedies.com wrote:
GRUB pre-boot environment can read grubenv from anything GRUB supports reading, which is practically anything including mdadm RAID. Your grubenv can be read, it just can be changed by GRUB in the pre-boot
^can't
!!
On 5/6/19 5:51 PM, Chris Murphy wrote:
But it's worth keeping an eye on anomalies. There is the potential for goofy things happening. Unrelated to this particular feature, rather it was grub.cfg being updated, in cases where that update happened very quickly followed by an immediate reboot, GRUB only saw the previous grub.cfg. On journaled file systems, the new file gets written out, and indicated only in the journal, and a particular set of circumstances preventing the root fs from being cleanly unmounted resulted in a hidden new grub.cfg that only became revealed after journal replay by the kernel code. The GRUB code can't read file system journals, so it was seeing the stale file as a result of reading stale file system metadata without the benefit of reading the journal. :P
I may indeed have tripped over that a time or two. I've had cases where strange things happened following a kernel update / immediate reboot. Given the relative non-volatility of /boot, it's starting to make sense to use a plain ext2 fs for /boot rather than a more modern fs.
If I ever have to reload this machine, I may switch to UEFI. While I don't like the FAT filesystems, their simplicity has an advantage in this application.
Steve
On Tue, May 7, 2019 at 8:29 AM Steven A. Falco stevenfalco@gmail.com wrote:
On 5/6/19 5:51 PM, Chris Murphy wrote:
But it's worth keeping an eye on anomalies. There is the potential for goofy things happening. Unrelated to this particular feature, rather it was grub.cfg being updated, in cases where that update happened very quickly followed by an immediate reboot, GRUB only saw the previous grub.cfg. On journaled file systems, the new file gets written out, and indicated only in the journal, and a particular set of circumstances preventing the root fs from being cleanly unmounted resulted in a hidden new grub.cfg that only became revealed after journal replay by the kernel code. The GRUB code can't read file system journals, so it was seeing the stale file as a result of reading stale file system metadata without the benefit of reading the journal. :P
I may indeed have tripped over that a time or two. I've had cases where strange things happened following a kernel update / immediate reboot. Given the relative non-volatility of /boot, it's starting to make sense to use a plain ext2 fs for /boot rather than a more modern fs.
If I ever have to reload this machine, I may switch to UEFI. While I don't like the FAT filesystems, their simplicity has an advantage in this application.
The reality is, you can't disable UEFI on a UEFI computer. Legacy just enables a compatibility support module on top of UEFI to present a faux-BIOS to the bootloader and the kernel. I only recommend it if there are UEFI bugs that can't be worked around, yet are masked by the CSM. Any reputable vendor is responsive with firmware updates to address the worst UEFI bugs. For laptops, not running UEFI natively can also have negative power management side effects.
On Mon, May 6, 2019 at 2:47 PM Chris Murphy lists@colorremedies.com wrote:
I suggest keeping things as is, with saved_entry set in the grubenv. And that's because GRUB and the grub-boot-success.service are able to do an automatic fallback to the previous working kernel if boot fails following a kernel upgrade.
Sorry for the confusion, I don't think this is correct. If boot is not successful, the GRUB menu will not be hidden at the next boot giving the user the option of picking another kernel.
-
Chris Murphy -
Nico Kadel-Garcia -
Steven A. Falco