It's reported that kdump kernel failed to boot and can't dump vmcore when crashkernel=192M and SME/SEV is active.
This is because swiotlb will be enabled and reserves 64M memory by default on system with SME/SEV enabled. Then kdump kernel will be out of memory after taking 64M away for swiotlb init.
So here add extra 64M memory to default crashkernel value so that kdump kernel can function well as before. When doing that, search journalctl for the "Memory Encryption Features active: AMD" to check if SME or SEV is active. This line of log is printed out in kernel function as below and the type SME is mutual exclusive with type SEV. ***: arch/x86/mm/mem_encrypt.c:print_mem_encrypt_feature_info()
Note: 1) The conditional check is relying on journalctl log because I didn't find available system interface to check if SEV is active. Even though we can check if SME is active via /proc/cpuinfo. For consistency, I take the same check for both SME and SEV by searching journalctl.
2) The conditional check is relying on journalctl log, means it won't work for crashkernel setting in anoconda because the installation kernel doesn't have the SME/SEV setting. So customer need manually run 'kdumpctl reset-crashkernel' to reset crashkernel to add the extra 64M after OS installation.
3) We need watch the line of log printing in print_mem_encrypt_feature_info() in kernel just in case people may change it in the future.
Signed-off-by: Baoquan He bhe@redhat.com --- v2->v3: - Redirect the stdin|out to /dev/null since "journalctl -q" can't suppress output completely. Suggested by Coiby. - Define local variable "_delta" to contain the value added on "$_ck_cmdline". This is suggested by Philipp.
kdump-lib.sh | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/kdump-lib.sh b/kdump-lib.sh index 16238c508f65..89b6d39506fb 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -38,6 +38,11 @@ is_aws_aarch64() [[ "$(lscpu | grep "BIOS Model name")" =~ "AWS Graviton" ]] }
+is_sme_or_sev_active() +{ + journalctl -q --dmesg --grep "^Memory Encryption Features active: AMD (SME|SEV)$" >/dev/null 2>&1 +} + is_squash_available() { local _version kmodule @@ -917,6 +922,7 @@ _crashkernel_add() kdump_get_arch_recommend_crashkernel() { local _arch _ck_cmdline _dump_mode + local _delta=0
if [[ -z "$1" ]]; then if is_fadump_capable; then @@ -932,9 +938,9 @@ kdump_get_arch_recommend_crashkernel()
if [[ $_arch == "x86_64" ]] || [[ $_arch == "s390x" ]]; then _ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M" + is_sme_or_sev_active && ((_delta += 64)) elif [[ $_arch == "aarch64" ]]; then local _running_kernel - local _delta=0
# Base line for 4K variant kernel. The formula is based on x86 plus extra = 64M _ck_cmdline="1G-4G:256M,4G-64G:320M,64G-:576M" @@ -958,7 +964,6 @@ kdump_get_arch_recommend_crashkernel() #4k kernel, mlx5 consumes extra 124M memory, and choose 150M has_mlx5 && ((_delta += 150)) fi - _ck_cmdline=$(_crashkernel_add "$_ck_cmdline" "$_delta") elif [[ $_arch == "ppc64le" ]]; then if [[ $_dump_mode == "fadump" ]]; then _ck_cmdline="4G-16G:768M,16G-64G:1G,64G-128G:2G,128G-1T:4G,1T-2T:6G,2T-4T:12G,4T-8T:20G,8T-16T:36G,16T-32T:64G,32T-64T:128G,64T-:180G" @@ -967,7 +972,7 @@ kdump_get_arch_recommend_crashkernel() fi fi
- echo -n "$_ck_cmdline" + echo -n "$(_crashkernel_add "$_ck_cmdline" "$_delta")" }
# return recommended size based on current system RAM size
Hi Baoquan,
On Thu, 21 Sep 2023 18:01:02 +0800 Baoquan He bhe@redhat.com wrote:
It's reported that kdump kernel failed to boot and can't dump vmcore when crashkernel=192M and SME/SEV is active.
This is because swiotlb will be enabled and reserves 64M memory by default on system with SME/SEV enabled. Then kdump kernel will be out of memory after taking 64M away for swiotlb init.
So here add extra 64M memory to default crashkernel value so that kdump kernel can function well as before. When doing that, search journalctl for the "Memory Encryption Features active: AMD" to check if SME or SEV is active. This line of log is printed out in kernel function as below and the type SME is mutual exclusive with type SEV. ***: arch/x86/mm/mem_encrypt.c:print_mem_encrypt_feature_info()
Note:
The conditional check is relying on journalctl log because I didn't find available system interface to check if SEV is active. Even though we can check if SME is active via /proc/cpuinfo. For consistency, I take the same check for both SME and SEV by searching journalctl.
The conditional check is relying on journalctl log, means it won't work for crashkernel setting in anoconda because the installation kernel doesn't have the SME/SEV setting. So customer need manually run 'kdumpctl reset-crashkernel' to reset crashkernel to add the extra 64M after OS installation.
We need watch the line of log printing in print_mem_encrypt_feature_info() in kernel just in case people may change it in the future.
Signed-off-by: Baoquan He bhe@redhat.com
v2->v3:
- Redirect the stdin|out to /dev/null since "journalctl -q" can't suppress output completely. Suggested by Coiby.
- Define local variable "_delta" to contain the value added on "$_ck_cmdline". This is suggested by Philipp.
kdump-lib.sh | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/kdump-lib.sh b/kdump-lib.sh index 16238c508f65..89b6d39506fb 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -38,6 +38,11 @@ is_aws_aarch64() [[ "$(lscpu | grep "BIOS Model name")" =~ "AWS Graviton" ]] }
+is_sme_or_sev_active() +{
- journalctl -q --dmesg --grep "^Memory Encryption Features active: AMD (SME|SEV)$" >/dev/null 2>&1
You can use '&>' as short hand to redirect both stdout and stderr, i.e. s;>/dev/null 2>&1;&>/dev/null;
Anyway, that's just a very minor code cleanup and you don't need to include it.
Reviewed-by: Philipp Rudo prudo@redhat.com
+}
is_squash_available() { local _version kmodule @@ -917,6 +922,7 @@ _crashkernel_add() kdump_get_arch_recommend_crashkernel() { local _arch _ck_cmdline _dump_mode
local _delta=0
if [[ -z "$1" ]]; then if is_fadump_capable; then
@@ -932,9 +938,9 @@ kdump_get_arch_recommend_crashkernel()
if [[ $_arch == "x86_64" ]] || [[ $_arch == "s390x" ]]; then _ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M"
is_sme_or_sev_active && ((_delta += 64))
local _delta=0# Base line for 4K variant kernel. The formula is based on x86 plus extra = 64M _ck_cmdline="1G-4G:256M,4G-64G:320M,64G-:576M"
@@ -958,7 +964,6 @@ kdump_get_arch_recommend_crashkernel() #4k kernel, mlx5 consumes extra 124M memory, and choose 150M has_mlx5 && ((_delta += 150)) fi
_ck_cmdline=$(_crashkernel_add "$_ck_cmdline" "$_delta")@@ -967,7 +972,7 @@ kdump_get_arch_recommend_crashkernel() fi fi
- echo -n "$_ck_cmdline"
- echo -n "$(_crashkernel_add "$_ck_cmdline" "$_delta")"
}
# return recommended size based on current system RAM size
On 09/28/23 at 03:15pm, Philipp Rudo wrote: ......
+is_sme_or_sev_active() +{
- journalctl -q --dmesg --grep "^Memory Encryption Features active: AMD (SME|SEV)$" >/dev/null 2>&1
j You can use '&>' as short hand to redirect both stdout and stderr, i.e.
s;>/dev/null 2>&1;&>/dev/null;
Anyway, that's just a very minor code cleanup and you don't need to include it.
Reviewed-by: Philipp Rudo prudo@redhat.com
Thanks for careful reviewing, Philipp.
On Sat, Oct 07, 2023 at 11:48:27AM +0800, Baoquan He wrote:
On 09/28/23 at 03:15pm, Philipp Rudo wrote: ......
+is_sme_or_sev_active() +{
- journalctl -q --dmesg --grep "^Memory Encryption Features active: AMD (SME|SEV)$" >/dev/null 2>&1
j You can use '&>' as short hand to redirect both stdout and stderr, i.e.
s;>/dev/null 2>&1;&>/dev/null;
Anyway, that's just a very minor code cleanup and you don't need to include it.
Reviewed-by: Philipp Rudo prudo@redhat.com
Thanks for careful reviewing, Philipp.
I've rebased the patch onto the latest rawhide branch and then merged it together with [PATCH v2] Allow _crashkernel_add to address larger memory ranges. Thanks to Baoquan and Philipp!
On 11/08/23 at 09:47am, Coiby Xu wrote:
On Sat, Oct 07, 2023 at 11:48:27AM +0800, Baoquan He wrote:
On 09/28/23 at 03:15pm, Philipp Rudo wrote: ......
+is_sme_or_sev_active() +{
- journalctl -q --dmesg --grep "^Memory Encryption Features active: AMD (SME|SEV)$" >/dev/null 2>&1
j You can use '&>' as short hand to redirect both stdout and stderr, i.e.
s;>/dev/null 2>&1;&>/dev/null;
Anyway, that's just a very minor code cleanup and you don't need to include it.
Reviewed-by: Philipp Rudo prudo@redhat.com
Thanks for careful reviewing, Philipp.
I've rebased the patch onto the latest rawhide branch and then merged it together with [PATCH v2] Allow _crashkernel_add to address larger memory ranges. Thanks to Baoquan and Philipp!
That's great! Thanks, Coiby.
Hi Baoquan and Philipp,
This patch has a conflict with latest rawhide branch. And with commit 64f2827a ("kdump-lib: Harden _crashkernel_add"), the delta should specify the unit explicitly, i.e. it should be echo -n "$(_crashkernel_add "$_ck_cmdline" "${_delta}M")"
But after manually resolving the conflict, it still fails the unit tests. It turns out the failures are not caused by this patch but issues in _crashkernel_add. I've sent "[PATCH] Fix two issues in _crashkernel_add" which should be better merged before this patch or together.
On Thu, Sep 21, 2023 at 06:01:02PM +0800, Baoquan He wrote:
It's reported that kdump kernel failed to boot and can't dump vmcore when crashkernel=192M and SME/SEV is active.
This is because swiotlb will be enabled and reserves 64M memory by default on system with SME/SEV enabled. Then kdump kernel will be out of memory after taking 64M away for swiotlb init.
So here add extra 64M memory to default crashkernel value so that kdump kernel can function well as before. When doing that, search journalctl for the "Memory Encryption Features active: AMD" to check if SME or SEV is active. This line of log is printed out in kernel function as below and the type SME is mutual exclusive with type SEV. ***: arch/x86/mm/mem_encrypt.c:print_mem_encrypt_feature_info()
Note:
- The conditional check is relying on journalctl log because I didn't
find available system interface to check if SEV is active. Even though we can check if SME is active via /proc/cpuinfo. For consistency, I take the same check for both SME and SEV by searching journalctl.
- The conditional check is relying on journalctl log, means it won't
work for crashkernel setting in anoconda because the installation kernel doesn't have the SME/SEV setting. So customer need manually run 'kdumpctl reset-crashkernel' to reset crashkernel to add the extra 64M after OS installation.
- We need watch the line of log printing in
print_mem_encrypt_feature_info() in kernel just in case people may change it in the future.
Signed-off-by: Baoquan He bhe@redhat.com
v2->v3:
- Redirect the stdin|out to /dev/null since "journalctl -q" can't
suppress output completely. Suggested by Coiby.
- Define local variable "_delta" to contain the value added on
"$_ck_cmdline". This is suggested by Philipp.
kdump-lib.sh | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/kdump-lib.sh b/kdump-lib.sh index 16238c508f65..89b6d39506fb 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -38,6 +38,11 @@ is_aws_aarch64() [[ "$(lscpu | grep "BIOS Model name")" =~ "AWS Graviton" ]] }
+is_sme_or_sev_active() +{
- journalctl -q --dmesg --grep "^Memory Encryption Features active: AMD (SME|SEV)$" >/dev/null 2>&1
+}
is_squash_available() { local _version kmodule @@ -917,6 +922,7 @@ _crashkernel_add() kdump_get_arch_recommend_crashkernel() { local _arch _ck_cmdline _dump_mode
local _delta=0
if [[ -z "$1" ]]; then if is_fadump_capable; then
@@ -932,9 +938,9 @@ kdump_get_arch_recommend_crashkernel()
if [[ $_arch == "x86_64" ]] || [[ $_arch == "s390x" ]]; then _ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M"
is_sme_or_sev_active && ((_delta += 64))
local _delta=0# Base line for 4K variant kernel. The formula is based on x86 plus extra = 64M _ck_cmdline="1G-4G:256M,4G-64G:320M,64G-:576M"
@@ -958,7 +964,6 @@ kdump_get_arch_recommend_crashkernel() #4k kernel, mlx5 consumes extra 124M memory, and choose 150M has_mlx5 && ((_delta += 150)) fi
_ck_cmdline=$(_crashkernel_add "$_ck_cmdline" "$_delta")@@ -967,7 +972,7 @@ kdump_get_arch_recommend_crashkernel() fi fi
- echo -n "$_ck_cmdline"
- echo -n "$(_crashkernel_add "$_ck_cmdline" "$_delta")"
}
# return recommended size based on current system RAM size
2.41.0
Hi Coiby,
On Tue, 10 Oct 2023 14:52:21 +0800 Coiby Xu coxu@redhat.com wrote:
Hi Baoquan and Philipp,
This patch has a conflict with latest rawhide branch. And with commit 64f2827a ("kdump-lib: Harden _crashkernel_add"), the delta should specify the unit explicitly, i.e. it should be echo -n "$(_crashkernel_add "$_ck_cmdline" "${_delta}M")"
True, I missed that...
Thanks for catching the problem Philipp
But after manually resolving the conflict, it still fails the unit tests. It turns out the failures are not caused by this patch but issues in _crashkernel_add. I've sent "[PATCH] Fix two issues in _crashkernel_add" which should be better merged before this patch or together.
On Thu, Sep 21, 2023 at 06:01:02PM +0800, Baoquan He wrote:
It's reported that kdump kernel failed to boot and can't dump vmcore when crashkernel=192M and SME/SEV is active.
This is because swiotlb will be enabled and reserves 64M memory by default on system with SME/SEV enabled. Then kdump kernel will be out of memory after taking 64M away for swiotlb init.
So here add extra 64M memory to default crashkernel value so that kdump kernel can function well as before. When doing that, search journalctl for the "Memory Encryption Features active: AMD" to check if SME or SEV is active. This line of log is printed out in kernel function as below and the type SME is mutual exclusive with type SEV. ***: arch/x86/mm/mem_encrypt.c:print_mem_encrypt_feature_info()
Note:
- The conditional check is relying on journalctl log because I didn't
find available system interface to check if SEV is active. Even though we can check if SME is active via /proc/cpuinfo. For consistency, I take the same check for both SME and SEV by searching journalctl.
- The conditional check is relying on journalctl log, means it won't
work for crashkernel setting in anoconda because the installation kernel doesn't have the SME/SEV setting. So customer need manually run 'kdumpctl reset-crashkernel' to reset crashkernel to add the extra 64M after OS installation.
- We need watch the line of log printing in
print_mem_encrypt_feature_info() in kernel just in case people may change it in the future.
Signed-off-by: Baoquan He bhe@redhat.com
v2->v3:
- Redirect the stdin|out to /dev/null since "journalctl -q" can't
suppress output completely. Suggested by Coiby.
- Define local variable "_delta" to contain the value added on
"$_ck_cmdline". This is suggested by Philipp.
kdump-lib.sh | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/kdump-lib.sh b/kdump-lib.sh index 16238c508f65..89b6d39506fb 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -38,6 +38,11 @@ is_aws_aarch64() [[ "$(lscpu | grep "BIOS Model name")" =~ "AWS Graviton" ]] }
+is_sme_or_sev_active() +{
- journalctl -q --dmesg --grep "^Memory Encryption Features active: AMD (SME|SEV)$" >/dev/null 2>&1
+}
is_squash_available() { local _version kmodule @@ -917,6 +922,7 @@ _crashkernel_add() kdump_get_arch_recommend_crashkernel() { local _arch _ck_cmdline _dump_mode
local _delta=0
if [[ -z "$1" ]]; then if is_fadump_capable; then
@@ -932,9 +938,9 @@ kdump_get_arch_recommend_crashkernel()
if [[ $_arch == "x86_64" ]] || [[ $_arch == "s390x" ]]; then _ck_cmdline="1G-4G:192M,4G-64G:256M,64G-:512M"
is_sme_or_sev_active && ((_delta += 64))
local _delta=0# Base line for 4K variant kernel. The formula is based on x86 plus extra = 64M _ck_cmdline="1G-4G:256M,4G-64G:320M,64G-:576M"
@@ -958,7 +964,6 @@ kdump_get_arch_recommend_crashkernel() #4k kernel, mlx5 consumes extra 124M memory, and choose 150M has_mlx5 && ((_delta += 150)) fi
_ck_cmdline=$(_crashkernel_add "$_ck_cmdline" "$_delta")@@ -967,7 +972,7 @@ kdump_get_arch_recommend_crashkernel() fi fi
- echo -n "$_ck_cmdline"
- echo -n "$(_crashkernel_add "$_ck_cmdline" "$_delta")"
}
# return recommended size based on current system RAM size
2.41.0
-
Baoquan He -
Coiby Xu -
Philipp Rudo