+ more people
On 12/20/18 at 04:49pm, lijiang wrote:
在 2018年12月20日 13:57, Dave Young 写道:
> On 12/20/18 at 01:06pm, Lianbo Jiang wrote:
>> By default, early kdump reboots the system after capturing the vmcore.
>> If the problematic system is continuously crashing due to some issue
>> during early boot stage, the system may fall into infinite loop restart
>> like this:
>>
>> boot -----> crash -----> early kdump (dump vmcore)
>> ^ |
>> '.........(reboot).............'
>>
>> But now, the system crash at early stage is only captured by early kdump,
>> and the rest is captured by normal kdump. That to say, when normal kdump
>> service starts, it will load it again and override early kdump. It is
>> helpful to control the logic of early kdump and normal kdump separately
>> in final action(it is called by kdump-capture.service). For example,
>> early kdump always passes the 'rd.earlykdump' to the second kernel when
>> early kdump is enabled, but normal kdump doesn't pass the
'rd.earlykdump'
>> to the second kernel at any time. So they can be distinguished in the
>> second kernel.
>
> Hmm, I'm confused about the param passing above.
>
I copy some messages from another email, please refer to this one:
[--->
The rd.earlykdump is added to kernel command line in grub.cfg. However, early kdump
and normal kdump can get the same parameters from /proc/cmdline in the first kernel.
Early kdump passes the rd.earlykdump to the second kernel, but normal kdump doesn't
need it, normal kdump needs to remove the rd.earlykdump.
So which can distinguish early kdump and normal kdump in the second kernel. It helps
to control the logic of kdump capture service. For example: default action/final action.
]
The description is confusing, "ealy kdump passes ... to the second
kernel", for example about this, the real thing is one person adds the
param in 1st kernel cmdline, kexec-tools takes/inherits and pass to 2nd
kernel.
Anyway this is patch log issue.
> Early or non early just means about the service loading phase, in
Yes. This patch used the same method what you said. When normal kdump service starts,
it will reload. At the same time, early kdump will be overwritten by normal kdump.
Probably "early kdump load" is better than "early kdump" in words.
> initramfs or not, I notice dracut/systemd will print some message about
> they are running in initramfs, so probably you can check how to get it
> with same way, if this is not initramfs then just unload before the
> check in kdump loading.
>
> The picture like below:
>
> Kernel boot ->
>
> initramfs ---
> early kdump load
> ---- Mark A ----
> initramfs switch root
>
> system startup (real root fs)
> service a
> service b ... (eg. networking etc.)
> kdump service start
> -----Mark B -----
> load kdump kernel again
>
>
> The problem will happen between Mark A and Mark B, during this period,
> there could be repeated crash -> earlykdump_load, there might be some
> random crash as well since during the real root fs service startup,
> for example after network is ready if some network workload cause a
> panic, it maybe not 100% reproducible, so it seems we still need to
> make the poweroff configurable. eg.
>
> default is poweroff, but one can choose if he can.
>
Yes, default is poweroff for early kdump. Unless kdump capture service
happens error or enters the emergency service, one can choose the default
action.(configure default=xxx in kdump.conf)
For default action instead of final action if you hardcode it, then even if
one set default as reboot it still poweroff.
[snip]
>>
>> +check_rd_earlykdump()
>> +{
>> + egrep "rd.earlykdump" /proc/cmdline
>> +}
>> +
>> start()
>> {
>> check_dump_feasibility
>> @@ -969,7 +974,13 @@ start()
>> check_current_status
>> if [ $? == 0 ]; then
>> echo "Kdump already running: [WARNING]"
>> - return 0
>> + check_rd_earlykdump
>> + #if earlykdump loaded, it will stop and start.
>> + if [ $? -eq 0 ]; then
>> + stop
>
> kdumpctl start can run not only by system startup services, one can also
> run it manually or in udev rule.
>
> The checking of kernel cmdline seems not enough.
>
Here it means that if kdump has beend loaded, check whether early kdump did it.
If yes, let normal kdump load it again, otherwise no need to do anything.
As we discussed you do not get my points here :)
check_rd_earlykdump will be always true once kernel bootup, so there is
no way to get the first time of normal kdump load and the later one.
The early boot time panic to address for this patch is the 100%
reproducible panic, for this kinds of panic admin should be able to see
it when he boot the machine. So rethinking about this the best way may
be just a wontfix.
Let's consider below use cases:
* First install:
install os ->
---A
reboot ->
...
---B
kdump service start
-> create kdump initrd
...
boot finished
recreate default initrd and enable early kdump
goto A
Panic happened between A and B, if it is predictable, eg. 100%
reproducible, then admin should already see it, then he/she can control
and stop the repeating crash/kdump loop
if the panic is not 100% reproducible then use reboot as final action is just fine
* Other use cases eg. updating kernel or some other components:
It is similar with the intall os use case because if one update kernel
or critical components it is likely they need regenerate kdump initrd,
and then repack it into early kdump default initrd, in this case admin
should be able to see the panic loop and handle it.
Also if the panic is not 100% reproducible then we are just fine.
If we choose to split early and late kdump load, there could be other
side effects, and make the logic even complicated.
So... the better way may be just leave it as is, and maybe add some
documentation.
Thoughts?
Thanks
Dave