On Fri, 19 Jan 2024 at 14:55, Lichen Liu <lichliu(a)redhat.com> wrote:
On Fri, Jan 19, 2024 at 1:48 PM Dave Young <dyoung(a)redhat.com> wrote:
>
> On Fri, 19 Jan 2024 at 12:57, Dave Young <dyoung(a)redhat.com> wrote:
> >
> > On Fri, 19 Jan 2024 at 12:01, Lichen Liu <lichliu(a)redhat.com> wrote:
> > >
> > > On Fri, Jan 19, 2024 at 9:18 AM Dave Young <dyoung(a)redhat.com>
wrote:
> > > >
> > > > On Wed, 17 Jan 2024 at 11:12, Lichen Liu <lichliu(a)redhat.com>
wrote:
> > > > >
> > > > > When using multipath devices as the target for kdump, if
user_friendly_name
> > > > > is also specified, devices default to names like
"mpath*", e.g., mpatha.
> > > > > In dracut, we obtain a persistent device name via
get_persistent_dev. However,
> > > > > dracut currently believes using /dev/mapper/mpath* could cause
issues, thus
> > > > > alternatively names are used, here it's
/dev/disk/by-uuid/<FS_UUID>.
> > > > >
> > > > > During the kdump boot progress, the
/dev/disk/by-uuid/<FS_UUID> will exist as
> > > > > soon as one of the path devices exists, but it won't be
usable by systemd,
> > > > > since multipathd will claim that device as a path device. Then
multipathd will
> > > > > get stopped before it can create the multipath device.
> > > > >
> > > > > Without user_friendly_name, /dev/mapper/<WWID> is
considered a persistent
> > > > > device name, avoiding the issue.
> > > > >
> > > > > The exit of multipathd is due to two dependencies in the current
dracut module
> > > > > 90multipath/multipathd.service,
"Before=initrd-cleanup.service" and
> > > > > "Conflicts=initrd-cleanup.service".
> > > > >
> > > > > As per man 5 systemd.unit, if A.service has
"Conflicts=B.service", starting
> > > > > B.service will stop A.service.
> > > >
> > > > Hi Lichen, still something not clear to me is the
kdump-capture.service has
> > > > Before=initrd-cleanup.service
> > >
> > > Hi Dave,
> > > I think kdump-capture.service has this dependency when it changes from
> > > script to service file.
> > > We want it to run after dracut-pre-pivot.service.
> > >
> > > From `man 7 dracut.bootup`, after dracut-pre-pivot.service and before
> > > initrd-cleanup.service is a good place.
> > > initrd.target
> > > |
> > > v
> > > dracut-pre-pivot.service
> > > |
> > > v
> > > initrd-cleanup.service
> > > isolates to
> > > initrd-switch-root.target
> >
> > Yes, this is clear to me.
> >
> > >
> > > > Probably some other service order issue caused the cleanup service
> > > > running early than the dump capture service?
> > >
> > > Through a simple grep, it can be found that initrd-parse-etc.service
> > > has "ExecStart=systemctl --no-block start
initrd-cleanup.service".
> > > Coiby found that this problem will disappear if we revert the
> > > patch(45bcfcb36c: units/initrd-parse-etc.service: only start units
> > > that are required)[1] on initrd-parse-etc.service.
> >
> > Ok, good catch. Actually I do not think initrd-parse-etc is useful in
> > kdump initramfs, any use case for it?
> > If this service is for mount the real root /etc/fstab mounts then why
> > we need it, it seemd cause some problems before but we did not have a
> > deep look.
> >
> > How about remove both initrd-parse-etc and initrd-cleanup, does
> > anyone see something could break?
>
> Hi Lichen, or maybe you can experiment with patching the
> initrd-parse-etc and initrd-cleanup with below:
> ConditionPathExists=!/proc/vmcore
>
> See if it works or not..
Hi Dave, I did several tests in my environment and found no error log
for the time being. vmcore was successfully captured.
I'll post a V2 patch after I figure out their dependencies.
Very good! I vaguely remember previously we got bug reports that the
system switched to the real root filesystem after capturing vmcore,
and later it can not be reproduced again, I suspect it is caused by
the subtle order issue maybe caused by the initrd-parse-etc and
initrd-cleanup..
>
>
> >
> > >
> > > >
> > > > In another way, can we just remove the initrd-cleanup.service from
> > > > kdump initramfs? Or make it after the kdump-capture.service
explicitly
> > > > in the initrd-cleanup.service file.
> > >
> > > initrd-parse-etc.service will fail if we simply remove the
> > > initrd-cleanup.service, and then kdump fails.
> > > initrd-cleanup.service is indeed run after kdump-capture.service,
> > > because the latter contains Before=initrd-cleanup.service and is
> > > oneshot type service.
> > > According to the man page of systemd, the service manager will
> > > consider the unit up after the main process exits if it is oneshot.
> > > So I guess the problem is not the dependency between
> > > kdump-capture.service and initrd-cleanup.service.
> >
> > kdump-capture service should only exit after kdump.sh finish its work
> > if the main process meas about the "kdump.sh", anyway it seems the
key
> > problem is the initrd-parse-etc and the initrd-cleanup from your
> > reply.
> >
> > >
> > > I lack knowledge about systemd, and it seems that systemd dependencies
> > > are not at least as simple as I imagined.
> > >
> > > [1]
https://github.com/systemd/systemd/commit/45bcfcb36cec9bf810686ed956ff215...
> > > >
> > > > >
> > > > > This is useful during normal boot. However, in kdump, after
capturing vmcore,
> > > > > we don't proceed with steps like switch-root. Therefore,
removing these
> > > > > dependencies to keep multipathd running during kdump is safe.
> > > > >
> > > > > Suggested-by: Benjamin Marzinski <bmarzins(a)redhat.com>
> > > > > Signed-off-by: Lichen Liu <lichliu(a)redhat.com>
> > > > > ---
> > > > > dracut-module-setup.sh | 8 ++++++++
> > > > > 1 file changed, 8 insertions(+)
> > > > >
> > > > > diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh
> > > > > index 1dc88dc..6d2176f 100755
> > > > > --- a/dracut-module-setup.sh
> > > > > +++ b/dracut-module-setup.sh
> > > > > @@ -1089,6 +1089,14 @@ install() {
> > > > >
's/\(^[[:space:]]*reserved_memory[[:space:]]*=\)[[:space:]]*[[:digit:]]*/\1 1024/'
\
> > > > > "${initdir}/etc/lvm/lvm.conf" &>
/dev/null
> > > > >
> > > > > + # For the multipath target under kdump, we can safely
remove
> > > > > + # "Before=initrd-cleanup.service" and
"Conflicts=initrd-cleanup.service"
> > > > > + # from multipathd.service to keep multipathd running under
kdump.
> > > > > + sed -i \
> > > > > + -e '/^Before=initrd-cleanup.service$/d' \
> > > > > + -e '/^Conflicts=initrd-cleanup.service$/d' \
> > > > > +
"${initdir}/${systemdsystemunitdir}/multipathd.service" &> /dev/null
> > > > > +
> > > > > # Save more memory by dropping switch root capability
> > > > > dracut_no_switch_root
> > > > > }
> > > > > --
> > > > > 2.43.0
> > > > > --
> > > > > _______________________________________________
> > > > > kexec mailing list -- kexec(a)lists.fedoraproject.org
> > > > > To unsubscribe send an email to
kexec-leave(a)lists.fedoraproject.org
> > > > > Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > > > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > > > List Archives:
https://lists.fedoraproject.org/archives/list/kexec@lists.fedoraproject.org
> > > > > Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
> > > >
> > >
>