On 06/03/14 at 10:37am, Vivek Goyal wrote:
On Tue, Jun 03, 2014 at 03:55:50PM +0800, WANG Chao wrote:
> When starting kdump service with dump target being ssh host, after
> network-online.target, we connect to ssh host and touch the dump
> directory to make sure the host is ready to be dumped to.
>
> Chances are after network-online.target, the particular network resource
> we interest in isn't ready for connecting to the specified ssh host.
> And at that time, we connect to ssh host and fail.
>
> What we should do is to wait for the specific network resource, not
> totally depending on network-online.target. But it's relatively
> complicated to implement. A simple and direct solution would be try as
> many time as it needs to connect to the configured ssh host. However to
> avoid a infinitely loop, we time out and fail. I set this time out value
> to be 180 seconds, and general speaking, 180 seconds would be enough for
> almost any kind of network to be up and ready.
>
> Vivek:
> - sleep 2 seconds after each retry.
> - output a brief message for each retry/failure.
>
> Signed-off-by: WANG Chao <chaowang(a)redhat.com>
> ---
> kdumpctl | 18 ++++++++++++++++--
> 1 file changed, 16 insertions(+), 2 deletions(-)
>
> diff --git a/kdumpctl b/kdumpctl
> index 9cae0c4..5e4b1b0 100755
> --- a/kdumpctl
> +++ b/kdumpctl
> @@ -381,9 +381,23 @@ function check_ssh_config()
> function check_ssh_target()
> {
> local _ret
> - ssh -q -i $SSH_KEY_LOCATION -o BatchMode=yes $DUMP_TARGET mkdir -p $SAVE_PATH
> - _ret=$?
> + local _start _delta
> +
> + # Timeout out after 180 seconds, hopefully it's enough.
> + _start=$(date +%s)
> + while : ; do
> + ssh -q -i $SSH_KEY_LOCATION -o BatchMode=yes $DUMP_TARGET mkdir -p $SAVE_PATH
> + _ret=$?
> + _delta=$(($(date +%s) - $_start))
> + if [[ $_ret -eq 0 || $_delta -gt 180 ]]; then
> + break
> + fi
> + echo "ssh to $DUMP_TARGET failed. Will retry after 2 seconds"
> + sleep 2
> + done
> +
> if [ $_ret -ne 0 ]; then
> + echo "ssh failed after multiple tries"
> echo "Could not create $DUMP_TARGET:$SAVE_PATH, you probably need to run
\"kdumpctl propagate\"" >&2
Hold on. So assume that network is up but keys are not propagated or keys
are not valid, we will still keep on retyring? That does not sound right.
We need to retry only if network interface is not up. If ssh fails because
of no keys or wrong keys, then we should not retry.
I'm not sure how can we do this, the return code from ssh is always 255
in any case of failure, ie. wrong key, no key, network issue.