Change in vdsm[master]: fencing: Introduce getHostLeaseStatus API
nsoffer at redhat.com
nsoffer at redhat.com
Sun Jun 22 14:06:32 UTC 2014
Nir Soffer has posted comments on this change.
Change subject: fencing: Introduce getHostLeaseStatus API
......................................................................
Patch Set 5:
>From https://git.fedorahosted.org/cgit/sanlock.git/tree/src/lockspace.c#n937
/*
* After the lockspace starts, there is a limited amount of
* time that we've been watching the other hosts. This means
* we can't make an accurate assessment of their state, because
* the state is based on monitoring the hosts for host_fail_seconds
* and host_dead_seconds, or seeing a renewal. When none of
* those are true (not enough time monitoring and not seeing a
* renewal), we return UNKNOWN.
*
* (Example number of seconds below are based on hosts using the
* default 10 second io timeout.)
*
* * For hosts that are alive when we start, we return:
* UNKNOWN then LIVE
*
* UNKNOWN would typically last for 10-20 seconds, but it's possible that
* UNKNOWN could persist for up to 80 seconds before LIVE is returned.
* LIVE is returned after we see the timestamp change once.
*
* * For hosts that are dead when we start, we'd return:
* UNKNOWN then FAIL then DEAD
*
* UNKNOWN would last for 80 seconds before we return FAIL.
* FAIL would last for 60 more seconds before we return DEAD.
*
* * Hosts that are failing and don't recover would be the same as prev.
*
* * For hosts thet are failing but recover, we'd return:
* UNKNOWN then FAIL then LIVE
*
*
* For another host that is alive when we start,
* the sequence of values is:
*
* 0: we have not yet called check_other_leases()
* first_check = 0, last_check = 0, last_live = 0
*
* other host renews its lease
*
* 10: we call check_other_leases() for the first time,
* first_check = 10, last_check = 10, last_live = 10
*
* other host renews its lease
*
* 20: we call check_other_leases() for the second time,
* first_check = 10, last_check = 20, last_live = 20
*
* At 10, we have not yet seen a renewal from the other host, i.e. we have
* not seen its timestamp change (we only have one sample). The host could
* be dead or alive, so we set the state to UNKNOWN. The way we know
* that we have not yet observed the timestamp change is that
* first_check == last_live, (10 == 10).
*
* At 20, we have seen a renewal, i.e. the timestamp changed between checks,
* so we return LIVE.
*
* In the other case, if the host was actually dead, not alive, it would not
* have renewed between 10 and 20. So at 20 we would continue to see
* first_check == last_live, and would return UNKNOWN. If the host remains
* dead, we'd continue to report UNKNOWN for the first 80 seconds.
* After 80 seconds, we'd return FAIL. After 140 seconds we'd return DEAD.
*/
I think we need to take couple of samples, starting when we suspect that a host
is not healthy, and finishing at least 80 seconds later. Then when we fence
a host, we can look at the samples and apply various policies. The simplest
policy would be to avoid fencing if a host is live on at least one domain after
waiting 80 seconds or so.
Maybe we should move this discussion to the mailing list?
--
To view, visit http://gerrit.ovirt.org/28873
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Iccd62e58a194aa0ceb0f5e2503b8ec7e4349971b
Gerrit-PatchSet: 5
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer at redhat.com>
Gerrit-Reviewer: Allon Mureinik <amureini at redhat.com>
Gerrit-Reviewer: Barak Azulay <bazulay at redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken at redhat.com>
Gerrit-Reviewer: Federico Simoncelli <fsimonce at redhat.com>
Gerrit-Reviewer: Itamar Heim <iheim at redhat.com>
Gerrit-Reviewer: Nir Soffer <nsoffer at redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski at gmail.com>
Gerrit-Reviewer: Saggi Mizrahi <smizrahi at redhat.com>
Gerrit-Reviewer: Xavi Francisco <xfrancis at redhat.com>
Gerrit-Reviewer: Yoav Kleinberger <ykleinbe at redhat.com>
Gerrit-Reviewer: automation at ovirt.org
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No
More information about the vdsm-patches
mailing list