cluster: RHEL6 - config: Fix ccs_tool create -n
by Fabio M. Di Nitto
Gitweb: http://git.fedorahosted.org/git/cluster.git?p=cluster.git;a=commitdiff;h=...
Commit: b4317363d5db48458a015ac4d4de63a5048a9509
Parent: 4039bf4817a96b6aab20de948389f43b89ce4a8e
Author: Christine Caulfield <ccaulfie(a)redhat.com>
AuthorDate: Fri May 21 09:09:26 2010 +0100
Committer: Fabio M. Di Nitto <fdinitto(a)redhat.com>
CommitterDate: Tue May 25 14:41:48 2010 +0200
config: Fix ccs_tool create -n
Make 'ccs_tool create -n <x>' create a skeleton with <x> nodes rather
than <x-1>.
Resolves: rhbz#594626
Signed-off-by: Christine Caulfield <ccaulfie(a)redhat.com>
---
config/tools/ccs_tool/editconf.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/config/tools/ccs_tool/editconf.c b/config/tools/ccs_tool/editconf.c
index 323c561..a0371b4 100644
--- a/config/tools/ccs_tool/editconf.c
+++ b/config/tools/ccs_tool/editconf.c
@@ -1031,7 +1031,7 @@ void create_skeleton(int argc, char **argv)
}
fprintf(outfile, " <clusternodes>\n");
- for (i=1; i< numnodes; i++) {
+ for (i=1; i <= numnodes; i++) {
fprintf(outfile, " <clusternode name=\"NEEDNAME-%02d\" votes=\"1\" nodeid=\"%d\">\n", i, i);
fprintf(outfile, " <fence>\n");
fprintf(outfile, " <method name=\"single\">\n");
13 years, 11 months
dlm: master - dlm_controld: wrong fencing time comparison
by David Teigland
Gitweb: http://git.fedorahosted.org/git/dlm.git?p=dlm.git;a=commitdiff;h=1ff741d5...
Commit: 1ff741d523bda31af31971ba8a898aa391c004ef
Parent: 6607f2d209adb983b4642faef8a66f91ba9c727d
Author: David Teigland <teigland(a)redhat.com>
AuthorDate: Thu May 20 16:40:20 2010 -0500
Committer: David Teigland <teigland(a)redhat.com>
CommitterDate: Fri May 21 11:22:14 2010 -0500
dlm_controld: wrong fencing time comparison
dlm_controld checks that a failed node has been fenced by comparing the
time it was fenced to the time it properly joined the cpg. In one of
Nate's QE tests, he found a case where the failed node rejoined the cpg
just before the fencing time was recorded. This causes dlm_controld to
get stuck waiting for a fencing to occur when it already has.
1274304702 dlm:ls:clvmd conf 2 0 1 memb 1 4 join left 3
1274304702 clvmd add_change cg 2 remove nodeid 3 reason 3
1274304702 clvmd add_change cg 2 counts member 2 joined 0 remove 1 failed 1
1274304702 clvmd stop_kernel cg 2
1274304702 write "0" to "/sys/kernel/dlm/clvmd/control"
1274304702 clvmd check_fencing 3 not fenced add 1274304347 fence 0
1274304702 dlm:ls:clvmd conf 1 0 1 memb 1 join left 4
1274304702 clvmd add_change cg 3 remove nodeid 4 reason 3
1274304702 clvmd add_change cg 3 counts member 1 joined 0 remove 1 failed 1
1274304702 clvmd check_fencing 4 not fenced add 1274304347 fence 0
....
1274304741 cluster node 3 added
...
1274304752 dlm:ls:clvmd conf 2 1 0 memb 1 3 join 3 left
1274304752 clvmd add_change cg 4 joined nodeid 3
1274304752 clvmd add_change cg 4 counts member 2 joined 1 remove 0 failed 0
1274304754 clvmd receive_start 3:1 len 80
1274304754 clvmd match_change 3:1 skip 2 sender not member
1274304754 clvmd match_change 3:1 skip 3 sender not member
1274304754 clvmd match_change 3:1 matches cg 4
1274304754 clvmd check_fencing 3 not fenced add 1274304754 fence 1274304753
1274304754 clvmd check_fencing 4 1274304347 fenced at 1274304753
...
1274304832 dlm:ls:clvmd conf 3 1 0 memb 1 3 4 join 4 left
1274304832 clvmd add_change cg 5 joined nodeid 4
1274304832 clvmd add_change cg 5 counts member 3 joined 1 remove 0 failed 0
1274304832 clvmd receive_start 4:1 len 84
1274304832 clvmd match_change 4:1 skip cg 2 created 1274304702 cluster add
12743
1274304832 clvmd match_change 4:1 skip 3 sender not member
1274304832 clvmd match_change 4:1 skip 4 sender not member
1274304832 clvmd match_change 4:1 matches cg 5
1274304832 clvmd receive_start 3:2 len 84
1274304832 clvmd match_change 3:2 skip 2 sender not member
1274304832 clvmd match_change 3:2 skip 3 sender not member
1274304832 clvmd match_change 3:2 skip 4 already start
1274304832 clvmd match_change 3:2 matches cg 5
This line: clvmd check_fencing 3 not fenced add 1274304754 fence
1274304753 shows that 3 was added to the cpg one second after fenced
recorded that fencing was done. The fenced log confirms that this is when
it received the message indicating the 3 was fenced:
1274304753 receive_victim_done 1:3 flags 2 len 80
1274304753 receive_victim_done 1:3 remove victim 3 time 1274304753 how 2
I've not been able to reproduce this, but I believe the fix is for
dlm_controld to check that the fencing time is later than the time it saw
the node fail instead of later than the time it saw the node most recently
join.
bz 594511
Signed-off-by: David Teigland <teigland(a)redhat.com>
---
group/dlm_controld/cpg.c | 19 +++++++++++++------
1 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/group/dlm_controld/cpg.c b/group/dlm_controld/cpg.c
index 2eb3d65..94a9b68 100644
--- a/group/dlm_controld/cpg.c
+++ b/group/dlm_controld/cpg.c
@@ -45,6 +45,7 @@ struct node {
int check_fs;
int fs_notified;
uint64_t add_time;
+ uint64_t fail_time;
uint64_t fence_time; /* for debug */
uint64_t cluster_add_time;
uint64_t cluster_remove_time;
@@ -356,12 +357,13 @@ static void free_ls(struct lockspace *ls)
has failed yet).
So, check that:
- 1. has fenced fenced the node after it joined this lockspace?
+ 1. has fenced fenced the node since we saw it fail?
2. fenced has no outstanding fencing ops
For 1:
- - record the time of the first good start message we see from node X
- node X fails
+ - we see node X fail and X has non-zero add_time,
+ set check_fencing and record the fail time
- wait for X to be removed from all dlm cpg's (probably not necessary)
- check that the fencing time is later than the recorded time above
@@ -499,6 +501,7 @@ static void node_history_fail(struct lockspace *ls, int nodeid,
node->check_fencing = 1;
node->fence_time = 0;
node->fence_queries = 0;
+ node->fail_time = time(NULL);
}
/* fenced will take care of making sure the quorum value
@@ -539,10 +542,12 @@ static int check_fencing_done(struct lockspace *ls)
if (rv < 0)
log_error("fenced_node_info error %d", rv);
- if (last_fenced_time > node->add_time) {
- log_group(ls, "check_fencing %d %llu fenced at %llu",
+ if (last_fenced_time > node->fail_time) {
+ log_group(ls, "check_fencing %d done "
+ "add %llu fail %llu last %llu",
node->nodeid,
(unsigned long long)node->add_time,
+ (unsigned long long)node->fail_time,
(unsigned long long)last_fenced_time);
node->check_fencing = 0;
node->add_time = 0;
@@ -550,9 +555,11 @@ static int check_fencing_done(struct lockspace *ls)
} else {
if (!node->fence_queries ||
node->fence_time != last_fenced_time) {
- log_group(ls, "check_fencing %d not fenced "
- "add %llu fence %llu", node->nodeid,
+ log_group(ls, "check_fencing %d wait "
+ "add %llu fail %llu last %llu",
+ node->nodeid,
(unsigned long long)node->add_time,
+ (unsigned long long)node->fail_time,
(unsigned long long)last_fenced_time);
node->fence_queries++;
node->fence_time = last_fenced_time;
13 years, 11 months
cluster: RHEL6 - dlm_controld: wrong fencing time comparison
by David Teigland
Gitweb: http://git.fedorahosted.org/git/cluster.git?p=cluster.git;a=commitdiff;h=...
Commit: 4039bf4817a96b6aab20de948389f43b89ce4a8e
Parent: 6dd191b5a8044c60caba7109b77583b04823bd30
Author: David Teigland <teigland(a)redhat.com>
AuthorDate: Thu May 20 16:40:20 2010 -0500
Committer: David Teigland <teigland(a)redhat.com>
CommitterDate: Fri May 21 11:21:33 2010 -0500
dlm_controld: wrong fencing time comparison
dlm_controld checks that a failed node has been fenced by comparing the
time it was fenced to the time it properly joined the cpg. In one of
Nate's QE tests, he found a case where the failed node rejoined the cpg
just before the fencing time was recorded. This causes dlm_controld to
get stuck waiting for a fencing to occur when it already has.
1274304702 dlm:ls:clvmd conf 2 0 1 memb 1 4 join left 3
1274304702 clvmd add_change cg 2 remove nodeid 3 reason 3
1274304702 clvmd add_change cg 2 counts member 2 joined 0 remove 1 failed 1
1274304702 clvmd stop_kernel cg 2
1274304702 write "0" to "/sys/kernel/dlm/clvmd/control"
1274304702 clvmd check_fencing 3 not fenced add 1274304347 fence 0
1274304702 dlm:ls:clvmd conf 1 0 1 memb 1 join left 4
1274304702 clvmd add_change cg 3 remove nodeid 4 reason 3
1274304702 clvmd add_change cg 3 counts member 1 joined 0 remove 1 failed 1
1274304702 clvmd check_fencing 4 not fenced add 1274304347 fence 0
....
1274304741 cluster node 3 added
...
1274304752 dlm:ls:clvmd conf 2 1 0 memb 1 3 join 3 left
1274304752 clvmd add_change cg 4 joined nodeid 3
1274304752 clvmd add_change cg 4 counts member 2 joined 1 remove 0 failed 0
1274304754 clvmd receive_start 3:1 len 80
1274304754 clvmd match_change 3:1 skip 2 sender not member
1274304754 clvmd match_change 3:1 skip 3 sender not member
1274304754 clvmd match_change 3:1 matches cg 4
1274304754 clvmd check_fencing 3 not fenced add 1274304754 fence 1274304753
1274304754 clvmd check_fencing 4 1274304347 fenced at 1274304753
...
1274304832 dlm:ls:clvmd conf 3 1 0 memb 1 3 4 join 4 left
1274304832 clvmd add_change cg 5 joined nodeid 4
1274304832 clvmd add_change cg 5 counts member 3 joined 1 remove 0 failed 0
1274304832 clvmd receive_start 4:1 len 84
1274304832 clvmd match_change 4:1 skip cg 2 created 1274304702 cluster add
12743
1274304832 clvmd match_change 4:1 skip 3 sender not member
1274304832 clvmd match_change 4:1 skip 4 sender not member
1274304832 clvmd match_change 4:1 matches cg 5
1274304832 clvmd receive_start 3:2 len 84
1274304832 clvmd match_change 3:2 skip 2 sender not member
1274304832 clvmd match_change 3:2 skip 3 sender not member
1274304832 clvmd match_change 3:2 skip 4 already start
1274304832 clvmd match_change 3:2 matches cg 5
This line: clvmd check_fencing 3 not fenced add 1274304754 fence
1274304753 shows that 3 was added to the cpg one second after fenced
recorded that fencing was done. The fenced log confirms that this is when
it received the message indicating the 3 was fenced:
1274304753 receive_victim_done 1:3 flags 2 len 80
1274304753 receive_victim_done 1:3 remove victim 3 time 1274304753 how 2
I've not been able to reproduce this, but I believe the fix is for
dlm_controld to check that the fencing time is later than the time it saw
the node fail instead of later than the time it saw the node most recently
join.
bz 594511
Signed-off-by: David Teigland <teigland(a)redhat.com>
---
group/dlm_controld/cpg.c | 19 +++++++++++++------
1 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/group/dlm_controld/cpg.c b/group/dlm_controld/cpg.c
index 2eb3d65..94a9b68 100644
--- a/group/dlm_controld/cpg.c
+++ b/group/dlm_controld/cpg.c
@@ -45,6 +45,7 @@ struct node {
int check_fs;
int fs_notified;
uint64_t add_time;
+ uint64_t fail_time;
uint64_t fence_time; /* for debug */
uint64_t cluster_add_time;
uint64_t cluster_remove_time;
@@ -356,12 +357,13 @@ static void free_ls(struct lockspace *ls)
has failed yet).
So, check that:
- 1. has fenced fenced the node after it joined this lockspace?
+ 1. has fenced fenced the node since we saw it fail?
2. fenced has no outstanding fencing ops
For 1:
- - record the time of the first good start message we see from node X
- node X fails
+ - we see node X fail and X has non-zero add_time,
+ set check_fencing and record the fail time
- wait for X to be removed from all dlm cpg's (probably not necessary)
- check that the fencing time is later than the recorded time above
@@ -499,6 +501,7 @@ static void node_history_fail(struct lockspace *ls, int nodeid,
node->check_fencing = 1;
node->fence_time = 0;
node->fence_queries = 0;
+ node->fail_time = time(NULL);
}
/* fenced will take care of making sure the quorum value
@@ -539,10 +542,12 @@ static int check_fencing_done(struct lockspace *ls)
if (rv < 0)
log_error("fenced_node_info error %d", rv);
- if (last_fenced_time > node->add_time) {
- log_group(ls, "check_fencing %d %llu fenced at %llu",
+ if (last_fenced_time > node->fail_time) {
+ log_group(ls, "check_fencing %d done "
+ "add %llu fail %llu last %llu",
node->nodeid,
(unsigned long long)node->add_time,
+ (unsigned long long)node->fail_time,
(unsigned long long)last_fenced_time);
node->check_fencing = 0;
node->add_time = 0;
@@ -550,9 +555,11 @@ static int check_fencing_done(struct lockspace *ls)
} else {
if (!node->fence_queries ||
node->fence_time != last_fenced_time) {
- log_group(ls, "check_fencing %d not fenced "
- "add %llu fence %llu", node->nodeid,
+ log_group(ls, "check_fencing %d wait "
+ "add %llu fail %llu last %llu",
+ node->nodeid,
(unsigned long long)node->add_time,
+ (unsigned long long)node->fail_time,
(unsigned long long)last_fenced_time);
node->fence_queries++;
node->fence_time = last_fenced_time;
13 years, 11 months
cluster: STABLE3 - dlm_controld: wrong fencing time comparison
by David Teigland
Gitweb: http://git.fedorahosted.org/git/cluster.git?p=cluster.git;a=commitdiff;h=...
Commit: b165ce47df6f03c340abf4baa48b74368d12bf54
Parent: 1878f0bbffcf60f00a93bf2ac806af3caae86a49
Author: David Teigland <teigland(a)redhat.com>
AuthorDate: Thu May 20 16:40:20 2010 -0500
Committer: David Teigland <teigland(a)redhat.com>
CommitterDate: Fri May 21 11:20:28 2010 -0500
dlm_controld: wrong fencing time comparison
dlm_controld checks that a failed node has been fenced by comparing the
time it was fenced to the time it properly joined the cpg. In one of
Nate's QE tests, he found a case where the failed node rejoined the cpg
just before the fencing time was recorded. This causes dlm_controld to
get stuck waiting for a fencing to occur when it already has.
1274304702 dlm:ls:clvmd conf 2 0 1 memb 1 4 join left 3
1274304702 clvmd add_change cg 2 remove nodeid 3 reason 3
1274304702 clvmd add_change cg 2 counts member 2 joined 0 remove 1 failed 1
1274304702 clvmd stop_kernel cg 2
1274304702 write "0" to "/sys/kernel/dlm/clvmd/control"
1274304702 clvmd check_fencing 3 not fenced add 1274304347 fence 0
1274304702 dlm:ls:clvmd conf 1 0 1 memb 1 join left 4
1274304702 clvmd add_change cg 3 remove nodeid 4 reason 3
1274304702 clvmd add_change cg 3 counts member 1 joined 0 remove 1 failed 1
1274304702 clvmd check_fencing 4 not fenced add 1274304347 fence 0
....
1274304741 cluster node 3 added
...
1274304752 dlm:ls:clvmd conf 2 1 0 memb 1 3 join 3 left
1274304752 clvmd add_change cg 4 joined nodeid 3
1274304752 clvmd add_change cg 4 counts member 2 joined 1 remove 0 failed 0
1274304754 clvmd receive_start 3:1 len 80
1274304754 clvmd match_change 3:1 skip 2 sender not member
1274304754 clvmd match_change 3:1 skip 3 sender not member
1274304754 clvmd match_change 3:1 matches cg 4
1274304754 clvmd check_fencing 3 not fenced add 1274304754 fence 1274304753
1274304754 clvmd check_fencing 4 1274304347 fenced at 1274304753
...
1274304832 dlm:ls:clvmd conf 3 1 0 memb 1 3 4 join 4 left
1274304832 clvmd add_change cg 5 joined nodeid 4
1274304832 clvmd add_change cg 5 counts member 3 joined 1 remove 0 failed 0
1274304832 clvmd receive_start 4:1 len 84
1274304832 clvmd match_change 4:1 skip cg 2 created 1274304702 cluster add
12743
1274304832 clvmd match_change 4:1 skip 3 sender not member
1274304832 clvmd match_change 4:1 skip 4 sender not member
1274304832 clvmd match_change 4:1 matches cg 5
1274304832 clvmd receive_start 3:2 len 84
1274304832 clvmd match_change 3:2 skip 2 sender not member
1274304832 clvmd match_change 3:2 skip 3 sender not member
1274304832 clvmd match_change 3:2 skip 4 already start
1274304832 clvmd match_change 3:2 matches cg 5
This line: clvmd check_fencing 3 not fenced add 1274304754 fence
1274304753 shows that 3 was added to the cpg one second after fenced
recorded that fencing was done. The fenced log confirms that this is when
it received the message indicating the 3 was fenced:
1274304753 receive_victim_done 1:3 flags 2 len 80
1274304753 receive_victim_done 1:3 remove victim 3 time 1274304753 how 2
I've not been able to reproduce this, but I believe the fix is for
dlm_controld to check that the fencing time is later than the time it saw
the node fail instead of later than the time it saw the node most recently
join.
bz 594511
Signed-off-by: David Teigland <teigland(a)redhat.com>
---
group/dlm_controld/cpg.c | 19 +++++++++++++------
1 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/group/dlm_controld/cpg.c b/group/dlm_controld/cpg.c
index 2eb3d65..94a9b68 100644
--- a/group/dlm_controld/cpg.c
+++ b/group/dlm_controld/cpg.c
@@ -45,6 +45,7 @@ struct node {
int check_fs;
int fs_notified;
uint64_t add_time;
+ uint64_t fail_time;
uint64_t fence_time; /* for debug */
uint64_t cluster_add_time;
uint64_t cluster_remove_time;
@@ -356,12 +357,13 @@ static void free_ls(struct lockspace *ls)
has failed yet).
So, check that:
- 1. has fenced fenced the node after it joined this lockspace?
+ 1. has fenced fenced the node since we saw it fail?
2. fenced has no outstanding fencing ops
For 1:
- - record the time of the first good start message we see from node X
- node X fails
+ - we see node X fail and X has non-zero add_time,
+ set check_fencing and record the fail time
- wait for X to be removed from all dlm cpg's (probably not necessary)
- check that the fencing time is later than the recorded time above
@@ -499,6 +501,7 @@ static void node_history_fail(struct lockspace *ls, int nodeid,
node->check_fencing = 1;
node->fence_time = 0;
node->fence_queries = 0;
+ node->fail_time = time(NULL);
}
/* fenced will take care of making sure the quorum value
@@ -539,10 +542,12 @@ static int check_fencing_done(struct lockspace *ls)
if (rv < 0)
log_error("fenced_node_info error %d", rv);
- if (last_fenced_time > node->add_time) {
- log_group(ls, "check_fencing %d %llu fenced at %llu",
+ if (last_fenced_time > node->fail_time) {
+ log_group(ls, "check_fencing %d done "
+ "add %llu fail %llu last %llu",
node->nodeid,
(unsigned long long)node->add_time,
+ (unsigned long long)node->fail_time,
(unsigned long long)last_fenced_time);
node->check_fencing = 0;
node->add_time = 0;
@@ -550,9 +555,11 @@ static int check_fencing_done(struct lockspace *ls)
} else {
if (!node->fence_queries ||
node->fence_time != last_fenced_time) {
- log_group(ls, "check_fencing %d not fenced "
- "add %llu fence %llu", node->nodeid,
+ log_group(ls, "check_fencing %d wait "
+ "add %llu fail %llu last %llu",
+ node->nodeid,
(unsigned long long)node->add_time,
+ (unsigned long long)node->fail_time,
(unsigned long long)last_fenced_time);
node->fence_queries++;
node->fence_time = last_fenced_time;
13 years, 11 months
cluster: STABLE3 - config: Fix ccs_tool create -n
by Christine Caulfield
Gitweb: http://git.fedorahosted.org/git/cluster.git?p=cluster.git;a=commitdiff;h=...
Commit: 1878f0bbffcf60f00a93bf2ac806af3caae86a49
Parent: 6b676e1ccc192a6c1396f72b637ba89aa254fd17
Author: Christine Caulfield <ccaulfie(a)redhat.com>
AuthorDate: Fri May 21 09:09:26 2010 +0100
Committer: Christine Caulfield <ccaulfie(a)redhat.com>
CommitterDate: Fri May 21 09:09:26 2010 +0100
config: Fix ccs_tool create -n
Make 'ccs_tool create -n <x>' create a skeleton with <x> nodes rather
than <x-1>.
Signed-off-by: Christine Caulfield <ccaulfie(a)redhat.com>
---
config/tools/ccs_tool/editconf.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/config/tools/ccs_tool/editconf.c b/config/tools/ccs_tool/editconf.c
index 323c561..a0371b4 100644
--- a/config/tools/ccs_tool/editconf.c
+++ b/config/tools/ccs_tool/editconf.c
@@ -1031,7 +1031,7 @@ void create_skeleton(int argc, char **argv)
}
fprintf(outfile, " <clusternodes>\n");
- for (i=1; i< numnodes; i++) {
+ for (i=1; i <= numnodes; i++) {
fprintf(outfile, " <clusternode name=\"NEEDNAME-%02d\" votes=\"1\" nodeid=\"%d\">\n", i, i);
fprintf(outfile, " <fence>\n");
fprintf(outfile, " <method name=\"single\">\n");
13 years, 11 months
cluster: RHEL55 - resource-agents: Add status_program attribute
by Lon Hohberger
Gitweb: http://git.fedorahosted.org/git/cluster.git?p=cluster.git;a=commitdiff;h=...
Commit: ea76067bc05359c1582bfd84be4c1b3eb05b230a
Parent: 61a7c0f26c4248778d656bcecb47375c419fafd6
Author: Lon Hohberger <lhh(a)redhat.com>
AuthorDate: Tue May 4 17:54:15 2010 -0400
Committer: Lon Hohberger <lhh(a)redhat.com>
CommitterDate: Thu May 20 16:43:47 2010 -0400
resource-agents: Add status_program attribute
Resolves: bz594476
Signed-off-by: Lon Hohberger <lhh(a)redhat.com>
---
rgmanager/src/resources/vm.sh | 33 ++++++++++++++++++++++++++++++++-
1 files changed, 32 insertions(+), 1 deletions(-)
diff --git a/rgmanager/src/resources/vm.sh b/rgmanager/src/resources/vm.sh
index d2932dd..2a995eb 100755
--- a/rgmanager/src/resources/vm.sh
+++ b/rgmanager/src/resources/vm.sh
@@ -213,6 +213,20 @@ meta_data()
<content type="string" default="0"/>
</parameter>
+ <parameter name="status_program" reconfig="1">
+ <longdesc lang="en">
+ Ordinarily, only the presence/health of a virtual machine
+ is checked. If specified, the status_program value is
+ executed during a depth 10 check. The intent of this
+ program is to ascertain the status of critical services
+ within a virtual machine.
+ </longdesc>
+ <shortdesc lang="en">
+ Additional status check program
+ </shortdesc>
+ <content type="string" default="0"/>
+ </parameter>
+
<parameter name="hypervisor">
<shortdesc lang="en">
Hypervisor
@@ -255,6 +269,10 @@ meta_data()
<action name="status" timeout="10" interval="30"/>
<action name="monitor" timeout="10" interval="30"/>
+ <!-- depth 10 calls the status_program -->
+ <action name="status" depth="10" timeout="10" interval="300"/>
+ <action name="monitor" depth="10" timeout="10" interval="300"/>
+
<!-- reconfigure - reconfigure with new OCF parameters.
NOT OCF COMPATIBLE AT ALL -->
<action name="reconfig" timeout="10"/>
@@ -543,7 +561,11 @@ virsh_status()
return 0
fi
- return 1
+ if [ "$state" = "shut off" ]; then
+ return $OCF_NOT_RUNNING
+ fi
+
+ return $OCF_ERR_GENERIC
}
@@ -923,6 +945,15 @@ case $1 in
validate_all || exit $OCF_ERR_ARGS
echo -n "Virtual machine $OCF_RESKEY_name is "
do_status
+ rv=$?
+ if [ $rv -ne 0 ]; then
+ exit $rv
+ fi
+ [ -z "$OCF_RESKEY_status_program" ] && exit 0
+ [ -z "$OCF_CHECK_LEVEL" ] && exit 0
+ [ $OCF_CHECK_LEVEL -lt 10 ] && exit 0
+
+ bash -c "$OCF_RESKEY_status_program" &> /dev/null
exit $?
;;
migrate)
13 years, 11 months
cluster: RHEL55 - rgmanager: Add per-resource status check tolerances
by Lon Hohberger
Gitweb: http://git.fedorahosted.org/git/cluster.git?p=cluster.git;a=commitdiff;h=...
Commit: 61a7c0f26c4248778d656bcecb47375c419fafd6
Parent: 7a1e2af8cc37409b3a38fbe9b5945d186be9f2ec
Author: Lon Hohberger <lhh(a)redhat.com>
AuthorDate: Tue May 4 17:51:45 2010 -0400
Committer: Lon Hohberger <lhh(a)redhat.com>
CommitterDate: Thu May 20 16:43:35 2010 -0400
rgmanager: Add per-resource status check tolerances
Resolves: bz594476
Signed-off-by: Lon Hohberger <lhh(a)redhat.com>
---
rgmanager/include/reslist.h | 1 +
rgmanager/src/daemons/restree.c | 47 +++++++++++++++++++++++++++++++++++++++
2 files changed, 48 insertions(+), 0 deletions(-)
diff --git a/rgmanager/include/reslist.h b/rgmanager/include/reslist.h
index 0275bfd..001f56b 100644
--- a/rgmanager/include/reslist.h
+++ b/rgmanager/include/reslist.h
@@ -128,6 +128,7 @@ typedef struct _rg_node {
resource_t *rn_resource;
resource_act_t *rn_actions;
restart_counter_t rn_restart_counter;
+ restart_counter_t rn_failure_counter;
int rn_state; /* State of this instance of rn_resource */
int rn_flags;
int rn_last_status;
diff --git a/rgmanager/src/daemons/restree.c b/rgmanager/src/daemons/restree.c
index d5ffea1..3f07f8f 100644
--- a/rgmanager/src/daemons/restree.c
+++ b/rgmanager/src/daemons/restree.c
@@ -565,6 +565,8 @@ do_load_resource(int ccsfd, char *base,
char *ref;
resource_node_t *node;
resource_t *curres;
+ time_t failure_expire = 0;
+ int max_failures = 0;
snprintf(tok, sizeof(tok), "%s/@ref", base);
@@ -659,6 +661,36 @@ do_load_resource(int ccsfd, char *base,
free(ref);
}
+ /* per-resource-node failures / expire times */
+ snprintf(tok, sizeof(tok), "%s/@__max_failures", base);
+#ifndef NO_CCS
+ if (ccs_get(ccsfd, tok, &ref) == 0) {
+#else
+ if (conf_get(tok, &ref) == 0) {
+#endif
+ max_failures = atoi(ref);
+ if (max_failures < 0)
+ max_failures = 0;
+ free(ref);
+ }
+
+ snprintf(tok, sizeof(tok), "%s/@__failure_expire_time", base);
+#ifndef NO_CCS
+ if (ccs_get(ccsfd, tok, &ref) == 0) {
+#else
+ if (conf_get(tok, &ref) == 0) {
+#endif
+ failure_expire = (time_t)expand_time(ref);
+ if ((int64_t)failure_expire < 0)
+ failure_expire = 0;
+ free(ref);
+ }
+
+ if (max_failures && failure_expire) {
+ node->rn_failure_counter = restart_init(failure_expire,
+ max_failures);
+ }
+
curres->r_refs++;
*newnode = node;
@@ -906,6 +938,10 @@ destroy_resource_tree(resource_node_t **tree)
restart_cleanup(node->rn_restart_counter);
}
+ if (node->rn_failure_counter) {
+ restart_cleanup(node->rn_failure_counter);
+ }
+
if(node->rn_actions){
free(node->rn_actions);
}
@@ -1204,6 +1240,15 @@ do_status(resource_node_t *node)
* completed. */
node->rn_actions[idx].ra_last = time(NULL);
+ /* If we have not exceeded our failure count threshold, then fudge
+ * the status check this round */
+ if (x && node->rn_failure_counter) {
+ if (!restart_threshold_exceeded(node->rn_failure_counter)) {
+ x = 0;
+ restart_add(node->rn_failure_counter);
+ }
+ }
+
node->rn_last_status = x;
node->rn_last_depth = node->rn_actions[idx].ra_depth;
node->rn_checked = 1;
@@ -1277,6 +1322,8 @@ clear_checks(resource_node_t *node)
node->rn_actions[x].ra_last = now;
}
+ restart_clear(node->rn_failure_counter);
+
node->rn_checked = 0;
node->rn_last_status = 0;
node->rn_last_depth = 0;
13 years, 11 months
cluster: RHEL55 - rgmanager: Fix max_restart threshold handling
by Lon Hohberger
Gitweb: http://git.fedorahosted.org/git/cluster.git?p=cluster.git;a=commitdiff;h=...
Commit: 7a1e2af8cc37409b3a38fbe9b5945d186be9f2ec
Parent: 75c7ecae61d9400d084227304d5b7068fdaa310b
Author: Lon Hohberger <lhh(a)redhat.com>
AuthorDate: Tue May 4 15:03:23 2010 -0400
Committer: Lon Hohberger <lhh(a)redhat.com>
CommitterDate: Thu May 20 16:43:18 2010 -0400
rgmanager: Fix max_restart threshold handling
Max_restart / restart_expire_time was not being correctly
handled, resulting in inconsistent amounts of restarts prior
to relocation (or in some cases, no relocation at all)
Resolves: bz594476
Signed-off-by: Lon Hohberger <lhh(a)redhat.com>
---
rgmanager/include/resgroup.h | 1 +
rgmanager/src/daemons/groups.c | 16 ++++++++++++++++
rgmanager/src/daemons/restart_counter.c | 5 +++++
rgmanager/src/daemons/rg_state.c | 6 +++++-
rgmanager/src/daemons/slang_event.c | 20 ++++++++++++++++++--
5 files changed, 45 insertions(+), 3 deletions(-)
diff --git a/rgmanager/include/resgroup.h b/rgmanager/include/resgroup.h
index ec3e67d..c8e7e2d 100644
--- a/rgmanager/include/resgroup.h
+++ b/rgmanager/include/resgroup.h
@@ -144,6 +144,7 @@ int svc_freeze(char *svcName);
int svc_unfreeze(char *svcName);
int svc_migrate(char *svcName, int target);
int check_restart(char *svcName);
+int clear_restart(const char *rg_name);
int add_restart(char *svcName);
int rt_enqueue_request(const char *resgroupname, int request,
diff --git a/rgmanager/src/daemons/groups.c b/rgmanager/src/daemons/groups.c
index 9caf746..9818aa3 100644
--- a/rgmanager/src/daemons/groups.c
+++ b/rgmanager/src/daemons/groups.c
@@ -1945,6 +1945,22 @@ check_restart(char *rg_name)
return ret;
}
+int
+clear_restart(const char *rg_name)
+{
+ resource_node_t *node;
+ int ret = 0;
+
+ pthread_rwlock_rdlock(&resource_lock);
+ node = node_by_ref(&_tree, rg_name);
+ if (node) {
+ ret = restart_clear(node->rn_restart_counter);
+ }
+ pthread_rwlock_unlock(&resource_lock);
+
+ return ret;
+}
+
void
kill_resource_groups(void)
diff --git a/rgmanager/src/daemons/restart_counter.c b/rgmanager/src/daemons/restart_counter.c
index c889c6d..d909713 100644
--- a/rgmanager/src/daemons/restart_counter.c
+++ b/rgmanager/src/daemons/restart_counter.c
@@ -107,6 +107,11 @@ restart_threshold_exceeded(restart_counter_t arg)
restart_info_t *restarts = (restart_info_t *)arg;
time_t now;
+ if (!arg)
+ /* No max restarts / threshold = always
+ ok to restart! */
+ return 0;
+
VALIDATE(arg, -1);
now = time(NULL);
restart_timer_purge(arg, now);
diff --git a/rgmanager/src/daemons/rg_state.c b/rgmanager/src/daemons/rg_state.c
index 49464d5..3e57748 100644
--- a/rgmanager/src/daemons/rg_state.c
+++ b/rgmanager/src/daemons/rg_state.c
@@ -812,7 +812,6 @@ svc_start(char *svcName, int req)
svcStatus.rs_transition = (uint64_t)time(NULL);
if (svcStatus.rs_state == RG_STATE_RECOVER) {
- add_restart(svcName);
svcStatus.rs_restarts++;
} else {
svcStatus.rs_restarts = 0;
@@ -2143,20 +2142,25 @@ handle_recover_req(char *svcName, int *new_owner)
get_recovery_policy(svcName, policy, sizeof(policy));
if (!strcasecmp(policy, "disable")) {
+ clear_restart(svcName);
return svc_disable(svcName);
} else if (!strcasecmp(policy, "relocate")) {
+ clear_restart(svcName);
return handle_relocate_req(svcName, RG_START_RECOVER, -1,
new_owner);
}
/* Check restart counter/timer for this resource */
if (check_restart(svcName) > 0) {
+ clear_restart(svcName);
clulog(LOG_NOTICE, "Restart threshold for %s exceeded; "
"attempting to relocate\n", svcName);
return handle_relocate_req(svcName, RG_START_RECOVER, -1,
new_owner);
}
+ add_restart(svcName);
+
return handle_start_req(svcName, RG_START_RECOVER, new_owner);
}
diff --git a/rgmanager/src/daemons/slang_event.c b/rgmanager/src/daemons/slang_event.c
index 8a5493c..086ac6b 100644
--- a/rgmanager/src/daemons/slang_event.c
+++ b/rgmanager/src/daemons/slang_event.c
@@ -1123,22 +1123,38 @@ int
S_service_event(const char *file, const char *script, char *name,
int state, int owner, int last_owner)
{
+ char policy[20];
int ret;
_service_name = name;
_service_state = (char *)rg_state_str(state);
_service_owner = owner;
_service_last_owner = last_owner;
- _service_restarts_exceeded = check_restart(name);
+ _service_restarts_exceeded = 0;
switch(state) {
case RG_STATE_DISABLED:
case RG_STATE_STOPPED:
case RG_STATE_FAILED:
- case RG_STATE_RECOVER:
case RG_STATE_ERROR:
/* There is no owner for these states. Ever. */
_service_owner = -1;
+ break;
+ case RG_STATE_RECOVER:
+ get_recovery_policy(name, policy, sizeof(policy));
+ if (!strcasecmp(policy, "restart")) {
+ _service_restarts_exceeded = check_restart(name);
+ add_restart(name);
+ } else if (!strcasecmp(policy, "relocate")) {
+ /* Restart threshold is always exceeded
+ * with 'relocate' recovery policy */
+ _service_restarts_exceeded = 1;
+ }
+ _service_owner = -1;
+ }
+
+ if (_service_restarts_exceeded) {
+ clear_restart(name);
}
ret = do_slang_run(file, script);
13 years, 11 months
cluster: RHEL55 - rgmanager: Fix resource start/check times
by Lon Hohberger
Gitweb: http://git.fedorahosted.org/git/cluster.git?p=cluster.git;a=commitdiff;h=...
Commit: 75c7ecae61d9400d084227304d5b7068fdaa310b
Parent: 5bde34c1fee9e5c5ba70d4997898328eb07d5b70
Author: Lon Hohberger <lhh(a)redhat.com>
AuthorDate: Tue May 4 14:58:58 2010 -0400
Committer: Lon Hohberger <lhh(a)redhat.com>
CommitterDate: Thu May 20 16:42:46 2010 -0400
rgmanager: Fix resource start/check times
Resolves: bz#594476
Signed-off-by: Lon Hohberger <lhh(a)redhat.com>
---
rgmanager/include/reslist.h | 1 -
rgmanager/src/daemons/restree.c | 3 +--
2 files changed, 1 insertions(+), 3 deletions(-)
diff --git a/rgmanager/include/reslist.h b/rgmanager/include/reslist.h
index 4ce3838..0275bfd 100644
--- a/rgmanager/include/reslist.h
+++ b/rgmanager/include/reslist.h
@@ -115,7 +115,6 @@ typedef struct _resource {
char * r_name;
resource_attr_t * r_attrs;
resource_act_t * r_actions;
- time_t r_started; /** Time this resource was last started */
int r_flags;
int r_refs;
int r_incarnations; /** Number of instances running locally */
diff --git a/rgmanager/src/daemons/restree.c b/rgmanager/src/daemons/restree.c
index c4634a5..d5ffea1 100644
--- a/rgmanager/src/daemons/restree.c
+++ b/rgmanager/src/daemons/restree.c
@@ -1265,9 +1265,8 @@ clear_checks(resource_node_t *node)
{
time_t now;
int x = 0;
- resource_t *res = node->rn_resource;
- now = res->r_started;
+ now = get_time((char *)"start", 0, node);
for (; node->rn_actions[x].ra_name; x++) {
13 years, 11 months
cluster: RHEL4 - ccsd: Fix bug 49898 - symlink attack vulnerabilities
by Jonathan Brassow
Gitweb: http://git.fedorahosted.org/git/cluster.git?p=cluster.git;a=commitdiff;h=...
Commit: bba9068b91f737b8f88cd26581eed5884f443bae
Parent: 6330bbe58b809aaf5dffc4be87e632b427a4b568
Author: Jonathan Brassow <jbrassow(a)redhat.com>
AuthorDate: Thu May 20 11:31:49 2010 -0500
Committer: Jonathan Brassow <jbrassow(a)redhat.com>
CommitterDate: Thu May 20 11:31:49 2010 -0500
ccsd: Fix bug 49898 - symlink attack vulnerabilities
Fix vulnerability in CCS that would allow someone to
prevent a 'ccs_tool update'.
---
ccs/ccs_tool/upgrade.c | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/ccs/ccs_tool/upgrade.c b/ccs/ccs_tool/upgrade.c
index b085407..cb844b9 100644
--- a/ccs/ccs_tool/upgrade.c
+++ b/ccs/ccs_tool/upgrade.c
@@ -193,7 +193,7 @@ static void ccs_dh_in(ccs_dh_t *dh, char *buf){
static int upgrade_device_archive(char *location){
int error = 0;
int dev_fd=-1, tmp_fd=-1;
- char tmp_file[64];
+ char tmp_file[128];
char *buffer = NULL;
ccs_dh_t dev_header;
@@ -229,9 +229,10 @@ static int upgrade_device_archive(char *location){
goto fail;
}
- sprintf(tmp_file, "/tmp/tmp_%d", getpid());
+ memset(tmp_file, 0, 128);
+ sprintf(tmp_file, "/tmp/ccs_tool_tmp_XXXXXX");
- tmp_fd = open(tmp_file, O_RDWR | O_CREAT |O_TRUNC, S_IRUSR|S_IWUSR);
+ tmp_fd = mkostemp(tmp_file, O_RDWR | O_CREAT |O_TRUNC);
if(tmp_fd < 0){
fprintf(stderr, "Unable to create temporary archive: %s\n", strerror(errno));
error = -errno;
13 years, 11 months