On Tue, Feb 28, 2012 at 5:00 PM, David Teigland <teigland@redhat.com> wrote:
On Tue, Feb 28, 2012 at 04:40:04PM +0100, Frido Roose wrote:
> 839496 s1 check_our_lease warning 79 last_success 839417
> 839497 s1 check_our_lease failed 80
> 841667 s1 renewed 841667 delta_length 2229 too long

It looks like i/o to your storage blocked for 2229 seconds, which is much
longer than the 80 seconds it's given.  If this is on nfs, you might check
if the nfs server was down.


Thanks for your quick reply.
The lockspace is on a GFS2 volume, but you are right that we had an issue with the GFS2 volume that was hanging for some reason:

Feb 24 13:49:23 maraikoh kernel: INFO: task gfs2_quotad:6113 blocked for more than 120 seconds.
Feb 24 13:49:23 maraikoh kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 24 13:49:23 maraikoh kernel: gfs2_quotad   D 0000000000000005     0  6113      2 0x00000080
Feb 24 13:49:23 maraikoh kernel: ffff880279901c20 0000000000000046 ffff880279901b90 ffffffffa043cd3d
Feb 24 13:49:23 maraikoh kernel: 0000000000000000 ffff880279fd9800 ffff880279901c50 ffffffffa043b4e6
Feb 24 13:49:23 maraikoh kernel: ffff8801f6ca30b8 ffff880279901fd8 000000000000f4e8 ffff8801f6ca30b8
Feb 24 13:49:23 maraikoh kernel: Call Trace:
Feb 24 13:49:23 maraikoh kernel: [<ffffffffa043cd3d>] ? dlm_put_lockspace+0x1d/0x40 [dlm]

That will have caused the timeout for sanlock.

Thanks for explaining this!

Best regards,
Frido