[PATCH] sanlock: host_message
David Teigland
teigland at redhat.com
Thu Mar 6 22:14:26 UTC 2014
On Thu, Mar 06, 2014 at 03:56:25PM -0500, Nir Soffer wrote:
> field bits
> -----------------
> host_id 12
> generation 32
> send_msg 8
> send_seq 12
Even if I didn't need to be consistent with the way this works elsewhere,
creating ad hoc field sizes like this would be unmanageable.
> Acknowledge is just another message, so we send in the same field.
>
> RESET = 0x01
> ACK = 0xFF
>
> The sender detects the ack by getting an ACK message in the lease of the
> receiver with the sender host_id and generation and the same send_seq as
> the sent message.
It doesn't work because we can easily have unregulated overlapping of
sending/receiving/acking messages, all trying to use the same fields at
once. The result is chaos. Perhaps in your very specific usage this
wouldn't happen, but this is at least a minimally generalized capability.
> > Receiving acknowledgement
> > -------------------------
> >
> > sanlock will not keep any state about the host messages it has sent or try
> > to match acknowledgements. But, sanlock does keep track of other host's
> > delta lease state, and that could include recv_from_host_id/recv_seq. We
> > can add an api for the caller to query the recv_from_host_id/recv_seq for
> > a given host_id.
>
> This means that the clients has to remember sent messages sequence, so
> implementing a simple fence agent script will be impossible. You will have
> to create another process running from start of fencing, remembering the sent
> message sequence, and polling sanlock daemon for the result.
I thought the program that sent the message (and got send_seq), would
itself want to watch for the ack (recv_seq). If it got the ack, it would
then procede to monitor the host status.
If they are different programs, there are ways of passing a number between
them. If it's truely difficult, then perhaps we could query both send_seq
and recv_seq from sanlock.
> If sanlock does remember sent messages and check for acks, it will be easier
> to use it from other tools.
I think it's too unrelated to sanlock's main job. I'm really aiming for
as minimal and primitive and unintrusive as possible. One reason I don't
like the idea of acks is because a system that needs acks probably wants a
level of sophistication which sanlock simply can't provide (and shouldn't
because it's not the purpose of sanlock.) So I want to add the absolute
minimum that we need to implement your function.
> > 1. It will not work with a fast reset option using /proc/sysrq-trigger
> > because there will not be enough time for the acknowledgement to be
> > written before the host is reset. (With another independent message
> > area, we could write an acknowledgement immediately, but borrowing the
> > lockspace lease means we do not have this option.)
>
> You can do a fast reset after the write to the storage finished, assuming
> that the write is not asynchronous.
I'm not sure how I'd use sysrq-trigger yet anyway -- I don't like the idea
of encoding such a specific feature directly into sanlock. So, I'd need
to figure that out, and maybe whatever is doing sysrq-trigger could add
some delay to given the next renewal (with ack) a chance to complete.
More information about the sanlock-devel
mailing list