On Wed, Sep 14, 2011 at 7:30 PM, Jeff Darcy <jdarcy@redhat.com> wrote:
>> 1. Forward the write to all N replicas 2. If all N replicas
>> indicate success, we're done 3. If any replica fails, add
>> information about the failed request (e.g. file, offset, length) to
>> journals on the replicas where it succeeded.

First, I would not say that the file on the server where the write
succeeded (there's no real primary in this architecture) is corrupt.
It's just more up to date that on a server where the same write failed.
 The key here is that it can update its journal *locally* as part of the
write itself.  This would involve extra disk I/O but not extra network
traffic.

I feel the above two paragraphs are not in sync.

Paragraph 1 says if all N replicas indicate success, we're done (i.e no more network calls. which means, no journal entries were made yet). -- (1)

Paragraph 2 says journals are local to the server as part of the write itself. But in anticipation of all writes succeeding, nothing journal entries are made along with the write. -- (2).

Paragraph 2 says, in the event of a write failing on some server, journal is written on those servers where write succeeded via an extra disk I/O but not a network I/O. But, due to (1) and (2), and due to the fact that a write failure is detected by the client (and not the other servers), an extra network call seems inevitable. Which means, there is scope for failure in that phase as well. Also, there is the window between write and journal update on the local server (assuming it is achieved somehow) where the data center can lose power.

I may have missed something obvious. Please correct me if so.

Avati