On Tue, Mar 11, 2014 at 9:34 AM, Martin Langhoff <martin.langhoff@gmail.com> wrote:
Ouch -- so updates fetch EVERY file regardless of whether it has changed between the snapshot installed and the target snapshot? That is kind of bad.

No.  Clients only fetch objects they don't have.

Are you aware of the work done on OLPC's os-builder, which used rsync "informed" by a per-snapshot metatada file?

I'd definitely looked at older versions of olpc-update, but not at the latest.

Bottom line though is that while rsync is a very good general purpose tool (for ad-hoc sync between two machines in particular), it spends CPU and I/O time on both client and server in order to attempt to minimize network bandwidth.  This is not good for the "pool of servers replicating to many clients" case.

Now rsync's "batch mode" allows precomputing on the server once, which helps a lot - and it starts to get closer to where I'm going with OSTree static deltas.  The main difference then is that OSTree static deltas are much more flexible - the rsync style "rolling checksum" is only one of several possible algorithms.

The OSTree static delta design was inspired by the network protocol part of:
http://www.chromium.org/chromium-os/chromiumos-design-docs/autoupdate-details

There are other details too, like the fact that static deltas are chunked to say 16MB, so an interrupted client won't have to re-download more than one chunk.  They also reuse the integrated GPG verification in OSTree, etc.

In our experience, the problem we had was network latency. On a local network, a small OS image, and with rsync client sending requests speculatively a bit ahead of receiving the response to its prev request... the lag for clients updating was horrible.

The main latency with OSTree comes from metadata fetches.

How long does an update of a full Fedora

You'd have to define "full Fedora" =)  And furthermore what you're updating from and to.

(Is there a good overview I should be reading?)

There's not a good overview of this particular level.  But once you understand that OSTree is like git, with commits, trees, and content objects, and that in the "archive" mode they're just stored as individual zlib-compressed files, the rest of the design kind of follows from there.

Two recent blog posts are:
http://samthursfield.wordpress.com/2014/01/08/os-level-version-control/
http://samthursfield.wordpress.com/2014/01/16/the-fundamentals-of-ostree/

I also have several here:
http://blog.verbum.org/