On Sat, Feb 12, 2022 at 8:32 AM Jamie Fargen <jamie(a)fargenable.com> wrote:
Not familiar with DejaDup, but with this setup on RAID0 do an rsync every 15 minutes to
the backup system.
rsync has some advantages: destination does not need to be btrfs, the
--inplace option for VM images
But for such very frequent backups, it's more like a replication use
case. And btrfs send/receive is very efficient at this because unlike
rsync, no deep traversal is required of either the source or
destination. Btrfs will increment the generation anytime a file is
modified, and the generation of the leaf the inode is contained in,
and the node the leaf is referenced in, all the way up to the file
tree root. This makes it very cheap when doing a diff between two
snapshots, for btrfs to figure out what has changed without having to
look at every inode. It just skips all the parts of the tree that
haven't changed, in effect it creates a "replay" list between the two
generations. An incremental send contains just the changes, and that
includes knowing when files are renamed or moved, so their data
doesn't need to be sent again.
So if you were to change just one file in 15 minutes, a btrfs send -p
stream (an incremental stream produced as a "diff" between two
snapshots) and receive will take a few seconds, even if the snapshot
contains millions of files. There'd be a straight line following the
nodes and leaves with the incremented generation leading straight to
the only changed file.
(You could use 'btrfs send -f' and place the stream as a file on a
non-btrfs file system. But you can't really look inside of it like a
snapshot received on a btrfs file system.)
--
Chris Murphy