On Jul 28, 2014, at 17:11, Will Woods <wwoods(a)redhat.com> wrote:
You're not quite right about how the overlay works.
The default in-memory overlay is just 512MB. And the device-mapper docs
note that "if it fills up the snapshot will become useless and be
disabled, returning errors."[1].
You should also note that the overlay is a block-level snapshot - so any
changes to existing files or filesystem metadata will cause data to be
written to the overlay. Furthermore, the default chunk size is 4kb - so
any change less than 4kb will take 4kb of space.
After hitting a wall several times today, I began to see what you're talking about
here. ;)
I assume you're using filesystem images 'cuz you don't
have a reliable
network connection (otherwise you'd probably be using NFS or iSCSI or
something)?
The network connection is reliable, but I'm working with thousands of nodes that need
a largely stateless system with only a few persistent items.
Since your systems have lots of RAM, why not just use a regular ext4
filesystem image as your root filesystem? Then you don't need to worry
about blowing up the overlay at all.
Are you suggesting an ext4 r/w filesystem stored in RAM? I haven't seen how to do
that in dracut with the existing scripts.
If you need compression to save RAM: why not use a squashfs image
directly, and mount/bind a tmpfs to the places you'll be writing data?
I'd be interested in that for sure but the dmsquash module in dracut seems to require
a real ext/btrfs/xfs filesystem for device mapper. I couldn't find a way to boot a
plain squashfs with a filesystem in it.
Is there a particular reason you need to use dmsquash-live, or is
this
just a case of the hammer making all your problems look like nails?
My goal is to live boot our servers since the majority of our systems would be stateless.
Being able to reboot into a known good, tested state would be advantageous. I've
worked with Debian's Live Systems project[1] and their strategy is to mount a squashfs
read only but then use aufs to provide a writeable filesystem overlay. It's handy
since you can fill up the overlay without causing the snapshot to overflow. However, AUFS
isn't in the upstream kernel and that makes things a bit challenging.
If there's a better strategy than using dmsquash-live in dracut, or if I need to do
some work to build a new dracut module, I'm certainly up for that. Finding
documentation on the internals of dracut has been a bit challenging for me so far.
[1]
http://live.debian.net/
--
Major Hayden