Hi,
> At 15:20 5/28/2004, Michael Mansour wrote:
> >Not only do I wonder if tar will respond the same
way
> >as rsync without the -S option, but I'm also
wondering
> >if backup software which use rsync to perform
backups
> >would also take this into account...
>
> I also wonder if there is any downside to using -S
or whether it might
> produce an undesirable result in some cases. If
not, perhaps one should
> always use it just in case?
Sparse files are a boon or a bust.
If you make a sparse file
open(); write(byte); lseek(way-out-there);
write(byte); close()
you have not reserved or allocated the disk blocks
for all the
possible stuff in the middle. If you expect those
blocks to be real
and there is not enough disk space in the future
things will bust.
In some cases if you compress the file then expand
it you can run
out of disk space on the uncompress pass (compress,
gzip, bzip2, zip).
Watch out for uncompress bombs.
Some backup and restore tools will run you out of
tape or out of disk
or both with sparse files. Gnu tar has a --sparse
flag. You might not
be able to restore the contents of a file system if
you are not
cautious/ aware.
Some data base files might be sparse but you could
run out of space
a year later when you do not expect and need the
blocks to be real.
mkswap (to files) might generate files that look
like they should be
sparse to backup and restore tools but should not
be. i.e. You should
not use swapon on a file with holes.
Holes on a file system with one size of allocation
block may not be
holes on another file system with a smaller
allocation block size.
Since allocation block size is heuristically
determined one could
be confused when moving files from one FS to another
of a different
size or type. mv does not have a --sparse flag that
I can see.
The big issue is backup and restore processes
(including cp, mv).
I guess a utility (holeify and holecount) might be
written.
--
T o m M i t c h e l l
/dev/null the ultimate in secure storage.
Thanks Tom, excellent explanation.
In regards to where these sparse files came from, I
set about to determine this and found out.
As mentioned in earlier posts, I use that data volume
under Samba, which is shared out to apps/clients to
store their files.
Azureus is a Java BitTorrent client, and when starting
torrent downloads on that data drive, the entire size
of the torrent is reserved on the data drive (so a
600Mb ISO shows up on the filesystem even though it is
empty). A "du -ks" shows the actual size (a few bytes
at the most), although an "ls -l" shows the 600Mb file
which the filesystem recognises as space taken.
So when I rsynced that data volume to another drive
(without the -S option), the rsync naturally copied
exactly what it saw, spacewise, but when using -S it
recognised the sparse files and therefore didn't take
up the space for them.
We learn something everyday :) ... I'm wondering now
if those sparse files that got "un-sparsed" on the
rsync will still be valid if, for example, Azureus was
to continue downloading to them.
Michael.
Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com