Hello List,
I am wanting to backup my Linux machine at work using Rsync. I have a mounted SMB share which it will be backing up to. I would like to rsync "into" an archive, so the backup is all in one file. I've done some googling and haven't found a real clear way to do an rsync backup into either a tar or gzip archive.
Does anyone have a couple of one-liners or links they could recommend?
Thanks.
Julian
Julian Underwood wrote:
Hello List,
I am wanting to backup my Linux machine at work using Rsync. I have a mounted SMB share which it will be backing up to. I would like to rsync "into" an archive, so the backup is all in one file. I've done some googling and haven't found a real clear way to do an rsync backup into either a tar or gzip archive.
Does anyone have a couple of one-liners or links they could recommend?
Thanks.
Julian
Why do you want to use rsync for this? Are you doing something that you can not do using the -u option of tar or the -f option or zip? If so, you may want to look at standard backup software.
Mikkel
On Tue, Jan 24, 2006 at 12:17:32PM -0600, Mikkel L. Ellertson wrote:
Julian Underwood wrote:
Hello List,
I am wanting to backup my Linux machine at work using Rsync. I have a mounted SMB share which it will be backing up to. I would like to rsync "into" an archive, so the backup is all in one file. I've done some googling and haven't found a real clear way to do an rsync backup into either a tar or gzip archive.
Does anyone have a couple of one-liners or links they could recommend?
Thanks.
Julian
Why do you want to use rsync for this? Are you doing something that you can not do using the -u option of tar or the -f option or zip? If so, you may want to look at standard backup software.
Mikkel
On Tue, 2006-01-24 at 15:22, akonstam@trinity.edu wrote:
I am wanting to backup my Linux machine at work using Rsync. I have a mounted SMB share which it will be backing up to. I would like to rsync "into" an archive, so the backup is all in one file. I've done some googling and haven't found a real clear way to do an rsync backup into either a tar or gzip archive.
Why do you want to use rsync for this? Are you doing something that you can not do using the -u option of tar or the -f option or zip? If so, you may want to look at standard backup software.
I agree that for backing up to a mounted filesystem archive rsync is not needed.
I , however, am having a blank in figuring out how to backup to a filesystem that is not mounted but is on another machine. It seems like some combination of rsync (or scp) and tar aught to be able to do this. But the method eludes me. I mean something other than creating the tar file on the local machine and then copying to a remote filesystem on another machine.
Rsync isn't going to back up to a tar/zip file no matter how you use it. It will copy to/from remote systems with the syntax: rsync -essh -av . user@remote_host:/path/to/dir (current directory to remote) rsync -essh -av user@remote_host:/path/to/dir . (remote dir to current)
If you want automatic backups controlled by a remote server, look at backuppc: http://backuppc.sourceforge.net/. It can use stock rsync on the client side while the server uses a special compressed format for storage.
Les Mikesell wrote:
On Tue, 2006-01-24 at 15:22, akonstam@trinity.edu wrote:
I am wanting to backup my Linux machine at work using Rsync. I have a mounted SMB share which it will be backing up to. I would like to rsync "into" an archive, so the backup is all in one file. I've done some googling and haven't found a real clear way to do an rsync backup into either a tar or gzip archive.
Why do you want to use rsync for this? Are you doing something that you can not do using the -u option of tar or the -f option or zip? If so, you may want to look at standard backup software.
I agree that for backing up to a mounted filesystem archive rsync is not needed.
I , however, am having a blank in figuring out how to backup to a filesystem that is not mounted but is on another machine. It seems like some combination of rsync (or scp) and tar aught to be able to do this. But the method eludes me. I mean something other than creating the tar file on the local machine and then copying to a remote filesystem on another machine.
Rsync isn't going to back up to a tar/zip file no matter how you use it. It will copy to/from remote systems with the syntax: rsync -essh -av . user@remote_host:/path/to/dir (current directory to remote) rsync -essh -av user@remote_host:/path/to/dir . (remote dir to current)
If you want automatic backups controlled by a remote server, look at backuppc: http://backuppc.sourceforge.net/. It can use stock rsync on the client side while the server uses a special compressed format for storage.
rsnapshot is another good one (http://www.rsnapshot.org), but it doesn't backup to an archive, however it works extremely well.
akonstam@trinity.edu wrote:
I agree that for backing up to a mounted filesystem archive rsync is not needed.
I , however, am having a blank in figuring out how to backup to a filesystem that is not mounted but is on another machine. It seems like some combination of rsync (or scp) and tar aught to be able to do this. But the method eludes me. I mean something other than creating the tar file on the local machine and then copying to a remote filesystem on another machine.
Something like:
tar -cf - <some files> | ssh <remote machine> "tar -uf <archive>"
You will end up transferring extra files because tar on the local machine does not know what is in the archive. If you just want to create an archive on the remote machine, then use:
tar -cf - <some files> | ssh <remote machine> "tar -cf <archive>"
You may need <user>:<remote machine> instead of just <remote machine>...
You can pipe just about any command that outputs to standard out over an ssh link to a program that accepts standard in.
Mikkel
On Tue, 2006-01-24 at 16:18, Mikkel L. Ellertson wrote:
If you just want to create an archive on the remote machine, then use:
tar -cf - <some files> | ssh <remote machine> "tar -cf <archive>"
You don't need tar on the other end - you just want to save the output to a file: tar -cf - . | ssh remote_machine 'cat >/path/to/file.tar' (sends the current directory) Don't forget to quote the command or the > redirection would happen locally.
Les Mikesell wrote:
On Tue, 2006-01-24 at 16:18, Mikkel L. Ellertson wrote:
If you just want to create an archive on the remote machine, then use:
tar -cf - <some files> | ssh <remote machine> "tar -cf <archive>"
You don't need tar on the other end - you just want to save the output to a file: tar -cf - . | ssh remote_machine 'cat >/path/to/file.tar'
"\ssh" please.
I have an alias set for ssh to expand ssh to "ssh -t" and that destroys tarballs. "\ssh" always works.
Howver, this sends everything every time. See my earlier post for a better way.
On Tue, 2006-01-24 at 12:17 -0600, Mikkel L. Ellertson wrote:
Why do you want to use rsync for this? Are you doing something that you can not do using the -u option of tar or the -f option or zip? If so, you may want to look at standard backup software.
Mikkel
Hi,
Yeah, you're right, I suppose the -u option of tar or the -f option of zip will probably work just fine. The reason I want to backup "changes only" is because I'm going be be doing nightly backups to a remote site and as the bandwidth is going to be limited.
Additionally, as the file systems I'm going to be backing up to are SMB, I wanted the data to be compressed and archived into one file so if there are files with unsupported SMB filenames/lengths, they won't get stripped when put onto the SMB volume. I'm going to be doing this for some OS X servers as well.
Does this make sense? You think this is an OK way to go about doing an "offsite" DR backup to a SMB volume?
Julian
At 9:23 PM -0500 1/26/06, Julian Underwood wrote:
On Tue, 2006-01-24 at 12:17 -0600, Mikkel L. Ellertson wrote:
Why do you want to use rsync for this? Are you doing something that you can not do using the -u option of tar or the -f option or zip? If so, you may want to look at standard backup software.
Mikkel
Hi,
Yeah, you're right, I suppose the -u option of tar or the -f option of zip will probably work just fine. The reason I want to backup "changes only" is because I'm going be be doing nightly backups to a remote site and as the bandwidth is going to be limited.
Additionally, as the file systems I'm going to be backing up to are SMB, I wanted the data to be compressed and archived into one file so if there are files with unsupported SMB filenames/lengths, they won't get stripped when put onto the SMB volume. I'm going to be doing this for some OS X servers as well.
Does this make sense? You think this is an OK way to go about doing an "offsite" DR backup to a SMB volume?
When you use tar, the destination filesystem won't get a chance to see or muck with any of the attributes of the backed-up files. Probably the only likely problem is with file-size limits.
FWIW, tar can back up changes only, by keeping a list of files it has backed up. See option --listed-incremental in "info tar", and the section on Backups. I find info to be inscrutable, so "info tar", U, cursor down to Backups, Enter.
If you compress the resulting files, you have more risk that a single error can make all of a backup (or at least any following incrementals) useless. I wonder if there is a tool to add redundancy, like .par files on usenet?
(Note that I have only read about this stuff; I have no actual experience.) ____________________________________________________________________ TonyN.:' mailto:tonynelson@georgeanelson.com ' http://www.georgeanelson.com/
On Fri, 2006-01-27 at 10:45, Tony Nelson wrote:
Yeah, you're right, I suppose the -u option of tar or the -f option of zip will probably work just fine. The reason I want to backup "changes only" is because I'm going be be doing nightly backups to a remote site and as the bandwidth is going to be limited.
Additionally, as the file systems I'm going to be backing up to are SMB, I wanted the data to be compressed and archived into one file so if there are files with unsupported SMB filenames/lengths, they won't get stripped when put onto the SMB volume. I'm going to be doing this for some OS X servers as well.
Does this make sense? You think this is an OK way to go about doing an "offsite" DR backup to a SMB volume?
When you use tar, the destination filesystem won't get a chance to see or muck with any of the attributes of the backed-up files. Probably the only likely problem is with file-size limits.
FWIW, tar can back up changes only, by keeping a list of files it has backed up. See option --listed-incremental in "info tar", and the section on Backups. I find info to be inscrutable, so "info tar", U, cursor down to Backups, Enter.
Yes, gnutar's --listed-incremental mode would work as long as you are sure you understand it and the way you have to restore the full first, then all subsequent incrementals where you specified the same filename. However it won't be as efficient about handling modifed or growing files. Each tar run will transfer a copy of the whole thing where rsync would find and send the changes. I'd recommend adding a drive to the box where you run the rsync to avoid the smb limitations, but depending on the circumstances you might try something more imaginative - like running a VMware virtual machine (runtime is free now) on the box where you planned to export the smb share so you could rsync to it's virtual drive contained in a file.
When you use tar, the destination filesystem won't get a chance to see or muck with any of the attributes of the backed-up files. Probably the only likely problem is with file-size limits.
FWIW, tar can back up changes only, by keeping a list of files it has backed up. See option --listed-incremental in "info tar", and the section on Backups. I find info to be inscrutable, so "info tar", U, cursor down to Backups, Enter.
If you compress the resulting files, you have more risk that a single error can make all of a backup (or at least any following incrementals) useless. I wonder if there is a tool to add redundancy, like .par files on usenet?
(Note that I have only read about this stuff; I have no actual experience.) ____________________________________________________________________ TonyN.:' mailto:tonynelson@georgeanelson.com ' http://www.georgeanelson.com/
Hi, I know this thread is basically dead, but I was thinking of simply using cp to perform a backup:
cp -dpRux source/ dest/
my question is how would I wrap this into a gzip archive (again to preserve file attributes which SMB cannot handle.) Additionally, would the whole archive have to be recreated each time or would it be able to insert the appropriate changes.
If you guys think it will have to recreate the entire archive each time, I guess I will just use tar then.
Thanks,
Julian
Julian Underwood wrote:
Hello List,
I am wanting to backup my Linux machine at work using Rsync. I have a mounted SMB share which it will be backing up to. I would like to rsync "into" an archive, so the backup is all in one file. I've done some googling and haven't found a real clear way to do an rsync backup into either a tar or gzip archive.
Does anyone have a couple of one-liners or links they could recommend?
rsync is a mirror program. It's purpose is to intelligently synchronise (sync) a local and a remote (r) copy of chosen elements of a filesystem.
It is capable of patching destination files to make them the same as source files. For this to work reasonably well, the source and destination need to be somewhat similar.
An example that works: Ubuntu, in its test cycle, creates daily builds of its install media. Each build contains great gobs of stuff that was the same yesterday, and a few things that got rebuilt since yesterday's build.
rsync can intelligently transfer the differences and massage your copy into an exact image of the new one, even (I'm told) to the extent of recording slabs of the destination.
This would actually work were the ISO actually an uncompressed tarball or any other kind of uncompressed archive.
And example that does not work: (We'll say) Ubuntu creates a daily compressed tarball of the entire source tree, and users round the world use rsync to update their local copies.
This does not work because the archive is one huge compressed file and contains no great chunks of data the same as yesterday's.
It would work, if they ran gzip (or bzip2 or other compression program) over the source tree and then created a tarball of that, because then there would be big pieces of compressed data in today's archive the same as yesterday's.
I have tried, and discussed with the rsync folk, to use rsync to mirror the filesystem on a server across an ADSL link. It uses enormous gobs of virtual storage (fortunately, in doing so its usage pattern does not induce thrashing even when virtual storage is five and more times real) and runs for hours; when I was trying to do it, often timing out and failing to finish, and on occasion I had it running for days: clearly not satisfactory for a daily backup.
What I do now is create an ISO of compressed files (I use mkzftree[1]) and I can sync that in under an hour.
1. The resultant ISOs can be burned to DVD (too big for CD), mounted on Linux and the individual files read as the kernel transparently decompresses them.
For a small filesystem, I'd just rsync the directory structure to replicate the filesystem, unless I particularly needed a single-file archive.