Show of hands: compress backups? Or 1:1 copy.
On Wed, 19 Mar 2008 17:24:21 -0400 Tom Holroyd tomh@kurage.nimh.nih.gov wrote:
Show of hands: compress backups? Or 1:1 copy.
1:1 - and test them.
Two reasons
- Most large data files (ogg etc) don't compress - If I have to go to backup I want to maximise the probability of success
Alan
On Wed, 19 Mar 2008 17:24:21 -0400 Tom Holroyd tomh@kurage.nimh.nih.gov wrote:
Show of hands: compress backups? Or 1:1 copy.
What are you backing up, to what media, and what tools are you doing it with?
The answer to your question depends on the above factors.
On 19Mar2008 17:24, Tom Holroyd tomh@kurage.nimh.nih.gov wrote: | Show of hands: compress backups? Or 1:1 copy.
My choice? No per file compression. Let the storage substrate compress if possible - eg modern tape drives do it on their own. Or if it doesn't, you might compress the "archive of everything" file (eg a tar or dump file). Otherwise you have to do "special" stuff on restore; it's untidy.
This position is a gross simplification of things of course.
Example: there are systems I backup with rsync to a new hardlinked tree from yesterday's snapshot. Obviously this is 1:1, with incremental cost. It goes to tape from an uncompressed tar file because the tape drive does some compression.
On Wed, Mar 19, 2008 at 3:29 PM, Frank Cox theatre@sasktel.net wrote:
On Wed, 19 Mar 2008 17:24:21 -0400 Tom Holroyd tomh@kurage.nimh.nih.gov wrote:
Show of hands: compress backups? Or 1:1 copy.
What are you backing up, to what media, and what tools are you doing it with?
The answer to your question depends on the above factors.
I'm using BackupPC to back up all of my stuff, and I decided in the end to not use compression. I have a big enough external hard drive that it's not an issue (even with a few full and partial backups) and I figure it's mostly just wasted CPU time and electricity for my big binary files, which as Alan mentioned, are what eat up most of the space.
reid
Cameron Simpson wrote:
On 19Mar2008 17:24, Tom Holroyd tomh@kurage.nimh.nih.gov wrote: | Show of hands: compress backups? Or 1:1 copy.
My choice? No per file compression. Let the storage substrate compress if possible - eg modern tape drives do it on their own. Or if it doesn't, you might compress the "archive of everything" file (eg a tar or dump file). Otherwise you have to do "special" stuff on restore; it's untidy.
Backuppc is something of a special case in this respect since it does everything transparently.
This position is a gross simplification of things of course.
Example: there are systems I backup with rsync to a new hardlinked tree from yesterday's snapshot. Obviously this is 1:1, with incremental cost.
Once again, backuppc is a special case. It can run a stock rsync on the remote side but work against its local compressed copy for the comparison, and it's hardlinked tree actually hardlinks all copies of identical files, not just the ones from previous runs of the same target. In any case it only has to compress one copy of new data and any subsequent matches are discarded and replaced with hardlinks.
If you have uncompressed log files or database dumps in the runs, compression will almost certainly be a win. If an overwhelming majority is pre-compressed (like most multi-media formats), then maybe not.
I use 1:1 copy. 1:1 is better for archiving because there is better chance of restoration at a later time. No matter what type of backup you have you need to check the backup to make sure it can be read and restored.
Regards. Frank
Show of hands: compress backups? Or 1:1 copy.
Dr. Tom Holroyd "A man of genius makes no mistakes. His errors are volitional and are the portals of discovery." -- James Joyce
-- fedora-list mailing list fedora-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
On Wed, 19 Mar 2008 17:24:21 -0400 Tom Holroyd tomh@kurage.nimh.nih.gov wrote:
Show of hands: compress backups? Or 1:1 copy.
I just backup to an external usb drive with rsync and use the --backup-dir option to make a new per-day directory of all the old files replaced in latest tree by the last rsync.
At one time I thought about making compressed copies of the older backup dirs to DVD when the USB drive started to fill up, but I never got that ambitious. I just delete the old backup dirs when the disk starts getting full (usually gives me a month or two of daily backups where I can find old copies of things if I screw something up).
1:1
Max Pyziur pyz@brama.com
On Wed, 19 Mar 2008, Tom Holroyd wrote:
Show of hands: compress backups? Or 1:1 copy.
Dr. Tom Holroyd "A man of genius makes no mistakes. His errors are volitional and are the portals of discovery." -- James Joyce
-- fedora-list mailing list fedora-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
Cameron Simpson wrote:
On 19Mar2008 17:24, Tom Holroyd tomh@kurage.nimh.nih.gov wrote: | Show of hands: compress backups? Or 1:1 copy.
My choice? No per file compression. Let the storage substrate compress if possible - eg modern tape drives do it on their own. Or if it doesn't, you might compress the "archive of everything" file (eg a tar or dump file). Otherwise you have to do "special" stuff on restore; it's untidy.
This position is a gross simplification of things of course.
Example: there are systems I backup with rsync to a new hardlinked tree from yesterday's snapshot. Obviously this is 1:1, with incremental cost. It goes to tape from an uncompressed tar file because the tape drive does some compression.
If the storage device does not do compression, then from a reliability standpoint, you are better off with per file compression as opposed to total backup compression. The reason for this is that if a compressed file gets corrupted, you lose that file, but if a compressed back gets corrupted, you have a hard time recovering the files after the corruption.
Mikkel
fchan wrote:
I use 1:1 copy. 1:1 is better for archiving because there is better chance of restoration at a later time.
That's not at all a sure thing. The uncompressed copy is going to take more media to store it, thus if you have a media error there is a larger chance that your uncompressed copy will be on that spot.
On 2008-03-19, Tom Holroyd tomh@kurage.nimh.nih.gov wrote:
Show of hands: compress backups? Or 1:1 copy.
I use tape with hardware compression and error correction. Before that I used afio, which compresses files individually as it places them in an archive. That way corruption in one file does not affect the rest of the archive.
On Thursday 20 March 2008 00:08:41 John Thompson wrote:
I use tape with hardware compression and error correction. Before that I used afio, which compresses files individually as it places them in an archive. That way corruption in one file does not affect the rest of the archive.
When I first started working in the early 80s I worked with an operations manager who had started when Turing was a boy and his view was it didn't matter what you did with backups so long as you verified them. At last then you would be certain that you had something to restore should the need arise.
Admittedly in those days, media was much more unreliable, which was his main concern, but it still seems good advice as it would pick up corrupt files or archives as well - and did for me last year when I had a bad DVD.
Nick
Nicholas Robinson wrote:
On Thursday 20 March 2008 00:08:41 John Thompson wrote:
I use tape with hardware compression and error correction. Before that I used afio, which compresses files individually as it places them in an archive. That way corruption in one file does not affect the rest of the archive.
When I first started working in the early 80s I worked with an operations manager who had started when Turing was a boy and his view was it didn't matter what you did with backups so long as you verified them. At last then you would be certain that you had something to restore should the need arise.
Alan Turing? The born in June 1912 and died in June 1954 Turing? He started what when Turing was a boy? Just wondering how old this operations manager was....
Admittedly in those days, media was much more unreliable, which was his main concern, but it still seems good advice as it would pick up corrupt files or archives as well - and did for me last year when I had a bad DVD.
Nick
Tom Holroyd wrote:
Show of hands: compress backups? Or 1:1 copy.
Compress on software, not on the device, backup to the most common device which will do the job. If you have room, don't compress at all.
And if you have sensitive data be sure you encrypt, and use something you can be sure of having available to decrypt. In practice that means open source to me, I had one company go out of business after a client standardized on their software.
I add software ECC to my most critical data, fortunately there's not all that much I have to handle.
On Wed, 2008-03-19 at 17:24 -0400, Tom Holroyd wrote:
Show of hands: compress backups? Or 1:1 copy.
Summary. More than 2:1 favor 1:1. If you do compress, compressing the entire backup is bogus, it should be done file by file, and preferably by the backup device.
The interesting news is backup verification. I typically look at the file sizes and go, "yeah, it ran." Prob'ly not good enough.
Also, one can argue in favor of FEC for archival storage, so if you lose a track or set of blocks you can still recover. I backup to RAID.
I just switched to Xubuntu 7.04 on my IBM Thinkpad G30 Laptop and it's dropping keystrokes! I never had that problem with Fedora. I switched to the lowlatency kernel and that helped but did not fix the problem. Is this an atkbd issue, or is there perhaps some other beastie out there turning interrupts off? More to the point, since this is a Fedora list, what is it in Fedora that does this right?
Tom Holroyd wrote:
On Wed, 2008-03-19 at 17:24 -0400, Tom Holroyd wrote:
Show of hands: compress backups? Or 1:1 copy.
Summary. More than 2:1 favor 1:1. If you do compress, compressing the entire backup is bogus, it should be done file by file, and preferably by the backup device.
The interesting news is backup verification. I typically look at the file sizes and go, "yeah, it ran." Prob'ly not good enough.
Backuppc keeps only one linked copy for all instances of the same file that appear - but full runs will replace any changed content. Even if use rsync and tell it to cache it's block checksums to improve speed, it will still compare a certain percentage of the stored file contents on each run.
Also, one can argue in favor of FEC for archival storage, so if you lose a track or set of blocks you can still recover. I backup to RAID.
I mirror the whole backuppc filesystem to disks that are rotated offsite. I'd like to have something like the zfs incremental block-level send/receive instead, though.
On Mon, 2008-03-24 at 11:00 -0400, Tom Holroyd wrote:
I just switched to Xubuntu 7.04 on my IBM Thinkpad G30 Laptop and it's dropping keystrokes! I never had that problem with Fedora. I switched to the lowlatency kernel and that helped but did not fix the problem. Is this an atkbd issue, or is there perhaps some other beastie out there turning interrupts off? More to the point, since this is a Fedora list, what is it in Fedora that does this right?
-- Dr. Tom Holroyd "A man of genius makes no mistakes. His errors are volitional and are the portals of discovery." -- James Joyce
Please don't hijack existing threads. Start a new topic.
poc
On 24/03/2008, Tom Holroyd tomh@kurage.nimh.nih.gov wrote:
I just switched to Xubuntu 7.04 on my IBM Thinkpad G30 Laptop and it's dropping keystrokes! I never had that problem with Fedora. I switched to the lowlatency kernel and that helped but did not fix the problem. Is this an atkbd issue, or is there perhaps some other beastie out there turning interrupts off? More to the point, since this is a Fedora list, what is it in Fedora that does this right?
I doubt anyone on this list will be able to pin point the exact difference between Fedora and Xubuntu that is causing your problem - it's rather a lot to ask given the scant information you've provided. I'd think you'd get much more help detailing your problem with much much more detail on an xubuntu mailing list.
Jonathan.
Patrick O'Callaghan wrote:
On Mon, 2008-03-24 at 11:00 -0400, Tom Holroyd wrote:
I just switched to Xubuntu 7.04 on my IBM Thinkpad G30 Laptop and it's dropping keystrokes! I never had that problem with Fedora. I switched to the lowlatency kernel and that helped but did not fix the problem. Is this an atkbd issue, or is there perhaps some other beastie out there turning interrupts off? More to the point, since this is a Fedora list, what is it in Fedora that does this right?
-- Dr. Tom Holroyd "A man of genius makes no mistakes. His errors are volitional and are the portals of discovery." -- James Joyce
Please don't hijack existing threads. Start a new topic.
Preferably on trhe right list
--- Jonathan Underwood jonathan.underwood@gmail.com wrote:
On 24/03/2008, Tom Holroyd tomh@kurage.nimh.nih.gov wrote:
I just switched to Xubuntu 7.04 on my IBM Thinkpad G30 Laptop and it's dropping keystrokes! I never had that problem with Fedora. I switched to the lowlatency kernel and that helped but did not fix the problem. Is this an atkbd issue, or is there perhaps some other beastie out there turning interrupts off? More to the point, since this is a Fedora list, what is it in Fedora that does this right?
I doubt anyone on this list will be able to pin point the exact difference between Fedora and Xubuntu that is causing your problem - it's rather a lot to ask given the scant information you've provided. I'd think you'd get much more help detailing your problem with much much more detail on an xubuntu mailing list.
Jonathan.
Just curious why the original poster wanted to switch. Fedora has a fabulous support structure with very helpful and very knowledgeable folks on the m/l (perhaps why the poster posted here) in addition to very high-quality latest software, which is not true of *ubuntu. if the goal was to get xfce (sorely lacking in fedora), i think rahul sundaram mentioned sometime ago that there will be a xfce version also from F9 onwards.
trotter
____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs