Forgot your password?
typodupeerror
Linux Software

Affordable Home Backups for 10-100G Systems? 690

Posted by Cliff
from the portable-media-that-can-store-GIGs dept.
MichaelJames asks: "Ok, I have my MP3's streaming, all our digital pictures up, and a file server running on one machine in the basement. What would be the best way to do simple backups of the system and data? Get a tape drive Get a CDRW or DVDRW to backup the MP3 and pics, but use the old Zip drive for the file server data?" With drives in the 10-20 gig range only getting smaller and less expensive, what are we to do for backups, that have yet to scale well in the same range. For home systems with up to 100G of storage, what do you use to back up that much data, with a solution that's affordable to the average computer user? Have DVD writers become cheap enough for serious consideration as a backup media?
This discussion has been archived. No new comments can be posted.

Affordable Home Backups for 10-100G Systems?

Comments Filter:
  • Hard Drives (Score:5, Insightful)

    by Tackhead (54550) on Tuesday December 11, 2001 @05:47PM (#2689349)
    > For home systems with up to 100G of storage, what do you use to back up that much data, with a solution that's affordable to the average computer user?

    Given that a 100G hard drive is cheaper than any removable media solution, why not just buy another hard drive and install it in a removable (not hot-swappable, just removable) rack?

    Racks are $20 at my local Fry's, and inserts for other hard drives are $10.

    • Re:Hard Drives (Score:4, Informative)

      by Marvin_OScribbley (50553) on Tuesday December 11, 2001 @05:54PM (#2689406) Homepage Journal
      This will also work for laptops. I recently purchased an external drive enclosure with a PCMCIA connector (also available in Firewire and USB), and a separate 3.5" hard drive. The cost of the two together was less than these external drives they advertise for backing up your notebook, plus I can reuse the drive enclosure for any 3.5" hard drive.

      The drive enclosure was a bit more expensive than the rack mentioned above (under $100 with shipping) however it did come with a two sets of enclosures - one for the drive itself to use externally, and another to put in a PC cabinet if you want to hot swap the drive with it instead of using the card slot.

    • Re:Hard Drives (Score:3, Informative)

      by Stoutlimb (143245)
      I did exactly just that for a small office that I consult for. It backs up the entire server hard drive, and all critical information on all networked workstations, such as e-mail, etc... And for the backup software, I simply used Powerquest's DataKeeper software, that happened to come free on a driver disk for a Firewire hard drive bay I bought on a whim and never used. Compared to the tape back-up and other options, this was a no-brainer, both in the price and ease of use departments. Running the backup was reduced to a one-click wonder app, perfect for the "mature" staff in this office. ;-)

      Does anyone have a better bang/VS/buck solution? I know CDRW's are quite cheap, but one has to factor in speed, as well as human intervention. Swapping disks requires attention, which requires a wage. The backup system simply is launched every friday after work, and does it's thing on it's own time, without need for people. And with 100GB, no sanely priced tape drive comes close.
      • What about fire? (Score:2, Insightful)

        by DAldredge (2353)
        What do you do when the building burns down?
        • I raised this issue with my clients. They considered it an unlikely enough occurence that they aren't worried. Further, they already keep archived copies of all projects on zip disks, in a an off-site storage area anyhow. This removable hard drive backup system is to primarily guard against deletions, viruses, hardware failiure, or your occasional smash&grab thief. So far it has already paid for itself when a partner had deleted a few directories by accident. Digital photos, taken of a site halfway across the country are expensive to replace. :-))
      • Hard drives was of course my first thought. They're cheap and widely supported. The problem though is that there are so many moving parts that can break. With something like a tape drive or DVD-Rs or something, the media is separate from the drive. If the drive goes bad, the media/data are still intact. Just remember redundancy is important whatever you choose to use.
    • The 100GB Western Digital Special Edition hard drive [theregister.co.uk] with 8MB of cache.

      Supposedly, it performs just like a SCSI drive, but it's IDE. A couple of these in the aforementioned rack in a mirrored RAID combo would make a perfect backup.

      I'm definitely swapping out my current configuration for two of these once I can afford the $600. :)
      • Well, it's only "just like SCSI" in that it has a larger cache. There's much more to SCSI than cache. Anyway, that site says $339 for the drive, where'd you get $600?
    • This is really only a short term backup. Since the storage media and the reader are one unit, if either fails the back up is toast. If you have two drives, you might as well setup mirrored RAID.

      Real backup is done on semipermanent media (>10 year storage) in a format that can be taken off site easily.
      • In Linux, and many other flavors of unix, you could buy three drives, set them all three up as a mirror set, then pull one of the drives and take it offsite. If one drive dies, you can replace it without interuption, if both fail, you can just bring in the one from offsite and plug it in. Given the speed / price / performance of modern IDE hard drives, you could have two offsite drives AND a mirror set for less than most medium to large tape drives, and as far as I know, hard drives have a pretty long shelf like (usually >10 years) assuming they are treated well.
      • RAID != Backup (Score:4, Insightful)

        by dasunt (249686) on Tuesday December 11, 2001 @06:47PM (#2689797)
        RAID is to provide either additional speed and/or hotswappable capability. RAID really stinks as a backup, since RAID doesn't care when some program deletes most of the hard drive, when some user removes too many files, or when the OS barfs. Sure, RAID will save your DATA if one HDD fails, as long as whatever caused it to fail didn't affect the other drive, but for the reasons already listed, this doesn't mean RAID is a valid method of backup.

        However, a HDD in an external enclosure could be considered a valid backup, however, for true redundancy, you better have two drives you swap, and you better be doing surface tests regularly. A drive, properly treated, should last many, many years. Also, you could combine a drive with monthly or quad-yearly backups to CD-R, just make sure you do your research on the inks used in CD-R disks, some don't last as long as others.

        Just my $.02
      • by Ogerman (136333) on Tuesday December 11, 2001 @07:11PM (#2689934)
        Since when is a harddrive not a semipermanent media that can be easily taken off site? I'm surprised this comment got modded up so high. And since when are tapes such a reliable media compared to a hard disk? So burn-in the drive for a few days before using it for backups. And use a S.M.A.R.T. utility to diagnose the drive before each backup to reduce the chance that something is getting ready to fail.

        Your best option is to put all data on a 2-disk mirrored RAID and use another drive as a removable for an off-site or fire-safe backup. The probability of 3 hard disks failing simultaneously, one not in use, is so incredibly small it's laughable. And for that non-zero chance, if it happens, you can pay to have the spindle of one of the failed drives transferred to a new drive in a clean room.
        • The chances aren't as bad as you make them sound...

          Let's just say you have an internal RAID system with, oooo, 4 drives, along with a removable drive to backup everything. The problem lies in the whole trusted/shared medium concept. If there is a surge passed along the case, the SCSI/IDE cable, or through the power-supply cabling, not only will ALL of your drives get toasted, but if you have the backup-harddrive connected to the system (actively archiving your data, finished archiving and waiting for you to remove it, or just because of a BAD practice of never actually removing the removable hard drive) you will loose your backup hard drive as well.

          While RAID is a good thing, multiple hard drives are still at the mersey of everything they are connected to. Using such a system as your only backup is a bad idea that happens too often. Having a removable hard drive is an option, but the work involved really makes other solutions much more viable, especially on a large-scale (and on a small-scale, people are lazy!).

          I propose a network-based automatic backup system for most people. You simply have your main system automatically backup it's data over the network to another system (systems with low number-crunching capabilities can be put back to work here). Of course, you would want to maintain at least 2 concurrent backups in-case the main system dies during the said backup. The benefit of network backup are that human intervention is not required (the user and administrator don't need to do much of anything after initial setup) and off-site backups can happen transparently (just send the data to the other office down the street, across town, whatever.

          Speed of the network may appear to be a problem, but 100 GigaBytes (UNCOMPRESSED) can be transfered in 2.22... hours over 100Base-Tx. First of all, it's likely you'll be compressing that data, which on average halves the size, and so the time is halved as well. Secondly, Gigabit over Cat-5 is available at $45 per NIC, making backups take one-tenth that time. And finally, an encrypted SSH, IPSec, PPTP, (etc) tunnel could be established that would ensure the data is kept private. Data security is much more difficult when you have multiple copies of it unencrypted in a conviently sized package. You are just saying 'steal me'.
      • by Tackhead (54550) on Tuesday December 11, 2001 @07:13PM (#2689950)
        > Real backup is done on semipermanent media (>10 year storage) in a format that can be taken off site easily.

        True -- but given the article's "affordable", "home", "10-100GB" parameters, I'd be quite happy regarding hard drives as a real solution.

        Don't expect one hard drive to last you 10 years, because 10 years from now, systems with 40-pin IDE won't exist. (And likewise, neither will readers for the tapes you purchase today. When was the last time you saw a DC600 cartridge tape drive available?)

        If you're talking longterm storage, leave your "backup" drive somewhere secure, and expect to replace it every 3-5 years. (That'll probably be a 500G serial IDE drive 5 years from now, a terabyte-range solid-state device 10 years from now, and a petabyte-range holocube 20 years from now.)

      • But I can't see any better solution affordable for the casual home user. CD-R's are even shorter-term media - I've already had 5-year-old CD-R's become unreliable to read, while my 8-year-old hard drive (it's not even an expensive one - some cheap Connor Peripherals thing that came with my Packard Bell) is working 100% perfectly.

        I'd call hard drives semi-permanent media that can be taken off-site easily, especially if they are mounted in a removable rack, as suggested. If a hard drive is used solely for backup (say, once a week?), MTBF should not be less than 10 years, even for a Maxtor.
    • Re:Hard Drives (Score:2, Redundant)

      by Uruk (4907)
      One of the reasons that hard drives are lousy back up medium is because they don't address some of the needs of serious backup systems.

      Sure if it's removable then you're probably mostly OK, but if some power accident fries your box, and your backup hard drive is in there, oh well.

      Another principle of good backups is to have another copy in another location, since having an extra hard drive won't help you if your house burns down.

      For average users, I think the only real solution to backups is going to come when network storage is dirt cheap and bandwidth is just as cheap. God how I'd love to backup my data with one entry in a crontab:

      0 * * * * scp -r / myaccount@bigcheapstorage.com:/home/myaccount

      (Using the SSH keysystem to avoid entering passwords, of course) :)
      • It also fails if you need to go back and grab a file from 6 months ago. Tapes and a good backup schema help take care of that.

        Of course, the poster was looking for a home backup, so data retention probably isn't high on his/her list.

        I have way too many god damn machines on a home network (windows, linux, Solaris/Sparc, OSX, etc) so simply backing up to a seperate drive doesn't look like it would cut it. Tapes would be best for a med/large *home* network, but it ain't exactly the cheapest. It might be worth looking at e-bay, dot bomb auctions to see if you can pick up a tape drive for cheap.
      • Sure if it's removable then you're probably mostly OK, but if some power accident fries your box, and your backup hard drive is in there, oh well.

        Another principle of good backups is to have another copy in another location, since having an extra hard drive won't help you if your house burns down.

        OK, so buy more than one back up HD for each application and don't leave it in the box. If you want, take the extra copy home or put it in a safe deposit box. His point is that the extra HD is still cheaper than removable media for most people right now.

        This is all lost on me right now. The largest projects I have easily fit on a CD, source executable and data files uncompressed. My photos are largely segregated by where and when they were taken and are archived that way. The one or two home movies I've made are no more than 10 minutes long and of such pathetic quality that they are less than 100MB each. This may change in time, but right now the 10 cent CD and a central FTP server are more than adequate. Thingy goes to FTP, then gets CDed several times and removed.

        I doubt I'll ever pay to have someone else paw through^H^H^H^H^H^H^H^H^H store my data.

    • Re:Hard Drives (Score:3, Interesting)

      by c.r.o.c.o (123083)
      I just hate it when somebody posts exactly what I was going to post. Had to erase a good two minutes of typing because of it. :)

      Anyway, that is exactly what I am doing. I have a 13Gb hdd for the system, and a 40Gb for storage (mp3s, movies, etc). Also I have a burner and one of the removable racks you mention. And in it, there's an identical 40Gb hdd used solely for backups. I keep it safe, and I plug it in every few days to copy the new stuff to it, remove the old, etc. I know that ideally I should have more backup space than hdds I'm using, but I never really run out of space. I am always writing the very important files to CDs, sometimes in duplicate. Call me paranoid, but after losing 3 years of data because of a hdd crash and a cheap CD which refuzed to be read, I'm not taking any chances. Also, all the stuff I don't need often (less than once a month) goes to CD.

      One very, very important thing though. Don't cheap out on the removable racks. Make sure that at least the lid on the one you get is mettal, and there's at least a fan in the hdd tray. All racks have one fan on the rack itself (the part that gets mounted in the case). But make sure you have another one in the tray.

      I used to have a rack made out of plastic completely, and with only one fan. My Maxtor 7200rpm drive was getting HOT. And I do mean hot! Then one day I ripped the IDE cable from its mount, and I had to buy another rack. This one is with metalic lid, and 2 fans inside. Now the hdd doesn't even get warm. And the difference in price between the racks was $10 canadian (about US$7.5).

      Those few extra bucks are probably going to prolong your hdd life by quite a bit.
    • Looks like the cheapest Kingston/StorCase offering [storcase.com] (which supports ATA/100, bonus!) costs around $80 mail-order [cnet.com].

      Are there any other reputable manufacturers that sell a cheaper solution for IDE?

    • Re:Hard Drives (Score:2, Insightful)

      by SecretAsianMan (45389)
      And then your house catches on fire, and your data is gone. Go with hot-swappable, and keep a weekly backup offsite.
    • Re:Hard Drives (Score:3, Informative)

      by cowbutt (21077)
      Given that a 100G hard drive is cheaper than any removable media solution, why not just buy another hard drive and install it in a removable (not hot-swappable, just removable) rack?

      Some thoughts:-

      1) IME, Racks cause heat build-up and kill HDDs. I won't use them and I recommend others don't either.

      2) If you're going to permanently mount and power the drive, then it stands as much chance of dying through power cycles and usage as your main drive.

      3) If you're going to leave the drive unpowered most of the time, then you probably won't bother backing up because it'll require a shutdown, an opened case, a boot and another shutdown.

      IMHO, If you're going to use another drive, you might as well go for RAID and have the backups done automatically and continuously.

      --

  • Tough problem (Score:3, Informative)

    by phr1 (211689) on Tuesday December 11, 2001 @05:48PM (#2689352)
    DVD media is about $6 per 4.7GB disk now, but do you really want to use 20 pieces of media to back up a 100 GB disk?

    One thing some people do is back up their HD to a second HD.

    Zip disks seem practically useless these days--recordable CD is just too cheap and universal by comparison.

    Tape drives are the high-end solution, but expensive.
    • Back in the day, people would use far more than 20 pieces of media to back up a 60 MB disk.
    • Re:Tough problem (Score:4, Informative)

      by zerocool^ (112121) on Tuesday December 11, 2001 @06:30PM (#2689703) Homepage Journal
      FYI, if there's anyone out there in retail sales working for a place that sells HP computers, you can log on to HP Info Lab [hpinfolab.com] and you can get a $400 rebate for the HP 100i DVD writer. There's also a $50 rebate available, which you can use in conjunction with the $400 rebate. The $50 rebate is available to the general public.

      This brings the price of a DVD burner down to $150, since the drive is 600 coon skins before rebate. At that price, and if the playstation 2 drops in price this christmas, i sense lots of burned games in my future...

  • by Strange_Attractor (160407) on Tuesday December 11, 2001 @05:48PM (#2689354) Homepage
    Just get a lot (A LOT) of 1.44MB floppy disks...
  • by meckardt (113120) on Tuesday December 11, 2001 @05:50PM (#2689374) Homepage

    Hey, I've got all these 5 1/4" floppy disks sitting in boxes in a back closet. I bet if I added them all up, they would amount to close to 100 GB.

    • Oh yeah...I've got a box of 5 1/4"'s I've been planning to burn to CD's forever.

      I can just see this as a backup solution ... "Just 4 more weeks and my Groundhog Day backup will be done...oh look - more trick or treaters" :)
    • I bet if I added them all up, they would amount to close to 100 GB.

      I bet they wouldn't! I remember seeing the stack of floppies that = 1 Travan tape.. It was 6 feet high.

    • 100GB of 5.25 inch floppies is like 28 cubic feet of floppy disk.

      At:
      2 MB/Floppy=.002GB/floppy
      5x5x1/25 Floppies/inch^3 = 1 Floppy/inch^3
      12x12x12=1728in^3/ft^3

      100GB of 5.25in floppies is:

      100/.002=
      50,000 floppies or

      50,000/1=
      50,000 cubic inches or

      50,000/1728=
      28.9 cubic feet

      That's a heck of a lot of floppy disks, espesically to back up one 5x3x10 = 150 cubic inch hard drive.
  • Not big enough (Score:3, Insightful)

    by paulywog (114255) on Tuesday December 11, 2001 @05:51PM (#2689377)
    I don't really think CDs and DVDs really aren't big enough for regular backup of large sets of files. It's just too inconvenient to have to setup a bunch of different 5GB backups, one per DVD (or swap DVDs). The only convenient solutions are to do what the first poster said: use a second harddrive, they're relatively cheap. Or buy a tape drive to store the backups.

    Personally, I backup to a second harddrive.
  • by still cynical (17020) on Tuesday December 11, 2001 @05:51PM (#2689383) Homepage
    CD-R/CD-RW are too slow and too small, plan on spending a day or so swapping disks. You can always mirror to another hard drive, get a basic RAID card or just use a Ghost-like program to do manual backups. But tape is still cheaper per megabyte and more reliable. Sure, you can damage a tape, but it's harder to do than with a hard drive. SCSI tape drives are more expensive than another drive, but fast enough, and allow you to keep multiple versions or copies of your backup. Try that with hard drives and you need arrays. Tape starts looking REAL cheap then.
    • You can find the Onstream ADR 30GB (firewire or ide) for about 200$US.

      Tapes are 113$US (shipped) for a 4 pack.

      For those of you following along at home, that is ~300$US for 120GB of backup.
      400$US for two sets of 120GB.

      Not bad, and it is a DAMN FAST drive...

  • Tarballz! (Score:5, Informative)

    by rantenki (66616) on Tuesday December 11, 2001 @05:52PM (#2689384)
    I have a 100BaseT network, and a server computer that resides in a different room from the rest of my systems. I rotate backups using those aluminum drive caddies. A pair of 60G drives turned out to be MUCH cheaper than the equivalent size tape backup. Every day, I rotate out the drive at the end of the day, and swap with the other. The spare I keep in a fireproof safe. Just tarball the appropriate directories. Done. Poof. Much faster than the average DDS3 tape drive too. Runs at night and I don't even notice it.
  • by weslocke (240386) on Tuesday December 11, 2001 @05:52PM (#2689390)
    I have to say that this is coming from someone with a total of around 280gig at the house, but...

    Out of 100gig, how much do you really NEED to back up?

    The vast majority of my space is taken up by MP3s (where I converted my CD collection), but that could easily be replaced. To tell you the truth, of the things that I would need (documents, pictures, etc), I could easily fit it all onto a CDR. Well, maybe two. (I take lots of pictures)

    Basically it boils down to, do you really need to shell out the money for that extra drive?

    :^)
    • This is exactly correct. There's no reason to back up what one already has on disk. In the event of a disk failure on, say, a Windows machine, there's no reason to need a backup of the whole disk when one has the install disks for the OS, Office, and whatever games you have.

      My backups consist of web design work, and that's basicly it. I have a rolling backup on a CDRW and it works great.
    • /home/dir (Score:5, Insightful)

      by xant (99438) on Tuesday December 11, 2001 @06:24PM (#2689655) Homepage
      This is where Unix's concept and implementation of HOME directory really shines. In the Windows paradigm, things can and do end up anywhere on the system, because you can write anywhere you want to. Application software is under no pressure to write to a standard place so you end up with things in: the desktop, the application's home directory, the user's home directory (if you're in win2k or later), a temp directory, etc. In Unix users have 2 places to write things: $HOME, and /tmp. If you don't want to keep a file around later, just remember not to put it in tmp. Then only back up $HOME. Everything else on your system can be restored automatically from either the net or the CD media that you purchased.

      Not to distro-bait, but Debian in particular shines here because apt makes it so damn easy to bring a system back to the state you wanted. For myself I have created a meta-package (.deb) which does nothing but depend on the applications I want installed on every desktop system: galeon, gnucash, xchat, gaim, xmms, vim-gtk, and a handful of others. Then I back up my meta-package, all of 10k including a few shell scripts I wrote for myself. Install my meta-package on a new system, and voilá, apt fetches and installs every app, that I need to continue working, dependencies included.

    • by geekoid (135745) <dadinportlandNO@SPAMyahoo.com> on Tuesday December 11, 2001 @06:25PM (#2689661) Homepage Journal
      digital movies can be hard to replace.
      Obsoleted OS's. If someone is usinf win95, and its got all the patches, you may want to back up everything, do to lack of support.
      is replacing 100's of megs of .mp3s really trivial? certainly not impossible, but still a pain.
      The ability to get everything back w/o reinstalling and downloading Service packs and patches is a huge plus to most people.
  • Another hard drive (Score:3, Informative)

    by Rick the Red (307103) <Rick.The.Red@gma i l . c om> on Tuesday December 11, 2001 @05:52PM (#2689391) Journal
    That's what I use. As you said, "drives in the 10-20 gig range [are] only getting smaller and less expensive," so buy two and use one as a backup. If your box is full, put the backup drive in an old 486.

    Actually, what I do is make the new (largest one I own) drive the backup drive, put the old backup drive into use as the primary drive, and retire the smallest one. Just make sure the new drive is as large as the others added together.

    CD-R's are OK, but why bother with the hassle? Just run a cron job to copy the files every evening/hour/whatever.

  • Onstream (Score:4, Informative)

    by wackysootroom (243310) on Tuesday December 11, 2001 @05:52PM (#2689393)
    Onstream 30 or 50 GB ADR Tape backup.

    Pros:
    Can be found for under $100
    Linux Support!

    Cons:
    Tapes are expensive
  • Get a USB drive (Score:2, Interesting)

    by biscuit67 (517991)
    I got an external usb drive for $248 for an 80G unit. It's the cheapest, fastest, least hassle way of backing up your data. Yeah, of course, you can buy internal drives at a much lower cost but I also needed a way of carting the data to other machines easily. Even though I don't have a USB2 interface, backing up my 40G or so was complete overnight. I'm sure if I had usb2, it would have only taken 2 or 3 hours. Sure as hell beats tape. Tape is a thing of the past. It's NOT cheap.
  • by tcc (140386) on Tuesday December 11, 2001 @05:53PM (#2689403) Homepage Journal
    Ultrium [ibm.com] tapes can backup 100GB Native but the price tag is way out of line for small buisness or home use (5000$+ a 100GB drive, ouch).. same goes with any dataloading systems... The only cheap tape backup I've found that was giving the best storage/price (aside from buying those used DSS 4/8 gig drives) is those 33GB Native VXA drives. [vxa.com]
  • Actually... (Score:3, Interesting)

    by JanneM (7445) on Tuesday December 11, 2001 @05:54PM (#2689407) Homepage
    First, on a typical system, not all data is really worth backing up; the OS and all applications can be reinstalled in the event of a crash (for Linux, it might even be slightly beneficial, as you'll reinstall newer versions and get rid of various cruft you've forgotten you have). Some data has been saved just because it's convenient or simply less bother than having to actively remove it (for me, I tend to collect old logs, various mails I never will look at again and documentation that's several revisions old). A lot of mp3:s and movies may already be burned onto CD:s. That filled 40Gb drive may actually 'only' contain 4-5 Gb of data that actually needs to be backed up.

    The data I actually need to back up I manage by having the important stuff an specified directories, then mirroring them over the net to my machine at work. By doing it incrementally, there is little time or bandwith wasted.

    /Janne
  • Another solution (Score:2, Interesting)

    by Outlet of Me (90657)
    Another solution might be to pair down what you backup. It isn't strictly necessary to backup everything on your hard drive. OSes and programs can be reinstalled, but the data that you create with them is the more precious commodity. Of course, figuring out exactly what you need to backup is the problem, and you still lose some information that isn't easily backed up or restored (like settings for programs in the Windows registry). It's taken several reinstalls of Windows before I figured out exactly what I needed to save...
  • by zutroy (542820)
    To minimize your storage costs, try using LZip: Lossy Compression [sourceforge.net]. Sure, you won't be able to restore your system to EXACTLY the same state, but you can compress your files to as little as 0% of their original size!
  • Why bother? (Score:5, Insightful)

    by crow (16139) on Tuesday December 11, 2001 @05:57PM (#2689431) Homepage Journal
    Why are you bothering to back up your data?

    That may seem like a stupid question, but you need to consider the reasons you want to have a backup before you settle on a method.

    Are you afraid of your drive failing? If so, then using a RAID solution should cover you.

    Are you afraid of losing your whole system (perhaps due to lightning or theft)? If so, then your backup must be kept physically isolated from your system.

    Are you afraid of accidentally deleting files (such as `rm -rf /` or a virus)? If so, then a RAID solution is useless.

    Are you afraid of having your system down for an hour or two while you replace a drive? If so, then regardless of other issues, you need a RAID setup.

    Do you want to use your MP3s with some other device? If so, you probably want CD-R copies.

    Of course, there are other considerations that I haven't mentioned or thought of.
    • by idonotexist (450877) on Tuesday December 11, 2001 @06:08PM (#2689518)
      Is there something you are trying to keep secure?

      Why do you want to keep your data safe?

      Is an encryption device utilized with a harddrive or an application?

      Where did you obtain all of your software?

      Are you looking to copy to a device that has the ability to encrypt files?

      If you are looking for a portable back-up device, why do you need it to be portable?

      Do you travel extensively?

      When you do travel, do you primarily travel by air?

      Do you have a digital camera?

      Do you have a mobile phone?

      Have you ever encrypted an email message?

      Have you ever deleted an email message?

      If so, have you had data rewrite over the sector(s) containing such message?

      What was the title of the last book you purchased?
    • Re:Why bother? (Score:2, Insightful)

      by foxtrot (14140)
      Well, I don't know about most of you, but I'm backing up data for a number of reasons:

      1) I'll never get another six weeks of vacation all at once to replace my digital pictures of Europe.

      2) Sure, I still have all the CDs I made .mp3s of, but it'd take me months to re-rip and encode 'em all.

      Some of it can't be replaced, some of it isn't worth my time and effort to replace it when a hundred bucks solves the problem.

      I've been working with the Addonics ExDrive stuff. PCMCIA, USB 2.0, or Firewire enclosures for normal hard disks. I don't know about Linux support; I went this route since I know it works on my Windows laptop and on my iPaq... It does kind of take a different mindset from what I had been using (read: "It's disk! Install it in something!") but it does make sure I've got redundant copies of, say, digital pictures that aren't on-line.

      -JDF
  • The cheapest backup media now is more hard drives. Really!

    So I got 4 x 60 GB Maxtors (cheapest MB/$ ratio when I purchased), two Promise Ultra TX2-100 controllers, and set up a 180 GB software RAID-5 under Linux. I'm running ReiserFS on it, so I don't have to fsck 180 GB if I crash the system running an experimental kernel, and I keep the whole thing on a big UPS. Total cost for hardware was about $500.

    I mirror data onto the RAID using rsync from my other computers, (or just drag-and-drop to the Samba server from the Win2K box). I think this is cheaper and more effective than a tape backup system for 180 GB of data.

    The Linux software RAID gives me the reliability - I've inadvertently tested it when a power connector popped loose from one of the hard drives - I didn't even notice until I read the kernel log (for a different reason). After powerering down, I plugged the drive back in, restarted, did the "raidhotadd" command, and away I went again. No data loss, no hassle.

    For stuff that I really, really want backed up beyond the reach of thieves and fire, I use CDR's and a safety deposit box. Luckily there is not too much stuff that falls into this category - source code and documents for projects I've worked on, my email, stuff like that - it still all fits on one CDR.

    If you don't use Linux, I think Win2K can do software RAID too. Never checked though.
  • by sulli (195030) on Tuesday December 11, 2001 @05:57PM (#2689435) Journal
    Clearly the answer, for easy backups of a 100G drive, is 21 iPods.
  • by igrek (127205) on Tuesday December 11, 2001 @05:57PM (#2689437)
    There are several programs allowing to use your DV camcorder for backups. For example:

    http://dvbackup.sourceforge.net
  • by nl (158427) on Tuesday December 11, 2001 @06:00PM (#2689453)
    Agreed that just backing up to another HD provides the best overall method for creating a complete backup of 100MB of disk storage.

    However, I would suspect that most users don't change a huge percentage of their HD's content on a daily basis, unless you are routinely d/l'ing or ripping MP3s and MPGs on a daily basis (and I note that when I do generate that kind of traffic, it is usually because I am making a compilation CD, and while this does generate a few GB of "new" files on my HD that day, that data doesn't need to be backup up because I've got the original CDs anyway).

    As a result, it seems to me that a reasonable solution is to create a "baseline" backup, say to a CD or DVD, at system install time, when there is (relatively) little on the disk, and then each day (or week, depending on needs), do an incremental backup of changed data only to another CD.

    This approach is obviously quite inefficient if you have a complete HD failure, in that you have to recreate a new drive by starting with the first backup CD and then restore EACH ONE thereafter until the final CD restores the disk to it's last backed-up state, but for a more common problem of losing or corrupting an individual file, since that is more likely to happen with a recently modified than a remotely modified file, you are likely to be able to restore a last good version within only a few CD's of the most recent incremental backups.
  • by pete-classic (75983) <hutnick@gmail.com> on Tuesday December 11, 2001 @06:00PM (#2689454) Homepage Journal
    A lot of people have mentioned that disk to disk backup seems to be the best way to go.

    I agree.

    What hasn't been mentioned is rsync [samba.org], which makes disk to (local or remote) disk backups fast and easy.

    It is trival to set up a second disk that is a "stale" mirror of your primary disk(s) that backs up nightly, and will boot off a floppy. This captures some of the advantage of RAID (quick recovery) while being an actual backup, not just fault tolerance.

    Rsync can use ssh as a transport, so you can securely back up remote disks as well.

    -Peter
    • I'd recommend the tape route over backing up to hard drive, but if you must, and if you want to use both ends of the sync try something like Unison [upenn.edu]. This thing makes synchronizing my laptop and desktop a piece of cake.
  • by greenfly (40953)
    I already have 2 30GB drives, and after hearing horror stories about hard drives crashing recently, I've decided my next step is to get another 30GB drive and run RAID5 across them all. Linux can do this in software by the way, and this way you can be assured that your data will stay intact if one of the hard drives crash or not, plus you won't lose half of your drives to backup.

    If you are concerned with recovering deleted files, simply use tar or something similar to backup to either a separate directory, or create a separate partition on the RAID array. Another advantage is that you can always increase your storage by slapping in an additional drive, partitioning it the way you want, and then adding it to the current array.
  • Backup solutions (Score:2, Interesting)

    by Haywood68 (531262)
    Go onto ebay... you can pick up a DLT (15/30GB) drive for around $150. scsi card $50. media are a little expensive (~$25 ea), but for around $275 and a little opensource backup software, you can get reliable backups.
  • by morzel (62033) on Tuesday December 11, 2001 @06:03PM (#2689480)
    I've seen some commenting that creating a RAID array will suffice to secure your data, but that's really not true:

    RAID offers good protection for some things: hardware failure (ie: HD crash) and uptime. That aren't the only woes, however... You can loose data in a lot of ways:

    Disaster (fire, quake, flood, nuff said)

    Hardware failure (disk, controller, ...)

    OS failure (FS corruption, ...)

    Application failure (User space applications malbehaving, virii, ...)

    User failure (accidental deletes, experimental children - trust me on this one ;-)

    RAID will protect you from the second, but will happily add nothing in case of any of the other failures. Backing up to another media is a necessity.

    Adding an extra disk (or two, or three), and some tar/cpio cronjobs will add basic protection. (No disaster recovery for you, unless it's off-site :)
    Removable harddrives (firewire, frames, ...) are a plus, but more cumbersome.

    Tape is considered a more 'trustworthy' backup medium because the mechanism and data storage are separated (ie: tape drive / tape), while in a HD it's in one single package, and it's not as easy to replace the logic board/stepper motor if this flunks. With tape it's easier: just get a new tapedrive.

    Anyhow: don't rely on RAID to save your data - it won't.

  • I would like go through a network backup solution and then do the removable hard drives.

    Of course, data file should not be installed on a local drive, so that you can implement some sort of a disk imaging solution for the base installation. The disk image should be of the main drive with the core installation folders or mounts.

    This way if someone screws up the system, you can blow out the main drive, replace it with a known good config, and then add the two or three apps you need, with the datafiles safely someplace else. This could even be done from a bootable CD, if needed.

  • 802.11 solution (Score:5, Interesting)

    by swordboy (472941) on Tuesday December 11, 2001 @06:10PM (#2689546) Journal
    I'm sharing my cable modem via 802.11 with all the neighbors and since I am the local "neighborhood helpdesk technician", they often come to me for advice. Recently, one of them wanted to know how to go about backing things up properly. It dawned on me that hard drive space is abundant and most people are buying much more than they need (the person in question has an 80 gig at about 20% capacity). So I worked out a deal so that everyone is backing up to each other's PC at night on a weekly basis. The 802.11b connection keeps drive thrashing to a minimum yet provides enough speed for complete backup on an overnight basis.

    I should start charging for these ideas... Can't wait for the proliferation of freenet!
    • But what happens when people shut their computers off, as users are wont to do, and user B wants to access his backups on user A's hard drive? Are you using some kind of equivalent to WoL for 802.11? (Does such a thing even exist?)
    • by doorbot.com (184378) on Tuesday December 11, 2001 @06:46PM (#2689791) Journal
      I'm sharing my cable modem via 802.11 with all the neighbors and since I am the local "neighborhood helpdesk technician", they often come to me for advice.

      I'm so sorry to hear that...

      So I worked out a deal so that everyone is backing up to each other's PC at night on a weekly basis.

      ...and I'm really sorry to hear that. I hope you have a good lawyer and some written authorization from your neighbors.

      Just the thought of what you've setup is enough to give me shivers.

      What were you thinking? You're setting yourself up for a huge lawsuit and/or war with your neighbors. What if someone gets pissed at you because your dog leaves a surprise on their lawn? (just an example). You're the one who's responsible here, and you get no benefit. Your neighbors are leeching off your cable modem and using you for your computer knowledge.

      Geeze, if I were you I'd be going insane! At least charge them for cable modem usage. And charge them for backups, but back it up to your own server. Get a tape drive and keep a recent set offsite (perhaps at a friendly neighbor's place, or a friend's place). And most of all... have a full contract signed by your neighbors clearing you of any responsibility!

      Excuse me while I have a heart attack...
  • Consider the technologies and what you expect from it. Mainly, the technologies are :

    Magnetic storage : the data is sensitive to electromagnetic fields, heat, and the magnetism in the ferrite particules decays over time (i.e. the data has a reasonable chance to get corrupted after 5 to 10 years, even if the media is properly stored). Also, reading a magnetic medium can wear out the medium, but that's not an issue if you just want backups. The bit density on these media is good, and the price per megabyte is excellent.

    Optical storage : there is no decay in theory, but I read somewhere that pressed CDs actually have a lifespan of 20 to 30 years even when properly stored, and CDR(W) even much less, especially when stored exposed to sunlight. I'm not certain there is much real-life data on the durability of DVDs. The bit density is good, but not as good as magnetic storage, and the price per megabyte, I guess, is comparable, but mastering optical media requires more effort.

    I'm not considering the fact that magnetic storage media are re-writable, since you're talking about doing backups. So I guess, the question really boils down to whether or not you want to have backups that are more reliable over time, and whether or not you want to spend the time and effort to create CDs or DVDs of your data.

    Then of course, there is the question of obsolescence : CDs have been around for years, and I don't think they're going to disappear any time soon. However, tapes for example can become unreadable because nobody makes readers anymore (ever tried to re-read a 80Mb Colorado tape recently ?). Same goes for hard-drives, although I'll admit IDE and SCSI interfaces will be around for quite a while longer.

    Then of course, if you really badly want your data stored safely for the longest time, you can get yourself an old punchcard deck, a very large box of punchcards, and there is a fair chance that some archeologist in the year 4000 will find them in a dig just above where you house used to be ;-)

  • For all those moderating the posts about setting up a RAID system as 'Redundant', please think first if such moderation is not 'Redundant' itself, since RAID is obviously 'Redundant'. Hmm. OK, it could be RAID-0, but noone is talking about stripping. :)
  • Try DLT... (Score:5, Informative)

    by bani (467531) on Tuesday December 11, 2001 @06:12PM (#2689562)
    There are two ways you can go relatively cheaply, and IMHO a far better solution than CD-R or CD-RW.

    Pick up a DLT2000XT (15gb native) off ebay for about $200. Tapes are dirt cheap, about $5/ea and the media is extremely durable, nearly indestructible.

    Pick up a DLT1 (40gb native) off ebay, about $500. Tapes are moderately expensive at around $20/ea, but again the media is extremely durable.

    DLT is industrial strength backup, the drives are built like tanks and the tapes can take incredible abuse.

    Its all standard SCSI and works great with linux, no problems whatsoever.

    I considered buying hard drives for backups, but they are far too fragile for long term backup and off-site storage. Most drives arent designed to be spun up and down lots of times either.

    Last thing you need is for your backup harddisk to go splat when youre trying to power it up to restore your main system from a data loss.

    With DLT, this isnt likely to happen.
  • For my home network, I use removable internal hard drives. There are several manuafacturers that make units that will turn an IDE or SCSI drive into a removable unit. I've got a dataport IV which has a component that fits in one of the 5" bays in my PC and connects to the motherboard via an IDE cable. A second component opens to hold a standard 3" IDE drive and plugs into the first component. I've got several of those. Back-ups are straight-forward: 1) Shut down the machine, pop in a drive module. 2) Boot 3) Do back-ups either locally, or across the network. 4) Shutdown, pop-out drive module and place it in a drawer.

    It would be nice if I could hot-plug the drives instead of having to reboot. If I was more thorough (or paranoid), I'd take the drive off-site and put it in a safe deposit box. Dataports are made by CRU-INC [cruinc.com], but there are other similar products.
  • I know this is a windows solution however I'm running two w2k advanced servers and have one on site one off site at my home.

    They are clustered. First created with a 100mb crossover link. Then moved one offsite with a modem link between them (direct modem to modem). AS the files don't change much there is not a lot of traffic. (1 file in 3 days maybe).

    There is also software link doubletake and surviveIT [ca.com] for the corporate world.
  • Doing backups (Score:4, Interesting)

    by digitalhermit (113459) on Tuesday December 11, 2001 @06:16PM (#2689588) Homepage
    One thing to do first is to separate the data from the OS. I.e., create a backup set with all your reinstallation files. For my systems this is a copy of the Linux installation iso and a directory of all the patches to it. Backup your system files (depends on if you're running Winders or something else). This takes a couple CDs, but is a lifesaver if you need to recreate the system or even upgrade it.


    Bringing up the system is less of a problem with newer OSes, since you can usually, at minimum, get to your data. Configuring the database, webserver, and firewalling depends on how good you are with the OS. However, when I worked at a former company there was no real plan to get a working system back in place. We were using Novell with Arcserve -- unfortunately, you couldn't get to the data without a working system.


    Next I usually try to segregate rapidly changing stuff versus things that are pretty much static. E.g, my mp3 collection is relatively static. I occasionally buy a fresh CD and rip it, but I'm pretty much satisfied with my collection as it is. I put these on CDROM. It takes a while to create them, but it's cheap and safe. If you want to keep everything up to date, you can run a script to save only files not included on the CDROM.


    Finally, I back up my constantly changing stuff such as CVS, MySQL database, etc. to 4MM tape. It's cheap (hardware and tape) and most drives are pretty well supported.

  • but sumthin like this was just posted on ask slashdot [slashdot.org] recently...

    actually, most of the points made in the earlier story apply here.

    and as i'm sure others will/have jumped in to say, there's a big gap between failover/redundancy afforded by a raid setup or external hard disk and tape backups. it, like so much else, depends on what you need.

    i'm pretty discouraged personally. i don't really see an affordable way to do real backups of a couple hundred gigs of data. it's probably going to have to be a mixed setup. most of the data is static: flac & mp3's. so maybe that just needs one offsite backup that's done to a couple hard disks--basically a mirror. then get some kind of tape system for the other 40gigs or so of slowly or quickly changing data that i also want to keep a couple snapshots of for historical purposes, as well as rotate offsite. still not sure what the latter solution should be.

    i have an old dds2 drive. but that's 8gb max compressed. dlt or ultrium is way out of budget. maybe an onstream. but i don't know anyone who's actually got one--any firsthand reports on these? and how likely is whatever onstream uses likely to exist/be supported in say 5 years or beyond?
  • If you need to protect against fault tolerance then just get another hard drive. IDE RAID on the cheap can be done via software or another drive controller.

    However, if you need to backup data with the ability to rollback changes or deletions then you are most definitely looking at a tape system.
  • I work for a professional photographer. We've been slowly phasing out film in favor of digital for about a year now, and currently we do everything digital except for certain outdoor things where certain issues make digital not feasible.


    Currently we've got a storeroom full of boxes and boxes of CDRs. I want to make an archival backup of them.


    Negatives will last for a hundred years if they're stored properly. Just the other day we made reprints of one of the first sessions our studio did, more than 20 years ago. However, CDRs do not last this long. Assuming nothing catastrophic (fire, CDs breaking) happens; CDRs are only made to last a few years at most. (Rough estimates put the shelf life of CDRs at 10 years or so.)


    What form of storage is the most archival?

    Consider that we've got probably a thousand CDRs sitting in boxes. Some way of doing a batch backup (CD towers?) would be great.

  • by AgTiger (458268) on Tuesday December 11, 2001 @06:23PM (#2689646) Homepage
    I'm classifiable as an audio addict, having taken my entire personal
    collection of CD's and ripped them to MP3's at 320 bit, and wanted to
    have them stored in a central place, accessible from any machine in my
    home. Currently this collection is at approximately 620 full CD's of
    music, and I'm pushing right at, or just above the 80 gigabyte limit.

    Now when you factor in personal files, financial records, games,
    downloaded material, installation software you don't want to lose,
    etc...etc... Well, see for yourself. Here's my space breakdown for the
    partitions on my main file server Fumus (Smoke, in Latin):

    fumus:/pub/mp3 # df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/hda3 3.0G 2.1G 804M 72% /
    /dev/hda1 129M 6.8M 115M 6% /boot
    /dev/hda5 9.8G 1.8M 9.3G 1% /home
    /dev/hda6 20G 13G 6.3G 67% /pchome
    /dev/hda8 40G 22G 17G 57% /pub
    /dev/hdb1 75G 38G 33G 53% /pub/mp3
    /dev/hda7 1.9G 20k 1.8G 1% /scratch
    /dev/hdc1 74G 34G 40G 46% /pub/mp3_2
    /dev/hdd1 74G 36G 37G 49% /pub/software

    So, here's what I looked at:

    Tape: For the size I'd need: Way WAY too expensive. When I brought
    the media down into the range I'd afford, I'd be swapping tapes all week
    to get a backup done. Not time effective.

    CD-R: Faster, yes, but at 650 megabytes per media, same problem as
    tape, only you've traded magne tic for optical.

    Extra hard drives in the same machine: Originally, this is exactly what
    I had done with a single file server running Reiser file systems in the
    more experimental days. I got the scare (and lesson) of my life when
    Reiser went a bit nuts, and started corrupting some of my data. I only
    lost about one percent, but I vowed, never never NEVER again would I
    backup data on a critical machine on live media in the same machine.

    Okay, so here's what I finally DID select as my solution: A second
    machine called Ignis (Fire in Latin) that uses the absolutely identical
    configuration, right down to the types and number of drives, partition
    sizes, everything. They both connect into my 100Mb network switch, and
    Ignis rsync's from Fumus every hour on the hour thanks to scripts in
    /etc/cron.hourly

    In fact, here's Ignis' /etc/cron.hourly/rsync_with_fumus script:

    rsync -arul --one-file-system --quiet fumus:/pub/mp3_2 /pub
    rsync -arul --one-file-system --quiet fumus:/pub/mp3 /pub
    rsync -azrul --one-file-system --quiet --delete --force fumus:/pub/software /pub
    rsync -azrul --one-file-system --quiet --delete --force fumus:/pub /
    rsync -azrul --one-file-system --quiet --delete --force fumus:/pchome /

    Is this a bit extreme? Yes. But... if, gods forbid, Fumus really does
    let out its magic smoke, or Ignis does catch on fire, and the physical
    media were actually damaged, hopefully the damage would be limited to
    *one* case, and wouldn't end up taking both machines out. Then I really
    would be crying the blues.

    Oh yes, and each machine is on their own 900VA UPS. I'm not playing
    THAT game. :-)
  • Backup has always been expensive. You have two choices:

    1) Go cheap (i.e. Zip drive, MO drive, CD-RW, etc.) and only backup the files you NEED (i.e. home directory, "My Documents" folder, etc.)

    2) Shell out for tape. This way you will be able to make multiple backups, keep them offsite, maintain them long-term, and back up most of your system. I find that the sweet-spot is usually to go 1-2 generations old (i.e. right now I'm using a DAT24 drive), that way you only have to pay about $400-600 instead of $1500+ for the mechanism.

    Don't fall for the "just use hard drives" trick. Hard drives have a number of problems.

    1) Mechanism + media in one unit. I've seen hard drives whose heads STUCK to the platters, rendering them useless, after only sitting around for a couple of months. Oops! Data gone! They are also sensitive to static and to environmental changes -- if your backup drive gets zapped accidentally, you can't just plug the media into a new mechanism! Data gone!

    2) Backups limited. If you backup a 100GB system onto a 100GB hard drive, you're limited to one physical backup. This creates all kinds of problems... Many's the time I've had to go back four or five generations in backups to find an old file that I didn't realize I'd deleted months ago. Not to mention that you are limited to one backup stored in one location -- no redundancy.

    3) Even RAID has problems -- I've had bad SCSI cables that filled a RAID filesystem with corruption before we were able to track down the problem and switch the cable. If that happens when you're just depending on RAID to preserve your important data... oops! There's a reason why many RAID-enabled datacenters also maintain backups of critical data in a second medium!

    So that's my personal take. I think either tape, maintaining at least 5-10 backup generations at a time, or MO/CD-RW just for keeping your critical files. Or both, even.
  • Digital Camcorder? (Score:3, Informative)

    by tjw (27390) on Tuesday December 11, 2001 @06:31PM (#2689708) Homepage
    dvbackup [sourceforge.net] is a utility that lets you store up to 13gb of data on commodity miniDV tapes. With the use of a Digital Camcorder.

    This is not exactly a dirt cheap solution, but if you have/want a digital camcorder anyway, there's only the cost of extra tapes.

    Make sure the camcorder works with dvbackup before buying one though. It doesn't work with any JVC's that I know of, or at least not mine :(.

  • by leto (8058) on Tuesday December 11, 2001 @06:36PM (#2689738) Homepage
    for i in `cat rsync.list| egrep -v "^#"`
    do
    HOSTNAME=`echo $i| awk -F: '{print $1;}'`
    DIRECTORY=`echo $i| awk -F: '{print $2;}'`
    DATE=`date +%A`
    install -d /vol/backup/$HOSTNAME/$DATE
    rsync --numeric-ids --compress --rsh=/usr/bin/ssh --recursive --archive --relative --sparse --one-file-system --compare-dest=/vol/backup/$HOSTNAME/current $HOSTNAME:$DIRECTORY /vol/backup/$HOSTNAME/$DATE
    done

    Then once a week we run a similar script that updates the 'current' directories and uses --delete

    (rsync.list contains entries like "hostname:/some/mounted/partition")
  • Some Metrics... (Score:3, Interesting)

    by pbryan (83482) <email@pbryan.net> on Tuesday December 11, 2001 @06:38PM (#2689747) Homepage
    To backup a 100GB drive, you require...

    - 6 DVD+RW (18 GB) discs, or
    - 20 DVD-RAM (5.2 GB) discs, or
    - 158 CD-R discs, or
    - 72,818 HD 3.5" floppy discs
  • Hey old-timer... (Score:5, Insightful)

    by plastik55 (218435) on Tuesday December 11, 2001 @06:52PM (#2689825) Homepage
    Back in the day, you would actually buy software packages dedicated to doing backup. And they all did s thing called "incremental backup." That meant that they kept a database of all the files on your disk, and they would only copy the files which had changed since your last backup.


    That worked really well for backing up our 80MB drives onto stacks of 1.44MB floppies, since you would really only need to insert about 5-10 floppies during your weekly backup, just to get the files that changed.


    So why not just do incremental backup onto CD-Rs? Even with 100GB of archives, most of those are static. You probably won't need to use more than one CDR per week (maybe two) to track the changes. It's cheap, relatively painless if you've got the right software (and it wouldn't be hard to throw together incremental backup/recovery scripts in Perl if you're into that sort of thing.) and you've probably already got a CD burner.


    If less than 650MB of files change in a week, the rest of the CDR can be filled up with files that were on earlier CDRs (this way your backup set can remain finite and you can throw out the earlier CDRs as they become obsolete. Or if you keep them all, you can reconstruct that state of your hard drive at *any* time, not just at the last backup.) This seems ideal to me--why is everyone else talking about expensive solutions like tape drives, DVD-RWs, and second hard drives?

    • Re:Hey old-timer... (Score:3, Informative)

      by glwtta (532858)

      Of course you'll still need something like 150 CDRs to get everything, inceremental or not - and that's just the first dump that will keep growing.

      BTW, to the other replies to this - one of the companies I worked for, we did full, weekly backups of all the important systems (ie the actual OS and software - not the data) like the domain controller, database servers, fileservers, firewall, the backup servers themselves, etc. and then daily incrementals on them.

      For the actual data on the file and database server, plus everyone's personal workstations (something like 100 people) it was a full backup once a month (maybe less) and nightly incrementals, which worked great because here people usually wanted just a couple of files their dumb asses deleted a week ago (usually perfectly timed to want the tape that just went off to off site storage :) ) All in all that achieved very good granularity. Of course the incremental tapes were recycled after about two months or so.

      The database servers were a bit more tricky so additional full dumps were done manually, when, let's say, 500,000 new compounds were imported. There were also databases attached to data collection robots that had their own separate system going.

      All in all, this took three backup servers (one for the data collection databases, one for individuals PCs and one for the other stuff) with their respective tape loaders, plus individual tape drives for the main database servers (we also had things like SGI workstations, but the scintists on those did their own thing), and the full backups did take up almost the entire weekend. But it was setup well enough that it took only a few minutes a week to maintain the whole thing - get a new tape, write on it what the server tells you to, put it in the loader, take the specified one out, send it off - you are done. And it did work - they had to recover after a fire once (before my time) and did it without any problems.

      Anyway, the point of the story was this - it's not as simple as "do incrementals" or "no! do full backups" when you are talking about actual companies, it's a bit more tricky than that.

  • 100GB? Whew! (Score:4, Insightful)

    by rnturn (11092) on Tuesday December 11, 2001 @06:53PM (#2689830)

    I have to wonder whether (first of all) why in the heck anyone would need to have 100GB of disk space on a home system. But then I have five systems networked together and have more storage than I would have thought sane a few years a go though I have a bit of a ways to go before I will run into the poster's backup problem. It wasn't too long ago that, if you could afford 100GB, you could probably afford a SCSI array controller that would let you do a lot of RAID, hot swapping, automatic drive replacement, etc. With today's cheap disk prices you don't have to be wealthy to have an ocean of disk space. (I can remember the days when we thought having 900MB on a MicroVAX II was extravagant.)

    You could always do it the traditional way and get some tape drives. Unfortunately, they're much more expensive than you might think when you have to backup that much disk space. You certainly wouldn't want to go cheap and be feeding 90m DAT cartridges into a drive all night (it'll start feeling like you're backing up to floppies before long). A good high capacity tape drive can get, what, 20GB onto a single cartridge? Not bad. And I think that at this point in time, tape is more cost effective than DVD-R. (Something tells me that the MPAA, and maybe the RIAA, will try to keep it that way too.)

    Mirroring disks can be helpful. Hard disks are getting cheaper and cheaper. Heck it's almost scary mow much disk space you get in a typical PC sold at Best Buy nowadays (and without a backup device; it's almost criminal). If you're running mirrored disks you'll forestall the inevitable disk crash that takes all your data with it. Question for the Linux folks using the `md' driver: Does it allow adding a third member to a mirrorset? And, if so, can it be done while the system is `live'? (The third member gets removed and taken offsite in case there's a disaster.)

    One final thought: The poster wasn't actually running a 100GB filesystem were they? I'm thinking that a power glitch could cause a world record to be set for the longest fsck-on-reboot run. Plus I'd think that backing up such a beast would be a challenge. I tend to keep my filesystem sizes no larger than what I can fit on a single tape cartridge... just to make life simple. (I'm used to having to pipe `df' commands through `more' at work so I don't mind lots of mount points. :-) )

  • NNTP backup (Score:3, Informative)

    by xant (99438) on Tuesday December 11, 2001 @07:12PM (#2689938) Homepage
    This may seem weird but a disconnected innd can act as a lovely backup server. Simply put a news posting client on every workstation to be backed up. Those workstations can use newspost or similar to post their important files to the server, along with commentary (making it easier to find the right file), timestamps and dates (provided automatically by the server when you post), etc. To back up the server, you backup one directory, where inn stores its files: /var/spool/news/news.archive

    Advantages:

    • Standardized TCP-based protocol, uses authentication.
    • Lots of free "backup software" available - e.g. pan, newspost, tin...
    • Centralized backups over the network! Only one server directory to back up.
    • Backups are automatically dated by the server, and can be signed and even encrypted if you integrate pgp into your posting script.
    Disadvantages:
    • Posts take up 35% more space than the original file did. This is adequately solved by compressing the inn directory before backing up.
    • Only solves backing up single-files cases, such as compressed archives. You cannot automatically restore a system to a particular state this way, only individual files. This is optimal for backing up your home directory, sub-optimal for system images. (See here [slashdot.org] for why this may not matter to you.)
    • If you need a secure pipe to the backup server, you either have to have a firewall in place (and you should), or you need to provide the encryption layer on your own. NNTP does not "naturally" provide encryption afaik.
  • by Julian Plamann (449854) on Tuesday December 11, 2001 @09:08PM (#2690607) Homepage Journal
    This may not be the best way to do it, but it works for me...

    I have a "backup" hard drive in my server. This drive is always unmounted so that there is no chance of filesystem corruption from the operating system.
    I just use a crontab to run a simple script that mounts the drive and coppies whatever specified backup files to it, then unmounts it. The same method slightly modified could be used to back up this same backup disk to another location on the network on regular intervals.
  • by skoda (211470) on Tuesday December 11, 2001 @10:11PM (#2690979) Homepage
    Unless you're creating 10 - 100 GBs of *new* data between every backup, it might be simpler (and cheaper) to use incremental backups.

    Instead of dumping all 100GB of files every time, 95% of which haven't changed since the last backup, use an incremental backup program to write only the 5% that actually changed. After the initial archive, the backup files will be significantly smaller, and could potentially saved on CDs.
  • Ecrix VXA-1 (Score:3, Informative)

    by Wdomburg (141264) on Wednesday December 12, 2001 @12:43AM (#2691728)
    This has been my choice for low-cost backup solutions for a couple of years now.

    The drives support three different flavours of media - 12GB, 20GB and 33GB, and come with IDE, SCSI, or Firewire interfaces. The IDE is cheapest, at $699, with the media costs being $80, $45, or $35 depending on the capacity.

    Is it the absoulte top of the line as far as tapes go? No. But the cost can't be beat. And you get a reasonably fast (3MB/sec) drive with very nice reliability (take a look at the independent testing on their site; e.g. soaking a tape in hot coffee for a minute, rinsing it, drying it and reading the data off.)

    They also recently merged with Exabyte, who will be positioning it as their new value solution. Hopefully the Exabyte name will expand the market enough to drive the prices down on these even further.

    Matt
  • Just FTP the files to the SAN at work. I hear they have tape drives and might even do backups!
  • by aoliva (310439) on Wednesday December 12, 2001 @01:16AM (#2691858) Homepage
    Being one of the maintainers of Amanda (www.amanda.org), I'd always been of the opinion that tape backups were the only way to do backups seriously.

    The recent explosion in disk capacities and decrease in prices got me to rethink this, just when it came the time for me to set up a home office. When I compared the cost of a reasonably-good tape drive and a number of tapes large enough for me to get at least a month of backups in rotation, and computed how many 60GB disks I could buy with that money, the solution was clear.

    I ended up setting up 3 machines with 4x60GB each. They're all on RAID 5, such that if any single disk fails, the machine keeps running (actually, I have /boot on RAID 1 over the 4 disks and / on RAID 1 over 2 of the disks and an alternate root to test upgrades over the other 2, but you get the point). This got me blazingly fast disk access, that tapes would never help me get :-)

    I get all my backup-worthy data rsynced over to the other machines daily or so. I plan to start playing with Inter-Mezzo soon, so that I don't have to remember to run these backups, and so that I don't run these backups on the wrong direction.

    But that's not all. With the mind-boggling amount of disk space I could afford, I could (actually, I will, but you get the idea) set up Amanda to backup interesting portions of my home directory to disk, and also replicate this to at least another of my local machines. Such backups can use software compression, such that they don't take as much space as live data. Also, I intend to use another form of compression: instead of backing up CVS trees (I've got loads of check outs), I'm going to back up only local changes to files, so that, in case of disaster, I can still download the original CVS tree and re-apply patches. But this is still a plan, not something I've got running.

    Finally, I've got yet another disk on a remote site, to which I rsync not only the interesting portions of my data, but also my backups. I could convince someone else to run this remote backup site for me by offering this person the speed up of RAID 0 over two disks (one of those mine). As for keeping the secrecy of the data on this remote backup site, I'd just get the backup files encrypted, no big deal.

    I can strongly recommend this solution: I got pretty much as much data safety as could be expected from a tape-based backup, without any of the hassle of having to switch tapes and moving them off-site and back on-site, and with the bonus of very fast access to local data, unlikely donw-time and fast recovery except in case of total disaster (i.e., having all of my local machines failing, in which case I'd have to either download my backups from the remote site over the net or, more likely, take a replacement machine over to the remote backup site and copy files over a fast local network connection, or from disk to disk.

    As for getting 4 IDE disks into a single machine, don't even think of using only the 2 IDE controllers that come on most motherboards these days (for RAID set-ups, you really want one IDE disk per controller). There are a few good motherboards that come with 4 IDE controllers, so that you can even have a CD-ROM and/or a CD-RW in addition to the 4 disks. If you can't find such a motherboard that suits your needs, you can always get one of those PCI cards that adds 2 IDE controllers to your machine.

    As for the problem of fitting so many disks in a standard ATX chassis, it can be done. Cooling may be a problem, but a good cooler has been good enough.

    All in all, I'm very happy with this arrangement. It was not cheap, but it was not as expensive as a tape-based solution, and it's far more flexible, way faster and it doesn't require any baby-sitting after you get it going. And I can keep far more backup history than I thought it was going to be possible.

"An open mind has but one disadvantage: it collects dirt." -- a saying at RPI

Working...