Forgot your password?
typodupeerror
Linux Software

Affordable Home Backups for 10-100G Systems? 690

Posted by Cliff
from the portable-media-that-can-store-GIGs dept.
MichaelJames asks: "Ok, I have my MP3's streaming, all our digital pictures up, and a file server running on one machine in the basement. What would be the best way to do simple backups of the system and data? Get a tape drive Get a CDRW or DVDRW to backup the MP3 and pics, but use the old Zip drive for the file server data?" With drives in the 10-20 gig range only getting smaller and less expensive, what are we to do for backups, that have yet to scale well in the same range. For home systems with up to 100G of storage, what do you use to back up that much data, with a solution that's affordable to the average computer user? Have DVD writers become cheap enough for serious consideration as a backup media?
This discussion has been archived. No new comments can be posted.

Affordable Home Backups for 10-100G Systems?

Comments Filter:
  • Tough problem (Score:3, Informative)

    by phr1 (211689) on Tuesday December 11, 2001 @05:48PM (#2689352)
    DVD media is about $6 per 4.7GB disk now, but do you really want to use 20 pieces of media to back up a 100 GB disk?

    One thing some people do is back up their HD to a second HD.

    Zip disks seem practically useless these days--recordable CD is just too cheap and universal by comparison.

    Tape drives are the high-end solution, but expensive.
  • raid 1 (Score:1, Informative)

    by Anonymous Coward on Tuesday December 11, 2001 @05:49PM (#2689363)
    I'd get a cheap raid card, set up raid 1 simple mirroring and poof... some backup but not off site...
  • Tarballz! (Score:5, Informative)

    by rantenki (66616) on Tuesday December 11, 2001 @05:52PM (#2689384)
    I have a 100BaseT network, and a server computer that resides in a different room from the rest of my systems. I rotate backups using those aluminum drive caddies. A pair of 60G drives turned out to be MUCH cheaper than the equivalent size tape backup. Every day, I rotate out the drive at the end of the day, and swap with the other. The spare I keep in a fireproof safe. Just tarball the appropriate directories. Done. Poof. Much faster than the average DDS3 tape drive too. Runs at night and I don't even notice it.
  • Another hard drive (Score:3, Informative)

    by Rick the Red (307103) <Rick.The.Red@gma[ ]com ['il.' in gap]> on Tuesday December 11, 2001 @05:52PM (#2689391) Journal
    That's what I use. As you said, "drives in the 10-20 gig range [are] only getting smaller and less expensive," so buy two and use one as a backup. If your box is full, put the backup drive in an old 486.

    Actually, what I do is make the new (largest one I own) drive the backup drive, put the old backup drive into use as the primary drive, and retire the smallest one. Just make sure the new drive is as large as the others added together.

    CD-R's are OK, but why bother with the hassle? Just run a cron job to copy the files every evening/hour/whatever.

  • Onstream (Score:4, Informative)

    by wackysootroom (243310) on Tuesday December 11, 2001 @05:52PM (#2689393)
    Onstream 30 or 50 GB ADR Tape backup.

    Pros:
    Can be found for under $100
    Linux Support!

    Cons:
    Tapes are expensive
  • by tcc (140386) on Tuesday December 11, 2001 @05:53PM (#2689403) Homepage Journal
    Ultrium [ibm.com] tapes can backup 100GB Native but the price tag is way out of line for small buisness or home use (5000$+ a 100GB drive, ouch).. same goes with any dataloading systems... The only cheap tape backup I've found that was giving the best storage/price (aside from buying those used DSS 4/8 gig drives) is those 33GB Native VXA drives. [vxa.com]
  • Re:Hard Drives (Score:4, Informative)

    by Marvin_OScribbley (50553) on Tuesday December 11, 2001 @05:54PM (#2689406) Homepage Journal
    This will also work for laptops. I recently purchased an external drive enclosure with a PCMCIA connector (also available in Firewire and USB), and a separate 3.5" hard drive. The cost of the two together was less than these external drives they advertise for backing up your notebook, plus I can reuse the drive enclosure for any 3.5" hard drive.

    The drive enclosure was a bit more expensive than the rack mentioned above (under $100 with shipping) however it did come with a two sets of enclosures - one for the drive itself to use externally, and another to put in a PC cabinet if you want to hot swap the drive with it instead of using the card slot.

  • Re:Hard Drives (Score:3, Informative)

    by Stoutlimb (143245) on Tuesday December 11, 2001 @05:55PM (#2689414)
    I did exactly just that for a small office that I consult for. It backs up the entire server hard drive, and all critical information on all networked workstations, such as e-mail, etc... And for the backup software, I simply used Powerquest's DataKeeper software, that happened to come free on a driver disk for a Firewire hard drive bay I bought on a whim and never used. Compared to the tape back-up and other options, this was a no-brainer, both in the price and ease of use departments. Running the backup was reduced to a one-click wonder app, perfect for the "mature" staff in this office. ;-)

    Does anyone have a better bang/VS/buck solution? I know CDRW's are quite cheap, but one has to factor in speed, as well as human intervention. Swapping disks requires attention, which requires a wage. The backup system simply is launched every friday after work, and does it's thing on it's own time, without need for people. And with 100GB, no sanely priced tape drive comes close.
  • Here (Score:1, Informative)

    by Anonymous Coward on Tuesday December 11, 2001 @05:56PM (#2689429)
    Get norton ghost, get a hard drive to fit your size.

    Once a week, remote dump your house to a central backupserver. No only does it offer compression, its fast and very easy.
  • by igrek (127205) on Tuesday December 11, 2001 @05:57PM (#2689437)
    There are several programs allowing to use your DV camcorder for backups. For example:

    http://dvbackup.sourceforge.net
  • Cheap CD backup. (Score:1, Informative)

    by Lefty2446 (232351) on Tuesday December 11, 2001 @05:59PM (#2689447) Homepage
    Ok, I have my MP3's streaming, all our digital pictures up

    Given that MP3's and digital pictures don't change with time like databases, documents etc, why not do what we do and back the whole lot up once. Then as you download new MP3's, pictures, whatever. Put them into a directory say Not_Backed_Up, and burn that to CD also when you have enough to fill a CD. Then migrate this with the rest of the data and start fresh with the next CD to be filled.

    FWIW if you have the money to buy 2 identical drives RAID 1 might be the way to go.

  • by nl (158427) on Tuesday December 11, 2001 @06:00PM (#2689453)
    Agreed that just backing up to another HD provides the best overall method for creating a complete backup of 100MB of disk storage.

    However, I would suspect that most users don't change a huge percentage of their HD's content on a daily basis, unless you are routinely d/l'ing or ripping MP3s and MPGs on a daily basis (and I note that when I do generate that kind of traffic, it is usually because I am making a compilation CD, and while this does generate a few GB of "new" files on my HD that day, that data doesn't need to be backup up because I've got the original CDs anyway).

    As a result, it seems to me that a reasonable solution is to create a "baseline" backup, say to a CD or DVD, at system install time, when there is (relatively) little on the disk, and then each day (or week, depending on needs), do an incremental backup of changed data only to another CD.

    This approach is obviously quite inefficient if you have a complete HD failure, in that you have to recreate a new drive by starting with the first backup CD and then restore EACH ONE thereafter until the final CD restores the disk to it's last backed-up state, but for a more common problem of losing or corrupting an individual file, since that is more likely to happen with a recently modified than a remotely modified file, you are likely to be able to restore a last good version within only a few CD's of the most recent incremental backups.
  • by pete-classic (75983) <hutnick@gmail.com> on Tuesday December 11, 2001 @06:00PM (#2689454) Homepage Journal
    A lot of people have mentioned that disk to disk backup seems to be the best way to go.

    I agree.

    What hasn't been mentioned is rsync [samba.org], which makes disk to (local or remote) disk backups fast and easy.

    It is trival to set up a second disk that is a "stale" mirror of your primary disk(s) that backs up nightly, and will boot off a floppy. This captures some of the advantage of RAID (quick recovery) while being an actual backup, not just fault tolerance.

    Rsync can use ssh as a transport, so you can securely back up remote disks as well.

    -Peter
  • by morzel (62033) on Tuesday December 11, 2001 @06:03PM (#2689480)
    I've seen some commenting that creating a RAID array will suffice to secure your data, but that's really not true:

    RAID offers good protection for some things: hardware failure (ie: HD crash) and uptime. That aren't the only woes, however... You can loose data in a lot of ways:

    Disaster (fire, quake, flood, nuff said)

    Hardware failure (disk, controller, ...)

    OS failure (FS corruption, ...)

    Application failure (User space applications malbehaving, virii, ...)

    User failure (accidental deletes, experimental children - trust me on this one ;-)

    RAID will protect you from the second, but will happily add nothing in case of any of the other failures. Backing up to another media is a necessity.

    Adding an extra disk (or two, or three), and some tar/cpio cronjobs will add basic protection. (No disaster recovery for you, unless it's off-site :)
    Removable harddrives (firewire, frames, ...) are a plus, but more cumbersome.

    Tape is considered a more 'trustworthy' backup medium because the mechanism and data storage are separated (ie: tape drive / tape), while in a HD it's in one single package, and it's not as easy to replace the logic board/stepper motor if this flunks. With tape it's easier: just get a new tapedrive.

    Anyhow: don't rely on RAID to save your data - it won't.

  • by zulux (112259) on Tuesday December 11, 2001 @06:03PM (#2689487) Homepage Journal
    RAID using rsync from my other computers, (or just drag-and-drop to the Samba server from the Win2K box).

    Cygwin has RSync for Windows. Works really well.
  • by Anonymous Coward on Tuesday December 11, 2001 @06:06PM (#2689498)
    Backing up your hard drives onto other hard drives sounds like a good idea (price, amount of storage, etc.).

    If you a backing up things that would be inconvenient to loose but not mission critical then this method may be fine.

    If your are backing up irreplacible data then I would suggest using a different method.

    I have experienced the meltdown of two systems where in both cases it was the power supply that failed. Unfortunately, the when the supplies failed they took out pretty much every component in the systems (including the hard drives). I can only guess at the voltage that was pumped through everything but between the two systems I found 5 chips that were cracked or blown apart (1 of them on a hard drive controller board)!

    These incidents happened at different times, in different locations, on different types of computers.

    If you do use hard drives for backup consider removable hard drive bays and don't leave the back up hard drives in your system.

    Just my 0.02.
    --
    Can't be bothered to login.
  • Try DLT... (Score:5, Informative)

    by bani (467531) on Tuesday December 11, 2001 @06:12PM (#2689562)
    There are two ways you can go relatively cheaply, and IMHO a far better solution than CD-R or CD-RW.

    Pick up a DLT2000XT (15gb native) off ebay for about $200. Tapes are dirt cheap, about $5/ea and the media is extremely durable, nearly indestructible.

    Pick up a DLT1 (40gb native) off ebay, about $500. Tapes are moderately expensive at around $20/ea, but again the media is extremely durable.

    DLT is industrial strength backup, the drives are built like tanks and the tapes can take incredible abuse.

    Its all standard SCSI and works great with linux, no problems whatsoever.

    I considered buying hard drives for backups, but they are far too fragile for long term backup and off-site storage. Most drives arent designed to be spun up and down lots of times either.

    Last thing you need is for your backup harddisk to go splat when youre trying to power it up to restore your main system from a data loss.

    With DLT, this isnt likely to happen.
  • Re:Hard Drives (Score:2, Informative)

    by tuxlove (316502) on Tuesday December 11, 2001 @06:14PM (#2689577)
    Don't rely on MTBF scores, they're not too reliable.


    That's actually pretty funny - the reliability metric isn't reliable.

    I agree, don't trust hard drives. They're a bad solution for backups, as they don't store as well as real backup media. However, RAID systems are also not suitable because you can't archive data in case your system burns, and it doesn't protect you from filesystem corruption or inadvertent "rm -rf" commands.

    They're steep in price, but DLT or AIT are optimal. The newer models are capable of 100GB, except you can't get that much on there if the data is uncompressible (like MP3s are). If you wanted to be able to back up less than 100GB, you could probably get an older DLT/AIT for a grand or less. I've seen 8 or 10-slot AIT changers for less than $3kUS, a config which can store almost a terabyte. If you can afford that kind of cash, you probably won't have to worry about backups for a long time to come.
  • by Anonymous Coward on Tuesday December 11, 2001 @06:18PM (#2689608)
    I have had bad luck trying to backups with
    Dantz Retrospect or Veritas to a yamaha cdrw, to a sony cdrw, and to a creative cdrw. IMHO, the firmware in these drives seems to hang if there is a media problem, and dantz just acts like its waiting. Veritas blue screens.

    If you are going to try this, use cdrw's or you may end up with 35 coasters.

    On the other hand, I shelled out $140 for Dantz because it supported my Onstream drive. This works the best, but sometimes the $250 computer i dedicated to doing backups hangs, expecially when I back up the full 120 gb.
  • by Anonymous Coward on Tuesday December 11, 2001 @06:29PM (#2689692)
    You may want to look at this paper by Sanders et. al. titled Redundant arrays of ide drives available on xxx.lanl.gov under hep-ex/0112003. They report tests of raid-5 under linux for backup of physics data and find it price comparable to tape at the 10^12 byte level
  • Re:Tough problem (Score:4, Informative)

    by zerocool^ (112121) on Tuesday December 11, 2001 @06:30PM (#2689703) Homepage Journal
    FYI, if there's anyone out there in retail sales working for a place that sells HP computers, you can log on to HP Info Lab [hpinfolab.com] and you can get a $400 rebate for the HP 100i DVD writer. There's also a $50 rebate available, which you can use in conjunction with the $400 rebate. The $50 rebate is available to the general public.

    This brings the price of a DVD burner down to $150, since the drive is 600 coon skins before rebate. At that price, and if the playstation 2 drops in price this christmas, i sense lots of burned games in my future...

  • Digital Camcorder? (Score:3, Informative)

    by tjw (27390) on Tuesday December 11, 2001 @06:31PM (#2689708) Homepage
    dvbackup [sourceforge.net] is a utility that lets you store up to 13gb of data on commodity miniDV tapes. With the use of a Digital Camcorder.

    This is not exactly a dirt cheap solution, but if you have/want a digital camcorder anyway, there's only the cost of extra tapes.

    Make sure the camcorder works with dvbackup before buying one though. It doesn't work with any JVC's that I know of, or at least not mine :(.

  • Disk or Amanda (Score:1, Informative)

    by Anonymous Coward on Tuesday December 11, 2001 @06:37PM (#2689746)
    If your data is fairly static and you just need to mirror it, buy an extra hard drive and run a dump periodically.

    If your data changes quite often, and you need the ability to do "point in time" recoveries of particular pieces of data, try Amanda [amanda.org].
  • by 4mn0t1337 (446316) on Tuesday December 11, 2001 @06:38PM (#2689750)
    You can find the Onstream ADR 30GB (firewire or ide) for about 200$US.

    Tapes are 113$US (shipped) for a 4 pack.

    For those of you following along at home, that is ~300$US for 120GB of backup.
    400$US for two sets of 120GB.

    Not bad, and it is a DAMN FAST drive...

  • ZIP, PGP, FTP (Score:2, Informative)

    by Lilkeeney (131454) on Tuesday December 11, 2001 @06:39PM (#2689760) Homepage
    I have over 100 GB of hard storage, however most of that is mp3s, videos, and other things that do not change. I have burned on CD all of my mp3s and other things that don't change, but I don't want to live without if my hard drive goes down. Then every night I have a script that does an incremental backup and zips it, PGPs it, and then ftps it to my friends hard drive, both at college with a fast connection and a 100 miles apart. Also once a week I do a full backup. I figure the chance of both of are computers getting struck by lighting, bruning in a fire, or failing at the same time is pretty rare.

    Its not the best of all solutions, but I don't have anything I would die without and it was really cheap (free).
  • Re:Hard Drives (Score:3, Informative)

    by cowbutt (21077) on Tuesday December 11, 2001 @06:44PM (#2689780) Journal
    Given that a 100G hard drive is cheaper than any removable media solution, why not just buy another hard drive and install it in a removable (not hot-swappable, just removable) rack?

    Some thoughts:-

    1) IME, Racks cause heat build-up and kill HDDs. I won't use them and I recommend others don't either.

    2) If you're going to permanently mount and power the drive, then it stands as much chance of dying through power cycles and usage as your main drive.

    3) If you're going to leave the drive unpowered most of the time, then you probably won't bother backing up because it'll require a shutdown, an opened case, a boot and another shutdown.

    IMHO, If you're going to use another drive, you might as well go for RAID and have the backups done automatically and continuously.

    --

  • Re:What about fire? (Score:1, Informative)

    by Anonymous Coward on Tuesday December 11, 2001 @07:03PM (#2689875)
    You say it's unlikely, and it is. It happens, though. It happened to me. I was working in a bulding that got torched after a squatter got evicted. The entire building is gone (99 Moody St. for those of you in Waltham, MA). We saved a ton of stuff *only* because a firefighter was fortunate enough to find a machine w/ an undamaged HDD. It happens.
  • by linuxbaby (124641) on Tuesday December 11, 2001 @07:09PM (#2689920)
    3ware 6800 Escalade IDE RAID is working great, and it's only $350 or so.

    It's a PCI card with *8* IDE drive slots, which you can configure in a RAID 5 array for huge, failsafe backups.

    I've got 8 60-gig IDE drives on it, in a RAID 5 array. Gives aboout 420 gigs. Shows up as a SCSI device in OpenBSD. Works great with Linux.

    Churning away wonderfully.
    I've backed-up 200 gigs of files on it so far.

    420 gigs of RAID5 storage = $1100 USD. ($300 for the card. $800 for 8 60-gig drives.)

    Here's my post on the OpenBSD list about it [monkey.org]
  • by Ogerman (136333) on Tuesday December 11, 2001 @07:11PM (#2689934)
    Since when is a harddrive not a semipermanent media that can be easily taken off site? I'm surprised this comment got modded up so high. And since when are tapes such a reliable media compared to a hard disk? So burn-in the drive for a few days before using it for backups. And use a S.M.A.R.T. utility to diagnose the drive before each backup to reduce the chance that something is getting ready to fail.

    Your best option is to put all data on a 2-disk mirrored RAID and use another drive as a removable for an off-site or fire-safe backup. The probability of 3 hard disks failing simultaneously, one not in use, is so incredibly small it's laughable. And for that non-zero chance, if it happens, you can pay to have the spindle of one of the failed drives transferred to a new drive in a clean room.
  • NNTP backup (Score:3, Informative)

    by xant (99438) on Tuesday December 11, 2001 @07:12PM (#2689938) Homepage
    This may seem weird but a disconnected innd can act as a lovely backup server. Simply put a news posting client on every workstation to be backed up. Those workstations can use newspost or similar to post their important files to the server, along with commentary (making it easier to find the right file), timestamps and dates (provided automatically by the server when you post), etc. To back up the server, you backup one directory, where inn stores its files: /var/spool/news/news.archive

    Advantages:

    • Standardized TCP-based protocol, uses authentication.
    • Lots of free "backup software" available - e.g. pan, newspost, tin...
    • Centralized backups over the network! Only one server directory to back up.
    • Backups are automatically dated by the server, and can be signed and even encrypted if you integrate pgp into your posting script.
    Disadvantages:
    • Posts take up 35% more space than the original file did. This is adequately solved by compressing the inn directory before backing up.
    • Only solves backing up single-files cases, such as compressed archives. You cannot automatically restore a system to a particular state this way, only individual files. This is optimal for backing up your home directory, sub-optimal for system images. (See here [slashdot.org] for why this may not matter to you.)
    • If you need a secure pipe to the backup server, you either have to have a firewall in place (and you should), or you need to provide the encryption layer on your own. NNTP does not "naturally" provide encryption afaik.
  • by kalinh (167661) on Tuesday December 11, 2001 @07:55PM (#2690169) Homepage
    rsync over ssh is a great solution. You should be able to find a friend with a spare IDE channel who you can convince to be your "backup buddy". Each of you can rsync your data to a drive how ever you want it: as a bootable copy or just essential data from several machines. Trade drives, setup accounts on eachother's machines; you may be able to figure out a way to encrypt the volume if you don't trust your friends.

    Then rsync over ssh at night, use RSA "passwordless" Authentication as explained here [ibm.com] , set up a nice little script, cron it you've got reasonably accessable cheap off-site backups.

    Kalin

  • by yerricde (125198) on Tuesday December 11, 2001 @07:59PM (#2690191) Homepage Journal

    You know, the funny thing about /usr/share is it is supposed to be where you put "data" which can be shared by users, right?

    No. /usr/share and /usr/local/share are for files installed by an application and shared between the different binary architectures the application can be compiled for (such as map files for a game). To share data among users, use /home/johndoe/pub, or use groups.

  • Re:/home/dir (Score:3, Informative)

    by NullAndVoid (181397) on Tuesday December 11, 2001 @08:38PM (#2690389)
    Anyway I think there should be a standard place for stuff which will be shared between users (everything from documentation to icons to MP3s to programs which users have compiled themselves)

    I believe this was the original purpose of /usr/local: /usr was for stuff that came from the vendor, /usr/local/ for stuff that the local site added to it. At the site I learned Unix /usr/local was often a separate partition from /usr to make it easier to share/duplicate/backup.
  • by Julian Plamann (449854) on Tuesday December 11, 2001 @09:08PM (#2690607) Homepage Journal
    This may not be the best way to do it, but it works for me...

    I have a "backup" hard drive in my server. This drive is always unmounted so that there is no chance of filesystem corruption from the operating system.
    I just use a crontab to run a simple script that mounts the drive and coppies whatever specified backup files to it, then unmounts it. The same method slightly modified could be used to back up this same backup disk to another location on the network on regular intervals.
  • Iomega Peerless (Score:2, Informative)

    by webprogrammer (518832) on Tuesday December 11, 2001 @09:14PM (#2690647) Homepage
    You might want to look into an Iomega Peerless [iomega.com]. The disks are pretty small (maybe about 5"x3"x.5") and the disks are 20 or 40 gb a piece. I'm running one a Windows 98 machine, I couldn't tell you about Linux compatability. It connects to a USB hub and has sustained data transfer of 12 mg a second, I think.
  • rdiff-backup (Score:1, Informative)

    by Anonymous Coward on Tuesday December 11, 2001 @09:29PM (#2690736)
    rdiff-backup and RAID it, dude...

    http://www.stanford.edu/~bescoto/rdiff-backup/

    From the web page:

    What is it?
    rdiff-backup is a script that backs up one directory to another. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves subdirectories, symlinks, special files, permissions, uid/gid ownership (if it is running as root), and modification times. Finally, rdiff-backup can operate in a bandwidth efficient manner over a pipe, like rsync. Thus you can use rdiff-backup and ssh to securely back a hard drive up to a remote location, and only the differences will be transmitted.
  • RAID5 (Score:1, Informative)

    by Jakyll (94797) on Tuesday December 11, 2001 @09:38PM (#2690797) Homepage
    Promise IDE RAID controllers [promise.com] saved my live more than once. Three 100GB drives in a RAID 5 config still leaves with you just under 200GB and a safety net.
  • by softwave (145750) <david.coppens@ad ... .be minus distro> on Tuesday December 11, 2001 @09:52PM (#2690869)
    It occured to me nobody mentioned online solutions such as Streamload [streamload.com] or MyPlay [myplay.com] (great for mp3 storage)..

    Too bad iDrive & Freespace.com went offline :(
  • by pete-classic (75983) <hutnick@gmail.com> on Tuesday December 11, 2001 @09:53PM (#2690879) Homepage Journal
    you may be able to figure out a way to encrypt the volume if you don't trust your friends.

    I rarely use the word impossible, but I think this is. He, presumably, is root. The system has to write to the disk . . . I don't see how to overcome this.

    OTOH, you could encrypt the files prior to transmission. This creates the new problems that 1. the efficiency of rsync is lost unless you do some kind of "chunky" encryption and 2. there is no obvious way to do this.

    I guess you could use dump [sourceforge.net] to do a full backup periodically, then encrypt and upload that, then do incremental dumps nightly and encrypt and upload that. Not pretty.

    -Peter
  • by spmkk (528421) on Tuesday December 11, 2001 @11:33PM (#2691384)
    I have been trying to implement an incremental backup solution (as outlined in a couple of posts in this thread) with the initial 'full' backup on DVD's and the 'increments' on CD's. I'm working with a small Windows network (slight pause for the boos and hisses from the Linux crowd).

    I've got all the hardware, but I can't seem to find a piece of software that's built for incremental backup to non-tape media. The closest I've found is NTI BackupNOW [ntibackupnow.com], but after much frustration I discovered that even their software won't support DVD-R(W) for some months to come.

    Has anyone actually **SUCCEEDED** in setting up such a backup system? Bonus question - has anyone had to restore data from this kind of setup?

  • Re:Hey old-timer... (Score:3, Informative)

    by glwtta (532858) on Tuesday December 11, 2001 @11:50PM (#2691476) Homepage

    Of course you'll still need something like 150 CDRs to get everything, inceremental or not - and that's just the first dump that will keep growing.

    BTW, to the other replies to this - one of the companies I worked for, we did full, weekly backups of all the important systems (ie the actual OS and software - not the data) like the domain controller, database servers, fileservers, firewall, the backup servers themselves, etc. and then daily incrementals on them.

    For the actual data on the file and database server, plus everyone's personal workstations (something like 100 people) it was a full backup once a month (maybe less) and nightly incrementals, which worked great because here people usually wanted just a couple of files their dumb asses deleted a week ago (usually perfectly timed to want the tape that just went off to off site storage :) ) All in all that achieved very good granularity. Of course the incremental tapes were recycled after about two months or so.

    The database servers were a bit more tricky so additional full dumps were done manually, when, let's say, 500,000 new compounds were imported. There were also databases attached to data collection robots that had their own separate system going.

    All in all, this took three backup servers (one for the data collection databases, one for individuals PCs and one for the other stuff) with their respective tape loaders, plus individual tape drives for the main database servers (we also had things like SGI workstations, but the scintists on those did their own thing), and the full backups did take up almost the entire weekend. But it was setup well enough that it took only a few minutes a week to maintain the whole thing - get a new tape, write on it what the server tells you to, put it in the loader, take the specified one out, send it off - you are done. And it did work - they had to recover after a fire once (before my time) and did it without any problems.

    Anyway, the point of the story was this - it's not as simple as "do incrementals" or "no! do full backups" when you are talking about actual companies, it's a bit more tricky than that.

  • Ecrix VXA-1 (Score:3, Informative)

    by Wdomburg (141264) on Wednesday December 12, 2001 @12:43AM (#2691728)
    This has been my choice for low-cost backup solutions for a couple of years now.

    The drives support three different flavours of media - 12GB, 20GB and 33GB, and come with IDE, SCSI, or Firewire interfaces. The IDE is cheapest, at $699, with the media costs being $80, $45, or $35 depending on the capacity.

    Is it the absoulte top of the line as far as tapes go? No. But the cost can't be beat. And you get a reasonably fast (3MB/sec) drive with very nice reliability (take a look at the independent testing on their site; e.g. soaking a tape in hot coffee for a minute, rinsing it, drying it and reading the data off.)

    They also recently merged with Exabyte, who will be positioning it as their new value solution. Hopefully the Exabyte name will expand the market enough to drive the prices down on these even further.

    Matt
  • by cgleba (521624) on Wednesday December 12, 2001 @12:47AM (#2691742)
    This is what I do (man levels of redundancy):

    1) Use resierfs -- it'd stable now and recovers better then ext2 in small "incidents"

    2) RAID 5 array

    3) Run a nightly script that hard links all the files into a . (hidden) directory -- protects against rm -rf''s

    4) Run mirror in every directory from cron-- if you lose or mangle one file you cna recontruct it from the contents of the mirror and the other files (works a lot like XOR in raid 5 arrays -- aka a RAID for files).

    5) DDS3 incremental back-up; complete backup at regular intervals -- one set of tapes stored off site.

    This protects you from multiple levels of failures -- with the catastrophic redundancy being the tapes. You don't always want to rely 100% on the tapes for all your redundancy.
  • DLT (Score:2, Informative)

    by Bamfsog (535812) on Wednesday December 12, 2001 @01:03AM (#2691805) Homepage
    You can get a DLT drive that will do 20-40G per tape for under $200 on Ebay. Add an external case if it doesn't have one, and grab a SCSI card. Cheap, reliable, tested.
  • by supersnail (106701) on Wednesday December 12, 2001 @07:16AM (#2692580)

    Don't listen to him. Tapes always let you down!

    Some of the problems with tape are:---

    Granularity -- i.e. you accidentally deleted "xmascard.lst" how do you know which tape it was on and how long will it take to scan several tapes to find your 2K file?

    How can you be sure the tape is readable. In my experience something like 5% of backup tapes cannot be read by any drive except the one that wrote the tape.

    No standard format, no backward compatabilty, short market life for thevarioustecnoligies. You may be very smug that you have an "acme" format backup stored in a safe deposit box, but after your house is flooded (mentioning the f*r* word upsets some people) you discover "acme" drives are no longer sold anymore, you just have to hope someone is trying to unload a working drive on e-bay.

    Expense -- the drives, the tapes etc. will all cost several times as much as a couple of external hard drives.

186,000 Miles per Second. It's not just a good idea. IT'S THE LAW.

Working...