Best Backup Server Option For University TV Station? 272
idk07002 writes 'I have been tasked with building an offsite backup server for my university's television station to back up our Final Cut Pro Server and our in-office file server (a Drobo), in case the studio spontaneously combusts. Total capacity between these two systems is ~12TB. Not at all full yet, but we would like the system to have the same capacity so that we can get maximum life out of it. It looks like it would be possible to get rack space somewhere on campus with Gigabit Ethernet and possibly fiber coming into our office. Would a Linux box with rsync work? What is the sweet spot between value and longevity? What solution would you use?'
Need more information (Score:5, Insightful)
A couple of details you'd need to fill in before people could give legitimate advice.
What's the rate of change of that 12TB. Is it mostly static or mostly dynamic. I would assume it's mostly write once read rarely video but maybe not.
Do you have a budget ? As cheap as practical or is there leeway for bells/whistles.
Is this just disaster recovery. You say if the station gets slagged you want a backup. How quickly do you want to restore. Minutes, hours, next day ?
Do you need historical dumps ? Will anybody want data as it existed last month ?
Is it just data you're dumping or some windows App complete with Windows registry junk that needs to be restored (don't know anything about Final cut pro)
If you just want to dump data and restore isn't critical, you just need to be able to do it in some time frame then sure rsync'ing to some striped 6 (or 12) TB SATA array is plenty good.
FreeBSD/Linux + Rsync (Score:4, Insightful)
That's all you need. We even use a script to create versioned backups going back six months using perl as a wrapper.
Assuming the same paths, edit to your liking. I've made the scripts available at http://www.secure-computing.net/rsync/ [secure-computing.net] if you're interested. It requires the system you're running the script for have root ssh access to the boxes it's backing up. We use password-less ssh keys for authentication.
The README file has the line I use in my crontab. I didn't write the script, but I've made a few modifications to it over the years.
Check with the university (Score:5, Insightful)
Does your university have a backup solution you can make use of? The one I work at lets researchers onto their Tivoli system for the cost of the tapes. I think I've got somewhere in the neighborhood of 100TB on the system and ended up being the driving force behind a migration from LTO-2 to LTO-4 this summer. If you are going to go and role your own and use disks, I'd recommend something with ZFS - you can make a snapshot after every backup so you can do point in time restores.
Also, I'd recommend more capacity on backup than you have now to allow versioning. I was the admin for a university film production recently (currently off at I believe Technicolor being put to IMAX) and I've lost track of the number of times I had to dig yesterday's or last week's version off of tape because someone made a mistake that was uncorrectable.
Just build a clone (Score:4, Insightful)
First of all, I love linux. Use it for my own file servers, and media machines, and routers, and pretty much everything except desktops.
That said...
For your task, I would probably just build an exact duplicate of the "real" machine and sync them nightly. Always keep in mind that if you have no way to quickly recover from a disaster, you don't actually have a backup.
That said, and if possible, I would also build the "backup" machine with more storage than the "real" machine. As someone else pointed out, you'll probably discover within a few days that your food-chain-superiors have no concept of "redundancy" vs "backup" vs "I can arbitrarily roll my files back to any second in the past 28 years". Having at least nightly snapshotting, unless your entire dataset changes rapidly, won't eat much extra disk space but will make you sleep ever so much better.
Re:Need more information (Score:4, Insightful)
>
Is it just data you're dumping or some windows App complete with Windows registry junk that needs to be restored (don't know anything about Final cut pro)
If you think Windows registry junk could possibly be involved with Apple's pro video software, you are quite right, you don't know anything about it.
Re:rdiff-backup: like rsync with versioning (Score:3, Insightful)
Do you know what -l does?
Re:Build a Backblaze Storage Pod. (Score:4, Insightful)
Remote storage at a provider like Backblace, Mozy, or Carbonite is a good tertiary level backup, just in case your site goes down, but you are limited by your Internet pipe. A full restore of terabytes of videos through a typical business Internet connection will take a long time, perhaps days. Of course, one could order a hard disk or several from the backup company, but then you are stuck waiting for the data to physically arrive.
Remote storage is one solution, but before that, you have to have local ones in place for a faster recovery should a disaster happen. The first line of defense against hard disk stuff is RAID. The second line of defense would be a decent tape drive, a tape rotation, and offsite capabilities. This way, if you lose everything on your RAID (malware or a blackhat formats the volume), you can stuff in a tape, sit on pins and needles for a couple hours, and get your stuff back, perhaps back a day or two.
For a number of machines, the best thing to have would be a backup server with a large array and D2D2T (disk to disk to tape) capabilities so you can do fast backups through the network (or perhaps through a dedicate backup fabric), then when you can, copy them to the tapes for offline storage and the tub to Iron Mountain.
Of course, virtually all distributed backup utilities support encryption. Use it. Even if it is just movies.
Re:rdiff-backup: like rsync with versioning (Score:4, Insightful)
Since we're talking about Final Cut data, it's safe to assume that it's all coming from Macs. The version of cp on Mac OS doesn't take either of those options, so it's a moot point.
Time Machine is probably the way to go. It's integrated into Mac OS, and it's ridiculously easy to set up. I don't know how it scales up, but I'd be very surprised if it couldn't handle 12TB.
Re:Just build a clone (Score:3, Insightful)
Why wouldn't you use it for your desktops?
Linux still doesn't have the "interface complexity vs implementation complexity," problem completely balanced on the desktop, just yet; although then again, to be fair, neither does anyone else. (Except maybe Apple, and that's a maybe)
Ubuntu can make a very pretty looking desktop, but updates will often hose the entire system, and in my experience, it can also crash if you give it a hard look.
On the other hand, you can use LFS, Slack, or Arch to make yourself something extremely hardware efficient and robust...but that isn't also going to please anyone who wants the eye candy.
Re:The rsync tool is called "rsnapshot" (Score:3, Insightful)
rsnapshot + mdadm raid6. Agreed 100%. That's what I'm currently using. Works like a charm for over 2 years now (and single HDD failure in meantime).
Re:Just build a clone (Score:2, Insightful)
Well, first you would need to define goals.
What do I want to backup? (only Data, or OS + Apps + Data)
Is my Data rather static or does it change a lot?
How fast does it change?
Do I have enough bandwidth to cope with the backup? (12TB is a lot! It would take more than a day to copy it over a GBit link... so, how much of the data changes over a day?)
Do i need daily backups? or even hourly?
How fast do i need to restore everything?
Do i need different versions? (Then the needed storage might be much higher than 12TB, ouch)
Who needs to restore files? (only the admins, or the users themself)
So it all boils down to: how much money do I have ;-)
Brgds
Michael
If you use ZFS, don't forget --inplace (Score:2, Insightful)
What you want is ZFS (Score:3, Insightful)
As for moving the data over there, sure use rsync and then use zfs's snapshot features so you have some rollback capability.
Why ZFS? So I'm envisioning that you're going to need a mid range machine (duel power supplies) and hanging off that you're going to have a whole pile of JBOD. You could spend the money on something that does hardware based raid, but if you're cost concious, your best route is to buy a JBOD box and fill it with 1.5TB disks. You could try to manage all of this with LVM and possibly XFS, but it would be nightmare. ZFS basically rolls RAID/LVM/FS into a single layer. Thus adding disks to your array becomes trivial. Also, I would recomment that each user/application get it's own sub filesystem on the array, that way you'll have much finer granularity for snapshots/quotas/etc.
I didn't intend this post to be an advertisement for ZFS but I have such a setup with ~14TB of disk on it right now and it works great. As for the OS on top, you could go with opensolaris, or netezza (which is just debian rolled ontop of the opensolaris kernel.
One solution total backup (Score:3, Insightful)
VMWare Snapshots
Are you backing up just data, or configurations or what? Backup Solutions are nice and all, but you're still missing something .... all the crap^H^H^H^H configurations that you've collected over the years of using that particular setup.
And once you go to VMWARE (or other VM product) you'll quickly realize that the abstraction away from specific Hardware is very nice indeed.
However, if one is REALLY concerned about backups, a duplicate Hardware setup in a seperate location sitting idle (or cold) is a necessity. And having a VMWare snapshot ready to load on backup hardware is just tits when things REALLY go south. You end up looking like a genius, and get to play Scotty (over engineered everything).
The difference between amateurs and professionals is not when things are going well, it is when the shit hits the fan. A weekend Geek can built the $8000 backupsever [backblaze.com] or whatever of storage, but once the drives start to fail (and they will) that solution starts to REALLY suck because you can't get to the freaking drives easily (and I doubt it will tell you that the drive even failed).
Let me just say it this way, if you can't afford "over engineered" equipment, you can't afford to do it right.
So, VMware, snapshots and spare hardware offsite are the way to go. Anything less these days is simply weekend geek pride.