What to think about when you need to back up vast amounts of data.
By Joe Kissell
MacworldSEP 16, 2015 8:12 pm PDT
Backing up a Mac with a terabyte or two of internal storage poses no particular challenge. (For Macworld’s recommended backup strategies, see
Bulletproof backups, and
How to restore data from the cloud.) Creative professionals, however, tend to generate massive quantities of new photo, video, or audio files on a daily basis, resulting in a multiple terabytes of data that may require a more elaborate backup approach.
My first recommendation for creative pros is to think of your backups as two separate operations: one for your large media files (which may be on external storage anyway), and one for everything else. You can tell Time Machine to exclude a folder or volume that stores your media files by opening System Preferences > Time Machine, clicking Options, dragging the item to the list, and clicking Save; most other backup software offers a similar capability. By doing this, you’ll keep your conventional backups relatively small and fast, and make it easier to give your media files special treatment.
Time Machine offers no way to back up a subset of your data to one destination and a different subset to another destination. So, for your media files, I recommend using a different backup app. There are
many fine choices, but if I had to pick just one, I’d suggest CrashPlan because it can back up to both local and cloud storage, as well as to other computers you own. And it lets you define multiple, independent backup sets, each with its own source and destination.
Run the numbers
Your next step is to figure out just how much data you’ll need to back up, so that you can select an appropriate storage device. Again, I suggest making this a different device from the one that holds your day-to-day versioned backups (such as those created by Time Machine) or bootable clones (such as those made by Carbon Copy Cloner), because that will make the process cleaner and give you more flexibility if you later need to add storage.
Start by noting how much data your media folder(s) or volume(s) currently occupy (which you can find by selecting the folder or volume in the Finder, choosing File > Get Info, and looking at the Used line).
Now estimate how much new data you generate on a typical day. One way to do this is to create a smart folder.
In the Finder, choose File > New Smart Folder. Make sure This Mac is selected.
Click the + (plus) button on the right, next to the Save button. From the first pop-up menu, choose Created Date; from the second, choose Yesterday (assuming yesterday was a typical day).
Click + (plus) again. Then choose Kind from the first pop-up menu in the new row and Other from the second; enter -folder into the field (meaning “show me just files, not folders”).
Press Command-A to select everything added yesterday, and then Command-Option-I to display the total size of that data.
Click Save to save your smart folder so you can check it again later.
If you want to back up every file you have now plus all those you add over, say, the next year, take your daily average, multiply it by the number of days you normally work per year, and add the amount of space your existing media files occupy. Multiply the result by 1.5 (or more!) to give yourself room for additional file versions and breathing space. If that number is bewilderingly large, you can reduce the backup space required by allowing your backup software to delete older versions of your files.
For example, if you work primarily in video, you’ll have your original footage, plus project files from your editing software and perhaps dozens or hundreds of intermediate versions of each video before the final cut. You should always back up the original footage—it’s easier to redo an edit than to reshoot. But you might want to remove older versions of project files from your backups in order to reduce your storage needs. (For example, in CrashPlan, go to Settings > Backup and click the Configure button next to “Frequency and versions” to specify how long CrashPlan should save old versions and deleted files.) Similarly, original photo files and raw audio tracks should be backed up, but you need not keep backups of every single edit to a photo or audio project; in most cases, just the few most recent ones will suffice.
Even so, you may need considerable storage space. Here are your two major options, which are the same as what you’re likely using for primary storage:
It’s fairly easy to find high-capacity hard drives (8TB drive mechanisms are easy to find, and 10TB drives exist but are uncommon). If your backups can fit on a single drive, that may be all you need, although a secondary backup is always a good idea.
RAIDs and other multi-drive products, such as
Drobo devices, let you combine two or more drives for greater capacity, performance, and/or data redundancy.
Both individual drives and multi-drive devices may be packaged with local interfaces (such as Thunderbolt, Thunderbolt 2, or USB 3) or network interfaces (Ethernet or Wi-Fi); a device in the latter category is called a NAS (for network-attached storage). A NAS makes it easier to share storage among multiple computers, whereas local interfaces (especially Thunderbolt) offer much faster performance.
I should mention that a RAID is not, by itself, the same thing as a backup—not even if you use RAID 1, which mirrors the same data onto two drives (or RAID 5, which requires three or more drives). A RAID can protect you against hardware failure but not accidental deletion, software bugs, theft, and other problems. So if you use a RAID as primary storage, you should almost certainly back it up to another RAID.
What about the cloud?
Perhaps you’re thinking that cloud backups—using, say, Amazon Cloud Drive or CrashPlan, which offer unlimited storage for a fixed annual fee—could be a less-expensive alternative. Maybe, but it depends on how much data you need to back up per day and whether your broadband connection can handle it. People who have or can obtain gigabit-or-faster fiber broadband are generally in good shape; most cable and DSL customers may find their service too limited to back up terabytes per month.
You can test your upload speed using a service such as
Speedtest. Then, to get a very rough estimate of how much data you can upload per day, take your upload speed in megabits per second (which is usually much lower than your download speed) and multiply it by 10.8. That will give you the approximate number of gigabytes you can upload in a day. (For example, if your upload speed is 15 Mbps, you can theoretically upload 15 x 10.8 = 162GB per day, although real-world results will almost invariably be worse.) If that’s considerably more than the amount of new data you’re creating each day—and if you aren’t constrained by a monthly data cap—then by all means, consider cloud backups. (But I still recommend at least one local backup, because it’s far faster to restore data locally than it is to download it from the cloud.)
Backing up large quantities of data is always going to require more money, time, and inconvenience than you’d prefer. Restoring that data may be more challenging too, in that raw photos, videos, and audio files don’t always have descriptive names, yet you may have to rely on filenames when retrieving data from your backups. And backing up while on location may mean lugging around a large, clunky device. But all of this hassle pales in comparison to the agony of losing the only copy of an irreplaceable media file that a customer or client is depending on.