We at Macworld are always looking to improve and streamline our product testing, and as hard drives boast bigger storage capacity and increasingly faster transfer times, we're updating the way we test these devices in our lab. We believe our new tests will provide more information to the reader, while also offering more insight into what separates these products. We’ll be breaking down our results into more specific categories—that means more tests, more data, and more ways to compare products.
Different teams, same tests
Our new testing methodology will be similar to that of our sister publication, PCWorld. For you, that means you’ll be able to compare the same type of test results for several different hard drives, whether you’re reading a Macworld or PCWorld review.
Instead of simple “copy” and “duplication” speed results, we’re now breaking up the new tests into separate “write” and “read” categories. Also, we’re moving away from calculating out results into raw minutes and seconds. Now, we’ll be calculating everything in “megabytes per second” (or MBps). With that change, larger numbers will now indicate the fastest hard drives, and you’ll have a better idea of how fast a device really is.
Better, faster technology
We’ve updated the computers that we use for our hard drive tests. Instead of the 2007 Clovertown Mac Pro dual quad core 3GHz Xeon 5400 that we used to run with Mac OS X 10.6.2 and 2GB of RAM, we’ve put together a new test platform that’s in line with the most recent Macs. Now, all of our hard drive tests run on a 2009 Mac Pro dual quad core 2.66GHz Xeon 5400 with Mac OS X 10.6.5 installed and 3GB of RAM. Of course, this meant that we had to retest a significant number of older hard drives in order to fairly compare upcoming hard drives with our previously reviewed devices—results that you can see here in our updated chart.
This also means that you’ll be seeing fewer tests results for FireWire 400, which has disappeared from all shipping Macs, including the Mac Pros that we use for hard drive tests. Also, eSATA tests will be on a case-by-case basis; since no Mac currently supports eSATA connectivity, we’ll only be recording results for that type of connection when the hard drive vendor provides an eSATA card that we can install in our Mac Pro’s card slots. (And that same rule goes for USB 3.0 compatible devices.)
How we (now) test hard drives
First, we copy a folder with 2GB worth of files to the hard drive that’s connected to our testing computer and record the length of time it takes for the drive to write the data. Then, we rename the same folder and move it back to the computer, once again timing it how long it takes for the hard drive to read the data. Once we’ve done that, we repeat the same process with a 2GB file in ZIP format in order to compare write/read speeds of a single large file versus many smaller files.
We’ve also introduced a new Photoshop test that uses the external drive as a scratch disk for processing images. Usually, Photoshop uses your computer’s main memory, but when a task is so demanding that it runs out of RAM, it switches to a scratch disk (a hard drive; in Photoshop, you can designate which attached drive can serve as the scratch disk) where it can temporarily process the data needed to handle certain tasks with huge files. We run a Photoshop Action script that includes the importing of seven large RAW files taken with a Canon EOS 5D Mark II camera, manipulating them in various ways. We limit Photoshop to using just 25 percent of available RAM and use the external hard drive as the scratch disk to test how it handles that kind of pressure.
Finally, our AJA System Test rounds out our new tests, and it’s the only part of our methodology that differs from PCWorld’s own process. Basically, the AJA System Test simulates the data transfer that occurs when you’re using their professional grade video equipment by creating files of certain sizes and resolutions. As the test runs, AJA monitors the rate and speed of data transfer, and the faster (and consistent) the results, the better a hard drive’s ability to handle professional grade video without dropping frames. With our tests, we bump up the video frame size to 1920-by-1080 pixels at 10-bit RGB, with the file size at 2GBs, and record the speed of data transfer afterwards.
What about everything else?
Don’t fear, we’ll also continue to evaluate hard drives by the other grading standards that we’ve previously used—How noisy is the drive? How much does it actually weigh? Is it too ugly for my desk at the office? What about Mac compatibility and ease of use? As much as we like to focus on the inner workings of our devices, we won’t skimp on the things you can’t put into charts, like design, ease of use, and general practicability.