First Look: Introducing Speedmark 5

Any time we test a new Mac desktop or laptop— last week’s MacBook tests are a perfect example—you’ll hear plenty of talk about a particular system’s Speedmark score. All well and good—unless you’re not sure what Speedmark is exactly.

Allow me to explain, then: Speedmark is Macworld Labs’ primary performance testing tool for all Mac systems. Though our main focus is testing and reviewing the latest Mac systems, we also like to use Speedmark on older Macs. That allows us to give readers a sense of how the computer they’re currently using—a Power Mac G5, say—stacks up against the latest crop of Apple hardware.

That goal requires us to ensure that the real world tests making up Speedmark need to be able to run on a vast array of systems. And that leads to some tough choices, balancing the desire to use current applications with the need to find programs that can run on many kinds of Macs so we can provide historical comparisons.

The release of a new operating system means we have to update Speedmark. And, prior to Leopard’s late October debut, we asked for your help in finding applications that could be a part of an updated testing suite. You responded, and we took that feedback into account when creating Speedmark 5.

So here’s the list of tests that make up the Speedmark suite and a couple of words about the reasons we’ve included it.

Mac OS X—Startup

Nobody likes to wait for their Macs to start up.

Mac OS X—Duplicate 1GB file

Our readers suggested we double the size of the files we work with so we did. This task tests both the read and write speeds of a drive.

Mac OS X—Compress/Decompress 2GB folder

Again, we switched to a folder that’s twice the size of what we tested in Speedmark 4.5. These tests involve the processor as well as the hard drive.

We did remove one Finder test, the Open Multiple Folders test was getting too fast on many systems. Unfortunately, doubling the amount of folders made the test results become erratic. So we removed it from the mix.

Microsoft Office 2004—Word Scroll/Entourage Download E-mail.

Like it or not, a lot of us depend on Office. These tests also require Rosetta to run on an Intel Mac. We felt was important to include at least a couple of non-Universal Binary applications in the suite.

Pages ’08—Search/Replace.

Readers asked for an iWork test to be included in Speedmark. In the previous version of our suite, we ran this test in Microsoft Word. Since we can duplicate it in Pages, we moved the test over to that application.

iTunes—Convert AAC files to MP3 from Hard Drive.

This test remains the same, though we’re using the latest version of iTunes (iTunes 7.5, as of this writing).

iMovie HD—Export to QuickTime for E-mail, Apply Aged video effect.

This was tricky. We thought of using iMovie ’08, but its minimum system requirements would have meant excluding a large number of Macs from being able to run all of the application tests in Speedmark. As Apple still offers iMovie HD for download, we decided to stick with it for now.

iPhoto—Import 100 photos from hard drive.

This test remains unchanged from the previous version of Speedmark, though we upgraded to iPhoto ’08.

Camino—Multiple Page Loading Test.

We keep trying to use Firefox or Safari for this test, but Camino remains the most stable browser for our purposes, and its results are the most consistent.

Unreal Tournament 2004—Antalus Botmatch

Yes, this game is a bit outdated, but much like iMovie HD, it runs on a wide range of Macs. Our readers suggested Quake 4, among other games, but newer games have fairly high minimum system requirements which would have prevented many Macs from being able to run Speedmark.

As a result, we’ve decided to stick with UT 2004 as the game we run on all Mac systems, but if a Mac meets the minimum requirements of Quake, we’ll include those results in their benchmark charts.

Incidentally, the Antalus Botmatch test is run at 1,024-by-768 Max Settings with sound and graphics enabled.

Cinema 4DXL 10.5—Scene Render

This test renders a 3-D scene and is one of the best multiprocessing tests we have. The test remains the same as in previous versions of Speedmark, though we’ve updated to the latest version of Cinema4D, 10.5

Compressor 3—Convert DV file to MPEG2 for DVD

This tests a system’s multiprocessing ability as well as hard drive performance.

Adobe Photoshop CS3—Actions Script with 14 Actions.

Many Mac users use Photoshop, and this test helps give a feeling for how the application runs across different Macs.

Also, since the last version of Speedmark was released before Photoshop ran natively on Intel-based Macs, we couldn’t include Photoshop results in the Speedmark score calculation. Now that Photoshop is a Universal Binary that runs on Intel systems as well as PowerPC Macs, we’ll factor the test results into the Speedmark score calculation.

HandBrake 0.9.1—Encode one chapter from DVD to H.264

Without a doubt, this was the most frequently suggested test from our readers, so we’ve included it.

I’ve included a chart with Macs tested with Speedmark 5.

 

Speedmark 5 Results

Speedmark 5 Adobe Photoshop CS3 Cinema 4D XL 10.5 Compressor 3 iMovie HD iTunes 7.5 Unreal Tournament 2004 Quake 4 Finder Handbrake
  OVERALL SCORE SUITE RENDER MPEG2 Encode Aged Effect MP3 ENCODE FRAME RATE FRAME RATE ZIP ARCHIVE H.264 ENCODE
MacBook Core 2 Duo/2.2GHz (black) 186 1:17 1:00 2:05 0:51 1:12 25.4 7.8 5:13 3:14
MacBook Core 2 Duo/2.2GHz (white) 185 1:17 1:01 2:11 0:53 1:11 23.3 7.7 5:09 3:14
MacBook Core 2 Duo/2GHz (white) 172 1:26 1:06 2:22 0:57 1:16 24.1 7.7 5:42 3:15
MacBook Core 2 Duo/2.16GHz (white) 179 1:16 1:04 2:17 0:53 1:13 18.5 4.5 5:11 3:18
MacBook Core 2 Duo/2GHz (white, May 2007) 165 1:31 1:11 2:37 0:59 1:16 18.5 4.5 5:51 3:31
15-inch MacBook Pro Core 2 Duo/2.2GHz 185 1:24 1:00 2:16 0:55 1:09 78 43.1 5:37 3:14
15-inch MacBook Pro Core 2 Duo/2.4GHz 199 1:17 0:55 2:08 0:50 1:06 69.9 39.1 5:01 3:02
17-inch MacBook Pro Core 2 Duo/2.4GHz 204 1:20 0:55 2:07 0:50 1:03 76.6 52.5 5:03 3:01
20-inch iMac Core 2 Duo/2GHz 210 1:03 1:05 2:17 0:56 1:09 73.7 21.3 5:13 3:21
20-inch iMac Core 2 Duo/2.4 GHz 245 0:54 0:54 1:57 0:47 0:59 88.7 31.3 4:24 2:56
24-inch iMac Core 2 Duo/2.4 GHz 245 0:55 0:54 1:57 0:47 0:59 88.6 31.2 4:25 2:56
24-inch iMac Core 2 Duo/2.8 GHz 262 0:47 0:47 1:42 0:41 0:51 93 31.1 3:46 2:39
Mac Pro Xeon/2.66 GHz Quad Core 260 0:51 0:28 1:19 0:39 0:52 93.5 51.4 4:16 1:55
Mac Pro Xeon/3 GHZ Quad Core 274 0:47 0:25 1:16 0:35 0:48 95.3 51.5 3:56 1:38
Mac mini Core 2 Duo/2 GHz 172 1:19 1:07 2:21 0:56 1:09 23.4 5.6 5:29 3:22
Mac mini Core 2 Duo/1.83GHz 159 1:24 1:14 2:37 1:02 1:25 23.6 5.6 5:57 3:34
Mac mini Core Solo/1.5GHz 100 2:16 3:00 8:03 2:09 2:50 19.2 n/a* 7:47 9:28
15-inch PowerBook G4/ 1.67Ghz 92 3:00 3:53 7:52 1:58 2:26 22.3 19.9 7:12 16:58
Power Macintosh G5/Quad 2.5GHz 225 1:12 0:32 1:28 0:37 0:48 32.6 37.7 4:47 2:25
Power Macintosh G5/Dual 2.7GHz 204 1:19 0:53 2:12 0:47 0:55 49.8 16.7 4:43 4:36
Power Macintosh G5/Dual 2.5GHz 193 1:22 0:57 2:18 0:47 0:58 47.3 18.1 4:45 4:57
Power Macintosh G5/Dual 2.0GHz 166 1:49 1:11 2:34 0:56 1:10 38.8 17.2 5:44 6:04
Power Macintosh G5/1.8GHz-single 112 2:25 2:33 5:40 1:31 1:58 19.3 5.8 7:01 13:54
  >Better >Better >Better >Better

Best results in bold. Recently added systems are in italics. Asterisk (*) Denotes a system that doesn’t meet the application’s minimum requirements.

Speedmark 5 scores are relative to those of a 1.5GHz Core Solo Mac mini, which is assigned a score of 100. Adobe Photoshop, Cinema 4D XL, iMovie, iTunes, and Finder scores are in minutes:seconds. All systems were running Mac OS X 10.5 with 2GB of RAM. The Photoshop Suite test is a set of 14 scripted tasks using a 50MB file. Photoshop’s memory was set to 70 percent and History was set to Minimum. We recorded how long it took to render a scene in Cinema 4D XL. We used Compressor to encode a 6minute:26second DV file using the DVD: Fastest Encode 120 minutes - 4:3 setting. In iMovie, we applied the Aged Film effect from the Video FX. menu to a one minute movie. We converted 45 minutes of AAC audio files to MP3 using iTunes’ High Quality setting. We used Unreal Tournament 2004’s Antalus Botmatch average-frames-per-second score; we tested at a resolution of 1,024-by-768 pixels at the Maximum setting with both audio and graphics enabled. We ran Quake 4 timedemo at a resolution of 1,024 by 768 pixels at High Quality settings. We created a Zip archive in the Finder from a 2GB folder.—MACWORLD LAB TESTING BY JAMES GALBRAITH, JERRY JUNG, AND BRIAN CHEN

 

Thanks to our readers for the test suggestions. Please, keep them coming !

[ James Galbraith is Macworld Lab director. ]

Comments