For the curious, OS X is like a birthday toy, promising hours of puzzle solving and fun. This beta is not fully optimized for performance, nor is it compatible with the full range of Macintosh peripherals. However, it does give us an indication of some of OS X’s potential. My goal was to find out how the public beta performed in terms of speed. Since there are very few Carbon or Cocoa applications currently available, I did most of my testing with OS 9 applications running in Classic.
Classic itself runs in its own window, and behaves much like PC emulation packages such as Virtual PC or SoftWindows. However, PC emulators must translate Intel instructions to PowerPC instructions, a slow and laborious process. Since OS 9 applications start out as PowerPC instructions, the applications should be almost as fast in Classic as they are in regular old OS 9. This was true for some tasks, such as encrypting a file in OS 9, and unstuffing a file with the Classic version of StuffIt Expander.
On a multiprocessing system, Classic gets to run on its own processor. However, Classic is not multithreaded. Currently, Mac OS 9 applications written to take advantage of the second processor are limited to one processor in Classic. However, Classic is separate from Mac OS X, so background apps in Mac OS X shouldn’t slow down Classic apps. Unfortunately, I wasn’t able to test this. My test machine, a Power Macintosh G4/500 dual processor, crashed repeatedly with iMovie and other Classic applications. Also, performance appeared to be slower than on a single processor Power Macintosh G4/500.
Macworld Lab’s standard testing suite, Speedmark 2.1, includes 2-D scrolling tests. I found 2-D scrolling performance was two to three times slower in Classic than in Mac OS 9. I checked scrolling in Adobe Acrobat, Microsoft Word, and Excel. Drawing windows in OS X was also noticeably slow. The problem may be that Mac OS X’s drivers for ATI graphics cards are incomplete. I also found that if the Dock is open, scrolling performance gets even worse — set the Dock to auto-hide to get the best results.
After the slow 2-D numbers, I was pleasantly surprised with the frame rates from the Quake III demo. 3-D operations are clearly being accelerated in Classic, although the frame rates are still not as fast as native OS 9.
Photoshop is an industry standard for performance testing. I wanted to install 1 GB of RAM and set Photoshop’s memory size to 900MB. However, I had problems setting Photoshop’s application size under OS X. I used the new version of Get Info, the Inspector, to change the application size. When I launched Photoshop in Classic, the setting appeared to be correct — but based on the performance numbers, the memory setting was still at its default. If I rebooted the computer into OS 9, changed the application size via Get Info, and then re-booted into OS X, Photoshop would quit before it even got to the splash screen.
Classic applications can’t communicate directly with Mac hardware — they must go through Mac OS X. This beta has support for most standard Apple hardware, and some USB and FireWire peripherals. However, OS 9 applications that are looking for specific hardware may not run. A VST FireWire drive and a USB Zip 250 worked fine. Even ones that didn’t mount were recognized on the bus — for instance, a USB Canon scanner, a QPS FireWire CD-RW, and a USB Epson printer. The only thing that wasn’t recognized at all was a Western Digital FireWire hard drive.
There were a couple of bright points. Steve Jobs was absolutely right in his Seybold keynote, despite the fact that his demo failed that day. Under OS X, PowerBooks do wake up from sleep in just a few seconds. Also. I saw good performance with the Carbonized OS X preview version of Casady & Greene’s SoundJam MP.
Macworld Lab will continue to test Mac OS X and report on the details. It’s an operating system that shows great promise, but it will only be as good as we make it.
OS X vs. OS 9Best results in bold. Reference system in italics. Speedmark 2.1 scores are relative to an iMac 350MHz which is assigned a score of 100. Photoshop results are in seconds. Cinema 4D XL and SoundJam results are in minutes:seconds. Quake 3 results are in frames per second.
| Speedmark 2.1 | Photoshop 5.5 | Cinema 4D XL 6.1 | SoundJam 2.5.1 | Quake III | ||||||
| Gaussian Blur 10 | Unsharp Mask 2.3 | Resize 50% | RGB to CMYK | Arbitrary Rotate .3 | Lighting Effects | Model Render 640×480 | MP3 Encode | Normal | ||
| Power Macintosh G4 500 w/OS X | 106 | 37.8 | 39.2 | 20.7 | 84.8 | 80.6 | 41.9 | 13:53 | 1:47 | 31.8 |
| Power Macintosh G4 500 w/ OS 9 | 159 | 23.5 | 26.8 | 5.9 | 69.1 | 39.2 | 26.6 | 13:40 | 1:41 | 43.1 |
For Speedmark and Quake, longer bars are better. For Photoshop, Cinema 4D XL, and SoundJam, shorter bars are better.
Behind Our TestsWe used Mac OS 9.0.4, a default system disk cache, and Virtual Memory disabled for all OS 9 applications tests. Displays were set to 1024 x 768 @ 24 bit color. We tested Photoshop with 1GB of RAM and a 200 MB test file. We tried to set Photoshop’s memory partition to 900 MB and History to minumum in OS X, with unclear results. Cinema 4D XL, SoundJam, and Quake 3 testing used 256 MB of RAM. 80MB of memory was allocated to Cinema 4D XL. We rendered a model at 680 x 480 with oversampling set to 4 x 4. A 9:25 track from an audio CD was used for our MP3 encoding test. It was converted using default settings of 128Kbps in SoundJam 2.1.1. Quake 3 framerates were recorded at 640 x 480 “Normal” mode.