Choosing a virtualization application

More Stories in this Series

Choosing a virtualization application

So now that you’ve read our reviews of Fusion 2.0 ( ), Parallels Desktop 4 ( ), and VirtualBox 2 ( ), you may be wondering just which program is the best virtualization solution for your Mac. The answer, unfortunately, isn’t quite as simple as simply picking one of the programs and saying, “There you go. That’s the best.” Each has its strengths and weaknesses, and as such, each program works better in some situations than in others.

To help you with the task of picking between these three programs, I’ve created this guide that makes recommendations based on your virtualization needs. Just skim the document until you find a category that closely matches what you intend to do, and then read about which program handles those tasks best.

The good news is that you can download and try all three of these programs free of charge, so even if you don’t agree with my conclusions in a particular category, you can try out the alternatives yourself before making your buying decision.

Performance

Before I get into the recommendations, though, I wanted to spend a bit of time talking about performance, specifically the performance of the virtualization apps with Windows. Generally speaking, all three of the virtualization solutions handle basic duties just fine in Windows XP Pro and Vista.

We’ll have some official Macworld benchmark results soon, but I wanted to do something on my own to get a sense for how each system performed.

After some hunting, I decided to use PassMark’s PerformanceTest 6.1, which tests CPU, 2D and 3D graphics, memory, disk, and CD-ROM performance for Windows XP Pro and Vista operating systems.

In addition to these standardized tests, I also tested playback of Windows high defintion video, using both a 720p version of the Coral Reef Adventure on Microsoft’s WMV HD Content Showcase page, and some random 720p demo video I found on the net. I also ran a number of hand-timed operational benchmarks, looking at the time required to start up, shut down, sleep, and resume both XP Pro and Vista.

image
PassMark's PerformanceTest was used to get a sense of overall virtual machine performance.

I tested on both a 2.66GHz Mac Pro quad core with 4GB of RAM and a 2.5GHz 15-inch MacBook Pro with 2GB of RAM. On each machine, I installed (as new virtual machines) versions of XP Pro and Vista Business (as well as Ubuntu Linux, though I didn’t run any Linux benchmarks) in Fusion, Parallels, and VirtualBox. If you’re counting at home, that’s two versions of Windows on two machines in three different virtualization applications, or a total of 12 Windows installations (on which I ran multiple benchmark tests, multiple times). So yes, I’m tired of installing and updating Windows!

To the extent I could, I made the OS X side of the test setup as clean as possible. On the Mac Pro, I used a separate partition with a totally stock OS X 10.5.5 installation. On the MacBook Pro, this wasn’t possible, so I went ahead and tested in my user account, with my usual suite of login items but without any other applications running. Because I ran these tests myself, and my testing setup is far from ideal, I’m not going to report on the exact benchmark results. Instead, I’ll share some general insights revealed by the testing results, and borne out by my experience with each system.

  • Generally speaking, I found that on both of my Macs in both versions of Windows, Parallels had the best CPU benchmarks; Fusion had the fastest 2D graphics in XP Pro while VirtualBox did best at 2D in Vista; Fusion had the best and fastest 3D support; and both memory and disk results were close in all three applications.
  • If you intend on running Windows, if at all possible, run Windows XP Pro. Although XP Pro is officially dead, it’s still available from many sources, such as these versions at NewEgg. When comparing XP Pro to Vista on the same machine, the overall benchmark results for Vista were anywhere from 6 to 20 percent worse than they were for XP Pro.

    You can feel this difference when using Vista over XP Pro—it’s never downright slow, but things just seem to happen more slowly than they do in XP Pro. Vista also wants more RAM than XP Pro, and is happier with a larger hard drive. Start up and shut down take longer in Vista, as do suspend and resume, and more idle CPU power is required even when doing nothing. So unless you absolutely positively need to use Vista, run XP Pro instead—regardless of which virtualization application you select.

  • Suspending and resuming virtual machines is generally quite speedy—most of the time, it took under 10 seconds either way, though I’d occasionally see much longer times for no apparent reason. Fusion and Parallels were speediest, with VirtualBox lagging. VirtualBox was also the only app that gave me troubles with suspend and resume—on the Mac Pro, I couldn’t get Vista to suspend or resume, even though it worked just fine in VirtualBox on the MacBook Pro.

  • Hardware matters, to some degree. In many of the CPU benchmarks, my MacBook Pro actually outscored my MacPro, most likely due to the much larger cache on the MacBook Pro. In real world use, though, the Mac Pro generally worked better than the MacBook Pro, especially if I wanted to work with OS X programs alongside the virtualization programs. My Mac Pro has twice the RAM, and with four CPUs, can better split workload between a virtualized OS and other OS X programs. That’s not to say things ran poorly on the MacBook Pro; they ran just fine. Though I no longer have a Mac mini in the house, I’ve used prior versions of both Parallels and Fusion on a mini, and they run fine there as well. It’s common sense, though, that the more horsepower you have in your Mac, the better these virtualization apps will perform.

  • Playing back high-definition Windows media files in Vista can be a troublesome experience. With seemingly no rhyme or reason, I found that different clips played with different levels of success on my various Vista installations. The Coral Reef Adventure clip was the most troublesome across all my test setups. In Parallels (on either Mac), for instance, it simply displays a black screen (with audio) when viewed in windowed mode. Shifting to full screen made the video appear, though the movie stuttered a bit.

    VirtualBox, on the other hand, was able to play the video in windowed mode, though it was quite jerky. When expanded to full-screen mode, the audio played fine, but the video frame rate dropped to a frame every couple of seconds—it was basically like watching a slideshow with audio. Fusion did the best job with this video, though it was still a bit jerky when viewed in full-screen mode.

    According to a contact at one of the virtualization companies, the problem is that the Vista media player disables hardware acceleration for full-screen playback if the video driver doesn’t include some Vista-specific digital rights management (DRM) features. Apparently none of the virtualization apps include those features, which would explain the playback issues.

    One alternative is to install the VLC media player, which has no such restrictions—hardware acceleration is enabled in full screen mode when using VLC. VLC, however, couldn’t play the audio to the Coral Reef Adventure clip, though it played the video reasonably well.

Overall, I was impressed with how well all three of these virtualization applications handled Windows—even Vista, with the exception of HD video playback. There were times while testing that I had to remind myself that I wasn’t using a native version of Windows. Unless you have some very specific needs (3D gaming, for one), there’s really no reason at all to use Boot Camp any more—any of these programs can probably do what you require in that “other” operating system without having to reboot your Mac.

Recommendations

So on to those recommendations…which virtualization application is right for you? To help you decide which program fits your needs, read the following statements and pick the one (or ones) that best matches your needs, then see which program I recommend for that task, based on my experiences. And remember, you can test all of these apps for free yourself, so don’t feel like you’re stuck with my recommendation.

I need to run Microsoft’s Office in Windows XP Pro.

All three of the virtualization apps will handle basic Office (any version) needs just fine. If you’re not working with gargantuan, complex Excel models and multi-hundred-megabyte PowerPoint presentations, VirtualBox may be all you need—it handled my basic Office tests without any problems, and never felt slow or overburdened by the workload. As models get more complex, you may want to switch to Parallels or Fusion, as their support for multiple CPUs may provide a speed bump. But for most users, VirtualBox will do just fine as a solution to running the Windows version of Office on their Mac.

One thing that matters in a program like Office is 2D graphics speed—you’ll appreciate speed as you scroll long pages of text, draw and move simple boxes, and other such tasks. To demonstrate how the various programs handle 2D graphics, I recorded a few movies of the 2D portion of the PerformanceTest benchmark suite on both the Mac Pro and the MacBook Pro. To make this a worst-case scenario, I chose to record them in Vista, which scored anywhere from 5% to 50% lower in each 2D test than did XP Pro. Click each image below to watch the movies—they run just over a minute or so each.

As you can see in the movie, the quickest program on the Mac Pro was not the quickest program on the MacBook Pro, which I found somewhat surprising. VirtualBox was unique in that it took the same amount of time on the Mac Pro as it did on the MacBook Pro; the other two were notably faster on the Mac Pro. In real-world testing, I found scrolling speed to be fine in all three programs in either version of Windows—Fusion and VirtualBox work best in XP Pro and Vista, respectively, but all three apps have more than usable 2D graphics speed for Office use.

I want to play 3D games in Windows.

By far, the best Windows gaming experience will be had by installing Apple’s Boot Camp, and rebooting into fully-native Windows to play your games. However, that’s really not the ideal solution. For the best no-reboot-required Windows gaming experience on the Mac, install VMware’s Fusion 2, and run Windows XP Pro.

With full DirectX 9.0c support, you can play many older games, and even a fair number of newer games such as Half Life 2 and Portal, with stellar results on your Mac within OS X. While Parallels also supports DirectX 9.0c, they don’t seem to have the same level of implementation as does Fusion. I saw different visuals in the same game when testing in each program. To demonstrate the differences, I created a brief movie using one of the 3D PerformanceTest tests—in this case, a scene of some aircraft flying over a mix of ocean and land.

As you watch the movie, compare the quality of the sky and the reflections and motion of the water in the Fusion clip to that of the Parallels clip, and it’s clear that Fusion has a nicer DirectX implementation. Frankly, watching a 3D game play at great frame rates within a Windows virtual machine window inside of OS X seems like magic, but it really works—I had a blast testing out some older titles like Combat Flight Simulator 3, Tony Hawk’s Pro Skater 4, and Kelly Slater’s Pro Surfer, as well as some newer titles like the FIFA ‘09 demo.

As an ultimate test, I thought I’d see how Far Cry ran, so I installed the Far Cry demo and recorded just a bit of the experience—click the image at right to see how well it ran. That video was recorded in Fusion at 800x600—the colors are washed out due to the compression of the clip; this 183KB image shows the real appearance.

While the video may look a little jumpy, consider that I was playing while recording the demo movie (at 30fps) using SnapzPro. The in-game frames-per-second indicator varied from a low of 24 to a high of 46, which isn’t bad at all. If I were to run Far Cry in Boot Camp, I’d see nicer visuals and much higher frame rates…but given it was running within a virtualized operating system, I think this is a pretty impressive demonstration of what’s possible.

I also tested my USB gamepad, joystick and steering wheel/pedal set, and all worked as expected, other than the force feedback feature on the steering wheel. In short, while gaming in Fusion probably won’t ever match Windows-native gaming performance via Boot Camp, the experience has gotten much better in the last couple of years, and you can now play many games with very usable frame rates without rebooting.

I want to use an OpenGL-based program in Windows.

Only Parallels offers OpenGL acceleration, so that’s your best bet for such programs. You may be able to run them in VirtualBox or Fusion, but the speed of the graphics will leave much to be desired. Until one of the other two programs steps up to the plate, Parallels is the only player in this niche.

I have a CPU-intensive Windows application that needs every bit of CPU power.

image
As of today, Parallels is probably your best solution, as it can support up to eight virtual CPUs on an eight-core Mac Pro—Fusion 2.0 can only support four cores on the same machine. In addition, Parallels’ CPU benchmark results in Windows were higher than those for VirtualBox or Fusion on both of my Macs, even with the same number of CPUs. To see the benefit of the added CPUs, I benchmarked a four-CPU Windows XP Pro Parallels virtual machine against a two-CPU Windows XP Pro Fusion virtual machine on my Mac Pro using the CPU portion of the PerformanceTest benchmark. The final score showed the Parallels four-CPU machine outpacing the two-CPU Fusion machine by 73%.

One thing to be aware of is that if you assign all the cores on your Mac to the virtual machine, the performance of your OS X programs will suffer—with all four cores assigned to my virtual Parallels Windows machine on my Mac Pro, I found mouse and window movements to be very choppy, and had trouble switching from one program to the next, as the CPUs were all busy with Windows. But if you’ve got numbers to crunch, throwing as much CPU power as you can at the problem makes a lot of sense.

I am a geeky hobbyist—I want to experiment with a bunch of different operating systems and web applications.

If you want to experiment with a wide range of operating systems and ready-to-use “appliances,” then VMware Fusion 2 is your best bet. The company’s Virtual Appliance Marketplace features nearly 1,000 ready-to-use virtual machines, many of which are completely free. Just download the appliance you’d like to test, launch Fusion, and point it at the downloaded file. It doesn’t get much simpler than that.

VirtualBox can also install VMware appliances format, too, but it takes more effort. The program can read the hard drive, but not the configuration file. So you’ll need to first create a virtual machine, and then point it at the downloaded appliance. It’s not a lot more effort, but it’s not click-and-go either. However, if you really are a geeky hobbyist, you may appreciate this hands-on nature to running the appliances. Just keep in mind that VirtualBox doesn’t support more than one CPU or 3D graphics; if you’re OK with those tradeoffs, perhaps the added setup work is worth the money saved.

I’d like to run OS X Server in a virtual machine.

Both Parallels and Fusion support virtualized OS X Server installations. Being a typical home user, however, I don’t have a copy of OS X Server for testing, and I haven’t looked at how well either programs handles Server. As noted earlier, though, you can download both for free and try it out with your own copy of OS X Server. (As of this writing, VirtualBox doesn’t support OS X Server as a guest operating system.)

I must have native support in the virtual machine for a FireWire device I use every day

Sorry, but none of the virtualization apps support FireWire directly—you can share files from an OS X-mounted FireWire drive, of course, but you can’t connect something like a FireWire scanner directly to a Windows virtual machine. For true FireWire support, you’ll need to use Boot Camp.

I want to watch high definition Windows video files.

If you really want to use high definition video in your virtual machine, the first thing I would tell you is that you really need to do it in Windows XP Pro. I had so many issues with HD playback in Vista in all three apps that it made me long for the days of black and white television and rabbit ear antennas! If you absolutely must have Vista and HD video playback, then Boot Camp is probably your best bet.

With that said, I found that Fusion did the best job with HD video playback in XP Pro. The videos I tried played back smoothly in both windowed and full-screen modes on the Mac Pro and the MacBook Pro. Parallels was close behind, but especially on my MacBook Pro, some videos didn’t play smoothly in full screen mode.

I want to do everything—run Windows games, experiment with new operating systems, run Office applications, and watch high definition video.

So you want, basically, my recommendation on the one virtualization application that’s the best all-around solution? As of today, my recommendation would be VMware’s Fusion 2. VirtualBox, while free, simply gives up too many features to the competition to win an all-around title—but it’s a solid performer for those with basic needs and a minimal budget. Parallels 4 is a solid upgrade with some compelling new features, but the bugs I ran into during my testing and its less-polished DirectX 3D performance relegate it to second place…for now. Going forward, I fully expect that the next round of updates to these three programs will both address current issues and bring exciting new features to the table. A little competition is a wonderful thing!

Subscribe to the Best of Macworld Newsletter

Comments