Parallels released Parallels Desktop 9, the latest version of the company’s virtualization software, early Thursday. Version 9 brings many new features to the virtualization game, but Parallels also promises significant performance improvements. As a follow-up to our virtualization benchmark showdown last fall, we began testing the new version immediately and we have some preliminary benchmarks to share.
Testing Hardware & Software
Our tests were performed on a 2012 15-inch MacBook Pro with Retina Display, equipped with a quad-core 2.7 GHz Intel Core i7 CPU, 16 GB of RAM, and a 256 GB SSD. Our host operating system is OS X 10.8.4 Mountain Lion, and our guest operating systems were Windows 7 Enterprise x64 and Windows 8 Pro x64. All operating systems were updated with the latest publicly available patches and updates.
For tests that cite “native” performance, we used Windows 7 Enterprise x64 installed via Boot Camp on the MacBook Pro. Note that most tests cite only Windows 7 results for the sake of simplicity in reading the charts. There were no significant differences in most tests between Windows 7 and Windows 8 virtual machines. For tests where a significant difference was apparent, Windows 8 results are included and noted.
Testing Methodology
We first installed the latest version of Parallels Desktop 8 and created two virtual machines, one each for Windows 7 and Windows 8, and performed all of the tests described below. We then performed a full uninstall of Parallels Desktop 8, installed Parallels Desktop 9, and then imported the existing virtual machines into version 9. All tests were then repeated with Desktop 9.
Each test was performed three times for each virtual machine on each version of Parallels Desktop, with the exception of the boot, shut down, suspend, and resume tests, which were performed five times. The results for each sequence of tests were averaged to obtain the reported results.
For time measurements, results were rounded to the nearest whole second. For all other tests, results were recorded in the same format as they were reported by their respective benchmark applications.
Geekbench 3.0.2
Primate Labs’ Geekbench is a cross-platform benchmark application that measures system processing and memory performance. It does not measure any aspect of a platform’s graphical or storage capabilities. Readers should note that Geekbench was recently updated to Version 3, which now reports both single- and multi-core results. We have a separate chart for each, below. Due to the new tests, Geekbench 3 scores cannot be compared to scores from earlier versions of the benchmark.
Geekbench results reveal a consistent pattern: native performance is the best, and Parallels Desktop 9 provides a modest improvement over Parallels 8. The most impressive performance boost is delivered via improved integer calculations, which score about 5 percent higher in Parallels 9.
Cinebench 11.5
Maxon’s Cinebench is another cross-platform benchmark that focuses on GPU performance and CPU rendering capabilities. It’s based on Maxon’s venerable Cinema 4D modeling and animation software.
Cinebench tests three primary areas: OpenGL GPU performance, measured in frames per second, and multi- and single-core rendering performance, measured in comparative “points.” For all categories, a higher score indicates better performance.
Just as we saw with Geekbench, the Cinebench results give us a performance ranking of Native > Parallels 9 > Parallels 8. However, once again, the performance improvements are not significant.
Heaven 4.0
The Unigine Heaven benchmark is a graphics-heavy test of a system’s GPU capabilities. When cranked up, it can bring even the most expensive dedicated gaming GPUs to their knees. Due to the limitations of the MacBook’s mid-level NVIDIA GPU, we opted to run the “basic” preset, which is a DirectX 9 test at 1280x720 with high-end options like tessellation disabled.
The test reports an overall average frame rate, minimum frame rate, and maximum frame rate, all measured as frames per second, with higher numbers indicating better performance.
Parallels Desktop 9 again offers only slight performance gains with DirectX 9 graphics, practically equal to the score in Parallels 8 and trailing average native performance significantly.
Perhaps the most important benefit offered in Parallels 9 is a higher minimum frame rate, almost equal to that of native performance.
3DMark (2013)
The new 3DMark benchmark, released earlier this year, contains three tests that are meant to cover the entire range of gaming hardware: Ice Storm, Cloud Gate, and Fire Strike. Ice Storm is an entry-level test designed for basic computers and mobile devices. A new “Extreme” mode adds a few additional effects and bumps the resolution of the test to 1080p, to accommodate slightly more powerful systems.
Cloud Gate is a mid-range test that covers most average gaming PCs and hardware, and Fire Strike is the most intense test, designed to punish not just this year’s gaming hardware, but next year’s as well.
As we mentioned earlier, Apple’s MacBooks don’t ship with top-of-the-line GPUs, and the inherent performance loss when using virtualized operating systems further limits the capabilities under which we can test. We therefore decided to stick with the Ice Storm and Cloud Gate tests.
Unfortunately, Ice Storm Extreme and Cloud Gate would not run correctly when using Parallels Desktop 8; there were graphical glitches during the tests that make any results unreliable for comparison purposes. We therefore tested only the basic, non-Extreme version of Ice Storm on Parallels 8, making the remaining higher-end tests a straight comparison between Parallels 9 and native performance.
As you can see from the basic Ice Storm test, Parallels 9 brings huge performance gains in terms of DirectX 10 capabilities. While still trailing native scores by a wide margin, Parallels 9 offers a nearly 430 percent increase in performance over its predecessor.
The 3DMark Physics test is primarily tied to CPU performance instead of the GPU so, even though Parallels 9 scores higher than Parallels 8, the difference is not nearly as dramatic as the GPU tests.
Looking at the higher-end tests, Parallels 9 is clearly outclassed by native performance, but still offers credible results. Your best bet for Windows gaming on a Mac is still Boot Camp, but with each generation of virtualization software, the prospect of gaming from within a Windows VM gets brighter.
iTunes 11.0.5 Encoding
To test how well Parallels handles audio encoding, a process which combines aspects of both the CPU and storage abilities of a virtual machine, we configured a test using iTunes 11.0.5 and a copy of disc one of M83’s Hurry Up, We’re Dreaming ripped to the virtual machine as lossless WAV files. The files, clocking in at 323 MB, were encoded to 192 kbps MP3 files using iTunes’ built-in encoder. We then timed the results.
The results reveal that audio encoding capability alone is not a sufficient reason to upgrade to Parallels 9. The new version beat Parallels 8 by only 2 seconds, and both took significantly longer than iTunes running via Boot Camp.
File Duplication
To test disk-intensive operations, we used a 3.5 GB ISO file and measured how long it took to duplicate it in the virtual machine’s file system. We deleted the copied file and rebooted the machine after each measurement to ensure that any file system cache operations would not contaminate the results.
Once again, there’s no change in performance between Parallels 8 and 9, with both versions taking 21 seconds on average to perform the operation, compared to 13 for native Windows via Boot Camp.
Virtual Machine Management
With practically every new version of virtualization software, be it Parallels Desktop or competitor VMware Fusion, the developer boasts improved startup, shutdown, suspend, and resume times. While these factors are of less import to most users, those who manage multiple virtual machines can realize definite gains to their workflow efficiency by not having to wait minutes for each machine to boot or resume.
Virtual machine management was already impressive in Parallels 8, so we wanted to see how things had improved, if at all, for Version 9. Note that there were some differences between Windows 7 and Windows 8 for this test, so both are included. Also note that, unlike our other tests which averaged three results, these tests were measured five times each and rounded to the nearest second.
Parallels 9 does indeed offer some nice performance improvements, most significantly in terms of cold boots. The time to boot Windows 7 dropped 5 seconds from version 8 to 9, while Windows 8 boot times improved to a blistering 8 seconds.
Aside from a few outliers of a second or two, the remaining functions took approximately the same time between Parallels 8 and Parallels 9. As always, your virtual machine times will vary based on your hardware and software configuration. Our machines were loaded from an SSD and had minimal software configured to load at launch (Steam, Microsoft Security Essentials, and the OneNote agent from Office 2013). Loading VMs from a mechanical hard drive, or packing them with auto-launching software will of course result in longer times.
Conclusions
Parallels 9 brings many great new features, as TMO’s John Martellaro discusses in his Quick Look Review, and does indeed offer significantly better performance in certain areas such as DirectX 10 rendering and cold VM boots. But, overall, the performance improvements are only minor compared to Parallels 8.
Those using virtualization software to run certain low- or mid-range productivity apps will likely find no performance-based reason to upgrade to Parallels 9. However, users should take a look at the new features, many of which add much value to the software beyond pure speed.
Finally, these benchmarks looked exclusively at Windows. While Windows is the most popular operating system to virtualize on OS X, we realize that many users virtualize other operating systems such as Linux and earlier versions of OS X itself. We’ll be taking a deeper look at these scenarios in the future, and we’ll also have product-to-product comparisons between Parallels 9 and the next versions of Fusion and VirtualBox once they are available.
Until then, enjoy running Windows on your Mac, but just make sure to install anti-virus software!
To learn more about Parallels Desktop 9, check out TMO's Quick Look Review.