Benchmarking Windows 7 Milestone 3
To test Windows 7, I employed the DMS Clarity Studio tools suite - the same tools I used in March to show that Vista was 40 percent slower than Windows XP. Using a combination of the Clarity Studio's ADO (ActiveX Data Objects), MAPI (Messaging Application Programming Interface), and WMP (Windows Media Player) Stress workload objects, I was able to simulate a complex, multiprocess workload under Windows 7 consisting of client/server database, workflow, and streaming media playback tasks. I used the DMS Clarity Tracker agent to record system and process metrics during the test scenarios.
All tests were conducted against a 2GB Core 2 Duo (T7200) laptop PC (the Dell XPS M1710) configured to dual-boot between Windows 7 Ultimate M3 and Windows Vista Ultimate SP1.
A nine-way test scenario, involving three concurrent instances of each workload object, turned in nearly identical average transaction times under Windows 7 M3 and Windows Vista. In fact, the scores were so close - less than a 5 percent delta (in favour of Vista) on the database tasks, and a roughly 2 percent delta (in favour of Windows 7) on the workflow tasks - that they fell within what I'd typically consider the margin of error for this sort of test.
An analysis of system and process metrics collected during testing showed both operating systems consuming a similar amount of RAM - 637MB for Vista and 658MB for Windows 7 - across all active processes, while the overall thread count showed a modest reduction under Windows 7 (712 versus 810 for Vista). Interestingly, Clarity Studio's process objects for the database workloads (ADO Stress) spent significantly less time (61 percent less) executing in kernel mode under Windows 7 than under Vista, perhaps indicating that some of the code paths related to Jet 4.0 database access have been moved into user mode.
This could conceivably explain the modest, 5 percent slowdown of these same workloads under Windows 7. The additional Local Procedure Call overhead of moving portions of MDAC (Microsoft Data Access Components) out of the kernel would most certainly be felt by a time-sensitive, looping transactional workload like ADO Stress.
In a nutshell, Windows 7 M3 is a virtual twin of Vista when it comes to performance. The few minor variations I observed during comparative testing are easily explained away by slight tweaks to the kernel (such as the aforementioned MDAC changes); they certainly don't indicate a significant performance overhaul. More important, these variations pale in comparison with the 40 percent gap in performance I've observed between Vista and Windows XP SP3.
From a raw throughput perspective, Windows 7 promises to perform as poorly as its predecessor. 'Pre-beta' notwithstanding, the reality is that any hope for closing of the performance gap with Windows XP is unlikely to materialise in Windows 7.