QuickTime Testers Called "Dubious" By Novell
[7:25 AM] Yesterday Microsoft announced that an independent company named Mindcraft had tested QuickTime. The results of that test supposedly determined that programming errors on the part of Apple were responsible for the error codes generated when Windows users installed QuickTime according to Microsoft. Mindcraft is an independent test lab. A reader pointed out that Microsoft has commissioned Mindcraft on other occasions as well. In one such case, Mindcraft tested Windows NT and Novell to determine which was faster. The results from that test led Novell to issue a counter report called "Reality Check- You can Fool Some of the People Some of the Time…."

This document strongly criticizes the company's testing methods and accused Mindcraft of producing a biased report. According to Novell:

"For the second time, Mindcraft, Inc. has published a product comparison of dubious origin, unprofessional methods and biased results. Novell is embarrassed to respond to such prejudiced work, but does have an obligation to inform the market of the facts and publish its own benchmarks with accurate and reproducible results."

Points included in the rebuttal include:

(1) Standard benchmarking practices were violated.
(2) The benchmark test parameters used were known to give faulty results.
(3) The server hardware configuration was contrived to put NT in it's best light.
(4) Major operational characteristics were applied unevenly.

Mindcraft, for its part, issued a counter rebuttal in which it accused Novell of hypocrisy and strongly defended its testing methods. The counter rebuttal also mentions that Novell had itself hired Mindcraft in the past.

Other recent reports from Mindcraft include a benchmark test of Microsoft Windows NT Server 4.0 on a Compaq ProLiant 3000 and Sun Solaris 2.6 with Sun WebServer 1.0 on a Sun Ultra Enterprise 450. Those tests also concluded that the Microsoft product was supposedly faster.

"Windows NT Server 4.0 Is Four Times Faster as a Web Server Than Solaris 2.6 with Sun Web Server 1.0 and Has 10.3 Times Better Price/Performance."

The Mac Observer Spin: Yesterday we said that we were in the midst of what could arguably be called the highest stake game the world has ever seen in business. Microsoft is in fact fighting for their very survival as the company it is known as today. Under those circumstances it is not uncommon for companies to use labs where they have gotten favorable results in the past. Trials of all sorts often include expert (paid) witnesses, for instance, that are known for giving the testimony needed by one party or the other.

Novell used strong words indeed to criticize Mindcraft, but in all fairness, Mindcraft's counter rebuttal was succinct and detailed in its defense. Apple has yet to release any kind of counter statement to Microsoft's assertions from yesterday, but it is our guess that this particular aspect of the battle between the two companies is not over yet.