< A href="http://www.apple.com/">Apple Computer defended its claim of having "the worldis fastest, most powerful personal computer" on the market by omitting certain speed tests it failed and claiming certain workstation computers werenit comparable, a detailed analysis by The Mac Observer of the Better Business Bureauis 26-page report on claims of false advertising brought forward by Dell Computer has found.
The report, first reported by CNet News, was the result of an investigation by the National Advertising Division (NAD) of the Better Business Bureau (BBB), the advertising industryis self-regulatory forum, after Dell challenged Appleis contention that its Power Mac G5 PC was the fastest and most powerful on the market. The report laid out three sides of the allegations against Apple - claims by Dell, Appleis responses to Dellis claims, and the findings by NAD as to whether the claims were valid or not.
In itis decision, NAD recommended Apple pull its advertisementis claiming to have the fastest and most powerful PC, saying readers could easily believe the claim applied to workstations, as well as personal computers, and that certain speed test results were purposely omitted to make Appleis test look better.
Apple accepted the NAD results, but responded to the report by saying it stood by its testing and advertisements, that the ad campaign had already ended and that "Apple will be mindful of NADis views in its future advertising."
The Dell challenge
Dell challenged Apple in three main areas - testing against two PCs that Apple knew wouldnit compete equally with the Power Mac G5, "optimizing" of the G5 to unfairly beat competitors PCs, and not fully disclosing test results that would show the G5 was beaten by a competitor.
While the report addressed all of Dell challenges, NAD primarily focused on three areas it concluded were indeed questionable - Appleis confusing use of the words ipersonal computeri and iworkstationi in its advertising, the testing of applications not routinely used by "a general audience" and its failure to disclose that its G5 system failed in beating its competitor in specific integer-base testing.
NAD felt that although the line between a personal computer and a workstation was becoming increasingly "blurred," some high-end PCs are now equivalent to low-end workstations and visa-versa. In its conclusion, however, NAD criticized Apple for claiming it shouldnit be judged against workstation PCs, when Apple did just that in its advertisements against the dual-processor Xeon workstation.
"...as presented in the print advertisements, the advertiser uses the terms personal computer and workstation interchangeably and, in this light, viewing the advertisement in its entirety, NAD concluded that the advertiseris claims in this advertisement (including the claim that its G5 is the worldis first 64-bit processor for personal computers) could reasonably be interpreted as applicable to competitive workstations -- which is not the case," the report concluded.
Photoshop and the general population
NAD also criticized Apple in its time testing of 600MB Photoshop files it concluded were bigger than what an average user would use and the use of scientific applications for testing when it should have used software apps more in use by the general population.
"NAD considered the challengeris position that the typical range for personal use of this application are file sizes between 10 to 30 MB and observed that PC World.comis testing of the advertiseris system (and others) using Photoshop employed file sizes of 50 and 150 MB," the report stated. "NAD concluded that the advertiseris application testing was outside of the range that the average user (to whom the advertisements at issue are directed) typically uses, thereby calling into question the consumer relevance of this test."
As for testing more appropriate applications, NAD agreed with Dell that testing with the scientific-analysis applications BLAST and HHMer was questionable.
"NAD similarly questioned the consumer relevance of these tests given that these programs are used for highly scientific purposes (i.e., DNA sequencing)," the report stated. "...NAD determined that the applications tested are not those typically engaged by users of personal computers, thereby leading NAD to, again, question the relevance of these application test results insofar as offered to support the advertiseris unqualified, broad superiority claim directed at a general audience."
Appleis defense
Apple defended its use of the two apps, saying its goal in performing such application testing was to test all of the capabilities of the G5 and competitive computers.
It was Appleis omission of integer testing from its advertisements, however, that became a main focus of the NAD report.
Appleis testing clearly admitted that in three out of four tests against Windows-based rivals, it was beaten in speed tests of the computeris processor to execute predetermined tasks - either "integer" calculations or "floating point" calculations.
In the tests, each type of calculation was tested in two different environments. The first, called the "base" test, isolated and tested a single processor within the system. The second test - the "rate" test - measured the entire system output, including the benefits of multiple processors when they are included.
Apple contended that the "rate" test, which recognized the presence of multiple processors and measured the full performance of a multiprocessor system (like the G5), was more relevant and a better indicator of the speed of the fastest computers on the market.
In fact, to run the base test on the G5, the tester had to actually disable part of its capacity, by disengaging one of the processors in the dual processor system to isolate the performance of the remaining processor. Therefore, Apple argued, the rate test was the more appropriate test because it measured the G5is full performance capability and that was the reason it showed those results and not others in which it failed to beat its competitor.
And the response
NAD acknowledged the validity of Appleis position that testing overall system performance is a valuable tool by which to compare competing systems to the extent that the test takes advantage of systems containing a second processor, but still questioned Appleis failure to publish all the results.
"While these results may well have been published on the advertiseris Web site (and are, therefore, available for public review), NAD questioned the absence of these results from the advertisements at issue and, more importantly, the relevance of these results insofar as concerns the advertiseris claim that the G5 is "the fastest, most powerful personal computer." NAD determined that, clearly, both are relevant for purposes of such a broad claim."
SPEC
Probably most damning for Apple in the argument over using "base" or "rate" testing was the conclusion of Appleis own testing group from which it based its claims. The Standard Performance Evaluation Corporation (SPEC), prefaced its own results by cautioning readers of the data to consider how a PC is going to be used.
"SPEC provides the benchmarks and results as tools for you to use. You need to determine how you use a computer or what your performance requirements are and then choose the appropriate SPEC metric," the document reads.
NAD therefore concluded, "While it may be true that the G5 outperformed its competitors on integer rate tests (testing overall system performance), the G5 was not the most powerful personal computer on the integer-base test. As such, the advertisement at issue, at best, tells only part of the story with respect to the results of the advertiseris VeriTest testing and there is nothing in the advertisement at issue from which a consumer could glean that other tests were, in fact, performed."
NADis conclusions
In the end, NAD came to the broad conclusion that Apple had somewhat conveniently used testing results that put Apple in the best light and ignored others that left open questions. In addition, NAD believed Apple should have either better differentiated PCs from workstations, or included them more prevalently in its tests.
Apple refused repeated attempts by The Mac Observer for further reaction to the NAD decision.