I wonder what ranking the Reputation Institute received. If they didn't make their own list, should we believe their report?

"I wonder what ranking the Reputation Institute received. If they didn't make their own list, should we believe their report?"

I don't think they need to make their own list to make the results valid. The survey involved 60,000 people... that is a pretty good survey size. I also don't see Apple as being #63 as a bad thing. To me, if you make the top 100 you are doing pretty good.

Quote

Guest wrote:
"I wonder what ranking the Reputation Institute received. If they didn't make their own list, should we believe their report?"

I don't think they need to make their own list to make the results valid. The survey involved 60,000 people... that is a pretty good survey size. I also don't see Apple as being #63 as a bad thing. To me, if you make the top 100 you are doing pretty good.

However, their questionnaire design and analysis method can seriously skew the results. They asked questions about several factors, but how did they put those factors together?

I regularly participate in polls conducted by the Harris company, usually on consumer questions. Often, the questions are designed to elicit particular answers and may not even have an answer that is close to mine. (For example, they asked if I had EVER been diagnosed with certain ailments. They then picked one and asked what I was taking for it. There was NO answer for "Nothing because I don't have that ailment any more."

Another example: when I was approaching retirement from the US Air Force, I did a lot of research on where I might like to live. Among the resources I used was a large paperback book, Places Rated Almanac. It had data and rankings on Costs of Living, Jobs, Housing, Transportation, Education, Health Care, Crime, The Arts, Recreation, and Climate. Each of 343 metropolitan areas in the US and Canada were ranked on each factor. The overall ranking was determined by adding the rankings for each factor and putting the areas in order. Sound good? Well, it's seriously flawed, though it's not as bad as the rankings produced by Money Magazine and others, which don't publish either their data or their methodology in detail.

1. The individual factor ratings included multiple sub-factors which were added together with some formula. How did they come up that formula? How sensitive would the factor rankings be to small changes in the formulas? (Answer: VERY sensitive--a small change in a multiplier could result in a 10-40 place change in ranking. Fortunately, much of the data used in their scoring was in the book, so I could make my own adjustments and assessments.

2. Focussing on ranking masked the real differences in scores. For example, consider these hypothetical scores:

City 1 100
City 2 50
City 3 49
City 4 7

The difference in score between City 1 and 2 is much greater than between City 2 & 3, but, the difference is the same in terms of ranking. Thus, City 1's score is effectively diminished.

3. The choice of factors affected the totals. Another rating scheme might combine "Jobs" and "Costs of Living," for example, or "Recreation" and "The Arts."

4. By adding all the rankings together, each factor was given the same weight. Would you do that? I didn't, as some of the factors were very important and others were irrelevant. (I did my own "ranking" of a subset of the areas by giving each factor a weight. I then varied the weights--used AppleWorks on an Apple //c--to see what happened.)

See the problem? It reduces the "meaning" of any such rating to somewhere between "very little" and "none," especially if they don't publish the raw data and analysis methodology, as would be required for a paper in sociology, for example.

It would also be interesting to know just who paid for the survey--who their clients are.

Interesting and thoughtful analysis. I would appreciate your take on our approach to measuring reputation. It's more of a social evaluation - tapping into the "gossip cloud" that surrounds each and every company, and quantifying what users share using Bayesian tools. By our methods, Apple falls in the lower 20% of peer companies relative to ethical aspects of reputation.

http://vanno.com/company/apple

Interesting and thoughtful analysis. I would appreciate your take on our approach to measuring reputation. It's more of a social evaluation - tapping into the "gossip cloud" that surrounds each and every company, and quantifying what users share using Bayesian tools. By our methods, Apple falls in the lower 20% of peer companies relative to ethical aspects of reputation.

http://vanno.com/company/apple

Interesting and thoughtful analysis. I would appreciate your take on our approach to measuring reputation. It's more of a social evaluation - tapping into the "gossip cloud" that surrounds each and every company, and quantifying what users share using Bayesian tools. By our methods, Apple falls in the lower 20% of peer companies relative to ethical aspects of reputation.

http://vanno.com/company/apple