iPhone 4 to 3GS Dropped Call Comparison Meaningless Without Context

| The Back Page

During Apple’s iPhone 4 “Antennagate” press conference, CEO Steve Jobs made what I think is a convincing case that Antennagate is mostly much ado about little. One factoid that he gave us — that the iPhone 4 drops just one more call per hundred than the iPhone 3GS — has little value without context, however, and that’s just how it was presented, without context.

So let’s examine it. One more than a hundred. It sounds small. It’s just one more, after all, right? For instance, let’s say the iPhone 4 dropped 57 calls out of a hundred, while the iPhone 3GS dropped 56 (there is no way the number is that high, but bear with me). That “just one more” would be a 1.78% increase, a tiny performance difference.

More realistically, is it  21 dropped calls on the iPhone 4 compared to 20 on the iPhone 3GS? That’s 5% more, which, as small as it is, is not a statistically small delta. Is it 11 compared to 10? That would be a 10% increase, which is considerable. Is it 5 compared to 4? That would represent a 25% increase, which would be staggering, if so.

It’s very understandable why AT&T won’t release the specific data in question. This is proprietary information that would be useful to competitors, and data that could be manipulated (see above) and subverted for use in advertising and marketing.

In other words, I am not accusing Apple of hiding anything here, but I am saying that it is little more than a platitude when presented without context. “Just one more than the other” sounds so benign, but “just one more” could be very little, a lot, a lot-er, or even “ZOMG! THAT’S JUST AWFUL!”

The only other thing Apple could have done would have been to express the delta in percentage terms, but even that level of specificity could be useful to the competition, and if it’s a high percentage (I’d personally say that anything over 15% is a high percentage), it’s information Apple wouldn’t want to have out there anyway.

The real point of my piece, though, is that we shouldn’t read too much into this particular factoid from Apple. Without the context all we really know is that the iPhone 4 drops more calls than the iPhone 3GS to some degree or another.

In comparison, the fact that 0.55% of iPhone 4 customers have called in with complaints is a tangible fact that strongly backs the argument that this issue effects few people. That 1/3 fewer iPhone 4s were returned to AT&T in the first few weeks of sales than the first few weeks of 3GS sales is meaningful number that further backs up Apple’s argument (Mr. Jobs told a reporter that Apple’s own customers returned even fewer in percentage terms than AT&T’s customers).

That Apple continues to sell the device, even with the supposed controversy in the blogosphere, in record numbers is proof (to me) that the vast majority of iPhone 4 owners have been happy with their experience.

Comments

Khaled

So who is going to release their own numbers? I’m surprised AT&T allowed these numbers to be announced.

DocRoss

You are correct and wildly wrong at the same time. The percentage difference between iPhone 4 and 3GS may be higher, as fewer calls are dropped, but that “higher” number represents a smaller percentage of the whole.

Let’s say that of every 100 calls, 1 is dropped by the 3GS. That would mean that 2 were dropped by the iPhone 4. OMG! That’s 100% more dropped calls. So what? It’s still only 2 dropped calls out of every 100.

Let’s say it’s .5 dropped calls for every 100. Holy Crap!!!! The iPhone 4 drops 300% more calls than the 3GS!!!!!

Your “context” blows things even further out of proportion.

The percentage difference between the two phones grows greater as the significance of the number gets smaller. NOT the other way around.

aardman

Actually Mr. Chaffin, you are the one who is afflicted with innumeracy.
Because if dropped calls were 0 with 3GS and 1 for iPhone4, then Omigod! that’s an increase of infinite percent!
The key statistic here is not the percentage increase, but the one in one hundred increase.  (Actually, less than one in a hundred increase.)  The percentage impact can range from near 0 to infinite but the actual impact is the same throughout:  Less than one additional dropped call per hundred.
Using percentage changes is actually one of the most common ways people lie using statistics.  (Our profits grew 3000%!  From $1 to $300 on $100,000 sales!)
I’m amazed your pronouncements about the importance of ‘context’ was the total opposite of the common sense conclusion.

aardman

tundraboy’s rule of forum posting.  A post complaining about grammatical errors will have grammatical errors.  A post complaining about mathematical errors will have . . .

3000% of $1 is $30 not $300.  Mea culpa.

Mike

Simply quoting numbers as you said “has little value without context”. Sometimes it’s not what is said but what’s left out. So let’s look at some of those numbers.

“the fact that 0.55% of iPhone 4 customers have called in with complaints” Of those how many didn’t want to wait on hold to be helped? How many went into the store to complain? How many aren’t even aware of the problem because their phones “bars” aren’t so sensitive? Too many unclear items.

“That 1/3 fewer iPhone 4s were returned to AT&T in the first few weeks of sales than the first few weeks of 3GS sales” So does returns mean for a refund? What about people who didn’t want to because of the restocking fee that was be fore they waived it. More importantly what about exchanges for faulty phones? At least one writer for PC mag has had 4 iPhones in 3 weeks due to various failures. What about Whoopi Goldberg who just this morning admitted to smashing her iPhone 4 in disgust for various problems and going back to a 3GS?

As I was watching a live blog of the press conference at Apple I was very tempted to cancel my order on my 32gb iPhone 4 simply because of Apple’s (Steve Jobs) attitude that there isn’t a problem because other phones do it. I just hope I get a “good one” unlike Whoopi and that PC mag writer, sigh.

aronnyc

Author of article is right. It’s easy to play and mislead with statistics. Ironically, Jobs introduced this number to put things in perspective about other statistics (i.e., to let people know that the iPhone 4 is comparable to 3GS, and not as bad as the media has made it out to be).

So, if you take this comparison, together with the other three he mentioned, that might help others see how things stand with the iPhone 4.

Photodan

I for one was glad that Apple (Steve Jobs) showed the comparison to other smart phones. It illustrates very well that Apple has been unfairly singled out for a reception problem that plagues many other phones. Jobs even said that, even though it happens to others, it was unacceptable. He never said it wasn’t a problem.

Mike, if defending your company and product is unacceptable to you and you share whoopi’s short fuse and childish behavior, then do everyone a favor and cancel your order now. If you destroy your iPhone, you won’t even be able to take advantage of Apple’s more-than-generous offer and return it for a full refund.

dwallin

I really struggle to see how anything a phone could do could possibly frustrate me enough to “smash” it!

It’s just a phone for goodness sake!!!

Lee Dronick

I really struggle to see how anything a phone could do could possibly frustrate me enough to ?smash? it!

I have wrestled with some horrid interfaces on dumb phones. But yeah, if Whoopi had trouble with an iPhone she probably pushes on doors that need to be pulled.

Photodan

dwallin said:I really struggle to see how anything a phone could do could possibly frustrate me enough to ?smash? it!
I have wrestled with some horrid interfaces on dumb phones. But yeah, if Whoopi had trouble with an iPhone she probably pushes on doors that need to be pulled.

Or she’s a crappy actress (with no career to speak of) who is desperately seeking attention any way possible. Just a thought.

YodaMac

I have to disagree with the author of this post.  He is trying to identify actual numbers when what is important is only the “comparison”.
Everyone was acting like the iphone 4 was dropping waaaaay more calls than the iphone 3GS.  And Mr. Jobs’ statement addresses exactly that comparison.

It doesn’t matter how many or what percentage it is…

it is still just a lowly “one more” call.  That’s the accurate, calculated “comparison” - which is what everyone wanted to know.  No biggie.

Mike

While I have once in my life smashed something in disgust I simply do not have the money to be so childish as Whoopi with a iPhone 4.

I have a friend in the UK has told me of some friends of his returning (for refund) their iPhone 4’s because of reception problems.

I have continued with my order simply because I’ll use a case and the antenna I won’t be touching so it wasn’t an issue for me. Now the antenna isn’t the only issue with the iPhone 4 either. I feel that the proximity sensor issue could be fixed with a software update that’s coming. What worries me is other problems that might be caused by hardware issues (ie quality of chips and manufacturing).

I didn’t see Steve give a full refund time frame. The policy is 30 days I think and AT&T said that was theirs but after that you can’t get out of the 2-year contract.

When this all started I went over to my local AT&T store to see for myself. I picked up the first one and held it in my hand just as I do with my original iPhone. I watched as the bars dropped from 5 to 1 just holding it normally. I thought well obviously this is a problem. After reading about others not experiencing this I went back to the store to try the same thing on their other ones. The others three I held dropped 1-2 bars. I went back and held the same one I did the first time in and it didn’t drop as much as before. BTW I also held in varying ways my original iPhone in the same location within the AT&T store and didn’t see an afeect on it’s signal bars.

What I gathered from going to the AT&T store was that there are several factos involved. First there is production variance (ie antenna tuning) between phones. Second that depending on what my hand is like at any given moment (work out, sweat, wash regularly, etc…) the affect on the phone can vary.

There is a design problem despite Steve’s smoke and mirrors routine. It’s common sense to me not to make a antenna design that people would be able to physically touch (ie no clear protective coating) in the process of using the device normally. It is a hand held device after all. I believe the problem exists with all the iPhone 4’s but either varies for the reasons above or people simply aren’t realizing the affect on data speeds (the numbers speak for themselves in others lab tests), bluetooth range, or call quality.

I love the looks and feel of the new phone and as I said before I’m hoping mine is going to be a good one. If not I will try exchanging it like the writer from PC mag did but at some point enough is enough. Not sure I’d give them 4 chances like she did.

I found this amusing. Steve Wozniak said “If you can afford it, carry a second Verizon phone for backup”, lol. the rest is here. http://www.mifieurope.com/2010/07/15/wozniak/

Duane Williams

Bryan’s fear of a large percentage delta increase in dropped calls is misplaced.  It is a very small percentage delta increase that he should fear.  A small percentage delta is generated when the number of 3GS dropped calls per 100 is large.  Suppose it’s 99 per 100.  Then the 1 more per 100 (just a small 1% delta) of the iPhone 4 would mean it drops 100 out of 100 calls!  Such a phone would be completely useless!

Nemo

The problem that I see is that Mr. Chaffin is incorrectly looking at the percentage increase in the number of calls dropped by the iPhone 4 versus the number of calls that iPhone 3GS drops, instead of the increase in the iPhone 4’s dropped calls per 100 calls versus the number of call the iPhone 3Gs drops per 100 calls.  Using Mr. Chaffin’s method, where the denominator is the number of calls dropped by the iPhone 3GS, so that if the iPhone 3GS dropped just 1 call per 100 and the iPhone 4 dropped just one more call per 100, that is, two calls, then the iPhone 4 has a 100% increased in dropped calls over the iPhone 3GS.  Of course, that is absurd. 

However, if you properly compare the number of dropped calls per 100 calls, using the example, supra, the iPhone 4’s dropped calls will have increased only 1%, or actually less than 1%, because the iPhone 4 drops less than 1 additional call per 100 calls than the iPhone 3GS.  So using the example, supra:  The iPhone 3GS drops 1 call and the iPhone 4 drops less than 1 additional (iPhone 4 drop<2) for a percentage increase in the iPhone 4’s dropped calls per 100 calls of less than 1%, that is, less than 1 dropped call per 100 calls is some percentage less than 1%. 

And so we have context that Mr. Chaffin incorrectly thinks is missing, because no matter how many call per 100 calls the iPhone 3GS drops, the iPhone 4 constantly drops less than 1% more calls per 100 calls than the iPhone 3GS.

jcn_13

The problem with all of this data (and this thread of discussion) is that Apple reported simple averages which tells you nothing about the real distribution of dropped calls. It’s possible that some users in weak signal areas are seeing dramatic increases in the number of dropped calls but since those users are in the minority (hopefully) it would mean that the average for all users would change very little.

For example, let’s say that we sample 1000 users and with the iPhone 3GS they each had 5 dropped calls per 100. Thus the total number of dropped calls would be:

1000 (users) * 5 dropped calls = 5,000 dropped calls out of 100,000 total calls.

Note, we know from our setup that the average was 5 dropped calls per 100.

Now let’s say that 100 of those 1000 users are in weak signal areas and they then switch to the iPhone 4. Next, assume that the iPhone 4 does drop more calls in weak signal areas and those 100 users now experience 15 dropped calls per 100 (a three fold increase). What does that do to the overall average for the total of 1000 users? It increases it by only one call per hundred:

900 (users) * 5 dropped calls + 100 (users) * 15 dropped calls = 6000 dropped calls out of 100,000 total calls.

However, that’s an overall AVERAGE of 6 dropped calls per 100 (only one more dropped call per 100).

Of course, the true distribution may be nothing like I’ve shown above and thus the “problem” could be worse or not nearly so bad as what I’ve shown (on an individual user basis).

Nemo

Dear jcn_13:  Because AT&T is the sole provider for virtually all the iPhone 3GS and iPhone 4s in the United States, AT&T data is virtually not a sampling of random samples but represent a descriptive statistic for the entire population of iPhone 3GS and iPhone 4s, which means that the number given by Jobs is nearly identical to the true Mean value for the entire populations to a level of precision so great that for all intents and purpose the value given by Jobs is the true value for the entire populations.  And even at a Confidence Level close to 100%, the value of iPhone 4 drops per 100 calls<1 more for iPhone 3GS per 100 calls is statistically significant.  Also, when using nearly the entire population, you would expect the distribution to be a Normal Distribution.

The bottom line is the value that Jobs gave will be a highly reliable estimate whether you are in an area with weak signal strength, average signal strength, or strong signal strength, variations in signal strength having been accounted for by both the huge size of the population and the measurements being taken across nearly every possible set of circumstances (e.g., signal strength, hand position on the phone, weather, urban areas, etc.). 

And the study from Anandtech indicates that the iPhone 4 did better than the iPhone 3GS in areas of weak signal strength, so if any thing the value given by Jobs is probably too high for the iPhone 4 in areas of weak signal strength.  And there is one other fact that suggest, if anything, the number of dropped calls for the iPhone 4 is even less or perhaps even fewer dropped calls per 100 calls than the iPhone 3GS:  many more users of the iPhone 3GS are using cases, which is something that AT&T stats can’t account for.  If some way can be found to correct for that, I wouldn’t be surprised to see that the iPhone 4 drops significantly fewer calls per 100 calls than the iPhone 3GS.

Rarely, in fact almost never, does a statistician get to sample the entire population, but here AT&T is sampling in real time virtually the entire populations of iPhone 4s and iPhone 3GS.  That will yield the smallest possible standard deviations from the Mean.  You can’t do better than that.

jcn_13

...which means that the number given by Jobs is nearly identical to the true Mean value for the entire populations to a level of precision so great that for all intents and purpose the value given by Jobs is the true value for the entire populations…

You seem to be suggesting that everyone gets the same distribution of signal strength and dropped calls. However, as I stated in my post the “...true distribution may be nothing like I’ve shown…” but you seem to be fully confident that the real distribution for each INDIVIDUAL could only favor the iPhone 4. My example wasn’t strictly concerned with the mean of the ENTIRE population (or the standard deviation or confidence interval), for that the reported values are entirely acceptable (a mean is a mean is a mean). What I was showing was that the performance on the iPhone 4 could be much worse for SOME users. Furthermore, there was nothing wrong with the example I provided (either statistically or otherwise), it COULD be fully representative of what is happening.

Lastly, it’s pretty apparent from the reports I’ve read that only some users are having problems with dropped calls (or slow data transfers) and even Apple has admitted that these types of problems are more likely to occur in areas of low signal strength. Therefore, your suggestion that the iPhone 4 may actually do better in weak signal areas (regardless of how it is held or whether it is in a case or not) seems rather at odds with the widely reported results and even with Apple’s own presentation.

Nemo

Dear jcn_13:  Yes, reception for the iPhone 4 could be much worse for some user under any give set of circumstances, but given the stat that Jobs presented, you would also expect that performance, by which I take you to mean reception, would also be much worse under those same circumstances for users of the iPhone 3GS, which everyone seems to agree has no unusual problem with reception.  So, it seems to me, that your example proves nothing about the issue of whether the iPhone 4 has worst reception than the iPhone 3GS, or can be expected to have worse reception than the iPhone 3GS.

So yes, given that we are virtually sampling entire populations of iPhone 3GSs and iPhone 4s in the United States, I would expect that the experience of individual users across a variety of relevant circumstances to at least reflect that the iPhone 4 drops<1 more additional call per 100 calls than the iPhone 3GS, even for individual users of the iPhone 4 across their entire range of measurements as they make calls, and with very little likelihood of deviation from that expectation for individual users.  So once again, your example that any individual user could have much worse reception than a user of an iPhone 3GS is highly unlikely for any particular user of an iPhone 4 and is, therefore, a trivial and irrelevant observation.

Lastly, I did not say or suggest that the iPhone 4 will have better reception regardless of how you hold it.  It is true for all modern smartphones with internal antennas that one can cause attenuation of reception by holding them in certain ways, respectively.  However, there is evidence to show that iPhone 4 does provide better reception in areas of low signal strength.  “From my day of testing, I’ve determined that the iPhone 4 performs much better than the 3GS in situations where signal is very low, at -113 dBm (1 bar). Previously, dropping this low all but guaranteed that calls would drop, fail to be placed, and data would no longer be transacted at all. I can honestly say that I’ve never held onto so many calls and data simultaneously on 1 bar at -113 dBm as I have with the iPhone 4, so it’s readily apparent that the new baseband hardware is much more sensitive compared to what was in the 3GS. The difference is that reception is massively better on the iPhone 4 in actual use.”  http://www.anandtech.com/show/3794/the-iphone-4-review/2. 

And you are patently wrong that there are widely reported results of dropped calls or other reception problems for the iPhone 4.  The stats that Steve Jobs presented at the press conference showed reports of reception problem of any kind are rare:  Only .55% of users have called Apple about any reception problems of any kind with the iPhone 4, which flatly refutes your statement that or widely reported results, by which I assume you to mean widely reported result about problems with the iPhone 4’s reception.  Though the results have been loudly reported, the stats from Apple, which is virtually the exclusive provider of support for the iPhone 4, show that, at slightly more than 1/2 of 1%, reports of reception issues with the iPhone 4 are rare, even if all those reports were about reception problems in areas of low signal strength.

The bottom line is that there is no observable hardware defect in the iPhone 4; reports of reception problems for the iPhone 4 are rare, and it may well be that the iPhone 4 provides better reception in areas of low signal strength than it immediate predecessor, the iPhone 3GS, and its dropped calls are at least not significantly worse than the iPhone 3GS.

jcn_13

...[iPhone 4] dropped calls are at least not significantly worse than the iPhone 3GS.

There is no absolute proof of that last statement. Once again, the fact that on average the iPhone 4 drops only one more call per hundred tells us nothing about the actual distribution of dropped calls. It COULD be much worse for a percentage of users who are in weak signal areas and who are using the so-called “grip of death.” That’s the only point I’ve been trying to make, I’m not claiming that the iPhone 4 has been shown to be worse, just that the dropped-call averages given by Apple do not prove that there is no problem.

Without seeing the actual distribution of dropped calls we have no way of knowing whether the “grip of death” users are a separate population or whether they should be assigned (statistically) to the overall average. 

You can illustrate this concept with a simple thought experiment. Take a group of rabbits and elephants (in our experiment both considered to be from a single “population”) and calculate the average amount of food they consume in a single day. Now take a group of bison and calculate that same figure. Let’s now say that the average amount of food consumed per individual is the same for both “populations.” Does that fact alone tell you the amount of food that an individual rabbit or elephant consumes in one day or even how those other individuals compare to the bison group? Of course not, because the distribution of food consumption in the rabbit/elephant group is different than in the bison group. Thus, in this example we’ve shown that you simply can’t assume that an average taken over a potentially non-homogeneous group will tell you anything about the distribution of values within that group or even the character of the extremes within that group.

I’m also surprised that you quote AnandTech since they have actually concluded that the iPhone 4 design does exhibit much greater signal loss when gripped than does the iPhone 3GS or any of the other phones that they have tested for this problem. Thus, even if the iPhone 4 has a more sensitive radio and performs better at “1 bar” that may not compensate completely for its greater loss of signal when gripped in a certain way.

In any case, let me repeat from my original post/conclusion. “Of course, the true distribution may be nothing like I?ve shown above and thus the ?problem? could be worse or not nearly so bad as what I?ve shown (on an individual user basis).” Thus, on a case-by-case user basis these simple averages don’t really prove anything one way or the other.

Nemo

Dear jcn_13:  I now see your problem.  We know or I thought that you and I both knew some external facts that show the stats provide by Jobs at the 7/16/10 news conference are sufficient to establish that the the iPhone 4?s reception is at least no worse than the iPhone 3GS in areas of low signal strength.  First, is the lesser point that we are comparing elephants to elephants, that is, we are comparing a smartphone to a smartphone across the range of signal strength.  The second much more important point, however, is this:  Since we are comparing a smartphone, iPhone 3GS to another smartphone, iPhone 4, the only way that a significant difference in reception in areas of low signal strength wouldn’t be reflected in the relative performance of dropped calls and, thus, in the Means for those two phones is for areas of weak signal strength to be insignificantly rare; otherwise, if areas of weak signal strength area a significant part of the distribution of signal strength, any significant difference in the iPhone 4 and iPhone 3GS reception would be reflected in a significant difference in the value of the mean stat for dropped calls that Jobs provided. 

However, we know that areas of weak signal strength are a major part of the distribution, and we don’t have to merely rely on anecdotal evidence to prove that.  Early this morning, I went to this website:  http://www.signalmap.com/.  There, I looked at the maps of signal strength for AT&T?s network and was readily able to discover that, at least for major cities, signal strength of only one or two bars was common for large geographic areas.  And for some cities, such as large parts of Chicago, which is a major market for AT&T, one could readily find areas that had an average signal strength of only two bars.  That show that areas of weak signal strength are so common and are such a significant part of the distribution of signal strength that, if the iPhone 4 and the iPhone 3GS?s reception was significantly differently in areas of low signal strength that would certainly be reflected in the mean stat on dropped calls.  But such a difference in reception in areas of low signal strength clearly isn’t reflected in the stat on dropped calls. 

Your objection would only be valid if the entire distribution of signal strength for AT&T’s network was as it is in Ann Arbor, MI, where the average signal strength is four bars, but that is rare.  What is common is an average of three bars in large cities with large areas of two bars and with many dead zones.  Thus, the mean stats for dropped calls for the iPhone 4 and the iPhone 3GS would reflect any significant difference in reception in areas of low signal strength, because such areas are a major part of the distribution of signal strength.  So the statement that the iPhone 4’s reception, as represented by dropped calls, is at least not significantly worse than the iPhone 3GS is true.

Duane Williams

The fact that AT&T’s network has large areas of low signal strength does not prove that such areas were a large part of the iPhone 3GS vs iPhone 4 comparison.

It is reasonable to suppose that people are less likely to own iPhones in areas where AT&T’s signal strength is low and more likely to own them in areas where AT&T’s signal is strong.

How much of a difference this makes is impossible to know without more data, but it argues against the assumption that merely looking at a signal strength map indicates the percentage of iPhones being used in low strength areas.

Nemo

Not so Mr. Williams:  AT&T numbers are from its entire network.  And we know from AT&T’s corporate data that they have plenty of users in areas of low signal strength.  Chicago, as I noted supra, is a major market for AT&T, yet there are large areas of low signal strength in the Chicago area.  It would be easy for me to show you others, such as the through the Midwest, SouthWest, and parts of the SouthEast, which are also major markets for AT&T.

Nemo

In fact Mr. Williams, AT&T has a much greater problem of more service areas with low signal strength, because it serves more areas of much lower geographic population density than, for example, Verizon, which has much high population density in more of its service areas and, thus, much better coverage by cell towers in more of its core markets in the NorthEast.

Duane Williams

Nemo:  All I can say to you is that if I were looking for a cell carrier and had a choice between one that offered a poor signal in my area and one that offered a good signal, I would be strongly inclined to choose the company that offered the stronger signal, knowing that that would likely improve my calling experience.  It may be that the iPhone’s significant advantages over other cell phones weighs against this someone.  No doubt some people will opt to have an iPhone in spite of having to live with a lower signal strength.  But I’m guessing that there are plenty of people who will reason that the iPhone is not worth the hassles that come with an always weak signal.  An added factor in the case of the iPhone is that it’s a smart phone and depends on heavily on the network than a simpler phone.

So, do you have data proving that the distribution of iPhones throughout AT&T’s network is independent of signal strength?

Duane Williams

Nema:  Regardless of how many phones AT&T has in low signal strength areas, how do you know they are iPhones?

Nemo

Dear Mr. Williams:  I know that the iPhone 4 and the iPhone 3GS, as is true of all of the phones on AT&T’s network, use all of AT&T’s network wherever the users happens to be.  There is no reason to expect that user of iPhones only make calls from areas of greater signal strength and have a different distribution for their calls than users of any other smartphone on AT&T’s network.  And certainly, users of the iPhone 3GS and the iPhone 4 would be making calls from areas of similar signal strength, and, as I said supra, there is not reasons to expect that users of iPhone would not make as many calls from areas of low signal strength as users of other smartphones on AT&T’s network.

A word about AT&T’s network.  I think it is clear that any network would be clobbered, as AT&T’s network has been, by the unprecedented bandwidth demands of the iPhones.  And even now, notwithstanding the introduction of the Android phones, Apple’s iOS devices use more than twice the bandwidth of the next best contender, the Android phones.  So AT&T is facing a unique amount of extraordinarily great demand for bandwidth.  I think, therefore, we should have some sympathy, some understanding, for AT&T’s network problems. 

But the demand that the Android phones places on Verizon’s network is growing to what many think are unsustainable levels.  We will see how well Verizon handles it.  Already, we can infer that Verizon said no to the new HTC EVO, because of the demand that its video chat apps would place on its network.

jcn_13

I now see your problem.? We know or I thought that you and I both knew some external facts that show the stats provide by Jobs at the 7/16/10 news conference are sufficient to establish that the the iPhone 4?s reception is at least no worse than the iPhone 3GS in areas of low signal strength…

I guess you are suggesting that if the iPhone 3GS and iPhone 4 behave similarly in one condition of signal strength and method of grip then they must behave similarly under all conditions (thus, they are both elephants). If that were true then I guess Apple has wasted a lot of time testing the iPhone 4 since they just could have compared it to the iPhone 3GS under one set of conditions and then assumed that it would be no worse under any other situation (that is, using my example, there could be no rabbits or bison in the population). Certainly, you can see the problem with that type of reasoning.

In any case, I would admit that the averages that have been given by Apple indicate with some certainty that the majority of users should not be seeing a significant increase in the number of dropped calls with the iPhone 4. However, that doesn’t eliminate the possibility that a subset of the group could be having problems and we can’t estimate the size of that subgroup without some knowledge about the distribution of results from within the sample population (i.e. a simple comparison of averages is not good enough).

Nemo

Dear jcn_13:  If I couldn’t directly observe the two population, then I might have to wonder about whether they were both elephants, that is, whether they were both smartphones.  But with the iPhone 3GS and iPhone 4, I can directly observe that they are both smartphones but one with an internal antenna and the other with a new external antenna system.  Since the aren’t identical—elephants but not identical elephants—I can’t use the iPhone 3GS as a model for the iPhone 4, but they are both smartphones on the same network. 

Next, I have mean data that tells me that they behave similarly on average, and finally I have knowledge not about how they operate under every condition of signal strength, but I have sufficient information about the distribution of signal strength on AT&T’s network to know what impact conditions of low signal strength must have on the means for dropped call for the iPhone 3GS and iPhone 4.  And what do we know about the distribution of low signal strength?  We know both from anecdote report and from direct measurement of the distribution of signal strength that low signals strength is prevalent so that if the iPhone 4 had significantly worse reception than the iPhone 3GS in conditions of low signal strength, we would see it in their respective means for dropped calls.  But we don’t measure any difference in the means that would support that the iPhone 4 has worst reception than the iPhone 3GS in areas of low signal strength; we, in fact, see the opposite:  That the iPhone 4 drops<1 more call per 100 calls than the iPhone 3GS, which means that the iPhone 4’s new external antenna is not causing any significantly observable problem with its reception as measured by dropped calls.   

That’s all you need to infer that the iPhone 4 at least a does not have significantly worse reception than the iPhone 3GS in conditions of low signal strength, and since it is agreed that the iPhone 3GS doesn’t have any problem with reception that is not common to all smartphones, that the iPhone 4’s mean for dropped calls is not significantly worse than the iPhone 3GS’s means implies that the iPhone 4’s has no observable defect in its reception.

jcn_13

If I couldn?t directly observe the two population…

Well, okay, I think you’ve just validated my point. We aren’t directly observing the two populations, we’re only observing the averages of the populations. Forget about the physical characteristics (we aren’t even observing those in my examples), I was just using elephants and rabbits as an example of what could happen when you try to compare the averages of two populations that could have different distributions.

I might also note that we have no proof that the distribution of dropped calls follows a strictly Normal Distribution. In fact, it’s most likely that the distribution is highly skewed since you can’t have fewer than zero dropped calls per hundred and I would expect that most users actually suffer very few dropped calls. However, the tail of the distribution could follow off to a very high number of dropped calls for any given individual who happens to make most of his calls in a weak signal area (and once again we’re concerned with individual results, not simple averages). In fact, my original post illustrates this exact situation where a small group of users (10% of total) could experience much worse performance with the iPhone 4 without greatly affecting the average of the entire population.

Nemo

Dear jcn_13:  I meant that you can see that both the iPhone 3GS and the iPhone 4 are both smartphones with internal antennas, so you aren’t, for example, comparing a smartphone to a pager or to a phone with an telescoping antenna.  Thus, you can see that the phones are both of the same class, i.e., smartphones with internal antennas.

Yes, you can have a normal distribution.  Symmetry does not require negative numbers.  Even if the means for the two iPhones was less than one dropped call, remember that we are sampling virtually entire populations.  Those are large number and will give you two symmetric tales on both sides of the mean, and, as the number of measurements gets larger, the distribution tends toward normal, if it wasn’t normal to begin with.  (This result derives from the Central Limit Theorem.)  And here we are sampling the entire populations for the two phones.

But if any statistically significant group of user is experiencing more dropped calls on the iPhone 4 in areas of low signal strength, that will cause the mean for the iPhone 4 to depart significantly from that of the iPhone 3GS, and we would see iPhone 4 drops<x more calls per 100 than the iPhone 3GS, where x is much greater than one, especially if 10% of the iPhone 4 groups was experiencing significantly more dropped calls.  Even without doing the math, I can tell you that to not affect the mean for the iPhone 4 drops more than we see with iPhone 4 drops<1 more call per 100 calls than the iPhone 3GS, at these sample sizes, the number iPhone 4 users experiencing more dropped calls would be such a small percentage as to be insignificant.

And perhaps this will help.  With large sample sizes, any percentage of iPhone 4 and iPhone 3GS users, who make most of their calls in areas of low signal strength will be offset by users who make most of their calls in areas of high signal strength.  However, if the iPhone 4 users experience significantly more dropped calls in areas of low signal strength than iPhone 3Gs users, that will shift the mean for the iPhone 4 to right, even with small percentages with a result where x from the equation, supra, will be a lot greater than 1.

Remember that the Confidence Level to reject the null hypothesis—that there is no difference in the means—is at least 95%, so your example of 10% experiencing systematically more dropped call will certain result in rejection of the null hypothesis.

sleepygeek

Just a theory:
The drop rate for the 3GS may be better only because it wouldn’t have started 1% of calls which the iPhone4 starts then drops.

It seems that iPhone4 reception is equal or better than 3GS under nearly all circumstances. Perhaps because of the external antenna. Thus the attenuation when cupping the antenna in your hand is greater for the iPhone 4 which is reduced from superior reception to equal reception.

The problem may arise because the iPhone4 is likely to start a call in weaker signal conditions (because reception is better), when a 3GS wouldn’t. Thereafter the same cupping of the antenna would drop the call in both circumstances. But the call was never started with the weaker phone.

jcn_13

Dear jcn_13:? I meant that you can see that both the iPhone 3GS and the iPhone 4 are both smartphones with internal antennas, so you aren?t, for example, comparing a smartphone to a pager or to a phone with an telescoping antenna.? Thus, you can see that the phones are both of the same class, i.e., smartphones with internal antennas.

Actually, that’s incorrect (I assume you’ve just made an unintentional mistake). The iPhone 4 has an external antenna, while the iPhone 3GS has an internal antenna. But I won’t try to claim that this fact alone is critically important to our discussion.

Not all distribution are classically NORMAL and all distributions do not suddenly become NORMAL just because you’ve taken a large sample. A large, perfectly random sample just means that your confidence level in the ability to predict the performance of the entire population has increased and in fact if you measure the entire population with high precision then you really don’t need to be too concerned about confidence levels (statistical analysis) since you already know the entire population (its distribution and average and other factors). Of course, if you are actually sampling a group that follows a Normal Distribution then the larger the sample the more closely you will conform to the theoretical model.

...Those are large number and will give you two symmetric tales on both sides of the mean, and, as the number of measurements gets larger, the distribution tends toward normal, if it wasn?t normal to begin with…

Frankly, I think you’ve finally “jumped the shark” in your arguments that similar averages indicate identical performance. If you’ve just described the actual distribution of the frequency of dropped calls then you’ve also just invalidated your defense of Apple’s reported averages. If the distributions are perfectly Normal (“two symmetric tales on both sides of the mean”) then what about the standard deviation? If (as you seem to be claiming) we are dealing with a perfectly Normal Distribution then the dropped call per hundred rate COULD have exactly the same mean and yet the standard deviation could be significantly different between the two models of phone (it’s unlikely that they are widely different, but they could be significantly different in terms of the individual experience). This could produce a distribution of dropped calls on the iPhone 4 which contained a larger number of users who are having problems with dropped calls (and the mean would be compensated by a larger number of users who are having less of a problem with dropped calls—but who is going to complain about that?). Of course, you could claim that the standard deviation is the same for both phones but we have no proof of that fact (nor do we know that the distribution is classically Normal, which I would say is the main fault in your argument). If Apple had said that the distribution of dropped calls followed a strictly Normal Distribution and that the standard deviation for both groups was the same and that the MEAN was within one dropped call per hundred then they would have proved their point (i.e. the iPhone 4 has no more of a “problem” than the iPhone 3GS). However, they only said that the AVERAGE was within one call per hundred.

If I had to GUESS at the distribution of dropped calls (rates) then I would say that it has a highly skewed peak and average trending toward a very low number of dropped calls (that is, most users are clustered around that peak and the majority of users are not dropping many calls). However, if we plotted better performance to the right then the distribution could have a very long tail of relatively low magnitude to the left toward a larger number of dropped calls per hundred (ending at point where calls can not be made—effectively 100 dropped calls per 100 for that individual user).

Even without doing the math…

Well, in my original post I already gave an example (with math) that showed that some users could be experiencing a greater number of dropped calls while keeping the average within one call per hundred. In fact, my math could be shown to be perfectly consistent with the type of distribution that I described above (peak to the far right—90% of users—with a long tail of worse performance to the left).

In any case, you seem to be clinging to a rigid belief that Apple’s stated averages can only be interpreted to show that there is no problem with dropped calls on the iPhone 4 (not even allowing for the possibility that a relatively small number of users could be having problems). However, I simply claim that we can’t tell with certainly whether there is a problem or not. Given these two alternatives, I think I have shown that my position is the stronger argument.

Allie

Or she?s a crappy actress (with no career to speak of) who is desperately seeking attention any way possible. Just a thought.

The fact that you even went there about an academy award winning actress is laughable…really…just a thought.

Sam

I believe others have said this already, but: wrong wrong wrong. Percentages are precisely the wrong thing to use here, natural frequencies are almost always more instructive. This is exactly why marketing people and lame journalists like to use percentages - they are easy to make sound big.

Your analysis would have an increase from 1 dropped call per hundred to 2 dropped calls per hundred as horrifying 50% difference, yet if there were 10 per hundred originally, we could have up to 14 and be happier. But think about it: which would you rather have, 2 or 14? The point is that we are already talking about small numbers in the first place.

Log-in to comment