How Siri (and Alexa) Continue to Fail Me

2 minute read
| Analysis

Siri is a fairly good AI, but the fact that it stumbled on some basics is a source of disappointment for me. In this test, Alexa stumbled as well.

Siri is very cool. Apple has put a lot of work into it. Siri is useful to me on an ongoing basis. But…

Apple Watch and Siri

Physics 101

An AI is designed to parse speech and do its best to respond. But when it stumbles on very basic concepts of speech, rich in context, it can be frustrating. For example, I asked Siri this question on my Apple Watch S4.

A physics conversion question for Siri.

Say what?

Just to be sure, I checked on my iPhone. Same answer. It’s almost like what the physicist Dr. Wolfgang Pauli (1900-1958) was fond of saying “That’s not even wrong.

Of course, if one leaves off “in pounds,” Siri (iPhone) presents a list of some website links. You’ll have to select one and hope it provides the right answer in plain sight. As an aside, here’s the correct answer, right out of Google search, very simply. I double checked myself, and 8.34 pounds is close to the correct. 8.3454. (It wasn’t rounded up correctly.)

Google's #1 result is right.

Google’s #1 result is right.

If one wants to drop back and let let Siri dictate the units of measure, then some progress is possible. I’m back to my AWS4/watchOS 5.0.1 here.

A simple physics question for Siri.

Let’s try something simpler.

Why does Siri report “3.79” and Google/Quora “3.78”? It’s because a more precise answer is 3.78541. Again, Google/Quora didn’t round up correctly. Siri did.

Alexa Up to Bat

I asked Alexa the identical question. “How much does a gallon of water weigh in pounds?” Alexa was better at figuring out that I wanted an answer in pounds. The answer I got was:

“9.92 pounds or 4.5 kilograms.”

I got that same answer three times. And that, as Dr. Pauli would have said, is indeed wrong.

What I Learned

  1. Siri couldn’t pick up the context of a desired answer in pounds. Alexa could.
  2. When Siri was given a simpler verbal context, it picked its own units of measure. At least the answer was right.
  3. Little attention was paid to rounding rules here by some sources.
  4. Alexa picked up on the context of desired units but presented a very wrong answer. This is alarming. Amazon?

Of course, one could do a lot more testing, and I’m sure all kinds of interesting effects would be noticed. But my simple goal here was to report some very odd things that I noticed right away. And wonder why these AIs both stumbled on something so simple.

4
Leave a Reply

Please Login to comment
2 Comment threads
2 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
4 Comment authors
CudaBoyJohn Martellaroduckduckgoosegeoduck Recent comment authors

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  Subscribe  
newest oldest most voted
Notify of
duckduckgoose
Member
duckduckgoose

It is possible that Alexa provided the weight of a British gallon of water rather than an American gallon of water. A US gallon is about 3.785 L and a UK gallon is about 4.546 L or about 4.5 kg.

geoduck
Member
geoduck

Of course you asked Google and Alexa. Expect ads for bottled water and where to convert dollars to pounds to start appearing in your web searches.