The Mac Observer

Skip navigational links

The Devil's Advocate - TMO Scoop: Apple Files Patent for Looking Glass
by - January 25th, 2006

From the out of left-field department, on June 21, 2004, Apple filed for patent application no. 20060007222 for an "Integrated Sensing Display."   Of course just because Apple filed for a patent application does not mean the company will release a product; but in this case, let's hope they do.

Basically this invention morphs a camera with a screen.  And I'm not talking about putting an iSight above the screen.  The screen's surface area is the camera. Each pixel would have an image capture element. So I theorize you could just put a piece of paper in front of the screen, and it could just appear like a reverse-mirror-dissolve onto the screen.  It could be quite the intuitive and "magic" way to capture images of all kinds.

Mirror mirror on the wall...

It also fixes a problem that cannot be fixed with an iSight as this panel would let  "a user ... maintain eye contact with someone on screen because the camera is 'in' the screen." 

The patent application shows that an image sensor would be located next to each pixel:


Display screen (300) with image sensors (305) forming a sensor array

It's not clear what the effective resolution of each image capture sensor would be, however, even if each element had some marginal resolution, the effective resolution of a 30" LCD capture screen would be very high.

Cost & Applications?

The application notes that this technology could be integrated into many display technologies and form factors including CRTs, Plasmas, LCDs and PDAs.


Individual image sensor

It's not clear just how much it would cost to use millions of image capture elements per screen; it likely would be cost prohibitive for consumers, at least initially. Also, I'm not sure how useful this would be to the average user. Indeed, the patent application notes that such a device could be useful for medical imaging.  

Anyway, I'm just happy to see something so creative come out of any company. If the display "sees" you, and can sense positions, new user interface advances like gesture-sensitive touch-screens (check out these videos) become more likely to come our way. In a world overstuffed with mediocrity, sometimes its enough if something is just cool and fun for its own sake.

is an attorney. Please don't hold that against him. This work does not necessarily reflect the views and/or opinions of The Mac Observer, any third parties, or even John for that matter. No assertions of fact are being made, but rather the reader is simply asked to consider the possibilities.

You can send your comments directly to me, or you can also post your comments below.

Most Recent Columns From The Devil's Advocate

The Devil's Advocate Archives

Observer Comments

Show: Subjects Only | Full Comments
Close Name:Mikuro Posts: 457 Joined: 15 Jun 2002
Subject: Cool....and kind of scary

It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.

Close Name:Small White Car Posts: 1960 Joined: 02 Jul 2004
Subject:

Quote
Mikuro wrote:
It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.


I have heard that the little green light by the camera in the iMacs is hard wired in such a way that it HAS to be on for the camera to work. So it would be impossible to hack...it's a hardware thing.

Now, that being said I can imagine a screen like this would be much more complicated and such a device might be impossible on a device like that.

But anyway, I thought I'd put your mind at ease about the current cameras out there at least.

Close Name:Rainy Day Posts: 607 Joined: 07 Jun 2005
Subject: Apple’s been thinking about this for years

Well, reminds me of three concept pieces by Apple: Parkbench Navigator, Knowledge Navigator and Future Shock, the latter two being discussed here.

Close Name:Mikuro Posts: 457 Joined: 15 Jun 2002
Subject: Re: Cool....and kind of scary

Quote
Small White Car wrote:
I have heard that the little green light by the camera in the iMacs is hard wired in such a way that it HAS to be on for the camera to work. So it would be impossible to hack...it's a hardware thing.

Now, that being said I can imagine a screen like this would be much more complicated and such a device might be impossible on a device like that.

But anyway, I thought I'd put your mind at ease about the current cameras out there at least.

Good to know. With the current stuff it's not so much of an issue, because you can just cover the lens or turn it off. I assume there would be some hardware "off" switch on the new thing, too, but...well, obviously the technology isn't made to be kept off all the time. If this technology becomes a vital part of the UI as Mr. Kheit suggests, then it would really have to be on whenever you use your computer. That's what makes it scary.

Close Name:Guest
Subject: Wow!

Quote
Rainy Day wrote:
Well, reminds me of three concept pieces by Apple:.


Wow, the future shock video is incredible for 1988! What's interesting is that some of this has come to pass. iSight isn't too far from this. But you're right. They talk right into the display and the display does track user gestures. It looks like MINORITY REPORT ripped off this 1988 apple video for its ideas on gestures.

Thanks for the link!

I love the funky 80's classes and hairdos!

Close Name:Guest
Subject: That's not what it's for!

So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.
No. The point of this is the holy grail of user interphase. The screen will be able to see what you are looking at, either by retinal reflection or by triangulation of your pupils. The mouse becomes mute. Imagine being able to drag files with your eyes, navigate with eye gestures, etc. Add voice recognition and you are now living Star Trek.
JH

Close Name:Biff Posts: 1479 Joined: 08 Apr 2004
Subject:

Quote
Guest wrote:
So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.
LOL. A simple chip to assemble the image before sending it to the main system would cost what like $5?

Close Name:bryson Posts: 79 Joined: 05 Mar 2002
Subject: ditto

Quote
Mikuro wrote:
It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.


I thought the exact same thing when I first read it. Cool technology though.

Close Name:bryson Posts: 79 Joined: 05 Mar 2002
Subject: LOL

"The mouse becomes mute."

Yeah, shut that damn mouse up.

Close Name:Guest
Subject: Not a scanner...

Problem is, as a scanner it would be WAY too low a resolution. It would work as a camera pretty well however.

Close Name:Guest
Subject: Check out the Dell XPS Mobile Concept!

This has beaten Apple to the punch. Might not be the same tech, but the camera is supposedly in the screen. go to www.dell.com, then go to gaming, and finally to 2006 Consumer Electronics Show. This is more like what I was hoping from Apple than a repackaged Powerbook with a lame-pro name.

Close Name:Guest
Subject: Touch screen

Could be used as a touch screen as well? Neat idea. Wonder how it will affect picture quality?

Close Name:Guest
Subject: No Scoop...but interesting take

Sorry, it's not a scoop:

http://www.macsimumnews.com/index.php/archive/new_apple_patent_integrated_sensing_display/

On the other hand, it's an interesting take of this patent, John.

Close Name:jimothy Posts: 612 Joined: 04 Jun 2004
Subject: Dell

Quote
Guest wrote:
This has beaten Apple to the punch. Might not be the same tech, but the camera is supposedly in the screen. go to www.dell.com, then go to gaming, and finally to 2006 Consumer Electronics Show. This is more like what I was hoping from Apple than a repackaged Powerbook with a lame-pro name.

What a wonderfully off-topic post! It includes a built-in camera; like the MacBook Pro or iMac. It does not contain anything at all like what this patent describes.

Close Name:Rainy Day Posts: 607 Joined: 07 Jun 2005
Subject: A solved problem already

Quote
Guest wrote:
So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.


This is essentially how a digital camera forms an image, so this is no big deal. In fact recent advances in the technology promise incredible performance gains.

Close Name:Guest
Subject:

Quote
Guest wrote:
The mouse becomes mute.
Hey, we just got more than one button. Leave that mouse alone!

Close Name:Guest
Subject: apple files patent for looking glass

You could also use this as a scanner, think what that could do for laptops.

Close Name:Rainy Day Posts: 607 Joined: 07 Jun 2005
Subject: Parkbench Navigator

Quote
Guest wrote:
You could also use this as a scanner, think what that could do for laptops.

See Parkbench Navigator above.

Close Name:Guest
Subject: COMPLETELY OUT OF 1984

It's really sad, and kind of scary, that Apple would be "first" to introduce this technology.

Can't you see a right-wing gov't passing legislation requiring such a sensor to be on and Internet-accessible, as long as the computer is being used, "for National Security purposes"... "to identify and track terrorists"...etc.?

Would that be out of character for the Cheney/Rove/Rumsfeld administration of today?

Close Name:gslusher Posts: 2088 Joined: 13 Nov 2002
Subject:

Quote
Rainy Day wrote:
Quote
Guest wrote:
So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.


This is essentially how a digital camera forms an image, so this is no big deal. In fact recent advances in the technology promise incredible performance gains.



Uh, no. The camera has a LENS, which focusses an image on the sensor array. Think of it this way: take a piece of photographic film and expose it for a short time to the light from a room. Would it pick up an image of the room? No. In order to create a recognizable image, each sensor must pick up light from only one small part of the view. That can be done with a lens or by putting the sensors at the bottom of a "deep" (relatively) hole, lined in black, so that light coming from directions other than straight ahead is blocked.

Close Name:Guest
Subject: Tim Coughlin

Why would the resolution be too low for use as a scanner? It seems to me thats where this tech would be best implemented, because I don't see how this could be a camera without a lens. Plus in scanning mode the screen would turn white just as a copier does now. An image sensor by each pixel sounds like a pretty decent resolution for scanning on a 15" powerbook, certainly good enough for text. Couple that with some image to text software and you have a killer app in my eyes, especially if the computer takes some form of tablet.

Tim Coughlin
http://timcoughlin.typepad.com

Close Name:Guest
Subject: iSight and any Mic hackable now

Quote
Small White Car wrote:
Quote
Mikuro wrote:
It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.


I have heard that the little green light by the camera in the iMacs is hard wired in such a way that it HAS to be on for the camera to work. So it would be impossible to hack...it's a hardware thing.

Now, that being said I can imagine a screen like this would be much more complicated and such a device might be impossible on a device like that.

But anyway, I thought I'd put your mind at ease about the current cameras out there at least.


----------------------
From what I understand it IS possible to hack microphones/cams at least as late as OSX 10.3. If a hacker were to modify your machines user privileges for user nobody, (dangerously easy if the machine in question is in single user mode)... You may well be on candid camera ALREADY.

This screen thing is EVEN worse. If you want a cam offer it on the side. If more and more of Apple's offerings "package" built in cams I'll probably opt out and use my old G4 with a CRT. LoL

Close Name:Steve W Posts: 482 Joined: 22 Nov 2002
Subject:

Quote
Guest wrote:
It's really sad, and kind of scary, that Apple would be "first" to introduce this technology.
Agreed. You'd think they could use all that talent and brainpower for something useful.

Quote
Can't you see a right-wing gov't passing legislation requiring such a sensor to be on and Internet-accessible, as long as the computer is being used, "for National Security purposes"... "to identify and track terrorists"...etc.?

Would that be out of character for the Cheney/Rove/Rumsfeld administration of today?
Yes, I can, and no, it would not. It would be completely in character for that cabal of criminals, and the even worse thing is that the entire right-wing neo-con chorus would chime in and agree. Anyone who disagrees would promptly be labeled as unpatriotic and accused of aiding and abetting terrorism. The phrase "I have nothing to hide" would be the automatic response from the lemmings.

I don't like having the built-in iSight on iMacs, and, no I don't have one. This would be worse, much worse.

Close Name:Guest
Subject: Imagine what this would do for malware & spyware!!!

Not only would criminals ba able to get your personal information, they could also ger a screen shot of you to help them fake an ID!

Comment on this Article


You cannot edit your comments.   You cannot delete your comments.

Comments are currently closed. Please email the author instead.


Recent Headlines - Updated July 5th

Fri, 10:29 AM
News - Apple Warns of Learning Interchange Security Breach
7:30 AM
News - Happy Fourth of July!
Thu, 6:07 PM
TMO Scoop - Psystar Moves to Drop Bankruptcy Ahead of Apple Legal Battle
5:37 PM
News - Uncomfirmed Reports Say Apple & Nvidia On The Outs
4:57 PM
News - Microsoft Sick Over Barf Ad
4:09 PM
Product News - KRK Ships R6 Passive Studio Monitor for Recording
3:45 PM
John Martellaro's Blog - Particle Debris (week ending 7/2)  Juiced, Joost and Goosed
3:12 PM
Product News - ExactScan 2 Pro Released
1:56 PM
Deal Brothers - Apple TV with 160GB Hard Drive:  $324.00 Delivered
12:46 PM
TMO Appearances - TMO Appearances Jeff Gamet Shares iPhone Apps on MacJury
10:41 AM
Product News - Art Text 2.2 Adds New Templates, Layer Options [Updated]
10:04 AM
Hot Forum Topic - Deciphering Mac Sales

The Mac Observer Reader Specials

  • __________
  • Buy Stuff, Support TMO!
  • Podcast: Mac Geek Gab
  • Podcast: Apple Weekly Report
  • TMO on Twitter!