The Mac Observer

Skip navigational links

The Devil's Advocate - TMO Scoop: Apple Files Patent for Looking Glass
by - January 25th, 2006

From the out of left-field department, on June 21, 2004, Apple filed for patent application no. 20060007222 for an "Integrated Sensing Display."   Of course just because Apple filed for a patent application does not mean the company will release a product; but in this case, let's hope they do.

Basically this invention morphs a camera with a screen.  And I'm not talking about putting an iSight above the screen.  The screen's surface area is the camera. Each pixel would have an image capture element. So I theorize you could just put a piece of paper in front of the screen, and it could just appear like a reverse-mirror-dissolve onto the screen.  It could be quite the intuitive and "magic" way to capture images of all kinds.

Mirror mirror on the wall...

It also fixes a problem that cannot be fixed with an iSight as this panel would let  "a user ... maintain eye contact with someone on screen because the camera is 'in' the screen." 

The patent application shows that an image sensor would be located next to each pixel:


Display screen (300) with image sensors (305) forming a sensor array

It's not clear what the effective resolution of each image capture sensor would be, however, even if each element had some marginal resolution, the effective resolution of a 30" LCD capture screen would be very high.

Cost & Applications?

The application notes that this technology could be integrated into many display technologies and form factors including CRTs, Plasmas, LCDs and PDAs.


Individual image sensor

It's not clear just how much it would cost to use millions of image capture elements per screen; it likely would be cost prohibitive for consumers, at least initially. Also, I'm not sure how useful this would be to the average user. Indeed, the patent application notes that such a device could be useful for medical imaging.  

Anyway, I'm just happy to see something so creative come out of any company. If the display "sees" you, and can sense positions, new user interface advances like gesture-sensitive touch-screens (check out these videos) become more likely to come our way. In a world overstuffed with mediocrity, sometimes its enough if something is just cool and fun for its own sake.

is an attorney. Please don't hold that against him. This work does not necessarily reflect the views and/or opinions of The Mac Observer, any third parties, or even John for that matter. No assertions of fact are being made, but rather the reader is simply asked to consider the possibilities.

You can send your comments directly to me, or you can also post your comments below.

Most Recent Columns From The Devil's Advocate

The Devil's Advocate Archives

Observer Comments

Show: Subjects Only | Full Comments
Close Name:Mikuro Posts: 457 Joined: 15 Jun 2002
Subject: Cool....and kind of scary

It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.

Close Name:Small White Car Posts: 1960 Joined: 02 Jul 2004
Subject:

Quote
Mikuro wrote:
It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.


I have heard that the little green light by the camera in the iMacs is hard wired in such a way that it HAS to be on for the camera to work. So it would be impossible to hack...it's a hardware thing.

Now, that being said I can imagine a screen like this would be much more complicated and such a device might be impossible on a device like that.

But anyway, I thought I'd put your mind at ease about the current cameras out there at least.

Close Name:Rainy Day Posts: 607 Joined: 07 Jun 2005
Subject: Apple’s been thinking about this for years

Well, reminds me of three concept pieces by Apple: Parkbench Navigator, Knowledge Navigator and Future Shock, the latter two being discussed here.

Close Name:Mikuro Posts: 457 Joined: 15 Jun 2002
Subject: Re: Cool....and kind of scary

Quote
Small White Car wrote:
I have heard that the little green light by the camera in the iMacs is hard wired in such a way that it HAS to be on for the camera to work. So it would be impossible to hack...it's a hardware thing.

Now, that being said I can imagine a screen like this would be much more complicated and such a device might be impossible on a device like that.

But anyway, I thought I'd put your mind at ease about the current cameras out there at least.

Good to know. With the current stuff it's not so much of an issue, because you can just cover the lens or turn it off. I assume there would be some hardware "off" switch on the new thing, too, but...well, obviously the technology isn't made to be kept off all the time. If this technology becomes a vital part of the UI as Mr. Kheit suggests, then it would really have to be on whenever you use your computer. That's what makes it scary.

Close Name:Guest
Subject: Wow!

Quote
Rainy Day wrote:
Well, reminds me of three concept pieces by Apple:.


Wow, the future shock video is incredible for 1988! What's interesting is that some of this has come to pass. iSight isn't too far from this. But you're right. They talk right into the display and the display does track user gestures. It looks like MINORITY REPORT ripped off this 1988 apple video for its ideas on gestures.

Thanks for the link!

I love the funky 80's classes and hairdos!

Close Name:Guest
Subject: That's not what it's for!

So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.
No. The point of this is the holy grail of user interphase. The screen will be able to see what you are looking at, either by retinal reflection or by triangulation of your pupils. The mouse becomes mute. Imagine being able to drag files with your eyes, navigate with eye gestures, etc. Add voice recognition and you are now living Star Trek.
JH

Close Name:Biff Posts: 1479 Joined: 08 Apr 2004
Subject:

Quote
Guest wrote:
So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.
LOL. A simple chip to assemble the image before sending it to the main system would cost what like $5?

Close Name:bryson Posts: 79 Joined: 05 Mar 2002
Subject: ditto

Quote
Mikuro wrote:
It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.


I thought the exact same thing when I first read it. Cool technology though.

Close Name:bryson Posts: 79 Joined: 05 Mar 2002
Subject: LOL

"The mouse becomes mute."

Yeah, shut that damn mouse up.

Close Name:Guest
Subject: Not a scanner...

Problem is, as a scanner it would be WAY too low a resolution. It would work as a camera pretty well however.

Close Name:Guest
Subject: Check out the Dell XPS Mobile Concept!

This has beaten Apple to the punch. Might not be the same tech, but the camera is supposedly in the screen. go to www.dell.com, then go to gaming, and finally to 2006 Consumer Electronics Show. This is more like what I was hoping from Apple than a repackaged Powerbook with a lame-pro name.

Close Name:Guest
Subject: Touch screen

Could be used as a touch screen as well? Neat idea. Wonder how it will affect picture quality?

Close Name:Guest
Subject: No Scoop...but interesting take

Sorry, it's not a scoop:

http://www.macsimumnews.com/index.php/archive/new_apple_patent_integrated_sensing_display/

On the other hand, it's an interesting take of this patent, John.

Close Name:jimothy Posts: 612 Joined: 04 Jun 2004
Subject: Dell

Quote
Guest wrote:
This has beaten Apple to the punch. Might not be the same tech, but the camera is supposedly in the screen. go to www.dell.com, then go to gaming, and finally to 2006 Consumer Electronics Show. This is more like what I was hoping from Apple than a repackaged Powerbook with a lame-pro name.

What a wonderfully off-topic post! It includes a built-in camera; like the MacBook Pro or iMac. It does not contain anything at all like what this patent describes.

Close Name:Rainy Day Posts: 607 Joined: 07 Jun 2005
Subject: A solved problem already

Quote
Guest wrote:
So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.


This is essentially how a digital camera forms an image, so this is no big deal. In fact recent advances in the technology promise incredible performance gains.

Close Name:Guest
Subject:

Quote
Guest wrote:
The mouse becomes mute.
Hey, we just got more than one button. Leave that mouse alone!

Close Name:Guest
Subject: apple files patent for looking glass

You could also use this as a scanner, think what that could do for laptops.

Close Name:Rainy Day Posts: 607 Joined: 07 Jun 2005
Subject: Parkbench Navigator

Quote
Guest wrote:
You could also use this as a scanner, think what that could do for laptops.

See Parkbench Navigator above.

Close Name:Guest
Subject: COMPLETELY OUT OF 1984

It's really sad, and kind of scary, that Apple would be "first" to introduce this technology.

Can't you see a right-wing gov't passing legislation requiring such a sensor to be on and Internet-accessible, as long as the computer is being used, "for National Security purposes"... "to identify and track terrorists"...etc.?

Would that be out of character for the Cheney/Rove/Rumsfeld administration of today?

Close Name:gslusher Posts: 2088 Joined: 13 Nov 2002
Subject:

Quote
Rainy Day wrote:
Quote
Guest wrote:
So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.


This is essentially how a digital camera forms an image, so this is no big deal. In fact recent advances in the technology promise incredible performance gains.



Uh, no. The camera has a LENS, which focusses an image on the sensor array. Think of it this way: take a piece of photographic film and expose it for a short time to the light from a room. Would it pick up an image of the room? No. In order to create a recognizable image, each sensor must pick up light from only one small part of the view. That can be done with a lens or by putting the sensors at the bottom of a "deep" (relatively) hole, lined in black, so that light coming from directions other than straight ahead is blocked.

Close Name:Guest
Subject: Tim Coughlin

Why would the resolution be too low for use as a scanner? It seems to me thats where this tech would be best implemented, because I don't see how this could be a camera without a lens. Plus in scanning mode the screen would turn white just as a copier does now. An image sensor by each pixel sounds like a pretty decent resolution for scanning on a 15" powerbook, certainly good enough for text. Couple that with some image to text software and you have a killer app in my eyes, especially if the computer takes some form of tablet.

Tim Coughlin
http://timcoughlin.typepad.com

Close Name:Guest
Subject: iSight and any Mic hackable now

Quote
Small White Car wrote:
Quote
Mikuro wrote:
It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.


I have heard that the little green light by the camera in the iMacs is hard wired in such a way that it HAS to be on for the camera to work. So it would be impossible to hack...it's a hardware thing.

Now, that being said I can imagine a screen like this would be much more complicated and such a device might be impossible on a device like that.

But anyway, I thought I'd put your mind at ease about the current cameras out there at least.


----------------------
From what I understand it IS possible to hack microphones/cams at least as late as OSX 10.3. If a hacker were to modify your machines user privileges for user nobody, (dangerously easy if the machine in question is in single user mode)... You may well be on candid camera ALREADY.

This screen thing is EVEN worse. If you want a cam offer it on the side. If more and more of Apple's offerings "package" built in cams I'll probably opt out and use my old G4 with a CRT. LoL

Close Name:Steve W Posts: 482 Joined: 22 Nov 2002
Subject:

Quote
Guest wrote:
It's really sad, and kind of scary, that Apple would be "first" to introduce this technology.
Agreed. You'd think they could use all that talent and brainpower for something useful.

Quote
Can't you see a right-wing gov't passing legislation requiring such a sensor to be on and Internet-accessible, as long as the computer is being used, "for National Security purposes"... "to identify and track terrorists"...etc.?

Would that be out of character for the Cheney/Rove/Rumsfeld administration of today?
Yes, I can, and no, it would not. It would be completely in character for that cabal of criminals, and the even worse thing is that the entire right-wing neo-con chorus would chime in and agree. Anyone who disagrees would promptly be labeled as unpatriotic and accused of aiding and abetting terrorism. The phrase "I have nothing to hide" would be the automatic response from the lemmings.

I don't like having the built-in iSight on iMacs, and, no I don't have one. This would be worse, much worse.

Close Name:Guest
Subject: Imagine what this would do for malware & spyware!!!

Not only would criminals ba able to get your personal information, they could also ger a screen shot of you to help them fake an ID!

Comment on this Article


You cannot edit your comments.   You cannot delete your comments.

Comments are currently closed. Please email the author instead.


Recent Headlines - Updated November 20th

Fri, 7:07 PM
Games - Soccer Sim Championship Manager 2010 Released for Mac
6:47 PM
Games - EA Publishes Original Monopoly for iPhone
6:15 PM
News - Original Apple I on Ebay for $50K, w/Letter from Steve Jobs
6:11 PM
Games - New iPhone Games: Secret of the Lost Cavern Ep 1, New DJ Nights, More
5:47 PM
Games - Star Trek D-A-C Game Headed to the Mac Next Month
4:57 PM
Product News - TidBITS Releases “Take Control of Syncing Data in Snow Leopard”
4:26 PM
John Martellaro's Blog - Particle Debris (week ending 11/20) Stationery Pads Go Poof
2:59 PM
Free on iTunes - Musée du Louvre, Art Lite, SketchBook Mobile X and More.
1:50 PM
Deal Brothers - Acer P215H bmid 21.5” Widescreen LCD Monitor:  $139.99
11:24 AM
TMO Appearances - Jeff Gamet Shares More Holiday Gift Ideas on MacJury
10:43 AM
Product News - Cocktail 4.5 for Leopard Adds QuickLook Cache Clearing
10:06 AM
News - Hack Enables Mac OS X 10.6.2 on Netbooks

The Mac Observer Reader Specials

  • TypeStyler For Mac OS X is Now Shipping! Download The Free Fully Functional 60 Day Tryout at www.typestyler.com
  • RamJet Memory: Mac Pro 8-core 8GB Kit $249.99, 4GB Kits $149.99! Sale on MacBook and MacBook Pro 8GB kits $497.99! New MacBook DDR3 4GB for $109.99. iMac and Mac mini 4GB Kits for $109.99! 1TB SATA Hard Drives for $109.99! Click here
  • OWC: Plug & Play Hardware RAID up to 8.0TB. High Performance, Data Redundant Solutions. FireWire 800, FireWire 400, USB2, or eSATA. Hot Swappable Bays, Data Rates over 200MB/s. Click here
  • Poker Mac If you're using a Mac, then you've gotta check out Full Tilt Poker for Mac. This Full Tilt Poker bonus code does the unthinkable, it actually rewards!
  • For the latest Apple products use Ciao, a price comparison website, to find laptops like MacBook Air. Then find the best prices on MP3 players and use our comparison tool to evaluate mobile phones like the Apple iPhone.

  • Laptop Hardware Provided by TechRestore - Overnight Mac & iPod Repairs.
  • __________
  • Buy Stuff, Support TMO!
  • Podcast: Mac Geek Gab
  • Podcast: Apple Weekly Report
  • TMO on Twitter!