The Devil's Advocate - TMO Scoop: Apple Files Patent for Looking Glass
by
- January 25th, 2006
From the out of left-field department, on June 21, 2004, Apple filed for patent application no. 20060007222 for an "Integrated Sensing Display." Of course just because Apple filed for a patent application does not mean the company will release a product; but in this case, let's hope they do.
Basically this invention morphs a camera with a screen. And I'm not talking about putting an iSight above the screen. The screen's surface area is the camera. Each pixel would have an image capture element. So I theorize you could just put a piece of paper in front of the screen, and it could just appear like a reverse-mirror-dissolve onto the screen. It could be quite the intuitive and "magic" way to capture images of all kinds.
Mirror mirror on the wall...
It also fixes a problem that cannot be fixed with an iSight as this panel would let "a user ... maintain eye contact with someone on screen because the camera is 'in' the screen."
The patent application shows that an image sensor would be located next to each pixel:

Display screen (300) with image sensors (305) forming a sensor array
It's not clear what the effective resolution of each image capture sensor would be, however, even if each element had some marginal resolution, the effective resolution of a 30" LCD capture screen would be very high.
Cost & Applications?
The application notes that this technology could be integrated into many display technologies and form factors including CRTs, Plasmas, LCDs and PDAs.

Individual image sensor
It's not clear just how much it would cost to use millions of image capture elements per screen; it likely would be cost prohibitive for consumers, at least initially. Also, I'm not sure how useful this would be to the average user. Indeed, the patent application notes that such a device could be useful for medical imaging.
Anyway, I'm just happy to see something so creative come out of any company. If the display "sees" you, and can sense positions, new user interface advances like gesture-sensitive touch-screens (check out these videos) become more likely to come our way. In a world overstuffed with mediocrity, sometimes its enough if something is just cool and fun for its own sake.
is an attorney. Please don't hold that against him. This work does not necessarily reflect the views and/or opinions of The Mac Observer, any third parties, or even John for that matter. No assertions of fact are being made, but rather the reader is simply asked to consider the possibilities.
You can send your comments directly to me, or you can also post your comments below.
Most Recent Columns From The Devil's Advocate
- Mac UI Ain't All That: The Future & History of the User Interface - August 15th
- MacWindows: The New Trojan Wars - April 4th
- TMO Scoop: Apple Files Patent for Looking Glass - January 25th
The Devil's Advocate Archives
Observer Comments
Thu Jan 26, 2006 12:59 am Subject: Cool....and kind of scary
It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.
QuoteMikuro wrote:
It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.
I have heard that the little green light by the camera in the iMacs is hard wired in such a way that it HAS to be on for the camera to work. So it would be impossible to hack...it's a hardware thing.
Now, that being said I can imagine a screen like this would be much more complicated and such a device might be impossible on a device like that.
But anyway, I thought I'd put your mind at ease about the current cameras out there at least.
Thu Jan 26, 2006 1:25 am Subject: Apple’s been thinking about this for years
Well, reminds me of three concept pieces by Apple: Parkbench Navigator, Knowledge Navigator and Future Shock, the latter two being discussed here.
Thu Jan 26, 2006 1:30 am Subject: Re: Cool....and kind of scary
QuoteSmall White Car wrote:
I have heard that the little green light by the camera in the iMacs is hard wired in such a way that it HAS to be on for the camera to work. So it would be impossible to hack...it's a hardware thing.
Now, that being said I can imagine a screen like this would be much more complicated and such a device might be impossible on a device like that.
But anyway, I thought I'd put your mind at ease about the current cameras out there at least.
Good to know. With the current stuff it's not so much of an issue, because you can just cover the lens or turn it off. I assume there would be some hardware "off" switch on the new thing, too, but...well, obviously the technology isn't made to be kept off all the time.
QuoteRainy Day wrote:
Well, reminds me of three concept pieces by Apple:.
Wow, the future shock video is incredible for 1988! What's interesting is that some of this has come to pass. iSight isn't too far from this. But you're right. They talk right into the display and the display does track user gestures. It looks like MINORITY REPORT ripped off this 1988 apple video for its ideas on gestures.
Thanks for the link!
I love the funky 80's classes and hairdos!
So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.
No. The point of this is the holy grail of user interphase. The screen will be able to see what you are looking at, either by retinal reflection or by triangulation of your pupils. The mouse becomes mute. Imagine being able to drag files with your eyes, navigate with eye gestures, etc. Add voice recognition and you are now living Star Trek.
JH
QuoteLOL. A simple chip to assemble the image before sending it to the main system would cost what like $5?Guest wrote:
So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.
QuoteMikuro wrote:
It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.
I thought the exact same thing when I first read it. Cool technology though.
This has beaten Apple to the punch. Might not be the same tech, but the camera is supposedly in the screen. go to www.dell.com, then go to gaming, and finally to 2006 Consumer Electronics Show. This is more like what I was hoping from Apple than a repackaged Powerbook with a lame-pro name.
Sorry, it's not a scoop:
http://www.macsimumnews.com/index.php/archive/new_apple_patent_integrated_sensing_display/
On the other hand, it's an interesting take of this patent, John.
QuoteGuest wrote:
This has beaten Apple to the punch. Might not be the same tech, but the camera is supposedly in the screen. go to www.dell.com, then go to gaming, and finally to 2006 Consumer Electronics Show. This is more like what I was hoping from Apple than a repackaged Powerbook with a lame-pro name.
What a wonderfully off-topic post! It includes a built-in camera; like the MacBook Pro or iMac. It does not contain anything at all like what this patent describes.
Thu Jan 26, 2006 3:19 pm Subject: A solved problem already
QuoteGuest wrote:
So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.
This is essentially how a digital camera forms an image, so this is no big deal. In fact recent advances in the technology promise incredible performance gains.
Thu Jan 26, 2006 6:58 pm Subject: Parkbench Navigator
It's really sad, and kind of scary, that Apple would be "first" to introduce this technology.
Can't you see a right-wing gov't passing legislation requiring such a sensor to be on and Internet-accessible, as long as the computer is being used, "for National Security purposes"... "to identify and track terrorists"...etc.?
Would that be out of character for the Cheney/Rove/Rumsfeld administration of today?
QuoteRainy Day wrote:QuoteGuest wrote:
So, this screen has like a gazillion eyes looking at you with fly like vision. It would take all of the processors' power to reconcile those images into one coherent image.
This is essentially how a digital camera forms an image, so this is no big deal. In fact recent advances in the technology promise incredible performance gains.
Uh, no. The camera has a LENS, which focusses an image on the sensor array. Think of it this way: take a piece of photographic film and expose it for a short time to the light from a room. Would it pick up an image of the room? No. In order to create a recognizable image, each sensor must pick up light from only one small part of the view. That can be done with a lens or by putting the sensors at the bottom of a "deep" (relatively) hole, lined in black, so that light coming from directions other than straight ahead is blocked.
Why would the resolution be too low for use as a scanner? It seems to me thats where this tech would be best implemented, because I don't see how this could be a camera without a lens. Plus in scanning mode the screen would turn white just as a copier does now. An image sensor by each pixel sounds like a pretty decent resolution for scanning on a 15" powerbook, certainly good enough for text. Couple that with some image to text software and you have a killer app in my eyes, especially if the computer takes some form of tablet.
Tim Coughlin
http://timcoughlin.typepad.com
QuoteSmall White Car wrote:QuoteMikuro wrote:
It's certainly cool, but I can't help thinking of 1984. It's awfully scary. I like being able to A) choose not to have a camera hooked up, or B) Just cover up the darned lense. I sure wouldn't want a virus-writer unknowingly accessing my monitor's "looking glass" (more a risk for Windows users, of course, but still). If the feature's there in the monitor, I don't know how such manipulation could really be avoided.
I have heard that the little green light by the camera in the iMacs is hard wired in such a way that it HAS to be on for the camera to work. So it would be impossible to hack...it's a hardware thing.
Now, that being said I can imagine a screen like this would be much more complicated and such a device might be impossible on a device like that.
But anyway, I thought I'd put your mind at ease about the current cameras out there at least.
----------------------
From what I understand it IS possible to hack microphones/cams at least as late as OSX 10.3. If a hacker were to modify your machines user privileges for user nobody, (dangerously easy if the machine in question is in single user mode)... You may well be on candid camera ALREADY.
This screen thing is EVEN worse. If you want a cam offer it on the side. If more and more of Apple's offerings "package" built in cams I'll probably opt out and use my old G4 with a CRT. LoL
QuoteAgreed. You'd think they could use all that talent and brainpower for something useful.Guest wrote:
It's really sad, and kind of scary, that Apple would be "first" to introduce this technology.
QuoteYes, I can, and no, it would not. It would be completely in character for that cabal of criminals, and the even worse thing is that the entire right-wing neo-con chorus would chime in and agree. Anyone who disagrees would promptly be labeled as unpatriotic and accused of aiding and abetting terrorism. The phrase "I have nothing to hide" would be the automatic response from the lemmings.Can't you see a right-wing gov't passing legislation requiring such a sensor to be on and Internet-accessible, as long as the computer is being used, "for National Security purposes"... "to identify and track terrorists"...etc.?
Would that be out of character for the Cheney/Rove/Rumsfeld administration of today?
I don't like having the built-in iSight on iMacs, and, no I don't have one. This would be worse, much worse.
Thu Feb 23, 2006 2:08 pm Subject: Imagine what this would do for malware & spyware!!!
Comments are currently closed. Please email the author instead.
Recent Headlines - Updated November 20th
- Fri, 7:07 PM
- Games - Soccer Sim Championship Manager 2010 Released for Mac
- 6:47 PM
- Games - EA Publishes Original Monopoly for iPhone
- 6:15 PM
- News - Original Apple I on Ebay for $50K, w/Letter from Steve Jobs
- 6:11 PM
- Games - New iPhone Games: Secret of the Lost Cavern Ep 1, New DJ Nights, More
- 5:47 PM
- Games - Star Trek D-A-C Game Headed to the Mac Next Month
- 4:57 PM
- Product News - TidBITS Releases “Take Control of Syncing Data in Snow Leopard”
- 4:26 PM
- John Martellaro's Blog - Particle Debris (week ending 11/20) Stationery Pads Go Poof
- 2:59 PM
- Free on iTunes - Musée du Louvre, Art Lite, SketchBook Mobile X and More.
- 1:50 PM
- Deal Brothers - Acer P215H bmid 21.5” Widescreen LCD Monitor: $139.99
- 11:24 AM
- TMO Appearances - Jeff Gamet Shares More Holiday Gift Ideas on MacJury
- 10:43 AM
- Product News - Cocktail 4.5 for Leopard Adds QuickLook Cache Clearing
- 10:06 AM
- News - Hack Enables Mac OS X 10.6.2 on Netbooks
The Mac Observer Reader Specials
- TypeStyler For Mac OS X is Now Shipping! Download The Free Fully Functional 60 Day Tryout at www.typestyler.com
RamJet Memory: Mac Pro 8-core 8GB Kit $249.99, 4GB Kits $149.99! Sale on MacBook and MacBook Pro 8GB kits $497.99! New MacBook DDR3 4GB for $109.99. iMac and Mac mini 4GB Kits for $109.99! 1TB SATA Hard Drives for $109.99! Click here
OWC: Plug & Play Hardware RAID up to 8.0TB. High Performance, Data Redundant Solutions. FireWire 800, FireWire 400, USB2, or eSATA. Hot Swappable Bays, Data Rates over 200MB/s. Click here
If you're using a Mac, then you've gotta check out Full Tilt Poker for Mac. This Full Tilt Poker bonus code does the unthinkable, it actually rewards!For the latest Apple products use Ciao, a price comparison website, to find laptops like MacBook Air. Then find the best prices on MP3 players and use our comparison tool to evaluate mobile phones like the Apple iPhone.
Laptop Hardware Provided by TechRestore - Overnight Mac & iPod Repairs.

