The Devil's Advocate - Mac UI Ain't All That: The Future & History of the User Interface (with Snazzy New UI Videos)
- August 15th, 2006
The current state-of-the-art User Interface (UI) we've been enjoying has remained largely stagnant since the 1980s. The greatest innovation that has been recently released is based on video card layering/buffering techniques like Apple's Expose. But, there is a large change coming. Rev 2 of the UI will be based on multiple gestures and more directly involve human interaction. Apple is clearly working in the area as some of the company's patent filings demonstrate. Nevertheless, these videos might make Mac (and Windows) users experience a huge case of UI envy, as a lot of UI development (in XGL in particular) makes the current Mac UI seem creaky and old fashioned.
I've been collecting these links to cool new UIs for a while (yea, I know, what a geek). Despite its geekiness, I thought some of you might enjoy them as well. It seems there is a good chance that multi-gesture/multi-input (MGMI) UIs will be in our future at least on some level (apparently they will be accompanied by no dearth of techno music).
Basically the rest of the article has a bunch of links, and brief descriptions to those links, to cool videos of various UI that include: Wobbly Window UIs (WWUIs), Multi-touch UIs, Mixed Physical UIs, Brain UIs, Art tools, and even a little bit of history on the UI. Enjoy!
Wobbly Window UIs
The "wobbly" UIs, largely represented by XGL, all seem to use layering techniques. All this layering of UI elements lets you see the desktop UI from more vantages. Wobbly Window UIs seemed to have started at SUN with Project Looking Glass (although they may have originated elsewhere, I'm unsure). It seems that Project Looking Glass "inspired" Apple's Exposé. Others have gone on to build on both SUN's and Apple's progress with the current state-of-the-wobbly-art with XGL. The 'gist' of how these UIs seem to work is they put a bitmap into a buffered backing plane and allow you to quickly move the resulting 2D bitmap layers through 3D space. Modern 3D video cards then allow you to move these relatively simple bit-mapped textured planes through 3D space rather easily.
- SUN Project Looking Glass seemed to get the ball rolling.
- BumpTop synthesizes a lot of work done with the notion of document piles. You can see that this technology, if coupled with multi-touch UIs will be quite useful.
- Multi-display XGL demo.
- XGL comparison with Mac OS X and Vista.
- Yet another XGL demo with interesting DVD interaction.
- Novell presentation of an XGL demo (long, thorough, and a bit dull).
Multi-touch UIs are much like the UIs on a touch pad or a tablet UI. You get to touch things directly on the display surface. However, this technology registers more than one touch, and it is capable of interpreting gestures as well. In reality, some of these UIs do not even require touching the surface, but just interpret your gestures near the surface. Multi-touMulti-touch UIs, along with mixed physical UIs, seems to be the direction where UI development is headed.
Multi-touch UI links:
- Perhaps an early form of this was first seen with this line of gesture keyboards that mapped to regular desktop functions. They have since ceased operation. Rumors abound; among them that Apple may have bought them out.
- One of the first Multi-touch UIs I saw (on OS X as well, though it mysteriously went "poof" a while ago).
- Here is a good intro to multi-touch UIs.
- Here is another version of the above multi-touch UI; it has more demo flash (and has been shown all over the place).
- I've seen this one in malls already.
- This is just a 42inch touch-screen, and not a multi-touch interface. However, I wanted to show that these new interfaces may force computers to take on new form factors and actually become our desktop surface.
- For you WoW fans.
Mixed Physical UIs
Mixed Physical UIs build upon multi-touch UIs, but they introduce physical objects into a multi-touch input plane. Some of the more advanced Mixed Physical UIs identify the types of physical objects that are placed onto the input surface, and then let you manipulate the computer environment when you manipulate the physical objects.
Mixed Physical UI links:
- Mixed reality interface.
- Sense table.
- Virtual art show.
- (Mostly useless) Bar tech!
- This one is a bit dull up until about the 5:13 second mark; a physically morphing table.
Brain UIs employ techniques of reading brain impulses in order to control a computer.
Brain UI links:
- Brain computer interface.
- This head gear is similar to the ones we used back in my research lab and developer days (when I was still developing using NeXT computers).
Art tools use UI techniques, already discussed above, to further artistic ends.
Art tools links:
Sadly, a great many people in the computer field have a pathetic sense (or rather ignorance) of history. They are pompous and narcissistic enough to ignore the great contributions of past geniuses. My hope is that more will watch some of these historical videos and learn from these innovators.
Perhaps, if more learned from the great computer science innovators of the past (who managed to make systems using computers with 192KB of memory that rival the responsiveness and features of modern systems having orders of magnitude more processing power), then later generations of computer scientists might adopt their elegant design philosophies, rather than the current philosophies responsible for the ill-thought out and implemented bloat software we "enjoy" today. It might be time to add a "History of Computers" class to the computer science curriculum so as to give new practitioners this much needed sense of history.
For example, it seems rare to find someone in the computer field who knows about the contributions of Ivan Sutherland and Douglas Engelbart. Ivan Sutherland came up with one of the earliest graphical interfaces (and threw in objected oriented programming techniques for good measure); he did this all in one year in the development of Sketchpad circa 1963. Amazing.
For a good historical overview of user interfaces and more, check out this video with Alan Key; it's fantastic.
Douglas Engelbart's demonstration of the online system, NLS, is THE demo that started them all. Most think that Apple/Microsoft stole their UI ideas from Xerox, but in reality, Douglas Engelbart and his team at Stanford made most of the conventions responsible for current computing (as well as current UIs). This is really the mother of all computer demos; Steve Jobs and others have basically copied this format, but no demo has ever equalled the thermonuclear impact of the NLS demo. Keep in mind, it was made in 1968! Douglas Engelbart is known as having invented the mouse, which is true, but that's a bit of a backhanded compliment as he did so much more. In reality, he and his team came up with our current paradigm of computer interactions, abstractions and development. It is beyond bizarre that he is not commonly credited with more.
Although a lot has come from Mr. Engelbart's work, I think he is actually disappointed with our current state of the art. I don't think he believed we should make UIs that are "simple" (or dumbed down), but instead, he was trying to develop UIs that empower us to embrace and engage in complexity; blame Jobs and team for our going astray with the one button mouse et al. Anyway, this is a long video and is somewhat tedious to those without a computer science background or sense of history as the technologies are not as whizbang by today's standards (and no techno track to keep you distracted). For its time, however, this was incredible; after this demo, many at the presentation left not believing the technology was real. Keep in mind this was achieved on a 192 Kilobyte (not Megabyte, not Gigabyte, but Kilobyte!) time share computer (that supported 20 users) with 0.5 MIPS of processing power.
This video is really the precursor to all modern demos and systems. In so many ways, it's frightening. Just setting up the demo was a technological marvel; it was projected on a huge display (using a very rare projector for the time) that they hacked to work with NLS; they even conducted a video teleconference with custom microwave based modem communications that they "just whipped up" to enable the demo.
How Douglas Engelbart, Ivan Sutherland, Doug Ross, and others have not collectively received a Nobel Prize for bringing forth modern computing is a mystery.
Xerox Smalltalk environment (one of the Xerox UIs from which everyone "borrowed").
I get the feeling we are about to enter a computer user interface renaissance. My guess is that somewhere in the next 2 years (and maybe as soon as with Mac OS X 10.5), we'll see the revitalization of the entire computer field because of UI advances. And with any UI renaissance, we will also see increases in new hardware sales supporting the new paradigms, which in turn will lead to further gains in personal productivity levels. If you find some other cool UI videos, share by adding links in the comments below.
is an attorney. Please don't hold that against him. This work does not necessarily reflect the views and/or opinions of The Mac Observer, any third parties, or even John for that matter. No assertions of fact are being made, but rather the reader is simply asked to consider the possibilities.
Most Recent Columns From The Devil's Advocate
- Mac UI Ain't All That: The Future & History of the User Interface - August 15th
- MacWindows: The New Trojan Wars - April 4th
- TMO Scoop: Apple Files Patent for Looking Glass - January 25th
The Devil's Advocate Archives
- Tue, 4:11 PM
- Apple TV+ Signs First-Look Deal with Maya Rudolph's Animal Pictures Following 'Loot' Premiere
- Tue, 3:31 PM
- Tales From the Rumor Mill: What's Next for Apple TV Including New Chip, RAM, Siri Remote and More
- Tue, 3:28 PM
- Apple Watch ECG Technology Infringes on AliveCor’s Patents, ITC Rules
- Tue, 1:51 PM
- Russia Fines Big Tech Companies Over User Data and Apple May Be Next
- Tue, 12:38 PM
- Foxconn Offers Summer Employee Bonus to Increase iPhone 14 Production
- Tue, 8:47 AM
- Rumored iPad mini 7 With 120Hz ProMotion Display Unlikely
- Tue, 7:37 AM
- Apple Creative Projects Showcases Artist Quentin Jones Creating Mantra Collages on iPad
- Tue, 7:32 AM
- Apple Shuts Down Hermit Spyware Distributed as Fake Enterprise App on iOS Devices
- Mon, 4:18 PM
- How Apple Watch Mirroring Works on iOS 16
- Mon, 3:33 PM
- Apple Suppliers Begin Shipping iPhone 14 Components
- Mon, 3:29 PM
- Tesla CarPlay Workaround Update Now Offers Support for All Tesla Models
- Mon, 2:39 PM
- Trailer for Apple TV+ San Fransisco-Thriller 'Surface' Goes Live