| This article serves two purposes. It is a discussion | | | | seeing more 3D incursions. Operating systems and |
| about the future of computer interfaces; and it is a | | | | applications are beginning to capitalize on what 3D has |
| means by which I can purge myself of thoughts that | | | | to offer. The precise nature of how and where 3D |
| have been accumulating on this topic for quite a few | | | | can best be incorporated is an open question, and a |
| years. Even if it fails as intelligent discourse on the first, | | | | framework to evaluate these questions seems |
| it will have succeeded in the second. Previously the title | | | | appropriate. As a rough starting point it seems |
| was Where Are Computer Interfaces Going? but | | | | reasonable to divide the attempts into two broad |
| after writing it I noticed a significant number of | | | | categories: those that are trying to simulate the |
| predictive passages and decided to be bold and move | | | | physical world and those that prefer more abstract |
| the "are". Of course now I feel obliged to add a | | | | representations. If you'll indulge me, I'd like to call these |
| disclaimer. I admit right here, or at least in the next | | | | two approaches, respectively, the "Physical Simulation |
| sentence, that I don't know where computer interfaces | | | | Approach" (PSA) and the "Abstract Representation |
| are going. I don't know.With that out of the way, I'd like | | | | Approach" (ARA).Developers in the PSA camp are |
| to start, as many interfaces do, with the metaphor. In | | | | taking physical simulations and hanging applications, |
| the 80s and 90s successful interface design and an | | | | web sites, movies, and pictures on simulated walls. |
| appropriate metaphor were taken to be nearly | | | | Simulated desks have functional simulated calculators |
| synonymous. Although a good metaphor is important, it | | | | on them. And, perhaps, there is a simulated sun outside. |
| imposes unnecessary and artificial restrictions. So why | | | | It's all very familiar and comes with a nice minimal |
| is it so important? The best, perhaps only, reason is | | | | learning curve.The ARA camp are working on strange |
| familiarity. Unfortunately, familiarity comes at a cost: the | | | | visualization techniques to view complexity and |
| shorter learning curve can require speed and ability to | | | | patterns in large amounts of data. They have general |
| be sacrificed.Consider the ubiquitous desktop | | | | graphs floating around in space with links joining |
| metaphor. What is more powerful, the abstract | | | | concepts and words together in arbitrary ways. They |
| construct of a tree, or a single flat surface to place | | | | have nifty algorithms that filter the salient |
| your papers on? Well, a tree is. In fact it is so much | | | | characteristics of large data sets so you don't get |
| more powerful that it is the cornerstone of all modern | | | | overwhelmed. Their attempts are, by far, much harder |
| file systems. Trees are great, they impose an | | | | to describe with these mere words.In practice many |
| organizational order that is common in natural systems. | | | | attempts will combine aspects of both philosophies. I |
| General graphs are, perhaps, too general. DAGs | | | | suspect that successful attempts at a 3D interface will |
| (Directed Acyclic Graphs) are a good contender; | | | | have to balance these two extremes in appropriate |
| largely because of their acyclicness, but also because | | | | ways. Objects in a functional 3D interface should |
| they extend trees in a well defined way. I suspect that | | | | probably be represented with models that are familiar, |
| trees are so useful because we can't move | | | | just like the icons on your desktop are often imitations |
| backwards in time. Species speciate, languages | | | | of familiar real-world objects. This is a PSA property. |
| extend, and software bloats. To fight these is to fight | | | | On the other hand, tree-based organizational systems |
| the increasing entropy of the universe.Would it be a | | | | would be well advised. Very much an ARA |
| good idea not to allow folders within folders within | | | | concept.Text should always be view-plane aligned, as |
| folders just because it would be physically | | | | should images. This is one of those 2D features |
| cumbersome, and at some point impossible? Probably | | | | mentioned earlier. Images and text may be scaled, but |
| not. Do icons have a real-world counterpart? Not really. | | | | they should not present themselves at an angle. |
| Metaphors should be, and have been, taken only so | | | | Vertical and horizontal edges need to remain vertical |
| far.So what does the future hold? Will interfaces be | | | | and horizontal. Of course, these features are trivially |
| 3D? Will we be stuck with rectangles forever? I think | | | | present with your desktop interface as well.And |
| it's reasonable to say both have their place. People on | | | | there's an important lesson: build on the backs of |
| the 3D side think that we humans see, work, live, and | | | | giants. The desktop UI is successful for a reason, not |
| play in 3D. We don't. They say they can't wait until | | | | simply because it has a familiar analogue in the |
| there are fully 3D monitors that you can walk around. | | | | physical world, but rather because it behaves in that |
| Why? Our retinas, as well as birds whose eyes are | | | | same useful way that real desks behave. It takes |
| plastered on the side of their heads, are | | | | advantage of a well-established ability; spatial memory. |
| two-dimensional surfaces. Birds have flatter vision than | | | | You put something down and it stays there.Useable |
| we do, if not as Euclidean, because they don't have | | | | interfaces need a certain amount of persistence in |
| the benefit of the tiny bit of 3D depth perception a | | | | their structure. Having objects stay where you leave |
| predator gets by overlapping images. I've heard | | | | them is one good way to achieve persistence. Placing |
| graphics programmers explain that their 3D scene | | | | objects manually, whether on your desktop or in a 3D |
| was being projected onto a flat 2D screen and so it | | | | environment, takes advantage of spatial memory. We |
| was no longer really 3D. But consider this: everything | | | | can remember, in context, where we've left hundreds |
| you see in this world is like that. It all gets projected | | | | of objects (notwithstanding car keys; they get moved |
| onto our flat retinas. We just have really big brains. A | | | | around too much). You probably know where your |
| 3D scene is constructed in our mind regardless of | | | | camera is and where the light switches in your home |
| whether what we're viewing is on a flat computer | | | | are. By positioning objects manually you can give them |
| monitor or in that nether-world known as real life. In | | | | some context; perhaps by placing pictures of your |
| fact, most brains do a decent job of scene | | | | family to the left, and panoramic vistas to the right. |
| construction even with one eye closed. From 2D to | | | | Contextual clues help you remember.I've heard the |
| 3D. Impressive!People on the 2D side think that we | | | | assertion that adding a single extra dimension doesn't |
| humans see, work, live, and play in 2D. We do, after all, | | | | buy you much organizational power and that the |
| have flat retinas, like playing tennis on flat tennis courts, | | | | added navigational complexity isn't worth it. Others |
| and eat dinners from flat plates on flat tables. But we | | | | think that we need an n-dimensional space to do a |
| don't live in 2D. Our brains are really big. 1.3 litres big. | | | | good job. Aside from the obvious observation that we |
| More than enough dendrites, axons and other | | | | seem to exist in a macroscopically three-dimensional |
| brain-things to contain a nice 3D representation of the | | | | world (macroscopically was added just to keep any |
| world we live in. Clues to build the scene abound: | | | | physicists-who-may-know-better reading) and are |
| motion, foreshortening, and the aforementioned depth | | | | therefore good at 3D manipulations, there is evidence |
| perception.The truth is some things are better in 2D | | | | that the jump from 2 to 3 dimensions is of a more |
| and some 3D. Writing a letter? Use a desk. Put a flat | | | | fundamental significance. If you draw a bunch of dots |
| piece of paper on it. Want to file that letter away? | | | | on a piece of paper you will not be able to draw lines |
| Wouldn't it be cool if you could just let it hover in some | | | | joining the dots in all possible configurations unless the |
| large 3D organizational space? Here's what I think. | | | | lines cross (given some sufficiently large number of |
| Text: 2D | | | | dots. I think 5 might do it). However, once you hit three |
| Reading and Writing: 2D | | | | dimensions, all configurations are possible without |
| Organizing and Grouping: 3D | | | | crossings. Adding a fourth or fifth doesn't have any |
| Visualization of Data: Depends | | | | further beneficial effect. Admittedly there is some |
| It has occurred to me that 2D representations should | | | | hand-waving going on here; but the result has |
| be considered a feature of an interface. It's beneficial | | | | implications for some possible interface designs; and it |
| that text documents are lined up nicely for you in a | | | | points to using three dimensions.So why haven't |
| window. If head-or-eye-tracking hardware were more | | | | interfaces changed much in the last 20 years? One |
| widespread, we'd have software that could | | | | possibility is that the desktop is in some way an optimal |
| compensate for (single) users who are not directly in | | | | representation. More likely, however, is that it is simply |
| front of their screens. Imagine looking at your monitor | | | | a functional representation; no need to change when |
| from an angle but still having the text of this article | | | | change takes effort, right? We expect to be able to |
| appear flat. That would be a pretty neat feature (on | | | | sit down in front of a new interface and immediately |
| the other hand, it might just look strange and make you | | | | be as productive as we were before. We have all |
| sick; hard to tell without trying it).Because the input is | | | | learned to use the desktop and menu-driven interfaces |
| essentially 2D, I predict pure 3D imaging devices will | | | | because we haven't had a choice. It has taken time; |
| prove to be a novelty even if the enormous bandwidth | | | | just as learning to read and write took years when we |
| problems can be solved. A graphics card that draws a | | | | were younger. Even the keyboard and the mouse, |
| 480x480 pixel scene at 60fps would take 8 seconds | | | | although perhaps easier than writing, have taken time |
| to update a 480x480x480 cube. Yes, I understand this | | | | and effort to master. New interfaces will face the |
| is a vast simplification. Somehow restricting rendering | | | | same hurdles. Their designs will need tweaking to |
| to the surfaces of an object might help, but it sounds | | | | reduce the learning curve as much as possible. The |
| tricky. Regardless, the same or better effect will be | | | | users of these new interfaces will need the patience |
| achievable by feeding a couple of 2D images to each | | | | to develop efficient usage patterns; and the interfaces |
| eye. Technology that takes this approach will be more | | | | themselves will need to be entertaining enough to |
| successful. Devices that project images directly onto | | | | mitigate the patience required. All these efforts will yield |
| the retina seem like a reasonable approach; along with | | | | interfaces that are not only more enjoyable, but faster |
| any tracking technology that may go with them.The | | | | and more useful.Tristan Grimmer is technical director at |
| next 10 years will be a transitional phase for interface | | | | Upper Bounds Interactive Inc. Previously a video-game |
| design. 3D rendering technologies already have a | | | | programmer, and much earlier than that a young boy |
| stable home in the entertainment, video game, | | | | trying to get his Vic-20 to accurately compute Pi, |
| simulation, and design sectors. Although 2D interfaces | | | | Tristan now spends his time working on Tactile 3D in |
| have dominated everything else, I expect we will start | | | | an attempt to rid the world of rectangles. |