| This article serves two purposes. It is a | | | | the entertainment, video game, simulation, |
| discussion about the future of computer | | | | and design sectors. Although 2D interfaces |
| interfaces; and it is a means by which I can | | | | have dominated everything else, I expect we |
| purge myself of thoughts that have been | | | | will start seeing more 3D incursions. |
| accumulating on this topic for quite a few | | | | Operating systems and applications are |
| years. Even if it fails as intelligent | | | | beginning to capitalize on what 3D has to |
| discourse on the first, it will have | | | | offer. The precise nature of how and where |
| succeeded in the second. Previously the | | | | 3D can best be incorporated is an open |
| title was Where Are Computer Interfaces | | | | question, and a framework to evaluate these |
| Going? but after writing it I noticed a | | | | questions seems appropriate. As a rough |
| significant number of predictive passages and | | | | starting point it seems reasonable to divide |
| decided to be bold and move the "are". Of | | | | the attempts into two broad categories: those |
| course now I feel obliged to add a | | | | that are trying to simulate the physical |
| disclaimer. I admit right here, or at least | | | | world and those that prefer more abstract |
| in the next sentence, that I don't know where | | | | representations. If you'll indulge me, I'd |
| computer interfaces are going. I don't | | | | like to call these two approaches, |
| know.With that out of the way, I'd like to | | | | respectively, the "Physical Simulation |
| start, as many interfaces do, with the | | | | Approach" (PSA) and the "Abstract |
| metaphor. In the 80s and 90s successful | | | | Representation Approach" (ARA).Developers in |
| interface design and an appropriate metaphor | | | | the PSA camp are taking physical simulations |
| were taken to be nearly synonymous. Although | | | | and hanging applications, web sites, movies, |
| a good metaphor is important, it imposes | | | | and pictures on simulated walls. Simulated |
| unnecessary and artificial restrictions. So | | | | desks have functional simulated calculators |
| why is it so important? The best, perhaps | | | | on them. And, perhaps, there is a simulated |
| only, reason is familiarity. Unfortunately, | | | | sun outside. It's all very familiar and |
| familiarity comes at a cost: the shorter | | | | comes with a nice minimal learning curve.The |
| learning curve can require speed and ability | | | | ARA camp are working on strange visualization |
| to be sacrificed.Consider the ubiquitous | | | | techniques to view complexity and patterns in |
| desktop metaphor. What is more powerful, the | | | | large amounts of data. They have general |
| abstract construct of a tree, or a single | | | | graphs floating around in space with links |
| flat surface to place your papers on? Well, | | | | joining concepts and words together in |
| a tree is. In fact it is so much more | | | | arbitrary ways. They have nifty algorithms |
| powerful that it is the cornerstone of all | | | | that filter the salient characteristics of |
| modern file systems. Trees are great, they | | | | large data sets so you don't get overwhelmed. |
| impose an organizational order that is common | | | | Their attempts are, by far, much harder to |
| in natural systems. General graphs are, | | | | describe with these mere words.In practice |
| perhaps, too general. DAGs (Directed Acyclic | | | | many attempts will combine aspects of both |
| Graphs) are a good contender; largely because | | | | philosophies. I suspect that successful |
| of their acyclicness, but also because they | | | | attempts at a 3D interface will have to |
| extend trees in a well defined way. I | | | | balance these two extremes in appropriate |
| suspect that trees are so useful because we | | | | ways. Objects in a functional 3D interface |
| can't move backwards in time. Species | | | | should probably be represented with models |
| speciate, languages extend, and software | | | | that are familiar, just like the icons on |
| bloats. To fight these is to fight the | | | | your desktop are often imitations of familiar |
| increasing entropy of the universe.Would it | | | | real-world objects. This is a PSA property. |
| be a good idea not to allow folders within | | | | On the other hand, tree-based organizational |
| folders within folders just because it would | | | | systems would be well advised. Very much an |
| be physically cumbersome, and at some point | | | | ARA concept.Text should always be view-plane |
| impossible? Probably not. Do icons have a | | | | aligned, as should images. This is one of |
| real-world counterpart? Not really. | | | | those 2D features mentioned earlier. Images |
| Metaphors should be, and have been, taken | | | | and text may be scaled, but they should not |
| only so far.So what does the future hold? | | | | present themselves at an angle. Vertical and |
| Will interfaces be 3D? Will we be stuck with | | | | horizontal edges need to remain vertical and |
| rectangles forever? I think it's reasonable | | | | horizontal. Of course, these features are |
| to say both have their place. People on the | | | | trivially present with your desktop interface |
| 3D side think that we humans see, work, live, | | | | as well.And there's an important lesson: |
| and play in 3D. We don't. They say they | | | | build on the backs of giants. The desktop UI |
| can't wait until there are fully 3D monitors | | | | is successful for a reason, not simply |
| that you can walk around. Why? Our retinas, | | | | because it has a familiar analogue in the |
| as well as birds whose eyes are plastered on | | | | physical world, but rather because it behaves |
| the side of their heads, are two-dimensional | | | | in that same useful way that real desks |
| surfaces. Birds have flatter vision than we | | | | behave. It takes advantage of a |
| do, if not as Euclidean, because they don't | | | | well-established ability; spatial memory. |
| have the benefit of the tiny bit of 3D depth | | | | You put something down and it stays |
| perception a predator gets by overlapping | | | | there.Useable interfaces need a certain |
| images. I've heard graphics programmers | | | | amount of persistence in their structure. |
| explain that their 3D scene was being | | | | Having objects stay where you leave them is |
| projected onto a flat 2D screen and so it was | | | | one good way to achieve persistence. Placing |
| no longer really 3D. But consider this: | | | | objects manually, whether on your desktop or |
| everything you see in this world is like | | | | in a 3D environment, takes advantage of |
| that. It all gets projected onto our flat | | | | spatial memory. We can remember, in context, |
| retinas. We just have really big brains. A | | | | where we've left hundreds of objects |
| 3D scene is constructed in our mind | | | | (notwithstanding car keys; they get moved |
| regardless of whether what we're viewing is | | | | around too much). You probably know where |
| on a flat computer monitor or in that | | | | your camera is and where the light switches |
| nether-world known as real life. In fact, | | | | in your home are. By positioning objects |
| most brains do a decent job of scene | | | | manually you can give them some context; |
| construction even with one eye closed. From | | | | perhaps by placing pictures of your family to |
| 2D to 3D. Impressive!People on the 2D side | | | | the left, and panoramic vistas to the right. |
| think that we humans see, work, live, and | | | | Contextual clues help you remember.I've heard |
| play in 2D. We do, after all, have flat | | | | the assertion that adding a single extra |
| retinas, like playing tennis on flat tennis | | | | dimension doesn't buy you much organizational |
| courts, and eat dinners from flat plates on | | | | power and that the added navigational |
| flat tables. But we don't live in 2D. Our | | | | complexity isn't worth it. Others think that |
| brains are really big. 1.3 litres big. More | | | | we need an n-dimensional space to do a good |
| than enough dendrites, axons and other | | | | job. Aside from the obvious observation that |
| brain-things to contain a nice 3D | | | | we seem to exist in a macroscopically |
| representation of the world we live in. | | | | three-dimensional world (macroscopically was |
| Clues to build the scene abound: motion, | | | | added just to keep any |
| foreshortening, and the aforementioned depth | | | | physicists-who-may-know-better reading) and |
| perception.The truth is some things are | | | | are therefore good at 3D manipulations, there |
| better in 2D and some 3D. Writing a letter? | | | | is evidence that the jump from 2 to 3 |
| Use a desk. Put a flat piece of paper on it. | | | | dimensions is of a more fundamental |
| Want to file that letter away? Wouldn't it | | | | significance. If you draw a bunch of dots on |
| be cool if you could just let it hover in | | | | a piece of paper you will not be able to draw |
| some large 3D organizational space? Here's | | | | lines joining the dots in all possible |
| what I think. | | | | configurations unless the lines cross (given |
| | | | some sufficiently large number of dots. I |
| Text: 2D | | | | think 5 might do it). However, once you hit |
| | | | three dimensions, all configurations are |
| Reading and Writing: 2D | | | | possible without crossings. Adding a fourth |
| | | | or fifth doesn't have any further beneficial |
| Organizing and Grouping: 3D | | | | effect. Admittedly there is some hand-waving |
| | | | going on here; but the result has |
| Visualization of Data: Depends | | | | implications for some possible interface |
| | | | designs; and it points to using three |
| It has occurred to me that 2D | | | | dimensions.So why haven't interfaces changed |
| representations should be considered a | | | | much in the last 20 years? One possibility |
| feature of an interface. It's beneficial | | | | is that the desktop is in some way an optimal |
| that text documents are lined up nicely for | | | | representation. More likely, however, is |
| you in a window. If head-or-eye-tracking | | | | that it is simply a functional |
| hardware were more widespread, we'd have | | | | representation; no need to change when change |
| software that could compensate for (single) | | | | takes effort, right? We expect to be able to |
| users who are not directly in front of their | | | | sit down in front of a new interface and |
| screens. Imagine looking at your monitor | | | | immediately be as productive as we were |
| from an angle but still having the text of | | | | before. We have all learned to use the |
| this article appear flat. That would be a | | | | desktop and menu-driven interfaces because we |
| pretty neat feature (on the other hand, it | | | | haven't had a choice. It has taken time; |
| might just look strange and make you sick; | | | | just as learning to read and write took years |
| hard to tell without trying it).Because the | | | | when we were younger. Even the keyboard and |
| input is essentially 2D, I predict pure 3D | | | | the mouse, although perhaps easier than |
| imaging devices will prove to be a novelty | | | | writing, have taken time and effort to |
| even if the enormous bandwidth problems can | | | | master. New interfaces will face the same |
| be solved. A graphics card that draws a | | | | hurdles. Their designs will need tweaking to |
| 480x480 pixel scene at 60fps would take 8 | | | | reduce the learning curve as much as |
| seconds to update a 480x480x480 cube. Yes, I | | | | possible. The users of these new interfaces |
| understand this is a vast simplification. | | | | will need the patience to develop efficient |
| Somehow restricting rendering to the surfaces | | | | usage patterns; and the interfaces themselves |
| of an object might help, but it sounds | | | | will need to be entertaining enough to |
| tricky. Regardless, the same or better | | | | mitigate the patience required. All these |
| effect will be achievable by feeding a couple | | | | efforts will yield interfaces that are not |
| of 2D images to each eye. Technology that | | | | only more enjoyable, but faster and more |
| takes this approach will be more successful. | | | | useful.Tristan Grimmer is technical director |
| Devices that project images directly onto the | | | | at Upper Bounds Interactive Inc. Previously |
| retina seem like a reasonable approach; along | | | | a video-game programmer, and much earlier |
| with any tracking technology that may go with | | | | than that a young boy trying to get his |
| them.The next 10 years will be a transitional | | | | Vic-20 to accurately compute Pi, Tristan now |
| phase for interface design. 3D rendering | | | | spends his time working on Tactile 3D in an |
| technologies already have a stable home in | | | | attempt to rid the world of rectangles. |