The Natural
Sean Moore
Clap your hands, say ‘yeah’, and hopefully your computer will respond.
I’ve said this before, but computer interaction has always proceeded through a metaphor. At times, that metaphor was language – and if you’re a developer or a sys. admin, it still is – typing commands into a black box and waiting to see a response. Later, that metaphor became the mouse and the graphical environment, translating physical actions into on-screen commands. And now, we are entranced by the touchscreen, and that almost real feeling of interacting with the pixels underneath of our fingertips.
We’ve always been constrained by these metaphors, in ways large and small, because they are defined by others and out of our control. But in a way they are also freeing: they are incredibly effective at managing the complexity of what a computer fundamentally is, and they limit us in such a way to channel our actions toward productive, rather than destructive work.
In an even greater sense, these metaphors for interaction give us a great deal of leverage: with a small amount of physical effort we can be massively creative. In the simplest sense, even, the tiny movements of my fingers and hands as I sit at a keyboard add up into a considerable corpus of text; saving those calories for the more important thinking process of my actions. Ad though many of these interactions require a considerable degree of precision and visual feedback – not ideal for certain disabilities, to be sure – the movements once learned are predictable and repeatable; that is to say, you always know how to perform a particular action, and you also know what a particular action will do.
The new kid on the block of human computer interaction is the Natural User Interface. You’ve seen it, somewhere, somehow: the Xbox Kinect, Siri, or the soon to be released Myo armband.
These systems heavily rely on high fidelity sensors, gesture recognition, and natural language/interaction processing. For one thing, that’s a lot of pipe for your inputs to travel through: which means a lot of pain points where things can go wrong, or, and this especially true on mobile devices where computation is typically offloaded to some faraway server, greatly increases the latency, and hence the frustration
But even ignoring everything else, just look at actual gestures needed to interact with these new systems. Waving your arm to select a control. Assuming a posture to perform a specific test. Asking a specific question to get a specific response. These actions are neither repeatable or reliable; you can’t be guaranteed to assume the same position every time, or ask the same question in the same way, nor can you expect the response of the system to be the same every time. By increasing the abstraction, the leverage has been oriented in the wrong way. A simple computing task now necessitates a grandiose, thought-out and complicated gesture. It’s hard to be productive when you have to remember the contortions or lines of inquest you need to generate a desired response from your computer.
Should we really be creating a system that makes us feel extravagant, or looks cool in a thirty-second YouTube clip? Or do we want a system that leverages our small actions to create something truly great?