Insert hand into box, watch it become a surreal digital object

There are rules in the physical world about what can and cannot happen. Newton might have described some of them, but all humans learn about what is possible and impossible. For example, fingers cannot magically appear or disappear from one’s hands. That’s not how the world works.

But in a digital environment, these rules do not apply. Less fingers are a cut away. More fingers a paste. Luckily, there are some rules in the digital world such as, one’s physical face does not change when one Photoshops a selfie.

But what happens when you combine physical reality, in this case one’s own hand, with real-time manipulations only possible in the digital realm. This is the premise of The Augmented Hand Series, created by Golan Levin, Chris Sugrue, and Kyle McDonald.

“[The system] consists of a box into which the visitor inserts their hand, and a screen which displays their ‘reimagined’ hand—for example, with an extra finger, or with fingers that move autonomously,” the artists explain. “Critically, the project’s transformations operate within the logical space of the hand itself, which is to say: the artwork performs ‘hand-aware’ visualizations that alter the deep structure of how the hand appears.”

The fluidity and strangeness of the interaction must be seen to be believed. Here is Levin demonstrating the system:

[vimeo 111951283 w=800 h=450]

Augmented Hand Series (Demonstration, 2014) from Golan Levin on Vimeo.

The project was conceived a decade ago, but received funding in 2013, and then debuted at the 2014 Cinekid Festival in Berlin.

Since I had my son, and I watched him figure out how to move, one of my fascinations has been the extent to which moving our bodies is a kind of thinking. Just try to build a system that moves, as roboticists have over the past decades: It’s really, really hard. The thinking we do with our bodies doesn’t just extend to moving our limbs, either. It structures the way that we understand all the world. How do you come to understand gravity’s effects? You fall. And you stand up. And you fall. These actions, which children repeat over and over, are more than idle fun. They are the literal data input mechanism for the brain’s understanding of physical reality.

The Augmented Hand Series artists conceived of their project as an intervention to get visitors thinking about what’s called embodied cognition. “Augmented Hand Series can be understood as an instrument for probing or muddling embodied cognition, using a ‘direct manipulation’ interface and the suspension of disbelief to further problematize the mind-body problem,” they write. “We see evidence of our instrument’s powers in the actions of young visitors who, uncertain whether to believe their eyes, peek into the box to double-check what is (not) really happening to their hand.”

Seeing this manipulation of the hand makes people want to see their hands again. Maybe they will even reconsider the hand’s structure and movement, the effortlessness of its grips and rotations.

And it is a good time to consider the hand. Our new smartphone computing paradigm depends on the subtlest hand motions. Our phones look like they do because our hands work like they do.