Paper Reading #15: Interactions in the air: adding further depth to interactive tabletops

Comments
Evin Schuchardt
Shena Hoffman

Reference Information
Title: Interactions in the air: adding further depth to interactive tabletops
Authors: Otmar Hilliges, Shahram Izadi, Andrew D. Wilson, Steve Hodges, Armando Garcia-Mendoza, Andreas Butz
Presentation Venue: UIST 2009: 22nd annual ACM symposium on User interface software and technology; October 4-7, 2009; Victoria, British Columbia, Canada

Summary
In this paper the researchers present a way to improve interaction with tabletops by allowing users to work above the surface to intuitively manipulate digital content in three dimensions.

In their first system they use a diffuser to extend the input space so that it can recognize what the users are gesturing. To detect a “pinching” gesture (used to pick objects up), they use an algorithm that identifies the hole that is formed by the thumb and pointer finger touching. The user can then pick up and manipulate the location and angle of the object. The algorithm also calculates the depth of the user’s hand based on the hole formed by the thumb and pointer finger.

They also describe a shadow-based technique and two ways to implement it. In the first they use a shadow-mapping technique that is not described in detail. In the second, more successful shadow-based technique they generated a shadow map for all the virtual objects and stored each pixel’s z-value to be compared when determining how a user is manipulating an object. A benefit to a shadow-based technique that is mentioned in the paper is that they provide users with a natural feedback mechanism.

While they have not done a formal user study, the authors explain that they have presented their system to many colleagues. Based on the results from their colleagues, the researchers found that without the shadows, the users had trouble learning how to work the system on their own. They also noticed that many users wanted to grab the objects, but grabbing was not supported by their system.

Image taken from paper

To address some of the issues presented by their colleagues, the researchers also include information on how they worked to improve the system to include more degrees of freedom in moving objects and a way for the system to recognize grasping objects with all the fingers rather than just “pinching” an object.

Discussion
I think finding ways to improve tabletop interactions is a good idea. The ideas they presented seem well thought out and interesting, but I did find it odd they hadn’t done a true user study and instead said they have demonstrated their “prototype to hundreds of colleagues” and thus “had the opportunity to observe hundreds of users interacting with their system.” Still, they did seem to get very good feedback, which helped them improve upon their system. In future studies, I think it would be interesting to see how this can be applied in different domains like gaming and computer-aided design.