Merrill, D. "Interaction with Embodied Media"

Abstract

The graphical user interface has become the de facto metaphor for the majority of our diverse activities using computers, yet the desktop environment provides a one size fits all user interface. This dissertation argues that for the computer to fully realize its potential to signicantly extend our intellectual abilities, new interaction techniques must call upon our bodily abilities to manipulate objects, enable collaborative work, and be usable in our everyday physical environment.

In this dissertation I introduce a new human-computer interaction concept, embodied media. An embodied media system physically represents digital content such as files, variables, or other program constructs with a collection of self-contained, interactive electronic tokens that can display visual feedback and can be manipulated gesturally by users as a single, coordinated interface. Such a system relies minimally on external sensing infrastructure compared to tabletop or augmented reality systems, and is a more general-purpose platform than most tangible user interfaces.

I hypothesized that embodied media interfaces provide advantages for activities that require the user to efficiently arrange and adjust multiple digital content items. Siftables is the first instantiation of an embodied media interface. I built 180 Siftable devices in three design iterations, and developed a programming interface and various applications to explore the possibilities of embodied media. In a survey, outside developers reported that Siftables created new user interface possibilities, and that working with Siftables increased their interest in human-computer interaction and expanded their ideas about the field. I evaluated a content organization application with users, nding that Siftables offered an advantage over the mouse+graphical user interface (GUI) for task completion time that was amplied when participants worked in pairs, and a digital image manipulation application in which participants preferred Siftables to the GUI in terms of enjoyability, expressivity, domain learning, and for exploratory/quick arrangement of items.