Dissertation Title: ‘Shells’ and ‘Stages’ for Actuated TUIs: Reconfiguring and Orchestrating Dynamic Physical Interaction
Research on Actuated and Shape-Changing Tangible User Interfaces (TUIs) in the field of Human Computer Interaction (HCI) has been explored widely to fuse the dynamism of digital information and tangibility of physical matter for embodied interaction. While advanced technical approaches, such as robotics and material science, have given many concrete instances of Actuated TUIs, a single reconfigurable hardware system is inherently limited to the fixed configurations, thus limits the interaction versatility and adaptability.
In my thesis dissertation, I introduce novel hardware interaction design modalities, 'Shells' and 'Stages,' for Actuated TUI hardware to dynamically reconfigure physical interaction. I introduce and define the concept and framework of 1. 'Shells' which are mechanical attachments for Actuated TUIs that can extend, reconfigure and augment the interactivity and functionality of the hardware, and 2. 'Stages', which allow Actuated TUIs to propel on as a platform for novel physical expression based on a duality of front and back stages. These concepts are built based on inspirations from theatrical performances, computational / robotic architecture, biological systems, and physical tools. Also, with the combination of ‘Shells’ and ‘Stages’, it enables Actuated TUIs to perform advanced physical expression and interaction such as teleportation, shape transition, and digital-physical transition. By introducing these novel modalities of ‘Shells’ and ‘Stages,’ the thesis expands and contributes to a new paradigm of Inter-Material /Device Interaction in the domain of Actuated TUI systems.
The thesis demonstrates and assesses these concepts and framework through prototypes primarily using self-propelled tabletop TUIs, one of the major instances of Actuated TUIs, whose behaviors can be orchestrated on top of tabletop surfaces. Other hardware approaches, using pin-based shape displays, are also introduced as precursors to build up the concepts of ‘Shells’ and ‘Stages.’ Design and implementation methods are introduced to fabricate mechanical shells with different properties, and orchestrate a swarm of robots on the stage with arbitrary configurations. Interactive applications are demonstrated based on the prototype systems, ranging from digital data interaction, reconfigurable physical environment, and storytelling. By introducing a design framework that incorporate the self-actuating generic hardware (Actuated TUIs) and passively actuated mechanical modules (Shells) together with reconfigurable platform (Stages), my research envisions the future of physical environment where computational robotic devices, passive mechanical material and environment work together to provide dynamic physical interaction to assist and enrich our everyday life in physical, dynamic and playful ways.
Prof. Hiroshi Ishii
The Jerome B. Wiesner Professor of Media Arts and Sciences, MIT Media Lab
Prof. Joseph Paradiso
The Alexander W. Dreyfoos (1954) Professor in Media Arts and Sciences, MIT Media Lab
Prof. Masahiko Inami
Research Center for Advanced Science and Technology, The University of Tokyo