Project

HERMITS: ‘Mechanical Shells’ for Robotic Tangible UIs

Ken Nakagaki

Project Contact:

Groups

By Ken Nakagaki, Joanne Leong, Jordan L Tappa, Joao Wilbert, and Hiroshi Ishii

Overview

Project HERMITS explores a way to greatly advance the versatility of Robotic Tangible Interfaces. Inspired by hermit crabs, we designed a modular system for table-top wheeled robots to dock to passive attachment modules, defined as "mechanical shells." Different types of mechanical shells can uniquely extend and convert the motion of robots with embedded mechanisms, so that, as a whole architecture, the system can offer a variety of interactive functionality by self-reconfiguration. We envision this novel interactive architecture to bring a rich application space including physical space organization, digital data physicalization, and entertainment and storytelling systems.

The general approach in HERMITS expands how physical interfaces and computers in our daily life can adapt and reconfigure for user interactions with passive attachments. We are in the age where robotic systems are emerging in our living space (e.g. robotic vacuum cleaner, drones). The idea presented in HERMITS has greater implications beyond our prototype, where everyday robotic systems may gain a significant amount of functionality, expressivity, and interactivity by switching mechanical shells.

UIST2020 Paper (PDF) ACM Digital Library 5 mins UIST2020 Presentation 15 mins UIST2020 Presentation Press Kit

For press inquiry, please reach out to ken_n [at] media.mit.edu.

Research Description

Concept of Mechanical Shell

We define a "mechanical shell" as "an external passive attachment to an existing actuated tangible interface that can be automatically attached or detached." As shown in the diagram below, the shell is an attachment that can modify, convert, or extend the interactive functionalities of actuated robotic tangible user interfaces (TUIs).

Mechanical Shells Design and Examples

We have defined the design space of mechanical shells which became the basis for all of our prototypes. These properties include single module properties (shape, motion, light, etc) and multi-module properties (Multi-DoF, Speed, and Force Aggregation). This basic vocabulary points towards an infinite number of design opportunities for the robotic TUIs to gain versatile interactivity.

Mechanical Shell activated by Single Robotic Module

Mechanical Shell activated by Multiple Robotic Modules

Overall System Design of HERMITS

The overall system of HERMITS includes the (1) robots, (2) mechanical shells, (3) an interaction stage, and (4) a computer to control the system, as shown in the figure. By using a computer to control the robots to dock, un-dock and control the mechanical shells, the system provides reconfigurable rich interactivity to users.

Implementation

Our prototype system of HERMITS is based on an off-the-shelf robotic toy, toio, a two-wheeled robotic system. We have built a Raspberry Pi-based hierarchical control architecture. A computer takes central control of the system, while a number of Raspberry Pi micro-controllers were used for connecting and controlling individual toios through Bluetooth (based on Python code).

Active Docking Mechanism

To achieve the active docking, we have modified the toio hardware with an additional vertically moving pin enabled by a micro-linear servo motor. This pin was designed to dock to a slot of mechanical shell for robust connection and motion transmission.

Applications

We demonstrate the applications including physical space reconfiguration (e.g. a desktop organization), animated story-telling, and tangible mobility simulation. We explore ways to leverage the dynamic reconfigurability of HERMITS with Mechanical Shells to enrich the interaction for the physical environment as well as digital information.

Next Step / Future work

Based on the concept of Mechanical Shell, we look at a variety of future work; including a range of robotic tangible interfaces, rapid fabrication of Mechanical Shells, design of interaction stage to moderate the 'backstage' reconfiguration, and advanced control for automated docking and closed-loop haptic control.

Special Thanks

Dr. Alexis Andre, Sony CSL
John Vincent, the Ford Motor Company
Dr. Pietro Buttoro, the Ford Motor Company

Dr. Artem Dementyev, Google
Hila Mor, MIT Media Lab
Zhipeng Liang, MIT Media Lab
Deema Qashat, MIT Media Lab
Jack Forman, MIT Media Lab

toio Maker community

Publication Research

HERMITS: Dynamically Reconfiguring the Interactivity of Self-propelled TUIs with Mechanical Shell Add-ons

Ken Nakagaki, Joanne Leong, Jordan L. Tappa, João Wilbert, and Hiroshi Ishii. 2020. HERMITS: Dynamically Reconfiguring the Interactivity of Self-propelled TUIs with Mechanical Shell Add-ons. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (UIST '20). Association for Computing Machinery, New York, NY, USA, 882–896. DOI:https://doi.org/10.1145/3379337.341583

Publication Research

Mechanical Shells: Physical Add-ons for Extending and Reconfiguring the Interactivity of Actuated TUIs

Ken Nakagaki. 2020. Mechanical Shells: Physical Add-ons for Extending and Reconfiguring the Interactivities of Actuated TUIs. In Adjunct Publication of the 33rd Annual ACM Symposium on User Interface Software and Technology (UIST '20 Adjunct). Association for Computing Machinery, New York, NY, USA, 151–156. DOI:https://doi.org/10.1145/3379350.3415801

Research Topics

#robotics #human-computer interaction