{{{gallery_title}}}

Type experiments

36 Days Of Type is an initiative that invites creatives to create their version of the letters and numbers of the Latin alphabet, one…

Archived

How x-ray vision is becoming a reality

Tara Boroushaki, a PhD student in the Signal Kinetics group, explains how she's combined AR and wireless signals to find hidden objects.

Post Research

StoryLine: Exploring the intersection of visual storytelling and machine learning

McKinsey & Co. and LSM ask: Can we understand, and even predict, the relationship between video story content and audience consumption?

Photorythms

Photorythms: a computational art-based inquiry of portrait photographyPhotorythms investigates whether computational methods such as facial…

Detecting and Mapping Transparent or Mirror-like Surfaces with Lidar

Although lidar is widely used for mapping the 3D geometry of surfaces, the technology has historically been challenged by specular, or mirr…

Archived

Bounce-Flash Lidar

In photography, illuminating a subject directly using an on-camera flash can result in unflattering photos. To avoid this, photographers o…

Archived

Liquid Learning

Continuing the work of Liquid Movies, we introduce Liquid Learning, an extension for not simply recommending videos but encouraging people …

Augmented Reality with X-Ray Vision

X-AR is an augmented reality (AR) system that gives humans "X-Ray Vision" X-AR is a new AR headset that enables users to see things th…

What can we learn from predicting shadows?

By predicting sparse shadow cues, our physics-inspired machine learning algorithm can reconstruct the underlying 3D scene. Abstract. W…

Modeling Contingent Behaviors in Multiparty Conversations

Recognizing and interpreting group behaviors is much more challenging than that of individual behaviors. Firstly, the system must perform w…

Objects as Cameras

Press: - MIT News: Using reflections to see the world from new points of view- TechXplore, EuropaPress, La Nacion …

New research from the Signal Kinetics research group is featured on "Science Without the Gobbledygook"

Physicist Sabine Hossenfelder discusses X-AR, the new research from the Media Lab’s Signal Kinetics group.

MIT researchers develop augmented reality headset to help users see hidden objects

Using augmented reality, the X-AR system developed by the Signal Kinetics research group can guide users to find hidden objects.

Archived

Research

Social Machines

Promoting deeper learning and understanding in human networks

Archived

Liquid Movies

Video has evolved from an esoteric production to the default means of online and broadcast communication. We accept a wrong or lengthy vide…

Archived

Using reflections to see the world from new points of view

A new computer vision system turns any shiny object into a camera of sorts, enabling an observer to see around corners or beyond obstruction

Augmented reality headset enables users to see hidden objects

The device could help workers locate objects for fulfilling e-commerce orders or identify parts for assembling products.

Archived

Research

Collective Learning

Transforming data into knowledge

Amazon MARS 2022: Day 2 Keynote

Be inspired by Rohit Prasad, SVP and Head Scientist, Alexa AI, Amazon, Tye Brady, Chief Technologist, Amazon Robotics, Dilip Kumar, VP, Phy…

Archived

Biomarker Imaging with Mobile Phones

We built a low-cost and open source 405 nm imaging device to capture red fluorescence signatures associated with the oral biomarker porphyr…

Archived

Near-Infrared Imaging for Detecting Dental Caries

Two-dimensional radiographs are commonly used for evaluating sub-surface hard structures of teeth, but they have low sensitivity for early …

Archived

Non-Ionizing Imaging for Medical Diagnosis Instead of 2D Radiography and Cone-Beam Computed Tomography

Two-dimensional radiographs are commonly used for evaluating sub-surface hard structures of teeth, but they have low sensitivity for early …

Archived

Machine Learning and Automated Segmentation of Oral Diseases Using Biomarker Images

We report a novel method that processes biomarker images collected at the point of care and uses machine learning algorithms to provide a f…

Archived

Machine Learning from Biomarker Signatures and Correlation to Systemic Health Conditions

Imaging fluorescent disease biomarkers in tissues and skin is a non-invasive method to screen for health conditions. We report an automated…

Archived

Artificial Intelligence and Machine Learning in Clinical Development: A Translational Perspective

Technical summary: Future of clinical development is on the verge of a major transformation due to convergence of large new digital data so…

Archived

Helping Emergency Care Physicians Diagnose Sepsis and Bacterial Infections with Machine Learning and Vision

General overview:Sepsis, a life-threatening complication of bacterial infection, leads to millions of worldwide deaths requires significant…

Archived

Research Area | Emergent Neural Networks to Redesign Clinical Testing of Medicines and Learning from Real World Evidence

Research studies led by Dr. Shah, in collaboration with the United States Food and Drug Administration (FDA), have develope…

Archived

Research Area | Application, Interpretability and Clinical Translation of Deep Learning Algorithms for Medical Images

Research studies led by Dr. Shah in his laboratory have created new paradigms for using low-cost images captured using simple optical …

Archived

Using Shadows to See Behind Objects

The full text of our paper is available here.Sometimes the thing that we want to see is hidden behind something else. A neighboring vehicl…

Archived

Privacy-Preserving Depressive Symptoms Estimation

The goal of this project is to develop techniques to remove identifying information from wearable and phone data to protect patients’ priva…

Archived

Robot overcomes uncertainty to retrieve buried objects

This robotic system uses radio frequency signals, computer vision, and complex reasoning to efficiently find items hidden under a pile.

Archived

ZoomADay - AI Synthesized Characters

#ZoomADay was a year long project exploring the creation and use of synthetic characters and deep fakes for use in online telepresence and …

Archived

Center for Civic Media

Archived

Research

Object-Based Media

Changing storytelling, communication, and everyday life through sensing, understanding, and new interface technologies

Archived

Actionable Auditing: Coordinated bias disclosure study

Algorithmic auditing has emerged as a key strategy to expose systematic biases embedded in software platforms, yet scholarship on the impac…

Archived

Gender Shades

The Gender Shades project pilots an intersectional approach to inclusive product testing for AI.Algorithmic Bias PersistsGender Shades is a…

Archived

DISSECT: Disentangled Simultaneous Explanationsvia Concept Traversals

One of the principal benefits of counterfactual explanations is allowing users to explore "what-if" scenarios through what does not and can…

Archived

UncNet: Modeling uncertainty in deep learning for inherently subjective tasks

Automatic emotion recognition has become a well-established machine learning task in recent years. The sensitive and subjective nature of e…

Archived

Behavioral Indications of Depression Severity

In collaboration with Massachusetts General Hospital, we are conducting a clinical trial exploring objective methods for assessing depressi…

Archived

Image2Reverb: Cross-Modal Reverb Impulse Response Synthesis

Image2Reverb: Cross-Modal Reverb Impulse Response Synthesis. Nikhil Singh, Jeff Mentch, Jerry Ng, Matthew Beveridge, Iddo Drori; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 286-295

Archived

Turning Objects into Cameras

Consider a small object sitting on a desk in your living room. The object is illuminated by light sources from all directions—this includes…

Archived

ELSA: Empathy learning, socially-aware agents

What is ELSA?ELSA is an AI-powered chatbot that acts as an empathetic companion, encouraging users to talk about their day through a form o…

Archived

RFusion: Robotic Grasping via RF-Visual Sensing and Learning

Tara Boroushaki, Isaac Perper, Mergen Nachin, Alberto Rodriguez, Fadel Adib. "RFusion: Robotic Grasping via RF-Visual Sensing and Learning." ACM Conference on Embedded Networked Sensor Systems 2021.

RFusion: Robotic Grasping via RF-Visual Sensing and Learning

RFusion is a robotic system that can search for and retrieve items in line-of-sight, non-line-of-sight, and fully occluded settings. It con…

Robotic Grasping of Fully-Occluded Objects using RF Perception

Boroushaki, Tara, et al. "Robotic Grasping of Fully-Occluded Objects using RF Perception." IEEE International Conference on Robotics and Automation (ICRA 2021).

Tangible Swarm: see the unseen

Tangible Swarm is a tool that displays relevant information about a robotics system (e.g., multi-robot, swarm, etc.) in real time whil…

A robot that finds your lost stuff and more AI-enabled gadgets to come

The WSJ covers research from the Signal Kinetics group and other labs, universities, and companies working on AI-enabled devices.

Automated end-to-end deep learning framework for classification and tumor localization from native nonstained pathology images

Akram Bayat, Connor Anderson and Pratik Shah. "Automated end-to-end deep learning framework for classification and tumor localization from native nonstained pathology images," Proc. SPIE 11596, Medical Imaging 2021: Image Processing, 115960A (15 February 2021); doi: 10.1117/12.2582303

RF-Grasp: Robotic Grasping of Hidden Objects

Can robots find and grasp hidden objects?Robots are not capable of handling tasks as simple as restocking grocery store shelves as th…

A robot that senses hidden objects

System uses penetrative radio frequency to pinpoint items, even when they’re hidden from view.

ACM ByteCast: Ramesh Raskar

Camera Culture head Ramesh Raskar talks to Rashmi Mohan about his interdisciplinary research and entrepreneurial endeavors.

Imaging Behind Occluders Using Two-Bounce Light

Henley, C., Maeda, T., Swedish, T., & Raskar, R. (2020). Imaging Behind Occluders Using Two-Bounce Light. Computer Vision – ECCV 2020 Lecture Notes in Computer Science, 573-588. doi:10.1007/978-3-030-58526-6_34

Bringing construction projects to the digital world

The startup OpenSpace is using 360-degree cameras and computer vision to create comprehensive digital replicas of construction sites.

Archived

Streetchange

Computer vision uncovers predictors of physical urban change

Archived

Algorithmic Justice League

www.ajl.orgAn unseen force is rising—helping to determine who is hired, granted a loan, or even how long someone spends in prison. This for…

Archived

SuperGlue Kinesics

The relationship between news content and its presentation has been a long-studied problem in the communications domain. Often, channe…

Archived