Project

Deep Angel: The AI behind the aesthetics of absence

Deep Angel

Project Contact:

Deep Angel is an artificial intelligence that erases objects from photographs. The algorithm is hosted on http://deepangel.media.mit.edu, which enables anyone to interact with the AI and explore what it can disappear.

Part philosophy, part technology, and part art, Deep Angel is designed to spark a series of conversations on technology in our daily lives and AI and media manipulation.

Deep Angel draws from Walter Benjamin's description of Paul Klee's Angelus Novus, the angel of history who has clairvoyance into the dark side of what appears to be progress. The angel sees the unravelling of all that matters in the world and would like to alert the world about his vision, but he's caught in the storm of progress and can't communicate any messages. The images that Deep Angel generates are intended to deliver the message that Angelus Novus would have sent if he could.

The algorithm applies computer vision techniques to automatically (1) detect and outline objects in images, (2) remove the outlined object from the image, and (3) imagine what the image would look like if that outlined object were removed from the image. Any image uploaded and transformed by Deep Angel can be published on the Deep Angel website by clicking the "Publish to Deep Angel" button.

The AI's performance varies across photographs. Sometimes, it's impossible to tell what has been disappeared. Other times, the images appear similar to the images from Adrian Piper's Everything series. The more people interact with the algorithm, the more attuned people will be to the potential and limitations of modern AI to manipulate the media. It's now possible to automate the vanishing commissar in Soviet photography, but the AI is not yet perfect. Below are two examples of the Deep Angel AI effect: (1) a gif generated by Deep Angel showing a father and daughter disappearing in the wilderness and (2) two images showing the before and after of Deep Angel peering into a photo of a professional surfer.

As Angelus Novus must have foreseen, technology mediates our daily lives. Unfortunately, it's not always for the best. There's a corporate competition over humankind's attention. Media (social, Youtube, television) is often designed to capture our attention rather than expand our world view. In a showcase of Deep Angel's artificial intelligence with the intention of holding a mirror to technology with technology, Scalable Cooperation produced a short film, The Broken Flaneur.

From a technical perspective, Deep Angel is an end-to-end targeted object removal pipeline consisting of three interfacing neural networks: an object mask generator (G), a generative inpainter (I), and a local discriminator (D). For every input image and class label pair, Deep Angel generates an object mask using G, which is combined with the image and fed to the inpainting network that produces the generated image. The inpainter, I, is trained from the loss of the discriminator, D, following the typical GAN pipeline. Below is an illustration of the Deep Angel neural network architecture. Of course, there are many additional optimizations that enable running this neural network in a production environment.

Deep Angel was a collaborative project with many contributors. In particular, we would like to thank Brian Denzel Bailey, Julian Kelly, Carl-Gustaf Ewerbring, and Micah Epstein for excellent contributions in ideation and design.

Research Topics

#artificial intelligence #social media #imaging