A visiting scientist from NHK, Takahito Ito joined the Media Lab as a company member and left as a crew member with a new direction: virtual reality.

By Takahito Ito

I work as a computer graphics creator at NHK, Japan’s public broadcaster. I knew that NHK was a Media Lab member company, but I had no idea that membership could give me personally the chance to benefit from the relationship. I found out two years ago when my NHK colleagues returned to Tokyo with stories of the Lab’s member week and told me about the opportunity to become a visiting scientist at the Lab.

As well, NHK hosted a meeting there for the Lab’s principal investigators (PIs) who were interested in 8K technology, which is very high resolution. That attracted my attention because NHK has been developing and expanding 8K as it navigates the changing television industry. So, I got a list of those PIs to find out more about what they were doing.

In the year that followed, when NHK became a corporate member of the Media Lab, two NHK employees were sent there as visiting scientists. That’s a benefit allowed to companies which have a certain level of Media Lab membership. And, I was able to represent NHK at the fall 2015 member meeting, which included a joint session of workshops and presentations by the Lab’s PIs for Japanese sponsor companies. Through these experiences, I decided I’d like to return to the Media Lab as a visiting scientist.

Why I went to the Media Lab

For years I have been creating computer graphics for TV programs, such as NHK’s special documentary series. The graphics focus on data visualization and range from earthquakes to sports to weather. With the Tokyo 2020 Olympics fast approaching, my goal is to pave our way to new services by exploring new media technologies, such as 8K—and virtual reality (VR), as I’d discover at the Media Lab. 8K has been a priority for NHK since 1995, when it became the first company to start research and development of 8K, which has a resolution that’s 16 times that of HDTV.

With that in mind, I visited the Media Lab in 2015 where I met professor César Hidalgo, who directs the Lab’s Collective Learning group and whose interests aligned with mine. Shaking hands with César turned out to be one of the most memorable moments of my life. What attracted me to his group was its focus on both data analytics and visualization and its potential applications for my work. So, back in Tokyo, I wrote a proposal and plan for a one-year Lab assignment, which my NHK boss approved. I was on my way. 

I arrived at the Media Lab in April last year, just in time for its spring member meeting. Within days, I took part in my first meeting with the Collective Learning group. I was in for a surprise: everyone in the group was busy getting organized for the member meeting. Among other activities, they were cleaning the walls!

At that time, I decided that as I’d joined this group I’d collaborate on anything related to it: from cleaning walls to crunching data to building prototypes. In my mind, that was important for good relationships with all group members. In fact, I even moved offices so I could be in the group’s shared area, not only to seek their feedback but also to observe their work. It was amazing to see how they formed collaborations on papers and projects—sometimes in meetings, other times over a casual meal. The importance of the group’s dynamic was my first learning experience at the Media Lab.

Turning data into stories and reality into VR

My second lesson also occurred early in my time at the Lab. I’d made the trip from Tokyo to Cambridge, expecting that I’d focus only on 8K tech. But César recommended that I also explore virtual reality (VR) as a way to tell stories. It had been more than a decade since I’d studied VR and human-machine interaction in college. So, this recommendation was another surprise for me—it wasn’t what I’d expected, though I’d always had an interest in VR.

César convinced me it would be a good path for me and NHK. He has a passionate approach to his research, and I absorbed that passion. He’s on the analytics side and I’m on the visualization side. Still, it was a challenge to figure out how to combine VR and 8K to help NHK transition into a new way of telling stories. But it’s important: although my company has started shooting 360-degree movies, most of my colleagues don’t know the effects of removing a 2D rectangle frame; neither do they know how to tell stories in nonlinear ways. 

I see my work as influencing new directions for my company. 

During my Media Lab year, I based my project on the mission of Collective Learning: improve learning by transforming data into stories. For my needs, I also had to find a way to transform the data into a user experience in VR, where a user would “feel” the data in a 3D environment. At first, I called my project Time Traveler. Then it became Sci Fi Story. Finally, we named it Biodigital. 

Immersing myself in VR

Biodigital is a fictional virtual reality experience that combines VR film, immersive 3D environments, and VR data visualization. It combines a science fiction story with data visualization in virtual reality, to turn data into a cinematic experience where a user is enmeshed as a character in the story. Biodigital tells the tale of humanity a hundred years from now, in a world where humans are embodied in machines that combine biological and digital parts. We put the user in this world, and encourage them to think “How should we live in the future?”

While creating Biodigital, I realized that you cannot make technology without considering its ethical and social ramifications. How should we use technology? How do we adapt it to society? We wove such questions throughout the Biodigital story.

Biodigital is not yet public, but we showed it privately during the Lab’s spring event last month and received fantastic feedback. It was a wonderfully collaborative project, not only with César, but also with people from other research groups, a local sound engineer, a designer in Chile, and a student from Harvard University. Before my time at the Lab, I’d never experienced that kind of collaboration, where researchers with different specialties share their expertise, helping me to come up with new ideas as a result. It was another lesson for me.

Before my time at the Lab, I’d never experienced that kind of collaboration.

I also discovered how sharing goes both ways. Often I was asked for feedback on other projects at the Lab, and last year, I got a chance to give back by being an advisor in the MIT Summer Research Program (MSRP.) My student Daniel Diaz-Etchevehere came from the University of Rochester in New York, and I helped him to create an interactive tour of Washington, DC. Daniel, in turn, helped me to improve my English!

Key takeaways

There are 27 research groups at the Media Lab, plus several other research initiatives and special interest groups. One of the first things I noticed was how freely people would move among the disparate disciplines. For instance, I was able to call on the special expertise of designers, artists, engineers, and scientists for input and feedback on my Biodigital project.

Of all the lessons from my time at the Lab, three priorities stand out:

• Story: The Lab’s principal investigators and researchers think about and explore what I call “the far future”—they’re creators who seek to transform science fiction into a real story. That amazed me.
  
• Purpose: I realized that it’s our responsibility to question how we use technology, and what we use it for. How do we connect our research to society? This was a year where I had time to explore and think about the world beyond.
  
• Prototype: Once I made a prototype of my project, I found it easier to demonstrate, discuss, and improve it. In fact, I’m still revising Biodigital based on the comments I received at the Lab’s most recent member meeting which took place just days before I was heading back to Japan.
  

Changing perspectives

Now that I’m back home in Tokyo, I shall continue to work on Biodigital, staying in contact with César Hidalgo and my fellow researchers in Collective Learning. I’ve come to see my group as a “ship” of sorts, with César as the captain, while students and other researchers are the crew. When I first joined, I was like a passenger on the ship. A year later, I see myself as a crew member.

But I’m a crew member with a difference: I now have a broader perspective of my contribution to NHK—a bird’s eye view that will help me do work that helps my company not only to survive the evolving broadcast industry, but also to adapt by exploring and creating breakthrough technologies.

My advice for any visiting scientist who joins the Media Lab is simple: dive right in! 

Takahito Ito was a visiting scientist in the MIT Media Lab’s Collective Learning group. He’s a CGI (computer-generated imagery) supervisor at NHK, the Japan Broadcasting Corporation, which is the largest broadcaster in the country.

Acknowledgments: Team Biodigital : César Hidalgo (Associate Professor of Media Arts and Sciences, head of the Media Lab’s Collective Learning research group), Daniel Maskit (Research Affiliate, Social Machines), Daniel Magnani (audio engineer in Cambridge, MA), Federico Wilckens Claussen (designer in Chile), Cristian Jara Figueroa (research assistant, Collective Learning), Kally Wu (Harvard University student), Mika Kanaya (NHK), and Daniel Diaz-Etchevehere (Rochester University student) and all my “crew” mates in Collective Learning.