Skip to main content
Part of complete coverage on

A camera that can see around corners

By Ramesh Raskar, Special to CNN
August 19, 2012 -- Updated 1305 GMT (2105 HKT)
STORY HIGHLIGHTS
  • Ramesh Raskar's team at the MIT Media Lab is re-imagining what photography can do
  • By creating movies of light in motion at a trillion frames per second, camera can see around corner
  • Raskar: Technology is years away from applications in the real world
  • He says it could be helpful in preventing car crashes and aiding rescues

Editor's note: Ramesh Raskar is Associate Professor at MIT Media Lab and head of the Camera Culture research group. He is an Alfred P. Sloan Research Fellow (2009), a recipient of the DARPA Young Faculty award, the co-author of "Spatial Augmented Reality", and the holder of more than 50 US patents. He spoke at the TEDGlobal conference in June. TED is a nonprofit dedicated to "Ideas worth spreading" which it makes available through talks posted on its website.

(CNN) -- A camera that can see around a corner?

I know this sounds like something in a sci-fi movie or a superhero comic, but this is a real-world technology we've made possible with a camera that is aware of the travel time of light, an imaging technique that can create movies of light in motion with an effective rate approaching a trillion frames per second: the speed of light.

Before I joined the MIT faculty in 2008, I had done deep research in "computational photography," a field of new imaging techniques dramatically improving the capture and synthesis of photos. But, I knew there was more to photography than just depicting what the eye can see. I wanted to create a camera that could see beyond the line of sight. The speed of light isn't infinite: light travels about a foot per billionth of a second.

Photography could make X-rays obsolete

If I could build a camera fast enough to analyze light at high speeds in room-sized environments, I knew we could then create cameras to solve major problems in scientific and consumer imaging, and enable completely new functionality.

I spoke to top researchers in ultrafast lasers and photonics to understand what was currently possible. When I did, most of them asked some version of: "Why? Why spend years building a camera to look around corners when no commercial application is screaming for it and no funding agency has a call for it?" In addition, it's rare to shoot light pulses and analyze at such high speeds in large environments. Ultra-fast imaging experiments are usually limited to centimeter- or smaller-size samples.

I continued the work and in the spring of 2008, with James Davis from UC Santa Cruz, wrote a proposal that laid out the mathematical foundation and various experimental solutions for exploration.

TED.com: Boaz Almog 'levitates' a superconductor

I also began working with Media Lab graduate student Matthew Hirsch to build a prototype, hoping that we would have something to demonstrate within a few months. The grant proposal was rejected for administrative reasons (I made a paperwork error!), which meant we had to wait nearly a year to apply again. But those two years didn't yield any meaningful results, as our lab components weren't designed to be used the way we wanted.

After nearly three years of experimental work, the team -- especially postdoctoral associate Andreas Velten and MIT professor Moungi Bawendi, many students, and several collaborators -- cobbled together pieces of the puzzle and built a software program to create a first demonstration of looking around corners. Very soon afterward, we could also start creating surreal movies of light in motion.

We call this new imaging technology femto photography because we capture a segment of the image with a flashlight (in this case, a laser pulse) on for a few millionths of a billionth of a second (or a few femtoseconds) and an exposure time approaching a trillionth of a second.

Just how fast is femto photography? Think of it this way: if we took one-thousandth of a second of footage from the femto camera video output and slowed it down to the speed of 30 seconds per frame -- the approximate speed of a standard TV broadcast -- it would take us a lifetime to watch.

Photographers know that at very short exposures and even at the most sensitive setting for dark scenes, we will record barely any light. So what about in a trillionth of a second? We actually record and average millions of photos to get enough light, each photo made to look the same via carefully timed synchronization with the light pulse. So even if our exposure time is indeed nearly a trillionth of a second, to get sufficient light we must take an average. Thus, as of now, we can only record repeatable events, but this is not a fundamental limitation.

TED.com: A robot that flies like a bird

Unlike conventional cameras, our femto camera captures an image as one thin slice at a time of one-dimensional space using a "streak tube," a laboratory instrument that is commonly used by chemists to study light passing through chemical samples. We then take hundreds of these narrow videos (each shot at a slightly different angle) and create a carefully synchronized, slow-motion composite of the light pulses. It takes about an hour to collect and aggregate the data (view a demonstration of a light pulse as it travels through an ordinary Coca-Cola bottle).

To see around the corner, we use femto photography to analyze scattered light. We bounce light off of visible parts into hidden parts and then measure the time and direction of returned light.

Usually the scattering of energy is considered a nuisance -- whether driving in fog or poor reception from a cell phone tower -- and most techniques either try to avoid it (by turning on fog lights) or reduce the impact of scattering (by selecting energy for the phone only from direct paths).

In contrast, we exploit the scattering. For the camera, a laser pulse is fired at a wall, and the impact of hitting the wall causes the particles of light to scatter. Some of the scattering particles return to the camera at different times. This is repeated about 60 times per image as the camera measures how long it takes for the light to travel back and where the particles land. An algorithm then crunches the data to reconstruct the hidden image. This technique even allows us to see a three-dimensional object such as a mock-up of running person.

TED.com: Test for Parkinson's with a phone call

As exciting as this work is, don't look for this technology to be in practice tomorrow -- we're still years away from bringing this to market. But, we can already imagine multiple ways that it could have a significant, positive impact on our everyday lives.

By allowing us to "see" around a corner, for example, this technology added to our cars could let us know if there's another vehicle approaching around a blind curve. It also could give us a new way to look deep inside our bodies without X-rays, or to look through a window into a burning building from a distance to see if anyone is left inside --without risking a firefighter's life.

When I gave a TEDGlobal talk on femto photography in June, I began with a reference to Doc Edgerton, a very popular MIT professor of electrical engineering who, in 1964, wowed the world with an image of a bullet in midair, having just passed through an apple. He accomplished this by using a strobe light to freeze the action of the bullet at a millionth of a second.

What we're talking about here -- the speed of light --is a million times faster, and is opening the door to a complete rethinking of what we mean by, and can do with, photography. It is a first step toward a new world of imaging that far exceeds human ability, resynthesizing data and depicting it in ways that are within human comprehensibility.

Follow @CNNOpinion on Twitter

Join us at Facebook/CNNOpinion

The opinions expressed in this commentary are solely those of the author, Ramesh Raskar, who would like to thank his collaborators: Moungi G. Bawendi, Professor, MIT Department of Chemistry; Andreas Velten and Christopher Barsi, postdoctoral associates, MIT Media Lab; Everett Lawson, Otkrist Gupta, Nikhil Naik, Amy Fritz, Di Wu, MIT Media Lab; Matt O'toole, Kyros Kutulakos, University of Toronto; Diego Gutierrez, Belen Masia, and Elisa Amoros, Universidad de Zaragoza; Kavita Bala, Shuang Zhao, Cornell University; Ashok Veeraraghavan, Rice University.; Thomas Willwacher, Harvard University.

ADVERTISEMENT
Part of complete coverage on
August 27, 2014 -- Updated 1626 GMT (0026 HKT)
The death of Douglas McAuthur McCain, the first American killed fighting for ISIS, highlights the pull of Syria's war for Western jihadists, writes Peter Bergen.
August 26, 2014 -- Updated 2242 GMT (0642 HKT)
Former ambassador to Syria Robert Ford says the West should be helping moderates in the Syrian armed opposition end the al-Assad regime and form a government to focus on driving ISIS out
August 27, 2014 -- Updated 1321 GMT (2121 HKT)
Ruben Navarrette says a great country does not deport thousands of vulnerable, unaccompanied minors who fled in fear for their lives
August 27, 2014 -- Updated 1319 GMT (2119 HKT)
Robert McIntyre says Congress is the culprit for letting Burger King pay lower taxes after merging with Tim Hortons.
August 26, 2014 -- Updated 2335 GMT (0735 HKT)
Wesley Clark says the U.S. can offer support to its Islamic friends in the region most threatened by ISIS, but it can't fight their war
August 27, 2014 -- Updated 1126 GMT (1926 HKT)
Jeff Yang says the tech sector's diversity numbers are embarrassing and the big players need to do more.
August 26, 2014 -- Updated 2053 GMT (0453 HKT)
America's painful struggle with racism has often brought great satisfaction to the country's rivals, critics, and foes. The killing of Michael Brown and its tumultuous aftermath has been a bonanza.
August 26, 2014 -- Updated 2019 GMT (0419 HKT)
Ed Bark says in this Emmy year, broadcasters CBS, ABC and PBS can all say they matched or exceeded HBO. These days that's no small feat
August 26, 2014 -- Updated 1919 GMT (0319 HKT)
Rick Martin says the death of Robin Williams brought back memories of his own battle facing down depression as a young man
August 26, 2014 -- Updated 1558 GMT (2358 HKT)
David Perry asks: What's the best way for police officers to handle people with psychiatric disabilities?
August 25, 2014 -- Updated 1950 GMT (0350 HKT)
Julian Zelizer says it's not crazy to think Mitt Romney would be able to end up at the top of the GOP ticket in 2016
August 25, 2014 -- Updated 2052 GMT (0452 HKT)
Roxanne Jones and her girlfriends would cheer from the sidelines for the boys playing Little League. But they really wanted to play. Now Mo'ne Davis shows the world that girls really can throw.
August 25, 2014 -- Updated 1629 GMT (0029 HKT)
Peter Bergen and Emily Schneider say a YouTube video apparently posted by ISIS seems to show that the group has a surveillance drone, highlighting a new reality: Terrorist groups have technology once only used by states
August 25, 2014 -- Updated 2104 GMT (0504 HKT)
Kimberly Norwood is a black mom who lives in an affluent neighborhood not far from Ferguson, but she has the same fears for her children as people in that troubled town do
August 22, 2014 -- Updated 2145 GMT (0545 HKT)
It apparently has worked for France, say Peter Bergen and Emily Schneider, but carries uncomfortable risks. When it comes to kidnappings, nations face grim options.
August 26, 2014 -- Updated 1727 GMT (0127 HKT)
John Bare says the Ice Bucket Challenge signals a new kind of activism and peer-to-peer fund-raising.
August 22, 2014 -- Updated 1231 GMT (2031 HKT)
James Dawes says calling ISIS evil over and over again could very well make it harder to stop them.
August 24, 2014 -- Updated 0105 GMT (0905 HKT)
As the inquiry into the shooting of Michael Brown continues, critics question the prosecutor's impartiality.
August 22, 2014 -- Updated 2247 GMT (0647 HKT)
Newt Gingrich says it's troubling that a vicious group like ISIS can recruit so many young men from Britain.
August 21, 2014 -- Updated 1450 GMT (2250 HKT)
David Weinberger says Twitter and other social networks have been vested with a responsibility, and a trust, they did not ask for.
August 22, 2014 -- Updated 1103 GMT (1903 HKT)
John Inazu says the slogan "We are Ferguson" is meant to express empathy and solidarity. It's not true: Not all of us live in those circumstances. But we all made them.
August 22, 2014 -- Updated 1223 GMT (2023 HKT)
Retired Lt. Gen. Mark Hertling says he learned that the territory ISIS wants to control is amazingly complex.
August 20, 2014 -- Updated 1951 GMT (0351 HKT)
Cerue Garlo says Liberia is desperate for help amid a Ebola outbreak that has touched every aspect of life.
August 21, 2014 -- Updated 1742 GMT (0142 HKT)
Eric Liu says Republicans who want to restrict voting may win now, but the party will suffer in the long term.
August 21, 2014 -- Updated 1538 GMT (2338 HKT)
Jay Parini: Jesus, Pope and now researchers agree: Wealth decreases our ability to sympathize with the poor.
August 21, 2014 -- Updated 1200 GMT (2000 HKT)
Judy Melinek offers a medical examiner's perspective on what happens when police kill people like Michael Brown.
August 19, 2014 -- Updated 2203 GMT (0603 HKT)
It used to be billy clubs, fire hoses and snarling German shepherds. Now it's armored personnel carriers and flash-bang grenades, writes Kara Dansky.
August 20, 2014 -- Updated 1727 GMT (0127 HKT)
Maria Haberfeld: People who are unfamiliar with police work can reasonably ask, why was an unarmed man shot so many times, and why was deadly force used at all?
ADVERTISEMENT