Physics to the Rescue -- Deep Non-line-of-sight Reconstruction for High-speed Imaging (original) (raw)

We consider non-confocal non-line-of-sight (NLOS) imaging based on the time-of-flight principle (left). A pulsed laser located at l0 illuminates a part of a relay wall. Light bounces off the wall, interacts with the occluded object, scatters to the wall again, and is eventually captured by a time-resolved sensor at position s0 (different from l0).

Our deep model (right) takes a transient measurement and reconstructs the hidden scene in the form of intensity and/or depth images. One key challenge in high-speed NLOS imaging is the approximations in light transport introduced by the hardware in exchange for data acquisition speed. This breaks existing learning based methods that (1) learn from synthetic data that assume an idealized image formation model, and (2) do not account for the domain gap between real and synthetic data in their model design. To this end, we embed two physics priors into our model to regularize its solution space. This ensures that our model, despite being trained exclusively on synthetic data, generalizes well on real captures.