Seeing Around Street Corners: Non-Line-of-Sight Detection (NLOS) and Tracking In-the-Wild Using Doppler Radar (CVPR 2020)

Conventional sensor systems record information about directly visible objects, whereas occluded scene components are considered lost in the measurement process. Non-line-of-sight (NLOS) methods try to recover such hidden objects from their indirect reflections — faint signal components, traditionally treated as measurement noise.

Existing NLOS approaches struggle to record these low-signal components outside the lab, and do not scale to large-scale outdoor scenes and high-speed motion, typical in automotive scenarios. In particular, optical NLOS capture is fundamentally limited by the quartic intensity falloff of diffuse indirect reflections.

In this work, we depart from visible-wavelength approaches and demonstrate detection, classification, and tracking of hidden objects in large-scale dynamic environments using Doppler radars that can be manufactured at low-cost in series production. To untangle noisy indirect and direct reflections, we learn from temporal sequences of Doppler velocity and position measurements, which we fuse in a joint NLOS detection and tracking network over time.

We validate the approach on in-the-wild automotive scenes, including sequences of parked cars or house facades as relay surfaces, and demonstrate low-cost, real-time NLOS in dynamic automotive environments.

Authors:
Nicolas Scheiner (Daimler AG), Florian Kraus, Fangyin Wei (Princeton), Buu Phan (Algolux), Fahim Mannan (Algolux), Nils Appenrodt (Daimler AG), Werner Ritter (Daimler AG), Jürgen Dickmann (Daimler AG), Klaus Dietmayer (Ulm University), Bernhard Sick (University of Kassel),  Felix Heide (Algolux)

Publication:
The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020

Read the paper