How Robot Dogs See the Unseeable: Improving Visual Interpretability via Peering for Exploratory Robots

If our eyes would be 1m in diameter, we would not need to do this.

Vegetation-induced occlusion remains a fundamental challenge for robotic scene understanding, regardless of the sensor modality used (e.g., cameras, LiDAR, or radar).

Unlike humans, many insects lack stereoscopic vision for depth perception. Instead of single-lens eyes, they possess compound eyes. To gauge the distance to prey or obstacles, they sway their bodies from side to side (peering). This movement generates motion parallax, which serves as a primary depth cue.

Applying this peering strategy to robotic occlusion removal is motivated by principles in optics and signal processing: A standard camera features a small lens and a wide depth of field, meaning objects across a broad distance range (including occluders) remain sharp. If a robot were equipped with a massive lens (e.g., 1 m in diameter), focusing behind occluding vegetation would yield an extremely shallow depth of field, optically blurring the occluders until they effectively disappear from the captured signal. Because such a physical lens is infeasible, we adopt synthetic aperture sensing from signal processing: we mimic a large lens by sequentially capturing images with a standard camera as the robot peers. The distance covered during peering defines the synthetic aperture (i.e., the virtual lens diameter). By computationally combining these frames using motion parallax, we synthesize an image with an extremely shallow depth of field.

If our eyes would be 1m in diameter, we would not need to do this. But since our eyes’ apertures are only a few millimeters, we need to move. Our brain combines these images to one with suppressed occlusion.

Ongoing thread. More from Oliver Bimber to follow.