Project: Diminished Reality

Description: 

We treat a video sequence as a space time 3D volume. Keeping the temporal coherence then becomes equivalent to preserving the smoothness of space-time motion trajectories of pixels. We construct the sub-manifolds (Figure 1 (c)) of the space-time volume, which we refer to as the motion manifolds, where the entire trajectory of a pixel is projected on to a 2D curve, thus reducing the search space from 3D to 2D. By enforcing spatial and temporal coherence in the propagation of the manifold to the areas of missing data due to object removal, we can restore a visually realistic video sequence with no or negligible visual artifacts. Our approach involves four main steps: (1) Separation of foreground and background, (2) Foreground volume rectification, (3) Motion pattern propagation, (4) Repairing background.

Keywords: Diminished Reality, Object Removal, Time-space Volume, Motion Manifold.

Publications: 

  • Yuping Shen, Fei Lu, Xiaochun Cao, and Hassan Foroosh, "Video Completion for Perspective Camera Under Constrained Motion", IAPR International Conference on Pattern Recognition (ICPR), 2006.

How does it work? 

Videos:

Example 1: moving Camera
Example 2: perspective distortion
Example 3: lighting variation