Project: Shadow Synthesis

Description: 

In this project, we present an approach for creating realistic shadows of objects composited into a novel background. Existing techniques either do not deal with the shadows or require that the relationship of the light source, reference plane, and camera be the same in the source and target scenes. We observe that scenes containing a ground plane and some up-right objects, e.g. walls, crowds, desks, street lamps, trees, etc., are common in natural environments, and show that two vertical lines and their cast shadows are enough to recover primary light source, which makes it possible to construct shadows of inserted planar objects in the single target image. We also demonstrate a technique to calibrate the camera when two views are available, which enables us to create shadows of general 3D objects in a target scene. The proposed method therefore advances the image based techniques one step further toward the synthesis of the scenes with variation of both lighting and viewpoints.

For synthesizing reflections, we assume that the target scene contains a ground plane and some up-right vertical objects, e.g. walls, crowds, desks, street lamps, trees, etc., which are common in both indoor and outdoor environments. We divide the problem of synthesizing reflections of inserted objects into two subproblems. From the geometric point of view, we need to synthesize reflections which would be seen by the same camera used in capturing the target scene and be reflected by the true reflective media in the target scene. Physically, we aim to infer the most likely rendered reflections given the set of known reflection patches in the target view. This problem is formulated as a Maximum A Posteriori (MAP) estimation problem.

Keywords: Shadow Synthesis, Reflection Synthesis, Homography, Cross-ratio, MAP.

Publications: 

  • Xiaochun Cao, and Hassan Foroosh, "Camera Calibration and Light Source Orientation from Solar Shadows", Computer Vision and Image Understanding (CVIU), to appear, 2006.
  • Xiaochun Cao, and Hassan Foroosh, "Synthesizing Reflections of Inserted Objects", Proc. International Conference on Pattern Recognition, Hong Kong, 2006.
  • Xiaochun Cao, Yuping Shen, Mubarak Shah and Hassan Foroosh, "Single View Compositing with Shadows", The Visual Computer, Vol. 21, Issue 8, pp. 639-648, 2005.

(a) and (b) two views from camera and light source points of view, (c) the object matte from light source view, (d) projection of the object mat in (c) onto the ground plane via the homography..

Starting from two views (a) and (b), we first calibrate the camera and compute the light source orientation. The square marks are the corresponding points between the two images, which are computed by using the method in [48]. As a result, we can render a virtual teapot with known 3D model in the real scene (b) shown in (c). Using this computed geometric information, we can also insert a second instance of a person seen in (d) into (b) as shown in (f). (e) is the view from light source and is used for shadow map.

Reflection synthesis of the source and target image illustrating the source, and the target composition with both shadow and reflection.