Computer Science 153 ~ Assignment 6
Due before Thanksgiving break (Wed., 11/26)



Tour into the Picture, i.e., 3d modeling from a single-view

Acknowledgments: the idea and scaffolding for this project is due to A. Efros

  

Overview

The goal of this project is to create a simple, planar 3D scene from a single photograph. The project will follow the description in Tour into the Picture by Horry et al. in modeling the scene as a 3D axis-parallel box.

First, your system should let the user specify simple constraints on that box (the back wall plus the vanishing point). Then, it's a matter of extracting the coordinates of the box in 3D, and texture-mapping the faces of the box. The paper has a rather poor description of the process, so you may want to consult the lecture notes or section 2.2 of this thesis for details. Here is a pdf version of only that section of the thesis.

For this project, you are free to use any programming language you like. OpenCV's input mechanisms (clicking to specify points and lines) will suffice to designate the box. Alternatively, you can do everything in Matlab... Alyosha Efros has sample starting code available at this link. The input interface is a bit messy (one option for an extension is to improve/rewrite it); the matlab code also shows how to setup texture-mapped surfaces in Matlab.

The project itself is not too involved: you will need to compute the 3D coordinates of each vertex of each of the five planes that make up the scene's bounding box. Then define the 3D geometry corresponding to these planes. Finally, use your homography warping code from previous projects to rectify the textures for the planes and then texturemap them onto the 3D model. You should demonstrate the ability to move and rotate the camera, looking at the scene from different viewpoints. The only other parameter that you will need to worry about is the focal length f. You can just guess it (a wrong f will simply mean that your scene is too deep or too shallow) and then adjust in order to make the scene look reasonable.

What to hand in

Extensions



Good luck with this hw #6!