Dense 3D Visual Mapping: A Rich Sensory Environment for Robotic Agents
With this project we hope to push the limits of what can be done with monocular vision, a rich and ubiquitous sensor for mobile robots. This work combines principles of robotics, particularly probabilistic robotics, with well-established results in computer vision, especially the subfield known as "structure from motion."
Mapping with a single camera is a compelling problem because we humans have no trouble wandering a new environment (even with one eye closed). What's more, we can subsequently use that experience to get around and perform tasks. In contrast to systems that use laser range finders, robots with a webcamera— in principle, at least—have the ability to create and use richer environmental maps at a fraction of the cost. The tradeoff, however, comes in processing power: the computational work of analyzing the input image stream is substantial. This is particularly true because our map representation is task-independent: it is simply a 3d texture-mapped model of the environment.
In 2007, S. Cord Melton (U. Chicago '09) and Lilia Markham (HMC '08) developed an end-to-end system for region-based visual reconstruction. Their wiki site contains many details, with Cord's and Lilia's journal entries giving a sense of what day-to-day REU experience is like. In addition, their paper was accepted and presented at the ISC '07 conference in Cambridge, MA.
To handle the inevitable difficulties in the "low-level" visual processing—image segmentation, feature extraction, and matching—the 2008 REU project will investigate machine-learning approaches to these problems. The REU participants will likely create and compare both supervised and unsupervised algorithms. The foundation from last year will allow us to use reconstruction fidelity as a basis for comparisons.
Progress on this problem requires continually developing and refining the software tools for analyzing and displaying image-based data. We will combine and extend existing software (OpenCV and others) using C++, Python, and OpenGL. The department has many different robot platforms available: currently we lean toward iRobot Creates for indoor applications and modified PowerWheels vehicles for outdoor autonomy.
Web summaries of previous HMC work in this area are linked here: Summer '05 robotics, Summer '05 vision and 2004-5. Related papers with student authors include ICARA '04, AAAI Robot Competition '05, AAAI Robot Exhibition '06, ICRA 2007, ISC 2007.
Mentor: Professor Zach Dodds
Zach has been a professor
at Harvey Mudd since 1999. His general research interests are in
computer vision and robotics. He received his B.A. in mathematics
and M.S. and Ph.D. in computer science from Yale University. In
addition to research and teaching, he likes to play in foam pits
with his children.
Required Background
While background in computer vision and robotics/robotics algorithms is wonderful, it's not required. Students interested in working on this project should have completed at least a course in C++ programming and data structures and should feel confident in applying their programming ability through new frameworks and for new problems (e.g., vision & robotics...). We will use Python a lot, too, but it is so easy to learn that it's not worth worrying about beforehand.
