-- CS635 (Spring 2010) --
Capturing and Rendering
Real-World Scenes
Have you ever wondered how to create models of 3D objects?
Have you ever wanted to create a model of an entire room, floor, or building?
Have you ever wanted to add real-world environments and objects to your games
and virtual worlds? If so, this is the course for you!!!
1.
Overview
The objective of this course/seminar is to understand the fundamental problems and challenges encountered when capturing, modeling, and rendering 3D structures and objects. The course covers several subjects within computer graphics, computer vision, and computer science so as to provide to the student a full understanding of the capture/model/render pipeline. From this understanding and cross-fertilization of ideas, it is expected that students will in the future be able to develop new and improved approaches.
Classroom: LWSN 1106
Time: TTh 3-4:15pm
Office hours: by appointment
Topics: a short list of the tentative topics to be covered in this course, based on the latest research results in the field, are:
-
Geometry-based
Acquisition
o Stereo Methods
o Structured Light Methods
-
Image-based
Acquisition
o Image-based Rendering Algorithms (Lumigraphs/Lightfields)
o Photometric Stereo Methods
-
Transport-
and Photogeometric-based Methods
o Light Transport-based methods
o Hybrid algorithms combining geometric and image-based methods
-
Calibration
and other Fundamentals
o Review of Optimization Methods
o Pose Estimation
o Pose-Free Calibration
Prerequisites: students are required to have previous programming experience and are recommended to have some previous computer graphics, computer vision/machine learning, or visualization programming experience.
2. Tentative Schedule
|
Date |
Lecture |
Assignment |
Final Project |
|
Jan 12 |
|
|
|
|
Jan 14 |
|
|
|
|
Jan 19 |
|
||
|
Jan 21 |
|
|
|
|
Jan 26 |
Asgn #0 in; Asgn #1 out |
|
|
|
Jan 28 |
|
|
|
|
Feb 2 |
|
|
|
|
Feb 4 |
|
|
|
|
Feb 9 |
Asgn #1 in; Asgn #2 out |
|
|
|
Feb 11 |
|
|
|
|
Feb 16 |
|
|
|
|
Feb 18 |
|
|
|
|
Feb 23 |
Asgn #2 in; Asgn #3 out |
|
|
|
Feb 25 |
|
||
|
Mar 2 |
Final Project Brainstorming |
|
|
|
Mar 4 |
|
|
|
|
Mar 9 |
Asgn #3 in |
Declare Project |
|
|
Mar 11 |
Review/TBA |
|
Provide Project Summary |
|
Mar 16 |
Spring Break |
|
|
|
Mar 18 |
Spring Break |
|
|
|
Mar 23 |
Project Background Presentations I |
|
Background Presentations |
|
Mar 25 |
Project Background Presentations II |
|
|
|
Mar 30 |
|
|
|
|
Apr 1 |
|
|
|
|
Apr 6 |
Mid-Project Presentation I |
|
Mid-Project Presentations |
|
Apr 8 |
Mid-Project Presentation II |
|
|
|
Apr 13 |
Space Carving/Voxel Coloring |
|
|
|
Apr 15 |
|
|
|
|
Apr 20 |
Recon. from Single Images, Art Gallery Problem |
|
|
|
Apr 22 |
|
|
|
|
Apr 27 |
Demo Rehearsals |
|
|
|
Apr 29 |
Public Demo Day |
|
Due Date |
Additional resources:
LSQR: http://www.stanford.edu/group/SOL/software/lsqr.html
Barycentric.txt
For more information about the general type of research, I recommend looking at:
CGVLab Webpage: http://www.cs.purdue.edu/cgvlab
My webpage: http://www.cs.purdue.edu/~aliaga
For graphics in general: http://kesen.huang.googlepages.com
3.
Workload
The course is divided into two parts.
· The first part describes, during the lectures, research methods to be presented by reviewing the latest works in the field. The students will also present informal presentations and summaries about work relevant to their projects. An exact schedule is to be determined once the semester starts.
· The second part of the course consists of a short set of assignments and then a substantial final project. The assignments provide guided programming projects that progressively implement a basic system to build 3D models from images. An initial software package is given so that the students can immediately focus on the algorithms. The core effort of the course is in the final project. There are multiple deliverables for the final project. Further, a publication/submission for the semester project would be an ideal goal.
The course grade is determined by the performance in the programming assignments, the final project, and class participation. Each assignment will be evaluated during an interactive session with the instructor. The grade depends on a combination of meeting the requirements, the presentation, and the sophistication of the solution. There will be no final exam but rather a public demo day at the end of the semester with all projects.
Assignment #0 –
Compiling Warm-up (Jan 19 to Jan 26)
Download, compile, and execute the provided software package. The deliverable includes a simple video sequence that is trivial to do with the provided software. The objective is just to “get you up and running for the assignments/project”. If you wish to use your our own framework, please see the instructor.
Assignment #1 – Camera
Calibration (Jan 26 to Feb 9)
Capture images (using your camera or a loaned camera) and “calibrate” the camera. The resulting calibrated camera should be used to verify correct pose estimation of a pair of images via simple visual feedback (correspondence can be established, for example, manually via mouse clicking).
Assignment #2 –
Example Real-world 3D Reconstruction (Feb 9 to Feb 23)
Using the previous assignment, reconstruct a 3D object (using triangles) and render the object within an OpenGL program where you can intuitively control the viewpoint and/or object position and orientation.
Assignment #3 –
View-dependent Texture-Mapping (Feb 23 to Mar 9)
Further extend the previous assignment to implement a view-dependent texture-mapping system so as to yield visually compelling imagery of captured objects.
Final Project (final
due date Apr 29)
Feb 25: on or before this date project ideas will be given to all students in written form
Mar 2: in-class project brainstorming session
Mar 9: declare project topic
Mar 11: provide a one-page project summary
Mar 23-25: in-class Powerpoint presentation of previous and related research papers to your project
Apr 6-8: in-class Powerpoint presentation of mid-project progress
Apr 27: in-class project demonstration rehearsals
Apr 29: projects due (Public Demo)
The grade distribution is tentatively:
Assignments: 30%
Final Project: 65%
Class Participation: 5%
A subset of relevant conferences that could be targeted with this semester’s work include:
· SIGAsia: ACM SIGGRAPH Asia 2010 (May submission deadline)
· ISMAR: IEEE/ACM Int’l Symposium on Mixed and Augmented Reality 2010 (May submission deadline)
· PG: Pacific Graphics 2010 (May submission deadline)
· EG: Eurographics 2011 (September submission deadline)
· Vis: IEEE Visualization 2010 (March submission deadline)
· ECCV: European Conference on Computer Vision 2010 (March submission deadline)
4.
Administrative Issues
All assignments must be handed-in by the specified due date/time. An assignment late by up to one day receives a 50% penalty (e.g., if maximum score is 10, it will be a maximum of 5), by up to two days a 75% penalty and after that a 100% penalty. The final project consists of 3 formal presentations (initial background research, a mid-project presentation, and final project presentation). The exact dates will be established once students and projects are settled. All final project related presentations must be on time; otherwise a grade of 0 is given for that component.
All assignments, presentations, and projects must be done individually unless otherwise indicated by instructor. In research, it is highly encouraged to “build upon the shoulders” of others, however due credit must be given to the sources. Unreported copying or plagiarism will give you a failing grade in the course and you will be subject to standard departmental and University policies. For the programming assignments, code obtained from the Internet, books, or other sources may *not* be used. For the final project, previously-written code is permissible pending instructor approval.