-- CS635 (Spring 2010) --

Capturing and Rendering Real-World Scenes

Have you ever wondered how to create models of 3D objects? Have you ever wanted to create a model of an entire room, floor, or building? Have you ever wanted to add real-world environments and objects to your games and virtual worlds? If so, this is the course for you!!!

Description: pic5  Description: pic4  Description: pic1  Description: pic6

Description: pic3  Description: pic2

[Course Teaser PDF]

[Course Summary PDF]

1. Overview

The objective of this course/seminar is to understand the fundamental problems and challenges encountered when capturing, modeling, and rendering 3D structures and objects. The course covers several subjects within computer graphics, computer vision, and computer science so as to provide to the student a full understanding of the capture/model/render pipeline. From this understanding and cross-fertilization of ideas, it is expected that students will in the future be able to develop new and improved approaches.

Classroom: LWSN 1106

Time: TTh 3-4:15pm

Office hours: by appointment

Topics: a short list of the tentative topics to be covered in this course, based on the latest research results in the field, are:

-          Geometry-based Acquisition

o   Stereo Methods

o   Structured Light Methods

-          Image-based Acquisition

o   Image-based Rendering Algorithms (Lumigraphs/Lightfields)

o   Photometric Stereo Methods

-          Transport- and Photogeometric-based Methods

o   Light Transport-based methods

o   Hybrid algorithms combining geometric and image-based methods

-          Calibration and other Fundamentals

o   Review of Optimization Methods

o   Pose Estimation

o   Pose-Free Calibration

Prerequisites: students are required to have previous programming experience and are recommended to have some previous computer graphics, computer vision/machine learning, or visualization programming experience.

2. Tentative Schedule

 

Date

Lecture

Assignment

Final Project

Jan 12

Introduction

 

 

Jan 14

OpenGL, GLUT, GLUI, OpenCV

 

 

Jan 19

Camera Models

Asgn #0 out

 

Jan 21

Geometric Camera Calibration

 

 

Jan 26

Color Models

Asgn #0 in; Asgn #1 out

 

Jan 28

Radiometric Camera Calibration

 

 

Feb 2

Stereo and 3D Reconstruction I

 

 

Feb 4

Stereo and 3D Reconstruction II

 

 

Feb 9

Images and Features

Asgn #1 in; Asgn #2 out

 

Feb 11

Image and View Morphing, Image Warping

 

 

Feb 16

Structured Light: overview, static scenes

 

 

Feb 18

Structured Light: dynamic scenes

 

 

Feb 23

Light Transport

Asgn #2 in; Asgn #3 out

 

Feb 25

Inverse Light Transport

 

Project Suggestions given

Mar 2

Final Project Brainstorming

 

 

Mar 4

Photometric and Photogeometric Stereo I

 

 

Mar 9

Photometric and Photogeometric Stereo II

Asgn #3 in

Declare Project

Mar 11

Review/TBA

 

Provide Project Summary

Mar 16

Spring Break

 

 

Mar 18

Spring Break

 

 

Mar 23

Project Background Presentations I

 

Background Presentations

Mar 25

Project Background Presentations II

 

 

Mar 30

Lumigraph/Lightfield I

 

 

Apr 1

Lumigraph/Lightfield II

 

 

Apr 6

Mid-Project Presentation I

 

Mid-Project Presentations

Apr 8

Mid-Project Presentation II

 

 

Apr 13

Space Carving/Voxel Coloring

 

 

Apr 15

Relighting and Lightfields

 

 

Apr 20

Recon. from Single Images, Art Gallery Problem

 

 

Apr 22

Aspect Graph Problem, Pose-Free

 

 

Apr 27

Demo Rehearsals

 

 

Apr 29

Public Demo Day

 

Due Date

 

Additional resources:      

BasicModeler.zip

BasicModeler-Models.zip

LMDif.zip

Triangulation.zip

LSQR: http://www.stanford.edu/group/SOL/software/lsqr.html

Barycentric.txt

 

 

For more information about the general type of research, I recommend looking at:

CGVLab Webpage: http://www.cs.purdue.edu/cgvlab

My webpage: http://www.cs.purdue.edu/~aliaga

For graphics in general: http://kesen.huang.googlepages.com

3. Workload

The course is divided into two parts.

·         The first part describes, during the lectures, research methods to be presented by reviewing the latest works in the field. The students will also present informal presentations and summaries about work relevant to their projects. An exact schedule is to be determined once the semester starts.

·         The second part of the course consists of a short set of assignments and then a substantial final project. The assignments provide guided programming projects that progressively implement a basic system to build 3D models from images. An initial software package is given so that the students can immediately focus on the algorithms. The core effort of the course is in the final project. There are multiple deliverables for the final project. Further, a publication/submission for the semester project would be an ideal goal.

The course grade is determined by the performance in the programming assignments, the final project, and class participation. Each assignment will be evaluated during an interactive session with the instructor. The grade depends on a combination of meeting the requirements, the presentation, and the sophistication of the solution. There will be no final exam but rather a public demo day at the end of the semester with all projects.

 

Assignment #0 – Compiling Warm-up (Jan 19 to Jan 26)

Download, compile, and execute the provided software package. The deliverable includes a simple video sequence that is trivial to do with the provided software. The objective is just to “get you up and running for the assignments/project”. If you wish to use your our own framework, please see the instructor.

 

Assignment #1 – Camera Calibration (Jan 26 to Feb 9)

Capture images (using your camera or a loaned camera) and “calibrate” the camera. The resulting calibrated camera should be used to verify correct pose estimation of a pair of images via simple visual feedback (correspondence can be established, for example, manually via mouse clicking).

 

Assignment #2 – Example Real-world 3D Reconstruction (Feb 9 to Feb 23)

Using the previous assignment, reconstruct a 3D object (using triangles) and render the object within an OpenGL program where you can intuitively control the viewpoint and/or object position and orientation.

 

Assignment #3 – View-dependent Texture-Mapping (Feb 23 to Mar 9)

Further extend the previous assignment to implement a view-dependent texture-mapping system so as to yield visually compelling imagery of captured objects.

 

Final Project (final due date Apr 29)

Feb 25: on or before this date project ideas will be given to all students in written form

Mar 2: in-class project brainstorming session

Mar 9: declare project topic

Mar 11: provide a one-page project summary

Mar 23-25: in-class Powerpoint presentation of previous and related research papers to your project

Apr 6-8: in-class Powerpoint presentation of mid-project progress

Apr 27: in-class project demonstration rehearsals

Apr 29: projects due (Public Demo)

 

The grade distribution is tentatively:

Assignments: 30%

Final Project: 65%

Class Participation: 5%

 

A subset of relevant conferences that could be targeted with this semester’s work include:

·                     SIGAsia: ACM SIGGRAPH Asia 2010 (May submission deadline)

·                     ISMAR: IEEE/ACM Int’l Symposium on Mixed and Augmented Reality 2010 (May submission deadline)

·                     PG: Pacific Graphics 2010 (May submission deadline)

·                     EG: Eurographics 2011 (September submission deadline)

·                     Vis: IEEE Visualization 2010 (March submission deadline)

·                     ECCV: European Conference on Computer Vision 2010 (March submission deadline)

 

4. Administrative Issues

All assignments must be handed-in by the specified due date/time. An assignment late by up to one day receives a 50% penalty (e.g., if maximum score is 10, it will be a maximum of 5), by up to two days a 75% penalty and after that a 100% penalty. The final project consists of 3 formal presentations (initial background research, a mid-project presentation, and final project presentation). The exact dates will be established once students and projects are settled. All final project related presentations must be on time; otherwise a grade of 0 is given for that component.

 

All assignments, presentations, and projects must be done individually unless otherwise indicated by instructor. In research, it is highly encouraged to “build upon the shoulders” of others, however due credit must be given to the sources. Unreported copying or plagiarism will give you a failing grade in the course and you will be subject to standard departmental and University policies. For the programming assignments, code obtained from the Internet, books, or other sources may *not* be used. For the final project, previously-written code is permissible pending instructor approval.