CS59200-TMP: Topics in Machine Perception (Fall 2022)

Images generated from DALL-E-2 with text prompt A robot learning to perceive the world

Course Information

This course covers the concepts and techniques for conducting research in the area of machine perception, i.e., how to enable machines to sense the world (with focus on deep learning methods with computer vision applications). The lectures are designed to lead discussions and facilitate student presentations on selected advanced topics in the area. We will cover three main areas, (1) predictive models, (2) generative models, and (3) other recent advances. The course aims to develop students' knowledge and analysis capabilities for understanding research publications in machine perception, e.g., papers from CVPR, ICCV, ECCV, NeurIPS, etc.

Pre-requisites:

Probability
Linear algebra
Proficiency in Python

Textbooks (Optional):

[Murphy2022a] Murphy, K. Probabilistic Machine Learning: An Introduction. (2022) [Available]
[Murphy2022b] Murphy, K. Probabilistic Machine Learning: Advanced Topics. (2022) [Available]
[Szeliski2022] Szeliski, R. Computer Vision: Algorithms and Applications, 2nd ed. (2022) [Available]
[Zhang2021] Zhang, Aston, et al. Dive into deep learning. (2021) [Available]

Grading:

Homework: 30%
Presentation: 35%
Final Project: 35%

Instructor

Raymond A. Yeh

Time & Location

Time: Tuesday, Thursday 10:30AM-11:45AM
Location: LWSN B134

Office Hour: By appointment

Other Materials

BrightSpace

Course Schedule

Date	Type	Description	Materials
---	Topic	Basics Select from the following:
Aug. 23	Lecture 1	Intro to Machine Perception
Aug. 25	Lecture 2	Linear Models
Aug. 30	Lecture 3	Optimization
Sept. 1	Lecture 4	Deep-Nets and Backpropagation
Sept. 1	Deadline	Presentation Preference Due Select from the following:
Sept. 6	Lecture 5	How to Read and Review Papers
---	Topic	Perception Select from the following:
Sept. 8	Lecture 6	Convolution Neural Networks-I
Sept. 13	Lecture 7	Convolution Neural Networks-II
Sept. 15	Discussion 1	Student Presentation Select from the following: Aggregated residual transformations for deep neural networks Densely Connected Convolutional Networks MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications DARTS: Differentiable Architecture Search Designing Network Design Spaces	[Expand]
Sept. 20	Lecture 8	Object Detection & Segmentation
Sept. 22	Discussion 2	Student Presentation Select from the following: Bottom-up Object Detection by Grouping Extreme and Center Points Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression Speed/accuracy trade-offs for modern convolutional object detectors Rethinking Atrous Convolution for Semantic Image Segmentation Soft-NMS -- Improving Object Detection With One Line of Code	[Expand]
Sept. 27	Lecture 9	Recurrent Neural Network
Sept. 29	Discussion 3	Student Presentation Select from the following: Pixel Recurrent Neural Networks Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences Joint Optimization of Masks and Deep Recurrent Neural Networks for Monaural Source Separation A multi-stream bi-directional recurrent neural network for fine-grained action detection. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling	[Expand]
Oct. 4	Lecture 10	Graph Neural Network
Oct. 6	Discussion 4	Student Presentation Select from the following: PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation Dynamic Graph CNN for Learning on Point Clouds Videos as Space-Time Region Graphs Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition	[Expand]
Oct. 11	---	October Break Select from the following:
Oct. 13	Lecture 11	Attention and Transformers
Oct. 18	Lecture 12	Vision Transformers
Oct. 20	Discussion 5	Student Presentation Select from the following: MViTv2: Improved Multiscale Vision Transformers for Classification and Detection Efficient Transformer for High-Resolution Image Restoration CvT: Introducing Convolutions to Vision Transformers Neighborhood Attention Transformer	[Expand]
Oct. 20	Deadline	Project proposal due Select from the following:
Oct. 20	Deadline	Homework (Review 1) Due Select from the following:
---	Topic	Generation Select from the following:
Oct. 25	Lecture 13	Variational Auto-Encoders
Oct. 27	Discussion 6	Student Presentation Select from the following: Semi-Supervised Learning with Deep Generative Models Neural Relational Inference for Interacting Systems Neural Discrete Representation Learning NVAE: A Deep Hierarchical Variational Autoencoder	[Expand]
Nov. 1	Lecture 14	Diffusion Models
Nov. 3	Discussion 7	Student Presentation Select from the following: High-Resolution Image Synthesis with Latent Diffusion Models Diffusion Autoencoders: Toward a Meaningful and Decodable Representation RePaint: Inpainting using Denoising Diffusion Probabilistic Models Classifier-Free Diffusion Guidance	[Expand]
Nov. 8	Lecture 15	Generative Adversarial Networks
Nov. 10	Discussion 8	Student Presentation Select from the following: Unpaired image-to-image translation using cycle-consistent adversarial networks A style-based generator architecture for generative adversarial networks Analyzing and Improving the Image Quality of StyleGAN Self-Attention Generative Adversarial Networks In-Domain GAN Inversion for Real Image Editing	[Expand]
---	Topic	Other Topics Select from the following:
Nov. 15	Lecture 16	Self-Supervised Learning
Nov. 17	Discussion 9	Student Presentation Select from the following: Unsupervised Feature Learning via Non-parameteric Instance Discrimination Space-Time Correspondence as a Contrastive Random Walk A Simple Framework for Contrastive Learning of Visual Representations Emerging Properties in Self-Supervised Vision Transformers Masked Autoencoders Are Scalable Vision Learners	[Expand]
Nov. 22	Lecture 17	Language and Vision
Nov. 24	---	Thanksgiving Break Select from the following:
Nov. 29	Discussion 10	Student Presentation Select from the following: An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion Language-driven Semantic Segmentation Simple Open-Vocabulary Object Detection with Vision Transformers Image-based CLIP-Guided Essence Transfer Grounded Language-Image Pre-training	[Expand]
Dec. 1	Lecture 18	Audio and Vision
Dec. 6	Discussion 11	Student Presentation Select from the following: Visual Acoustic Matching MERLOT Reserve: Multimodal Neural Script Knowledge through Vision and Language and Sound SoundNet: Learning Sound Representations from Unlabeled Vide The Sound of Pixels	[Expand]
Dec. 6	Deadline	Homework (Review 2) Due Select from the following:
Dec. 8	Lecture 19	Final Project Spotlights
Dec. 8	Lecture 20	Final Project Report Due

CS59200-TMP: Topics in Machine Perception (Fall 2022)

Course Information

Pre-requisites:

Textbooks (Optional):

Grading:

Instructor

Raymond A. Yeh

Time & Location

Other Materials

Course Schedule

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Select from the following:

Policies

Late Policy

Academic Honesty

Classroom Guidance Regarding Protect Purdue

University Policies