CS58700: Foundations Of Deep Learning (Spring 2024)
A class on Deep Learning, digital art.
Course Information
This course provides an integrated view of the key concepts of deep learning (representation learning) methods. This course focuses on teaching principles and methods needed to design and deploy novel deep learning models, emphasizing the relationship between traditional statistical models, invariant theory, and the algorithmic challenges of designing and deploying deep learning models in realworld applications. This course has both a theoretical and coding component. The course assumes familiarity with coding in the language used for stateoftheart deep learning libraries, linear algebra, probability theory, and statistical machine learning.
Prerequisites:
 CS 37300 Data Mining & Machine Learning
 MA 26500 Linear Algebra
 STAT 41600 Probability
Textbook:
 [DL] Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, MIT Press, 2016
 [DLFC] Christopher M. Bishop, Hugh Bishop, Deep Learning  Foundations and Concepts, Springer Cham, 2023
Grading:
The final grade will be curved and no stricter than the cutoff: A+: 97100, A: 9396, A: 9092, B+: 8789, ..., etc.The percentage is computed following (without any rounding):
 Assignments: 50% (12.5% each, 5 assignment with lowest dropped)
 Midterm: 25%
 Final Project: 25%
FAQ:
 Lecture slides will be posted on Brightspace. Some materials are from and copyright by Professor Bruno Bibeiro, do not redistribute.
 The instructor & TAs can be best reached through Ed Discussion. Please post your questions there instead of emailing TAs.
 During office hours or on Ed Discussion, please avoid posting partial homework solutions or asking TAs to "review" your code/solution.
 Tutorial for learning Latex with Overleaf: [Link]
Instructor & TAs
Raymond A. Yeh
Instructor
Email: rayyeh [at] purdue.edu
Office Hour: Wednesday 9AM10AM
Location: Zoom
Md Ashiqur Rahman
Teaching Assistant
Email: rahman79 [at] purdue.edu
Office Hour: Monday 11AM12PM
Location: HAAS 143
Simon Zhang
Teaching Assistant
Email: zhan4125 [at] purdue.edu
Office Hour: Friday 1PM2PM
Location: HAAS G072
Time & Location
 Time: Tuesday & Thursday (9:00 am  10:15 am)
 Location: Krannert Building G018
Other Resource
Course Schedule
The following schedule is tentative and subject to change.
Date  Event  Description  Readings 

Jan 9  Lecture 1  Introduction & Overview


Jan 11  Lecture 2  Supervised Learning


Jan 15  Info.  Assignment 1 Released
Select from the following: 

Jan 16  Lecture 3  Multilayer Perceptron (MLP)


Jan 18  Lecture 4  Optimization


Jan 23  Info.  Cancelled due to weather
Select from the following: 

Jan 25  Lecture 5  Training Deep Neural Networks


Jan 30  Lecture 6  How to Read Papers + Invariant MLP


Feb 1  Lecture 7  Invariant MLP Part 2


Feb 2  Deadline  Assignment 1 Due (Friday Feb 2, 11:59PM)
Select from the following: 

Feb 5  Info.  Assignment 2 Released
Select from the following: 

Feb 6  Lecture 8  Convolution Neural Network Part 1


Feb 8  Lecture 9  Convolution Neural Network Part 2


Feb 13  Lecture 10  Recurrent Neural Network


Feb 15  Lecture 11  Set Representation Learning


Feb 20  Lecture 12  Graph Neural Network


Feb 22  Lecture 13  Attention Layer + Transformer Architectures


Feb 23  Deadline  Assignment 2 Due (Friday Feb 23, 11:59PM)
Select from the following: 

Feb 26  Info.  Assignment 3 Released
Select from the following: 

Feb 27  Lecture 14  Variational AutoEncoder


Feb 29  Lecture 15  Diffusion Model


March 5  Lecture 16  Generative Adversarial Network


March 7  Lecture 17  Structuring Machine Learning Projects


Mar 8  Deadline  Assignment 3 Due (Friday Mar 8, 11:59PM)
Select from the following: 

Mar 8  Deadline  Project Proposal Due (Friday Mar 8, 11:59PM)
Select from the following: 

March 12    Spring Break
Select from the following: 

March 14    Spring Break
Select from the following: 

March 19  Lecture 18  Midterm Review


March 21  Exam  Midterm (In class)
Select from the following: 

March 25  Info.  Assignment 4 Released
Select from the following: 

March 26  Lecture 19  Optimization Layers


March 28  Lecture 20  Hyperparameter optimization


Apr 2  Lecture 21  Meta Learning


Apr 4  Lecture 22  Selfsupervised learning


Apr 5  Deadline  Assignment 4 Due (Friday Apr 5, 11:59PM)
Select from the following: 

Apr 8  Info.  Assignment 5 Released
Select from the following: 

Apr 9  Lecture 23  Applications 3D Vision


Apr 11  Lecture 24  Applications Language and Vision


Apr 16  Lecture 25  Applications Detection & Segmentation


Apr 18  Lecture 26  Final Project Presentation 1


Apr 19  Deadline  Assignment 5 Due (Friday Apr 19, 11:59PM)
Select from the following: 

Apr 23  Lecture 27  Final Project Presentation 2


Apr 25  Lecture 28  Final Project Presentation 3


Apr 30  Deadline  Final Project Report Due (Tuesday April 30, 11:59PM)
Select from the following: 