CS58700: Foundations Of Deep Learning (Spring 2024)
A class on Deep Learning, digital art.
Course Information
This course provides an integrated view of the key concepts of deep learning (representation learning) methods. This course focuses on teaching principles and methods needed to design and deploy novel deep learning models, emphasizing the relationship between traditional statistical models, invariant theory, and the algorithmic challenges of designing and deploying deep learning models in real-world applications. This course has both a theoretical and coding component. The course assumes familiarity with coding in the language used for state-of-the-art deep learning libraries, linear algebra, probability theory, and statistical machine learning.
Pre-requisites:
- CS 37300 Data Mining & Machine Learning
- MA 26500 Linear Algebra
- STAT 41600 Probability
Textbook:
- [DL] Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, MIT Press, 2016
- [DLFC] Christopher M. Bishop, Hugh Bishop, Deep Learning --- Foundations and Concepts, Springer Cham, 2023
Grading:
The final grade will be curved and no stricter than the cutoff: A+: 97-100, A: 93-96, A-: 90-92, B+: 87-89, ..., etc.The percentage is computed following (without any rounding):
- Assignments: 50% (12.5% each, 5 assignment with lowest dropped)
- Midterm: 25%
- Final Project: 25%
FAQ:
- Lecture slides will be posted on Brightspace. Some materials are from and copyright by Professor Bruno Bibeiro, do not redistribute.
- The instructor & TAs can be best reached through Ed Discussion. Please post your questions there instead of emailing TAs.
- During office hours or on Ed Discussion, please avoid posting partial homework solutions or asking TAs to "review" your code/solution.
- Tutorial for learning Latex with Overleaf: [Link]
Instructor & TAs
Raymond A. Yeh
Instructor
Email: rayyeh [at] purdue.edu
Office Hour: Wednesday 9AM-10AM
Location: Zoom
Md Ashiqur Rahman
Teaching Assistant
Email: rahman79 [at] purdue.edu
Office Hour: Monday 11AM-12PM
Location: HAAS 143
Simon Zhang
Teaching Assistant
Email: zhan4125 [at] purdue.edu
Office Hour: Friday 1PM-2PM
Location: HAAS G072
Time & Location
- Time: Tuesday & Thursday (9:00 am - 10:15 am)
- Location: Krannert Building G018
Other Resource
Course Schedule
The following schedule is tentative and subject to change.
| Date | Event | Description | Readings |
|---|---|---|---|
| Jan 9 | Lecture 1 | Introduction & Overview
|
|
| Jan 11 | Lecture 2 | Supervised Learning
|
|
| Jan 15 | Info. | Assignment 1 Released
Select from the following: |
|
| Jan 16 | Lecture 3 | Multilayer Perceptron (MLP)
|
|
| Jan 18 | Lecture 4 | Optimization
|
|
| Jan 23 | Info. | Cancelled due to weather
Select from the following: |
|
| Jan 25 | Lecture 5 | Training Deep Neural Networks
|
|
| Jan 30 | Lecture 6 | How to Read Papers + Invariant MLP
|
|
| Feb 1 | Lecture 7 | Invariant MLP Part 2
|
|
| Feb 2 | Deadline | Assignment 1 Due (Friday Feb 2, 11:59PM)
Select from the following: |
|
| Feb 5 | Info. | Assignment 2 Released
Select from the following: |
|
| Feb 6 | Lecture 8 | Convolution Neural Network Part 1
|
|
| Feb 8 | Lecture 9 | Convolution Neural Network Part 2
|
|
| Feb 13 | Lecture 10 | Recurrent Neural Network
|
|
| Feb 15 | Lecture 11 | Set Representation Learning
|
|
| Feb 20 | Lecture 12 | Graph Neural Network
|
|
| Feb 22 | Lecture 13 | Attention Layer + Transformer Architectures
|
|
| Feb 23 | Deadline | Assignment 2 Due (Friday Feb 23, 11:59PM)
Select from the following: |
|
| Feb 26 | Info. | Assignment 3 Released
Select from the following: |
|
| Feb 27 | Lecture 14 | Variational AutoEncoder
|
|
| Feb 29 | Lecture 15 | Diffusion Model
|
|
| March 5 | Lecture 16 | Diffusion Model II
|
|
| March 7 | Lecture 17 | Generative Adversarial Network
|
|
| Mar 8 | Deadline | Assignment 3 Due (Friday Mar 8, 11:59PM)
Select from the following: |
|
| Mar 8 | Deadline | Project Proposal Due (Friday Mar 8, 11:59PM)
Select from the following: |
|
| March 12 | --- | Spring Break
Select from the following: |
|
| March 14 | --- | Spring Break
Select from the following: |
|
| March 19 | Lecture 18 | Midterm Review
|
|
| March 21 | Exam | Midterm (In class)
Select from the following: |
|
| March 25 | Info. | Assignment 4 Released
Select from the following: |
|
| March 26 | Lecture 19 | Self-supervised Learning
|
|
| March 28 | Lecture 20 | Optimization Layers
|
|
| Apr 2 | Lecture 21 | Hyperparameter Optimization
|
|
| Apr 4 | Lecture 22 | Structuring Machine Learning Projects
|
|
| Apr 5 | Deadline | Assignment 4 Due (Friday Apr 5, 11:59PM)
Select from the following: |
|
| Apr 8 | Info. | Assignment 5 Released
Select from the following: |
|
| Apr 9 | Lecture 23 | Applications--- 3D Vision
|
|
| Apr 11 | Lecture 24 | Applications--- Language and Vision
|
|
| Apr 16 | Lecture 25 | Applications--- Detection & Segmentation
|
|
| Apr 18 | Lecture 26 | Final Project Presentation 1
|
|
| Apr 19 | Deadline | Assignment 5 Due (Friday Apr 19, 11:59PM)
Select from the following: |
|
| Apr 23 | Lecture 27 | Final Project Presentation 2
|
|
| Apr 25 | Lecture 28 | Final Project Presentation 3
|
|
| Apr 30 | Deadline | Final Project Report Due (Tuesday April 30, 11:59PM)
Select from the following: |