Statistical Machine Learning
CS59000-006 • Fall 2008 • Time: TR 10:30 am - 11:45 am •
Location: Electrical Engineering Bldg 234
(This course will be a Qual 1 course for CS Ph.D. students.)
Instructor
Professor Alan Qi
Lawson 1207 • alanqi[at]cs.purdue.edu
Teaching assistant
Yao Zhu
Lawson B116F • zhu36[at]cs.purdue.edu •
Office hours: MW 2:00 pm - 3:15 pm or by appointment
Course Description
This introductory course will cover many concepts,
models, and algorithms in machine learning. Topics include classical
supervised learning (e.g., regression and classification), unsupervised
learning (e.g., principle component analysis and K-means), and recent
development in the machine learning field such as variational Bayes,
expectation propagation, and Gaussian processes. While this
course will give students the basic ideas and intuition behind modern
machine learning methods, the underlying theme in the course is
probabilistic inference.
Tentative Topics
Review on probability distributions and basic concepts in information
theory
Linear regression and classification
Probabilistic graphical models: Bayesian networks, Markov random fields
and conditional random fields
K-means Clustering, mixture models and Expectation Maximization
Hidden Markov models, state space models, and forward-backward
algorithms
Sampling methods, importance sampling, and Markov Chain Monte Carlo
Deterministic approximate inference: Laplace's method, Variational
Bayes, and expectation propagation
Kernel methods
Selected topics:
Nonparametric Bayesian: Dirichlet
process mixture models
Combining models (or weak
learners): Boosting
Recent papers from NIPS, ICML,
UAI, JMLR, etc.
Prerequisites
Calculus, basic linear algebra and probability, or permission of
instructor.
Textbooks (recommended)
Pattern Recognition and Machine Learning, Christopher M. Bishop, 2007
Information Theory, Inference, and Learning Algorithms, David MacKay,
2003. Available on-line here.
Assignments
- Homework (links will be activated as homework is assigned).
Copying will not be tolerated.
- Midterm in mid October
- Review
of recent research
Students
will choose a subtopic of machine learning research,
select three recent conference papers on the topic, and write a 2 page
report outlining the main ideas of papers and relate them to the
context of the course.
- Final
project: You are required to complete a class project. The choice
of the topic is up to you so long as it clearly pertains to the course
material. To ensure that you are on the right track, you will have to
submit a one paragraph description of your project a month before the
project is due. You are encouraged to collaborate on the project, but .
We expect a four page write-up about the project, which should clearly
and succintly describe the project goal, methods, and your results.
Each group should submit only one copy of the write-up and include all
the names of the group members (a two person group will have 6 pages, a
three person group will have 8 pages, and so on). The projects will be
graded on the basis of your understanding of the overall course
material (not based on, e.g., how brilliantly your method works).
Grading
- Class participation: 5%
- Homework: 35%
- Midterm: 25%
- Paper Review: 10%
- Final project: 25%
Late policy
Assignments will be accepted up to 5 days late with a penalty of 10%
per day. No assignment will be accepted more than 5 days late.