CS59000: Statistical Machine Learning

Statistical Machine Learning

CS59000-005 / STAT598N • Fall 2009 • Time: TR 3:00 pm - 4:15 pm • Location: Lawson Computer Science Bldg 1106

Lecture Notes

This course has been approved by the department as a Qual 1 course for CS graduate students.

Instructor

Professor Alan Qi
Lawson 2142L • alanqi[at]cs.purdue.edu
Office hours by appointment

Teaching assistant

Yao Zhu
Lawson B116F • zhu36[at]cs.purdue.edu •
Office hours: MW 4:15 pm - 5:30 pm or by appointment

Course Description

This introductory course will cover many concepts, models, and algorithms in machine learning. Topics include classical supervised learning (e.g., regression and classification), unsupervised learning (e.g., principle component analysis and K-means), and recent development in the machine learning field such as variational Bayes, expectation propagation, and Gaussian processes. While this course will give students the basic ideas and intuition behind modern machine learning methods, the underlying theme in the course is probabilistic inference.

Tentative Topics

Review on probability distributions and basic concepts in information theory

Linear regression and classification

Probabilistic graphical models: Bayesian networks, Markov random fields and conditional random fields

K-means Clustering, mixture models and Expectation Maximization

Hidden Markov models, state space models, and forward-backward algorithms

Sampling methods, importance sampling, and Markov Chain Monte Carlo

Deterministic approximate inference: Laplace's method, Variational Bayes, and expectation propagation

Kernel methods

Selected topics:
        Nonparametric Bayesian: Dirichlet process mixture models
        Combining models (or weak learners): Boosting
        Recent papers from NIPS, ICML, UAI, JMLR, etc.

Prerequisites

Calculus, basic linear algebra and probability, or permission of instructor.

Textbooks (recommended)

Pattern Recognition and Machine Learning, Christopher M. Bishop, 2007

Information Theory, Inference, and Learning Algorithms, David MacKay, 2003. Available on-line here.

Assignments

Homework (links will be activated as homework is assigned). Copying will not be tolerated.
Midterm in mid October
Review of recent research
Students will choose a subtopic of machine learning research, select three recent conference papers on the topic, and write a 2 page report outlining the main ideas of papers and relate them to the context of the course.
Final project: You are required to complete a class project. The choice of the topic is up to you so long as it clearly pertains to the course material. To ensure that you are on the right track, you will have to submit a one paragraph description of your project a month before the project is due. You are encouraged to collaborate on the project, but . We expect a four page write-up about the project, which should clearly and succintly describe the project goal, methods, and your results. Each group should submit only one copy of the write-up and include all the names of the group members (a two person group will have 6 pages, a three person group will have 8 pages, and so on). The projects will be graded on the basis of your understanding of the overall course material (not based on, e.g., how brilliantly your method works).

Grading

Class participation: 5%
Homework: 25%
Midterm: 25%
Paper Review: 5%
Final project: 10%
Final exam: 30%

Late policy

Assignments will be accepted up to 5 days late with a penalty of 10% per day. No assignment will be accepted more than 5 days late.

Emergency policy

In the event of a major campus emergency, course requirements, deadlines and grading percentages are subject to changes that may be necessitated by a revised semester calendar or other circumstances beyond the instructor’s control. Here are ways to get information about changes in this course. Course web page: http://www.cs.purdue.edu/homes/alanqi/Courses/CS590_2009.html
Instructor’s and TA's emails: alanqi [at]cs.purdue.edu or .zhu36 [at]cs.purdue.edu •

To avoid the spread of pandemic influenza, a student is not recommended to come to class with a fever or for seven calendar days after recovering from influenza. Mitigation practices can be found at www.itap.purdue.edu/tlt/faculty.

The development of the course material is partially supported by the National Science Foundation under Grant No. 0916443. Any opinions, findings and conclusions or recomendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation (NSF).