Statistical Machine Learning

CS59000 • Spring 2011 • Time: TR 3:00 pm - 4:15 pm • Location:  Forney Hall of Chemical Engr B124

Lecture Notes

This course has been approved by the department as a Qual 1 course for CS graduate students.


Professor Alan Qi
Lawson 2142L • alanqi [at]
Office hours:  4:15-5:15pm Tues

Teaching assistant

Syed Abbas Zilqurnain Naqvi
Lawson   2149-1 • naqvi [at]
Office hours:  2-3pm Tues and 3:30-4:30pm Fri

Course Description

This introductory course will cover many concepts, models, and algorithms in machine learning. Topics include classical supervised learning (e.g., regression and classification), unsupervised learning (e.g., principle component analysis and K-means), and recent development in the machine learning field such as variational Bayes, expectation propagation, and Gaussian processes.  While this course will give students the basic ideas and intuition behind modern machine learning methods, the underlying theme in the course is probabilistic inference.

Tentative Topics

Review on probability distributions and basic concepts in information theory

Linear regression and classification

Probabilistic graphical models: Bayesian networks, Markov random fields and conditional random fields

K-means Clustering, mixture models and Expectation Maximization

Hidden Markov models, state space models, and forward-backward algorithms

Sampling methods, importance sampling, and Markov Chain Monte Carlo

Deterministic approximate inference: Laplace's method, Variational Bayes, and expectation propagation

Kernel methods

Selected topics:
        Nonparametric Bayesian: Dirichlet process mixture models
        Combining models (or weak learners): Boosting
        Recent papers from NIPS, ICML, UAI,  JMLR, etc.


Calculus, basic linear algebra and probability, or permission of instructor.

Textbooks (recommended)

Pattern Recognition and Machine Learning, Christopher M. Bishop, 2007

Information Theory, Inference, and Learning Algorithms, David MacKay, 2003. Available on-line here.



Late policy

Assignments will be accepted up to 5 days late with a penalty of 20% per day. No assignment will be accepted more than 5 days late.

Emergency policy

In the event of a major campus emergency, course requirements, deadlines and grading percentages are subject to changes that may be necessitated by a revised semester calendar or other circumstances beyond the instructor’s control. Here are ways to get information about changes in this course. Course web page:
Instructor’s and TA's emails: alanqi [at] or .naqvi [at]•

To avoid  the spread of pandemic influenza, a student is not recommended to come to class with a fever or for seven calendar days after recovering from influenza. Mitigation practices can be found at

The development of the course material is partially supported by the National Science Foundation under Grant No.  0916443. Any opinions, findings and conclusions or recomendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation (NSF).