a logo for the course

Computational methods in optimization

David Gleich

Purdue University

Spring 2012

Course number CS 59000-OPT

Tuesday and Thursday, 3:00-4:15pm

Lawson B134

CS 59000-OPT Syllabus

Course information

Spring 2012
T-Th 3:00-4:15pm
BRNG B243
http://www.cs.purdue.edu/homes/dgleich/compopt

Instructor

David F. Gleich
Lawson 1207
dgleich@purdue.edu
http://www.cs.purdue.edu/homes/dgleich

Office hours

Monday 1-3pm in Lawson 1207.

TA

Wei-Yen Day
wday@purdue.edu

Office Hours

Thursday 10:30am-12:30pm in Haas G16.

Description

This course is a introduction to optimization for graduate students in any computational field.
It will cover many of the fundamentals of optimization and is a good course to prepare those who wish to use optimization in their research and those who wish to become optimizers by developing new algorithms and theory. Selected topics include:

Prerequisites

This class is a graduate lecture class. Students should have taken a graduate level numerical linear algebra or matrix analysis class that covers: QR factorizations, the singular value decomposition, null-spaces, and eigenvalues. David will assume you know this stuff, or can learn it yourself, if you take this class. He will also assume an undergraduate level understanding of mathematical analysis, i.e. limits, convergence sequences, sub-sequences. Multi-dimensional Taylor series will also play a large role in the class.

Goals and objectives

Optimization is an incredibly rich and deep field. This graduate class only scratches the surface to show how to use optimization to solve problems, and peeks at the details of how optimization algorithms work.

When we reach the end of the class, you, the student, will:

Requirements

The formal requirements and percentage of the total course grade are:

Quizzes

These will be worth 5% of the total grade and are graded as taken/missed. Students can miss up to 5 quizzes with no penalty. Make sure you contact the instructor (David) if you anticipate missing more than 5 quizzes due to any planned absenses.

References

The required text book is:

Numerical Optimization. Jorge Nocedal and Stephen J. Wright. 2nd edition. Springer, 2006. Available online! doi:10.1007/978-0-387-40065-5

Policies

Conduct

Students are expected to maintain a professional and respectful classroom environment. In particular, this includes:

You may use any non-disruptive personal electronics during class. One exception is during in-class quizzes, for which personal electronics are prohibited.

Correspondence

Please feel free to email me with any questions, but please prefix all email titles with the string CS-59000 OPT: to aid in filtering email.

I will make every effort to respond promptly, however, replies could be delayed due to circumstances outside of my control. In particular, do not rely on a response between the hours of 8pm and 8am.

Please do not attempt to call, google chat, skype, facebook, or tweet with me without prior arrangement.

Assignment clarity

I expect all assignments to be legible and well-written. For this, I will suggest using a computer with LaTeX to prepare all submitted materials.

If you do not plan to use a computer and LaTeX to prepare solutions to homeworks, you must let me know.

Missing or late work

Except as discussed below, and or by prior arrangement, missing or late work will be counted as a zero.

Collaboration

Collaboration on homework is allowed. The final assignments must contain a list of all collaborators. However, students must prepare solutions individually. As an example of the ideal scenario, the following situation is permissible:

A group of students meets to develop the solution to a problem on a white board. Each student records individual notes from this problem solving meeting. All students then prepare solutions individually and without further collaboration. These solutions show the names of all members in the initial group.

Examples of collaborations that are not allowed include, but are not limited to:

Collaboration on the project is not encouraged, but students (you!) should feel free to make a case if you think it’s a good idea.
Be prepared to justify why collaboration will result in a better project.

Computer codes

The assignments will involve producing computer codes. These need to be documented and written in accordance with best software engineering practices. Failure to follow this advice may result in solutions receiving zero points. Moreover, these should be prepared individually. Groups may discuss implementation strategies, algorithms, and approaches; but codes, like written homework solutions, must be prepared separately.

Grading

The anticipated grade ranges are:

These may be adjusted downward by up to 10% to achieve a reasonable grade distribution. They also may be minor upward adjustments.

Dishonesty

Behavior consistent with cheating, copying, and academic dishonesty is not tolerated. Depending on the severity, this may result in a zero score on the assignment, and could result in a failing grade for the class.

Purdue prohibits “dishonesty in connection with any University activity. Cheating, plagiarism, or knowingly furnishing false information to the University are examples of dishonesty.” (Part 5, Section III-B-2-a, University Regulations) Furthermore, the University Senate has stipulated that “the commitment of acts of cheating, lying, and deceit in any of their diverse forms (such as the use of substitutes for taking examinations, the use of illegal cribs, plagiarism, and copying during examinations) is dishonest and must not be tolerated. Moreover, knowingly to aid and abet, directly or indirectly, other parties in committing dishonest acts is in itself dishonest.” (University Senate Document 72-18, December 15, 1972)

Please review the Purdue’s guide to academic integrity

Attendance

Students are expected to be present for every meeting of the classes in which they are enrolled. Only the instructor can excuse a student from a course requirement or responsibility. When conflicts or absences can be anticipated, such as for many University sponsored activities and religious observations, the student should inform the instructor of the situation as far in advance as possible. For unanticipated or emergency absences when advance notification to an instructor is not possible, the student should contact the instructor as soon as possible by email, or by contacting the main office that offers the course. When the student is unable to make direct contact with the instructor and is unable to leave word with the instructor’s department because of circumstances beyond the student’s control, and in cases of bereavement, the student or the student’s representative should contact the Office of the Dean of Students.

Grief Absence Policy

Purdue University recognizes that a time of bereavement is very difficult for a student. The University therefore provides the following rights to students facing the loss of a family member through the Grief Absence Policy for Students (GAPS). GAPS Policy: Students will be excused for funeral leave and given the opportunity to earn equivalent credit and to demonstrate evidence of meeting the learning outcomes for misses assignments or assessments in the event of the death of a member of the student’s family.

Violent Behavior Policy

Purdue University is committed to providing a safe and secure campus environment for members of the university community. Purdue strives to create an educational environment for students and a work environment for employees that promote educational and career goals. Violent Behavior impedes such goals. Therefore, Violent Behavior is prohibited in or on any University Facility or while participating in any university activity.

Students with Disabilities

Purdue University is required to respond to the needs of the students with disabilities as outlined in both the Rehabilitation Act of 1973 and the Americans with Disabilities Act of 1990 through the provision of auxiliary aids and services that allow a student with a disability to fully access and participate in the programs, services, and activities at Purdue University. If you have a disability that requires special academic accommodation, please make an appointment to speak with me within the first three (3) weeks of the semester in order to discuss any adjustments. It is important that we talk about this at the beginning of the semester. It is the student’s responsibility to notify the Disability Resource Center (http://www.purdue.edu/drc) of an impairment/condition that may require accommodations and/or classroom modifications.

Emergencies

In the event of a major campus emergency, course requirements, deadlines and grading percentages are subject to changes that may be necessitated by a revised semester calendar or other circumstances beyond the instructor’s control. Relevant changes to this course will be posted onto the course website or can be obtained by contacting the instructors or TAs via email. You are expected to read your @purdue.edu email on a frequent basis.

Nondiscrimination

Purdue University is committed to maintaining a community which recognizes and values the inherent worth and dignity of every person; fosters tolerance, sensitivity, understanding, and mutual respect among its members; and encourages each individual to strive to reach his or her own potential. In pursuit of its goal of academic excellence, the University seeks to develop and nurture diversity. The University believes that diversity among its many members strengthens the institution, stimulates creativity, promotes the exchange of ideas, and enriches campus life.

Purdue University prohibits discrimination against any member of the University community on the basis of race, religion, color, sex, age, national origin or ancestry, marital status, parental status, sexual orientation, disability, or status as a veteran. The University will conduct its programs, services and activities consistent with applicable federal, state and local laws, regulations and orders and in conformance with the procedures and limitations as set forth in Executive Memorandum No. D-1, which provides specific contractual rights and remedies.

Schedule

A tentative list of lectures follows. This list may evolve throughout the course. See the web-page for an up-to-date list of lectures, under the “readings” section.

  1. 10-Jan – Introduction
  2. 12-Jan – Optimization software
  3. 17-Jan – Introduction to mimization
  4. 19-Jan – Least squares 1
  5. 24-Jan – Least squares 2
  6. 26-Jan – Matrix calculus
  7. 31-Jan – Newton methods
  8. 2-Feb – Line search
  9. 7-Feb – Line search
  10. 9-Feb – Professor absent
  11. 14-Feb – Trust regions
  12. 16-Feb – Trust regions
  13. 21-Feb – Quasi-newton methods
  14. 23-Feb – Quasi-newton methods
  15. 28-Feb – Nonlinear equations
  16. 1-Mar – Midterm
  17. 6-Mar – Constrained theory
  18. 8-Mar – Constrained theory 13-Mar – Spring vacation 15-Mar – Spring vacation
  19. 20-Mar – Linear programming 1
  20. 22-Mar – Linear programming 2
  21. 27-Mar – Linear programming 3
  22. 29-Mar – Linear programming 4
  23. 3-Apr – Quadratic optimization
  24. 5-Apr – Quadratic optimization
  25. 10-Apr – Buffer-class
  26. 12-Apr – Large scale optimization
  27. 17-Apr – Convex modeling
  28. 19-Apr – Integer programming
  29. 24-Apr – Stochastic methods in optimization
  30. 26-Apr – Buffer-class 1-May – Finals 3-May – Finals

Makeup classes

In the event the schedule above changes, we may need to schedule make-up classes. Students will be expected to attend these classes, and David will make every effort to include everyone.

Changes

This syllabus is subject to change. Updates will be posted on the course website.