Computer Science 410

Seminar in AI: Optimization

Spring 2025

Instructor: Dan Gildea

Homework

Required text: Boyd and Vanderberghe, Convex Optimization

In addition to this text, we will cover additional numerical methods taken from Nocedal and Wright and Numerical Recipes. The second half of the course will focus on applications of optimization in the context of machine learning.

Tu 1/21 IntroBoyd ch 1
Th 1/23Convex SetsBoyd ch 2
Tu 1/28Convex FunctionsBoyd ch 3
Th 1/30Convex Optimization ProblemsBoyd ch 4
Tu 2/4 Convex Optimization ProblemsBoyd ch 4
Th 2/6 DualityBoyd ch 5
Tu 2/11DualityBoyd ch 5
Th 2/13DualityBoyd ch 5
Tu 2/18ApproximationBoyd ch 6
Th 2/20Statistical EstimationBoyd ch 7
Tu 2/25Geometric ProblemsBoyd ch 8
Th 2/27Unconstrained MinimizationBoyd ch 9
Tu 3/4 Equality Constrained MinimizationBoyd ch 10
Th 3/6 Interior Point MethodsBoyd ch 11
Tu 3/18Interior Point Methods (Arvin)Boyd ch 11
Th 3/20BFGS (Shunzhi)Nocedal ch 8
Tu 3/25Simulated Annealing (Matt)Kirkpatrick
Th 3/27Deterministic Annealing (Shunzhi)Rose
Tu 4/1 Numerical Methods (Matt)Press 10.4 - 10.6
Th 4/3 Integer Programming (Aidan)Nemhauser ch II.4
Tu 4/8 AdaGradDuchi
Th 4/10The Marginal Value of Adaptive Gradient Methods in Machine Learning (Arvin)Wilson et el. (2017)
Tu 4/15no class
Th 4/17training transformersKunster ICLR 2023
Tu 4/22Graphical Models (Aidan)Wainwright and Jordan ch 3
Th 4/24Graphical Models cont'dWainwright and Jordan ch 4
Tu 4/29Project Presentations
Th 5/1 Project Presentations

Projects due: 5pm Friday 5/2

Grading

Students will have written assignments, present two lectures, and complete a final project.
  • Homework: 30%
  • Lectures and participation: 30%
  • Final project: 40%

gildea @ cs rochester edu
March 18, 2025