Computer Science 410

Seminar in AI: Optimization

Spring 2025

Instructor: Dan Gildea

Homework

Required text: Boyd and Vanderberghe, Convex Optimization

In addition to this text, we will cover additional numerical methods taken from Nocedal and Wright and Numerical Recipes. The second half of the course will focus on applications of optimization in the context of machine learning.

Tu 1/21 IntroBoyd ch 1
Th 1/23 Convex SetsBoyd ch 2
Tu 1/28 Convex FunctionsBoyd ch 3
Th 1/30 Convex Optimization ProblemsBoyd ch 4
Tu 2/4 Convex Optimization ProblemsBoyd ch 4
Th 2/6 DualityBoyd ch 5
Tu 2/11 DualityBoyd ch 5
Th 2/13 Approximation and Statistical EstimationBoyd ch 6, 7
Tu 2/18 Geometric ProblemsBoyd ch 8
Th 2/20 Unconstrained MinimizationBoyd ch 9
Tu 2/25 Equality Constrained MinimizationBoyd ch 10
Th 2/27 Interior Point MethodsBoyd ch 11
Tu 3/4 Interior Point MethodsBoyd ch 11
Th 3/6 BFGSNocedal ch 8
Tu 3/18Simulated AnnealingKirkpatrick
Th 3/20
Tu 3/25Deterministic AnnealingRose
Th 3/27 Numerical MethodsPress 10.4 - 10.6
Tu 4/1 Integer ProgrammingNemhauser ch II.4
Th 4/3AdaGradDuchi
Tu 4/8The Marginal Value of Adaptive Gradient Methods in Machine LearningWilson et el. (2017)
Th 4/10training transformersKunster ICLR 2023
Tu 4/15
Th 4/17
Tu 4/22
W 12/3Graphical ModelsWainwright and Jordan ch 3
Th 4/24Graphical Models cont'dWainwright and Jordan ch 4
Tu 4/29 Project Presentations
Th 5/1 Project Presentations

Projects due: 5pm Friday 12/12

Grading

Students will have written assignments, present one lecture, and complete a final project.
  • Homework: 30%
  • Lectures and participation: 30%
  • Final project: 40%

gildea @ cs rochester edu
January 30, 2025