Computer Science 446

Machine Learning

Fall 2023

Instructor: Dan Gildea office hours T/Th 3:30-4:30pm, 3019 Wegmans

TAs:

  • Yuan Yao, yyao39@UR.Rochester.edu, office hours W 3:30-4:30pm, Th 1-2 pm, 3504 Wegmans
  • Pinxin (Andy) Liu, pliu23@u.Rochester.edu, office hours M/W 9:30-10:30, 2215 Wegmans

Location: Tu/Th 11:05-12:20am, Goergen 109

Prereqs: Probability, Linear Algebra, Vector Calculus.

Homeworks

Lecture notes

Required text: Christopher M. Bishop, Pattern Recognition and Machine Learning.

The following are useful references in addition to the reading material assigned for each class:

  • Stuart Russell and Peter Norvig, Artificial Intelligence, A Modern Approach.
  • Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
  • Larry Wasserman, All of Statistics, 2004.

Syllabus

Onwe will coverwhich means that after class you will understandif before class you have read
8/31 Regression and Classification constrained optimization, perceptron bishop 1.2, 1.4, app E
9/4 Logistic Regression stochastic gradient descent bishop 4.3
9/7 Backpropagation DP for gradient descent bishop 5.1, 5.2, 5.3
9/12 Deep Learning drop-out Krizhevsky 2012
9/14 Support Vectors max-margin bishop 3.1, 4.1
9/19 Support Vectors strong duality bishop 7.1
9/21 Support Vectors the kernel trick bishop 6.1, 6.2
9/26 Hidden Markov Models forward-backward bishop 13.2
9/28 Graphical Models bayes ball
10/3 Probabilistic Inference message passing bishop 8.4
10/5 Tree decomposition cyclic graphs koller and friedman
10/10 Tree decomposition cont'd vertex elimination
10/12 Review
10/19 Midterm
10/24 Midterm Solutions
10/26 Fairness in machine learning Kleinberg, Corbett-Davies
10/31 Expectation Maximization L = Q + H + D bishop 9
11/2 Expectation Maximization mixture of gaussians bishop 9
11/7 EM for HMM minimum bayes risk
11/9 Sampling Markov Chain Monte Carlo bishop 11.2
11/14 Metropolis Hastings detailed balance bishop 11.2
11/16 Gibbs sampling annealing bishop 11.3
11/21 Learning Theory PAC Kearns and Vazirani
11/28 Gradient Descent SGD convergence Ruder 2016
11/30 Transformers deep generative models
12/5 Reinforcement Learning q-learning sutton ch 3, 4.3, 4.4, 6.1, 6.5, 7.2, 11.1
12/7 Something fun VAE, Diffusion Kingma and Welling 2014, Ho et al. 2020
12/12 Review come to class with questions!
Final exam: Th Dec 21, 4-7pm, Goergen 109.

Grading

  • Homeworks: 50%
  • Final exam: 30%
  • Midterm: 20%
gildea @ cs rochester edu
December 7, 2023