Instructor:
Dan Gildea office hours Tu/Th 1-2pm Wegmans Hall 3019
TAs:
Location: M/W 9-10:15am, Wegmans 1400
The course is full. If spaces become available, I will
add people who hand in the first homework. If you would like to
try to add, write "NEED TO ADD" on the top of your assignment, along
with your name, program (eg, BS in CS) and year.
Prereqs: Probability, Linear Algebra, Vector Calculus.
Homeworks
Lecture notes
Required text: Christopher M. Bishop, Pattern Recognition and Machine Learning.
The following are useful references in addition
to the reading material assigned for each class:
- Stuart Russell and Peter Norvig, Artificial Intelligence, A Modern Approach.
- Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
- Larry Wasserman, All of Statistics, 2004.
Syllabus
On | we will cover | which means that after class you will understand | if before class you have read |
1/16 |
Regression and Classification |
constrained optimization, perceptron
| bishop 1.2, 1.4, app E |
1/23 |
Logistic Regression |
stochastic gradient descent
| bishop 4.3 |
1/28 |
Backpropagation |
DP for gradient descent
| bishop 5.1, 5.2, 5.3 |
1/30 |
Deep Learning |
drop-out
| Krizhevsky 2012 |
2/4 |
Support vectors |
max-margin
| bishop 3.1, 4.1 |
2/6 |
Support Vectors |
strong duality
| bishop 7.1 |
2/11 |
Support Vectors |
the kernel trick
| bishop 6.1, 6.2 |
2/13 |
Hidden Markov Models |
forward-backward
| bishop 13.2 |
2/18 |
Graphical Models |
bayes ball
| |
2/20 |
Probabilistic Inference |
message passing
| bishop 8.4 |
2/25 |
Tree decomposition |
cyclic graphs
| koller and friedman |
2/27 |
Tree decomposition cont'd |
vertex elimination
| |
3/4 |
Review |
| |
3/6 |
Midterm |
| |
3/18 |
Midterm Solutions |
| |
3/20 |
Expectation Maximization |
L = Q + H + D
| bishop 9 |
3/25 |
Expectation Maximization |
mixture of gaussians
| bishop 9 |
3/27 |
EM for HMM |
minimum bayes risk
| |
4/1 |
Sampling |
Markov Chain Monte Carlo
| bishop 11.2 |
4/3 |
Metropolis Hastings |
detailed balance
| bishop 11.2 |
4/8 |
Gibbs sampling |
annealing
| bishop 11.3 |
4/10 |
Learning Theory |
PAC
| Kearns and Vazirani |
4/15 |
Learning Theory |
VC dimension
| |
4/17 |
Optimization |
SGD convergence
| Ruder 2016 |
4/22 |
PCA |
Eigenvectors
| |
4/24 |
Reinforcement Learning |
q-learning
| sutton ch 3, 4.3, 4.4, 6.1, 6.5, 7.2, 11.1 |
4/29 |
no class |
| |
5/1 |
Review |
come to class with questions!
| |
Final Exam: Wed May 8, 8:30-11:30am, Douglass Ballroom.
Grading
- Homeworks: 50%
- Final exam: 30%
- Midterm: 20%
gildea @ cs rochester edu
April 22, 2019
|