Text: David J. C. MacKay, Information Theory, Inference, and Learning Algorithms
Recommended: Trevor Hastie, Robert Tibshirani, Jerome Friedman, The elements of statistical learning: data mining, inference, and prediction. Dana Ballard, Natural Computation.
| On | we will cover | which means that after class you will understand | if before class you have read |
|---|---|---|---|
| 1/13 | Introduction | ||
| 1/18 | Probability Theory | independence, bayes rule | charniak |
| 1/20 | Information Theory | entropy, kl-distance, coding | mackay ch 2 |
| 1/25 | Probabilistic Inference | priors: bayesian reasoning, MAP | heckerman |
| 1/27 | Probabilistic Inference | priors on continuous variables | mackay ch 24 |
| 2/1 | Minimum Description Length | decision trees | mackay ch 28 |
| 2/3 | Probabilistic Inference | polytree | mackay ch 26 |
| 2/8 | Expectation Maximization | latent variable clustering | bilmes § 1-3 |
| 2/10 | Independent Component Analysis | source separation | mackay ch 34 |
| 2/15 | Learning Theory | probably approximately correct | kearns&vazirani ch 1 |
| 2/17 | Learning Theory | VC dimension | kearns&vazirani ch 2, 3 |
| 2/22 | Eigenvectors | least squares, PCA | bishop 310-314, appendix E |
| 2/24 | Nonlinear Dimensionality Reduction | isomap, locally linear embedding | roweis; tenenbaum |
| 3/1 | Optimization | conjugate gradient | shewchuk § 1-9 |
| 3/3 | Optimization | Gibbs Sampling, MCMC | mackay ch 29 |
| 3/15 | Review | ||
| 3/17 | Midterm | ||
| 3/22 | Midterm Solutions | aspect model | |
| 3/24 | MCMC, Gibbs | (continued from before midterm) | mackay ch 38, 39 |
| 3/29 | Backpropagation | the chain rule | bishop 140-148 |
| 3/31 | Support Vectors | the wolfe dual | hastie ch 12 |
| 4/5 | Support Vectors | the kernel trick | |
| 4/7 | Hidden Markov Models | forward-backward | ballard ch 10; bilmes § 4 |
| 4/12 | Reinforcement Learning | q-learning | ballard ch 11 |
| 4/14 | Reinforcement Learning | partial observability | ballard ch 11 |
| 4/19 | Games | dana ballard | |
| 4/21 | Games | learning to co-operate | hauert, zhu |
| 4/26 | Review | come to class with questions! |