Supplementary references are given for each lecture, either to research papers or to
the following textbooks:
Stuart Russell and Peter Norvig, Artificial intelligence: a modern approach (second edition).
Judea Pearl, Probabilistic reasoning in intelligent systems.
Richard Duda, Peter Hart, and David Stork, Pattern classification.
Lecture 2: Frequentist vs. Bayesian learning; maximum likelihood; conjugate priors [March 31]
Russell and Norvig, 20.1 and 20.2.
Lecture 3: Product distributions; the multivariate Gaussian [April 5]
Duda, Hart, and Stork, 2.5.
Lecture 4: Multivariate Gaussian (cont'd); latent variable models [April 7]
C. Bishop. Latent variable models.
Lecture 5: Factor analysis; mixture models; EM [April 12]
Russell and Norvig, 20.3.
Lecture 6: Hidden Markov models [April 14]
Duda, Hart, and Stork, 3.10.
Russell and Norvig, 15.1, 15.2, 15.3, 15.6.
L.R. Rabiner. "A tutorial on hidden Markov models and selected applications in speech recognition", Proceedings of the IEEE, 77(2), Feb 1989.
Lecture 7: Conditional independence [April 19]
Lecture 8: Directed graphical models: examples and semantics [April 21]
Lecture 9: Directed graphical models: semantics [April 26]
Russell and Norvig, 14.2 and 14.3.
Lecture 10: Undirected graphical models: examples and semantics [April 28]
Lecture 11: Some examples; converting between directed/undirected models; conditional probability functions [May 3]
Blei, Ng, Jordan, Latent Dirichlet allocation.
Lecture 12: Inference: complexity results, variable elimination [May 5]
Russell and Norvig, 14.4.
Lecture 13: Inference: tree decompositions [May 10]
Lecture 14: Inference: Monte-Carlo Markov chain methods [May 12]
Russell and Norvig, 14.5.
MacKay, Introduction to Monte-Carlo methods.
Geman and Geman, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images.
Lecture 15: Inference: MCMC, cont'd [May 17]
No class on May 19.
Lecture 16: Inference: belief propagation [May 24]
Lecture 17: Entropy: axiomatic formulation and AEP [May 26]
Lecture 18: Maximum entropy; information projection [May 31]
Lecture 19: Exponential families; iterative scaling [June 2]