Discriminative Learning
Graduate course to be given Winter 2008.
Course Index: 259C. Section no. 610860. Tue, Thu 12:30 - 1:50, Room 4109 in CSE Building
Course Wiki Page
Registered students will recieve a grade of Satisfactory / Unsatisfactory
Description
A course on statistical machine learning algorithms for classification, detection and ranking.
The emphasis is on discriminative, rather than generative models. The problems we will look into have a long history, however, the
traditional algorithms do not work well for high dimensional data. New algorithms such as Support Vector Machines and Adaboost,
prove to be very effective for high dimensional problems. So effective as to require new mathematical theory to
explain their behavior. I will describe these algorithms, the theory for explaining them, and new problems and solutions that arise out of this
approach.
I plan to cover (some of) the following:
- Theory of the generalization error: epsilon covers, metric entropy, VC dimension, Rademacher dimension.
- Support Vector Machines.
- Adaboost, BrownBoost, NormalBoost.
- Margin-based bounds on the generalization error.
- Bagging, "How to be a Bayesian without believing".
- Online discriminative learning for individual sequences.
- Active learning and co-training.
- Using unlabeled data to aid discriminative learning.
Prerequisites
You should have a good foundation in math, i.e. linear algebra,
probability, integrals, taylor expansions. You will also need to
devote a significant amount of time between classes to reading papers
from a variety of fields. Notation and terminology in these papers
varies a lot, which makes reading a challange. However, mastering
these different terminologies and understanding the relationships
between them will provide you with an invaluable insight into this
exciting and evolving research area.