DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
UNIVERSITY OF CALIFORNIA, SAN DIEGO
CSE 250B: Principles of Artificial
Intelligence: Learning
Winter 2012
Please ask
questions using Piazza. Lectures are
in WLH 2207 on Tuesdays and Thursdays at 11am. Please arrive
in class on time. There will be a quiz at the start of class
every Thursday, beginning on January 12.
OVERVIEW
CSE 250B is
a graduate course devoted to the basic concepts and algorithms of
supervised and unsupervised learning from data. 250B is open to
Ph.D. and M.S. students in computer science, engineering,
cognitive science, and related areas. Other prospective
participants, including enthusiastic undergraduates, are welcome,
but should contact the instructor at elkan@cs.ucsd.edu. The
prerequisite for 250B is graduate status at UCSD, or consent of
the instructor for undergraduates. CSE 250A (taught by Prof.
Lawrence Saul) and 250B are complementary. Students may take one
or both courses: neither is a prerequisite for the other, and
there will be little overlap.
The
specific topics discussed in CSE 250B will include, not
necessarily in this order,
- maximum likelihood, conditional maximum likelihood
- logistic regression
- stochastic gradient descent training
- performance evaluation: precision, recall, cross-validation
- overfitting, Occam's razor, and regularization
- loss functions, proper loss functions
- kernel methods
- support vector machines (SVMs)
- log-linear models
- conditional random fields (CRFs)
- unsupervised learning
- generative models
- mixture models
- expectation-maximization (EM)
- bag of words model for text
- multinomial and Dirichlet distributions
- topic modeling
- latent Dirichlet allocation (LDA)
The
instructor is Charles Elkan, Professor.
For office hours, please send email to arrange an
appointment. The teaching assistant is Aditya Menon, whose office
hours will be Wednesdays from 10am to 12pm in the basement room
B240A in the CSE building.
LECTURES
Lectures
will be on Tuesdays and Thursdays from 11am to 12:20pm in Warren
Lecture Hall, room 2207. For lecture notes from the previous
version of 250B, see http://www.cs.ucsd.edu/users/elkan/250Bwinter2011. The last
lecture will be on Thursday March 15.
January 10
|
Maximum
likelihood (ML), maximum log conditional likelihood
|
January 12
|
Logistic regression model,
partial derivatives
|
January 17
|
convex minimization,
stochastic gradient (SG) in general
|
January 19
|
SG algorithm, cross-validation
|
January 24
|
Log-linear
models, feature functions
|
January 26
|
Conditional random fields
(CRFs)
|
January 31
|
Viterbi algorithm for CRFs
|
February 2
|
Gradients for CRFs
|
February 7
|
Forward and backward vectors
|
February 9
|
Gibbs sampling, contrastive
divergence
|
February 14
|
Text mining, bag of words
representation, multinomial distribution
|
Clickable
links in the table above are to lecture notes in PDF. Notes for
days or topics without a link are included in the notes for an
earlier topic. The quizzes and assignment descriptions are
included in the PDF lecture notes.
TEXTBOOKS
The course
will not be based on a book. The
following textbooks are recommended as references:
For a price
comparison among web booksellers use addall.com with the ISBN
numbers. Some topics discussed in class will not be in any
textbook, and many will be explained differently, so coming to
lectures and taking notes carefully is important. Quizzes and the
final exam will be based on the online lecture notes.
ASSIGNMENTS AND
GRADING
Instead of a
midterm exam, there will be a seven-minute in-class quiz at the
start of every Thursday lecture (10% of your overall grade), a
final examination (30%), and four project assignments (15% each).
You should do each project with one or two partners,
so individual work will count for 40% of your grade and joint
work for 60%. You are free to change partners, or not, between
projects.
Each project
will last between two and three weeks and will
require coding, experimenting with data, and writing a
report. Using a high-level environment such as Matlab or R is encouraged. Projects will be
graded based exclusively on the written report; see this writing advice. Each team should hand in its joint
report at the start of class on the day that the report is due.
Each day or part of a day that a report is late will cost 20% of
the maximum score available for the project. Reports will
be evaluated using grading criteria similar to those in
this form. Complete academic honesty is always required.
The first project
will be handed out on January 12, and will be due back on Thursday
January 26, 2012. Later projects will be due on Thursday February
16, Tuesday March 6, and Thursday March 22. The final exam will be
on Thursday March 22, 2012, at 11:30am.
There is no a
priori correspondence between letter grades and numerical
scores on the assignments or on the exam. You can evaluate your
performance in the class by comparing your scores with the means
and standard deviations, which will be announced. However there is
also no fixed correspondence between letter grades and standard
deviations above or below the mean. If all students do well in the
absolute, then all students will get a good grade.
You should not
drop CSE 250B just because you are unhappy with the score that you
receive on a project or quiz. Instead, you should make an
appointment to discuss with the TA or the instructor how you can
do better on following projects and quizzes.
Most recently
updated on February 13, 2012 by Charles Elkan, elkan@cs.ucsd.edu.