CSE 250B Winter 2014

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
UNIVERSITY OF CALIFORNIA, SAN DIEGO

CSE 250B: Principles of Artificial Intelligence: Learning

Winter 2014

Lectures are on Tuesdays and Thursdays from 3:30pm to 4:50pm, in Center Hall room 113.

Please ask questions at https://piazza.com/ucsd/winter2014/cse250b/home

OVERVIEW

CSE 250B is a graduate course devoted to algorithms for supervised and unsupervised learning from data. 250B is open to Ph.D. and M.S. students in computer science, engineering, cognitive science, and related areas. Other prospective participants, including enthusiastic undergraduates, are welcome, but should contact the instructor at elkan@cs.ucsd.edu. The prerequisite for 250B is graduate status at UCSD, or consent of the instructor for undergraduates. CSE 250A and 250B are complementary. Students may take one or both courses: neither is a prerequisite for the other, and there will be little overlap.

There will be a quiz at the start of class every Tuesday, beginning on January 14. Please arrive in class on time.

A unique feature of 250B is that students will do four substantial projects during the quarter. The projects will involve implementing logistic regression with stochastic gradient descent, conditional random fields, latent Dirichlet allocation, and recursive autoencoder neural networks. Each project will be based on algorithms published in the last few years. For each project, students will work in a team of two or three, and submit a report that is similar to a research publication. The course includes training in designing computational experiments and writing research papers.

The specific topics discussed in CSE 250B will include, not necessarily in this order,

maximum likelihood, conditional maximum likelihood
logistic regression
stochastic gradient descent optimization
performance evaluation: precision, recall, cross-validation
overfitting, Occam's razor, and regularization
loss functions, proper loss functions
kernel methods
support vector machines (SVMs)
log-linear models
conditional random fields (CRFs)
unsupervised learning
generative models
mixture models
expectation-maximization (EM)
bag of words model for text
multinomial and Dirichlet distributions
topic modeling
latent Dirichlet allocation (LDA)
neural networks and backpropagation
recursive and deep neural networks

The instructor is Charles Elkan, Professor. For office hours, please send email to arrange an appointment.

The teaching assistants (TAs) are Brendan Duncan (50% time) and Sahil Goyal (25% time). They will hold discussion sections on Mondays and on Wednesdays; attendance is optional but recommended. Both sections will cover the same material, so students are welcome to attend either or both. Monday sections are at 2pm in Warren Lecture Hall (WLH) room 2205. Wednesday sections are at noon (12pm) in Center Hall room 222.

Brendan's office hours are every Friday from 11am to 1pm in the CSE building, either room 4146 or the 4th floor lobby. Sahil's office hour is every Monday from 11am to noon in the CSE basement room B240A.

LECTURES

Lectures will be on Tuesday and Thursday afternoons from 3:30pm to 4:50pm in Center Hall room 113. For lecture notes from the previous version of 250B, see http://www.cs.ucsd.edu/users/elkan/250Bwinter2013. The first lecture will be on Tuesday January 7, 2014, and the last lecture will be on Thursday March 13.

January 7, 2014	Maximum likelihood (ML)
January 9	Conditional ML, logistic regression model	Project 1 due on January 23
January 14	Stochastic gradient descent/ascent (SGD)	Quiz 1 with answer
January 16	Regularization
January 21	General log-linear model, feature functions	Quiz 2 with answer
January 23	Conditional random fields (CRFs)	Project 2 due on February 13
January 28	Viterbi algorithm for prediction with a CRF	Quiz 3 with answer
January 30	Partial derivatives for CRFs, forward and backward vectors
February 4	Stochastic gradient and Collins perceptron for CRF training	Quiz 4 with answer
February 6	Gibbs sampling, contrastive divergence
February 11	Text mining, bag of words representation, multinomial model	Quiz 5 with answer
February 13	The latent Dirichlet allocation (LDA) generative model	Project 3 due on February 27
February 18	Overview of Gibbs sampling for LDA	Quiz 6 with answer
February 20	Derivation of collapsed Gibbs sampling formula
February 25	Introduction to recursive neural networks	Quiz 7 with answer
February 27	Due date for Project 2	Project 4A or Project 4B, due on Monday March 17 at 10am
March 4	Intro to backpropagation	Quiz 8 with answer
March 6	Backpropagation in general
March 11	Review of backpropagation for scalar nodes	Quiz 9 with answer
March 13	Review of backpropagation for vector-valued nodes
March 18 (Tuesday)	Final exam from 3pm to 6pm

Clickable links in the table above are to lecture notes in PDF. Notes for days or topics without a link are included in the notes for an earlier topic. The assignment descriptions are included in the lecture notes.

TEXTBOOKS

The course will not be based on a book. The following textbooks are recommended as references:

Pattern Recognition and Machine Learning by Christopher M. Bishop, 2006, ISBN 0387310738
Machine Learning: A Probabilistic Perspective by Kevin P. Murphy, 2012, ISBN 0262018020
Machine Learning: The Art and Science of Algorithms that Make Sense of Data by Peter Flach, 2012, ISBN 1107422221.

For a price comparison among web booksellers use addall.com with the ISBN numbers. Some topics discussed in class will not be in any textbook, and many will be explained differently, so coming to lectures and taking notes carefully is important. Quizzes and the final exam will be based on the online lecture notes.

ASSIGNMENTS AND GRADING

Instead of a midterm exam, there will be a seven-minute in-class quiz at the start of every Tuesday lecture (10% of your overall grade), a final examination (30%), and four project assignments (15% each). You should do each project and quiz with one partner, so individual work will count for 30% of your grade and joint work for 70%. You are free to change partners, or not, for each project and for each quiz. For the first project, because students will be adding and dropping the class, you may be part of a team of three.

Each project will last between two and three weeks and will require coding, experimenting with data, and writing a report. Using a high-level language such as Python, Matlab (available for use free on university-owned computers) or R is encouraged. Projects will be graded based exclusively on the written report; see this writing advice. Each team should hand in its joint report at the start of class on the day that the report is due. Each day or part of a day that a report is late will cost 20% of the maximum score available for the project. Reports will be evaluated using grading criteria similar to those in this form. Complete academic honesty is always required.

The three-hour final exam will be on Tuesday March 18 at 3pm. There is no a priori correspondence between letter grades and numerical scores on the assignments or on the exam. You can evaluate your performance in the class by comparing your scores with the means and standard deviations, which will be announced. However there is also no fixed correspondence between letter grades and standard deviations above or below the mean. If all students do well in the absolute, then all students will get a good grade.

You should not drop CSE 250B just because you are unhappy with the score that you receive on a project or quiz. Instead, you should make an appointment to discuss with a TA or the instructor how you can do better on following projects and quizzes.

Most recently updated on March 13, 2014 by Charles Elkan, elkan@cs.ucsd.edu.