DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
UNIVERSITY OF CALIFORNIA, SAN DIEGO


CSE 250B: Principles of Artificial Intelligence: Learning

Winter 2013


Lectures are on Tuesdays and Thursdays from 3:30pm to 4:50pm, in Warren Lecture Hall room 2111.

Please ask questions at https://piazza.com/class#winter2013/cse250b

OVERVIEW

CSE 250B is a graduate course devoted to algorithms for supervised and unsupervised learning from data. 250B is open to Ph.D. and M.S. students in computer science, engineering, cognitive science, and related areas. Other prospective participants, including enthusiastic undergraduates, are welcome, but should contact the instructor at elkan@cs.ucsd.edu. The prerequisite for 250B is graduate status at UCSD, or consent of the instructor for undergraduates. CSE 250A and 250B are complementary. Students may take one or both courses: neither is a prerequisite for the other, and there will be little overlap.

There will be a quiz at the start of class every Tuesday, beginning on January 15.
Please arrive in class on time.

A unique feature of 250B is that students will do four substantial projects during the quarter. The projects will involve implementing logistic regression with EM to handle missing data, conditional random fields, latent Dirichlet allocation, and recursive autoencoder neural networks. Each project will be based on algorithms published in the last few years. For each project, students will work in a team of two and submit a paper similar to a research publication. The course includes training in designing computational experiments and writing research papers.

The specific topics discussed in CSE 250B will include, not necessarily in this order,

The instructor is Charles Elkan, Professor. For office hours, please send email to arrange an appointment.

The teaching assistant is Brendan Duncan.
He will hold discussion sections on Mondays from 2pm to 3pm in Solis Hall room 110 and on Wednesdays from 1pm to 2pm in Pepper Canyon Hall room 121. The two sections will cover the same material. Students are welcome to attend either or both. Brendan will also hold an office hour every Friday from 11am to noon in CSE 4146.

LECTURES

Lectures will be on Tuesday and Thursday afternoons from 3:30pm to 4:50pm in Warren Lecture Hall room 2111. For lecture notes from the previous version of 250B, see http://www.cs.ucsd.edu/users/elkan/250Bwinter2012. The last lecture will be on Thursday March 14.

January 8, 2013
Maximum likelihood (ML)

January 10
Conditional log likelihood, logistic regression
Project 1 due on January 24
January 15
Gradient descent
Quiz 1 with answer
January 17
Stochastic gradient descent
January 22
The general log-linear model, feature functions
Quiz 2 with answer
January 24
Conditional random fields (CRFs), feature functions
Project 2 due on February 12
January 29
Viterbi algorithm for inference, Collins perceptron for training
Quiz 3 with answer
January 31
Forward and backward algorithms for CRFs

February 5
Computing CRF expectations, contrastive divergence
Quiz 4 with answer
February 7
Gibbs sampling, overview of text mining Project 3 due on February 28
February 12
Multinomial distribution, generative processes
Quiz 5 with answer
February 14
Dirichlet distribution, latent Dirichlet allocation (LDA)

February 19
Derivation of Gibbs sampling for LDA
Quiz 6 with answer
February 21
Optimization of Gobbs sampling for LDA

February 26
Learning semantics for sentences
Quiz 7 with answer
February 28
Neural networks and autoencoders
Project 4 due on March 18
March 5
Derivatives for SGD, choosing a tree structure, loss functions
Quiz 8 with answer
March 7
Backpropagation for feedforward networks

March 12
Backpropagation for vector-valued nodes
Quiz 9 with answer
March 14
Review of recursive autoencoder networks for the project

March 19 (Tuesday)
Final exam in the usual lecture room from 3pm to 6pm


Clickable links in the table above are to lecture notes in PDF. Notes for days or topics without a link are included in the notes for an earlier topic. The quizzes and assignment descriptions are included in the PDF lecture notes.

TEXTBOOKS

The course will not be based on a book. The following textbooks are recommended as references:
For a price comparison among web booksellers use addall.com with the ISBN numbers.  Some topics discussed in class will not be in any textbook, and many will be explained differently, so coming to lectures and taking notes carefully is important. Quizzes and the final exam will be based on the online lecture notes.

ASSIGNMENTS AND GRADING

Instead of a midterm exam, there will be a seven-minute in-class quiz at the start of every Tuesday lecture (10% of your overall grade), a final examination (30%), and four project assignments (15% each). You should do each project and quiz with one partner, so individual work will count for 30% of your grade and joint work for 70%. You are free to change partners, or not, for each project and for each quiz. 

Each project will last between two and three weeks and will require coding, experimenting with data, and writing a report. Using a high-level language such as Python, Matlab or R is encouraged. Projects will be graded based exclusively on the written report; see this writing advice. Each team should hand in its joint report at the start of class on the day that the report is due. Each day or part of a day that a report is late will cost 20% of the maximum score available for the project. Reports will be evaluated using grading criteria similar to those in this form. Complete academic honesty is always required. 

The first project will be handed out on January 10, and will be due back on Thursday January 24, 2012. Later projects will be due on ??. The three-hour final exam will be on Tuesday March 19 at 3pm.

There is no a priori correspondence between letter grades and numerical scores on the assignments or on the exam. You can evaluate your performance in the class by comparing your scores with the means and standard deviations, which will be announced. However there is also no fixed correspondence between letter grades and standard deviations above or below the mean. If all students do well in the absolute, then all students will get a good grade.  

You should not drop CSE 250B just because you are unhappy with the score that you receive on a project or quiz. Instead, you should make an appointment to discuss with the TA or the instructor how you can do better on following projects and quizzes.



Most recently updated on March 14, 2013 by Charles Elkan, elkan@cs.ucsd.edu.