CSE 254: Log-linear models and conditional random fields

Spring 2007

Please use this discussion board to ask questions about CSE 254, especially questions about the assignments.

CSE 254 is a graduate seminar devoted to recent research on AI learning methods and applications.  This is not an introductory course, so the prerequisite is at least one graduate-level course (at UCSD or elsewhere) in machine learning or a closely related area such as statistics or pattern recognition.  Appropriate courses at UCSD include CSE 250B (Principles of AI: Learning), CSE 291 (Probabilistic Methods in AI and Machine Learning), and Cognitive Science 260 (Pattern Recognition).

The room for CSE 254 is HSS 2321.  The class meets on Tuesdays and Thursdays from 3:30pm to 4:50pm, beginning on April 3.  Because several students have conflicts, we will not change the time or room.  The last meeting will be on June 8.

This year CSE 254 has a specific topic: conditional random fields, known as CRFs.  See here for tutorials and papers on CRFs, and see here for available software.  We will start by covering log-linear methods, which are the foundation of CRFs.  We may also cover related research on other methods for solving structured prediction problems, and/or on topic models.

For the first few weeks, the instructor will lecture on log-linear methods, topic models, and CRFs.  There are two assignments Then, in each class meeting, one student will give a talk lasting about 60 minutes presenting a recent technical paper in detail.  In questions during the talk, and in the final 20 minutes, all seminar participants will discuss the paper and the issues raised by it. 

date topic or presenter (not confirmed)
reading or handout
April 3 overview, log-linear models
April 5
No class: Rockwood lecture at 4pm by Tomaso Poggio
April 10
log-linear models continued Assignment 1: naive Bayes vs log-linear
April 12 nonlinear optimization; CRFs

April 17 CRFs continued Assignment 2: CRFs
April 19 Sanjoy Dasgupta
Maximum entropy and all that: information geometry
April 24 Eric Wiewiora
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data by J Lafferty, A McCallum, F Pereira, ICML 2001.
April 26 Peter Shin Michael Collins.  Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms.  Proceedings of the ACL-02 conference on Empirical methods in natural language processing, pp.1-8, 2002.  
May 1 Guilherme Hoefel
Ben Taskar, Carlos Guestrin and Daphne Koller.  Max-Margin Markov Networks.  In Advances in Neural Information Processing Systems 16 (NIPS 2003), 2004.
May 3 lecture

May 8 Panagiotis Voulgaris Andrew McCallum.  Efficiently Inducing Features of Conditional Random Fields.  In Proceedings of the 19th Conference in Uncertainty in Articifical Intelligence (UAI-2003), 2003.
May 10 Tom Griffiths Mandler Hall, room 3545, 4pm: "Everyday Inductive Leaps: Making Predictions and Detecting Coincidences"
May 15 Pouya Bozorgmehr Xuming He, Richard Zemel, and Miguel Á. Carreira-Perpiñán.  Multiscale conditional random fields for image labelling.  Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2004.
May 17 Simone McCloskey Multi-labelled classification using maximum entropy method by S Zhu, X Ji, W Xu, Y Gong, SIGIR 2005.
May 22 Tingfan Wu Ioannis Tsochantaridis, Thorsten Joachims, Thomas Hofmann, Yasemin Altun.  Large Margin Methods for Structured and Interdependent Output Variables.  JMLR, December 2005.
May 24 Nicholas Butko
Sham Kakade, Yee Whye Teh, Sam T. Roweis.  An Alternate Objective Function for Markovian Fields.  ICML 2002.
May 29 Alexander Loukissas Samuel Gross, Olga Russakovsky, Chuong Do, and Serafim Batzoglou.  Training conditional random fields for maximum labelwise accuracy.  In Advances in Neural Processing Systems 19 (NIPS), December 2006.
May 31
June 5
June 7
finals week project presentations

Each student who takes the course for four units will do a term project following specific guidelines.  The project should be at the frontier of current research, and closely inspired by at least one of the papers discussed in the class.  Many projects will be devoted to reproducing the results of a selected recent high-quality published paper.  Doing this has very high educational value and provides an excellent starting point for subsequent high-quality original research. 

Project reports will be evaluated using these grading criteria.  There is a schedule for handing in a detailed project proposal, a draft project report, and then the final report.  The seminar will have no final exam.  Letter grades will be based mostly on the final project report, but the presentations, participation in class and in the web-based discussions, and the intermediate project deliverables are all important also.

The instructor is Charles Elkan (Professor), whose office is in the CSE building, room 4134.  Feel free to send email to arrange an appointment, or telephone (858) 534-8897. 

Note:  The seminar will run in parallel with a data mining contest sponsored by Fair Isaac, with cash prizes.



Students may take the seminar for a letter grade for four units, or for one or two units S/U: For four units, a student should register for CSE 254, section id 588815 for a letter grade.  For one or two units, a student should register for the instructor's CSE 293, section id 588846.  Students who took a previous version of CSE 254 may take it again.  Papers will be different this year.



Papers will be recent technical articles from ICML and similar venues.  Each paper will be made available on the web as the quarter progresses.  Students will choose papers in consultation with the instructor.  Each presentation will cover a single paper, to ensure that it is explained and discussed in sufficient depth.

The procedure for each student presentation is as follows: Please read, reflect upon, and follow these presentation guidelines.  Presentations will be evaluated, in a friendly way but with high standards, using this feedback form.

Each  presentation should be prepared using LaTeX or Powerpoint, and should consist of about 40 slides.  You must copy all important equations, diagrams, charts, and tables from the paper into your slides.

For each paper, we will have a web-based discussion area.  Each student is expected to contribute at least one message to the discussion, before the presentation.  A message may ask an interesting question, point out a strength or weakness of the paper, or answer a question asked by someone else.  Messages should be thoughtful!

The schedule of presentations will be determined as early as possible.  Students should choose a date first, and then agree with the instructor about a paper to present.  To find ideas, students can look at this list of possible papers and contact the instructor.

If you want to change your presentation date, please arrange a swap with another student and notify the instructor at least two weeks in advance.

Most recently updated on May 14, 2007 by Charles Elkan,