CSE 254 Spring 2007

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
UNIVERSITY OF CALIFORNIA, SAN DIEGO

CSE 254: Log-linear models and conditional random fields

Spring 2007

Please use this discussion board to ask questions about CSE 254, especially questions about the assignments.

CSE 254 is a graduate seminar devoted to recent research on AI learning methods and applications. This is not an introductory course, so the prerequisite is at least one graduate-level course (at UCSD or elsewhere) in machine learning or a closely related area such as statistics or pattern recognition. Appropriate courses at UCSD include CSE 250B (Principles of AI: Learning), CSE 291 (Probabilistic Methods in AI and Machine Learning), and Cognitive Science 260 (Pattern Recognition).

The room for CSE 254 is HSS 2321. The class meets on Tuesdays and Thursdays from 3:30pm to 4:50pm, beginning on April 3. Because several students have conflicts, we will not change the time or room. The last meeting will be on June 8.

This year CSE 254 has a specific topic: conditional random fields, known as CRFs. See here for tutorials and papers on CRFs, and see here for available software. We will start by covering log-linear methods, which are the foundation of CRFs. We may also cover related research on other methods for solving structured prediction problems, and/or on topic models.

For the first few weeks, the instructor will lecture on log-linear methods, topic models, and CRFs. There are two assignments Then, in each class meeting, one student will give a talk lasting about 60 minutes presenting a recent technical paper in detail. In questions during the talk, and in the final 20 minutes, all seminar participants will discuss the paper and the issues raised by it.


date	topic or presenter (not confirmed)	reading or handout
April 3	overview, log-linear models
April 5	No class: Rockwood lecture at 4pm by Tomaso Poggio
April 10	log-linear models continued	Assignment 1: naive Bayes vs log-linear
April 12	nonlinear optimization; CRFs
April 17	CRFs continued	Assignment 2: CRFs
April 19	Sanjoy Dasgupta	Maximum entropy and all that: information geometry
April 24	Eric Wiewiora	Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data by J Lafferty, A McCallum, F Pereira, ICML 2001.
April 26	Peter Shin	Michael Collins. Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms. Proceedings of the ACL-02 conference on Empirical methods in natural language processing, pp.1-8, 2002.
May 1	Guilherme Hoefel	Ben Taskar, Carlos Guestrin and Daphne Koller. Max-Margin Markov Networks. In Advances in Neural Information Processing Systems 16 (NIPS 2003), 2004.
May 3	lecture
May 8	Panagiotis Voulgaris	Andrew McCallum. Efficiently Inducing Features of Conditional Random Fields. In Proceedings of the 19th Conference in Uncertainty in Articifical Intelligence (UAI-2003), 2003.
May 10	Tom Griffiths	Mandler Hall, room 3545, 4pm: "Everyday Inductive Leaps: Making Predictions and Detecting Coincidences"
May 15	Pouya Bozorgmehr	Xuming He, Richard Zemel, and Miguel Á. Carreira-Perpiñán. Multiscale conditional random fields for image labelling. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2004.
May 17	Simone McCloskey	Multi-labelled classification using maximum entropy method by S Zhu, X Ji, W Xu, Y Gong, SIGIR 2005.
May 22	Tingfan Wu	Ioannis Tsochantaridis, Thorsten Joachims, Thomas Hofmann, Yasemin Altun. Large Margin Methods for Structured and Interdependent Output Variables. JMLR, December 2005.
May 24	Nicholas Butko	Sham Kakade, Yee Whye Teh, Sam T. Roweis. An Alternate Objective Function for Markovian Fields. ICML 2002.
May 29	Alexander Loukissas	Samuel Gross, Olga Russakovsky, Chuong Do, and Serafim Batzoglou. Training conditional random fields for maximum labelwise accuracy. In Advances in Neural Processing Systems 19 (NIPS), December 2006.
May 31
June 5
June 7
finals week	project presentations

Each student who takes the course for four units will do a term project following specific guidelines. The project should be at the frontier of current research, and closely inspired by at least one of the papers discussed in the class. Many projects will be devoted to reproducing the results of a selected recent high-quality published paper. Doing this has very high educational value and provides an excellent starting point for subsequent high-quality original research.

Project reports will be evaluated using these grading criteria. There is a schedule for handing in a detailed project proposal, a draft project report, and then the final report. The seminar will have no final exam. Letter grades will be based mostly on the final project report, but the presentations, participation in class and in the web-based discussions, and the intermediate project deliverables are all important also.

The instructor is Charles Elkan (Professor), whose office is in the CSE building, room 4134. Feel free to send email to arrange an appointment, or telephone (858) 534-8897.

Note: The seminar will run in parallel with a data mining contest sponsored by Fair Isaac, with cash prizes.

REGISTRATION

Students may take the seminar for a letter grade for four units, or for one or two units S/U:

For one unit, a student will present one research paper and participate in all class meetings.
For two units, a student will make one presentation and do both Assignment 1 and Assignment 2
Four units will require a presentation, both assignments, and a project.

For four units, a student should register for CSE 254, section id 588815 for a letter grade. For one or two units, a student should register for the instructor's CSE 293, section id 588846. Students who took a previous version of CSE 254 may take it again. Papers will be different this year.

PAPERS AND PRESENTATIONS

Papers will be recent technical articles from ICML and similar venues. Each paper will be made available on the web as the quarter progresses. Students will choose papers in consultation with the instructor. Each presentation will cover a single paper, to ensure that it is explained and discussed in sufficient depth.

The procedure for each student presentation is as follows:

One week in advance: Finish a draft of about 40 slides that present clearly the work in the paper. Make an appointment with the instructor to discuss the draft slides. Email the slides to elkan@cs.ucsd.edu.
Several days in advance: Meet for about one hour to discuss improving the slides, and how to give a good presentation.
Day of presentation: Give a good presentation with confidence, enthusiasm, and clarity.
Less than three days afterwards: Make changes to the slides suggested by the class discussion, and email the slides in PDF, two slides per page, to the instructor for publishing. Try to make your PDF file less than one megabyte.

Please read, reflect upon, and follow these presentation guidelines. Presentations will be evaluated, in a friendly way but with high standards, using this feedback form.

Each presentation should be prepared using LaTeX or Powerpoint, and should consist of about 40 slides. You must copy all important equations, diagrams, charts, and tables from the paper into your slides.

For each paper, we will have a web-based discussion area. Each student is expected to contribute at least one message to the discussion, before the presentation. A message may ask an interesting question, point out a strength or weakness of the paper, or answer a question asked by someone else. Messages should be thoughtful!

The schedule of presentations will be determined as early as possible. Students should choose a date first, and then agree with the instructor about a paper to present. To find ideas, students can look at this list of possible papers and contact the instructor.

If you want to change your presentation date, please arrange a swap with another student and notify the instructor at least two weeks in advance.

Most recently updated on May 14, 2007 by Charles Elkan, elkan@cs.ucsd.edu