CSE 291 B00: Statistical Natural Language Processing

CSE 291: Statistical Natural Language Processing

Term: Spring Qtr 2017
Instructor: Ndapa Nakashole, EBU3B 4108
Lecture: Tuesday and Thursday 3:30pm-4:50pm, EBU3B 2154
Credits: 4
Office Hours: Monday 1pm - 2pm; Tuesday and Thursday 4:50pm-5.30pm

Announcements

June 6: Peer review discussions have started on TritonEd.
May 9: Project proposal upload link posted.
April 24: Project meeting slots posted.
April 20: Discussion leaders have been posted.
April 15: Paper preference selection form has been posted.
March 19: Enrollement questionnaire posted.
March 19: Tentative list of readings posted.

Course Description

This is a research-oriented course on statistical natural language processing (NLP). The course will focus on understanding and extending state-of-the-art machine learning algorithms for a series of important topics in NLP. The course involves reading and discussing current research papers. This course assumes background in basic machine learning. Prior NLP experience is helpful, but not required.

Enrollment

Class size will be capped at 20 students (PhD and MS). Enrollment decisions will be made by the instructor by the first week of class.
Enrollment decisions will be based on your answers to this questionnaire. The last day to fill it out is April 3.

Grading

20%: Class participation
20%: Paper reviews
10%: Peer review of other class project proposals
20%: Paper presentation
30%: Class project

Course Format

Each week, there will be two lectures where we will discuss reseach papers.
For each class, homework consists of writing a review for one focus paper, due before class.
During class we will have a lively discussion.
Review links will be posted next to focus papers.

Class Project

For the class project, your task is to write a 4-6 page paper, written by you alone, to extend a past NLP research contribution.
Pick a NLP paper among class readings or any paper from NLP and ML conferences.
Formatting: Please use the ACL 2017 style files. Papers longer than 6 pages of the ACL 2017 format will not be considered.
Your class paper should describe:

What prior research contribution you want to build on or extend.
What limitations, of a prior research contribution, you are addressing.
How you aim to address the limitations you have identified. This should include evaluation methodology, experimental design, evaluation resources.
Why your approach has a good chance of working backed up by an analysis of published related work in the area, or (optionally) preliminary experimental results.

Timeline:

First draft: due May 15. Worth 1/4 of project grade. Instructor will provide feedback on your first draft. Upload deadline has now passed. Style files can be downloaded from the ACL website.
Second draft. due ~~June 1~~ June 5. Worth 1/4 of project grade. Other participants will provide feedback on your second draft. You will be asked to read and comment on two other class projects. First, upload your project proposal here (Upload deadline has now passed). You'll pick two other projects to comment on, we will go over how to comment on a project during class (June 9). Note that for this part of the project you will be graded both on your own proposal, and on the feedback you provide on other people's proposals. Peer review feedback due June 9.
Final paper. due June 12. Worth 1/2 of project grade.

Books
Recommended texts are:

[J&M]: Jurafsky and Martin, Speech and Language Processing, 2nd edition (amazon)
[M&S]: Manning and Schütze, Foundations of Statistical Natural Language Processing (amazon) (online)

[J&M] 3rd edition free chapters online
[M&S] is free online.

Machine Learning background: Andrew Ng's Coursera course

Syllabus (Tentative, subject to substantial change!)

Date	Topic/Readings	Discussion Leader	Homework
Apr 6	Introduction
Apr 6	J&M Chapter 1	Ndapa Nakashole
Representation Learning for NLP
Apr 11	Goldberg, JAIR 2016 A Primer on Neural Network Models for NLP. Sections 3 & 5 (Background)	Ndapa Nakashole
	Mikolov et al., NIPS 2013 Distributed Representations of Words and Phrases and their Compositionality		Review (04/10)
	Le and Mikolov, ICML 2014 Distributed Representations of Sentences and Documents (Optional)
Machine Reading, & QA
Apr 13	Sukhbaatar et al., NIPS 2015 End-To-End Memory Networks	Ndapa Nakashole	Review (04/12)
Apr 13	Weston et al., ICLR 2015 Memory Networks (Background)
Apr 18	Carlson et al., AAAI 2010 Toward an Architecture for Never-Ending Language Learning	Ndapa Nakashole	Review (04/17)
Apr 18	Mitchell et al., AAAI 2015 Never-Ending Learning
Apr 20	Herman et al., NIPS 2015 Teaching Machines to Read and Comprehend	Ndapa Nakashole	Review (04/19)
Apr 20	Chen et al., ACL 2016 A Thorough Examination of the CNN/Daily Mail Reading ...
Cross-lingual projection, & Morphology
Apr 25	Lazaridou et al., ACL 2015 Delving into Cross-Space Mapping for Zero-Shot Learning	Ndapa Nakashole	Review (04/24)
Apr 25	Hermann et al., ACL 2014 Multilingual Models for Compositional Distributed Semantics
Apr 27	Soricut and Och, NAACL 2015 Unsupervised Morphology Induction Using Word Embeddings	Ndapa Nakashole	Review (04/26)
Semantics
May 2	Kiros et al., NIPS 2015 Skip-Thought Vectors	Shuai Tang	Review (05/01)
May 2	Hill et al., NAACL 2016 Learning Distributed Representations of Sentences from Unlabelled Data
May 4	Bowman, et al., ACL 2016 A Fast Unified Model for Parsing and Sentence Understanding	Mingyang Wang	Review (05/03)
May 4	Yogatoma et al., ICLR 2017 Learning to Compose Words into Sentences using Reinforcement Learning
Part-of-Speech Tagging, NER, & Phonology
May 9	Ling et al., EMNLP 2015 Finding Function in Form: Compositional Character Models for ...	Shashankar Sudarsan	Review (05/08)
May 11	Chiu and Nichols, TACL 2016 Named Entity Recognition with Bidirectional LSTM-CNNs	Kan Xu	Review (05/10)
	Goldberg, JAIR 2016 A Primer on Neural Net Models for NLP. Sec 9, pp. 41-45 (CNNs Background)
	Kim, EMNLP 2014 Convolutional Neural Networks for Sentence Classification (Optional)
	Rativov, CONLL 2009 Design Challenges and Misconceptions in Named Entity Recognition (Optional)
May 16	Tsvetkov et al., NAACL 2016 Polyglot Neural Language Models: A Case Study in ...	Sindhura Raghavan	Review (05/15)
Sentiment Analysis
May 18	Socher et al., EMNLP 2013 Recursive Deep Models for Semantic Compositionality ...	Utkarsh Simha	Review (05/17)
May 18	Irsoy and Cardie, ACL 2014 Opinion Mining with Deep Recurrent Neural Networks	Caroline Kim
Machine Translation
May 23	Sutskever, et al., NIPS 2014 Sequence to Sequence Learning with Neural Networks	Nishant Gurnani	Review (05/22)
May 23	Bahdanau et al., ICLR 2015 Neural Machine Translation by Jointly Learning to Align and Translate	Mainak Biswas
May 25	Lee, et al., TACL 2017 Fully Character-Level Neural Machine Translation...	Rishikesh Ghewari	Review (05/24)
May 25	Wu et al., arxiv 2016 Google's Neural Machine Translation System: Bridging the ...	Mridul Garg
Structured Prediction
May 30	Collins, EMNLP 2002 Theory and Experiments with Perceptron Algorithms	Cuong Luong	Review(05/29)
May 30	Dong and Lapata, ACL 2016 Language to Logical Form with Neural Attention	Swathi V. Hoysala
Vision-Robotics-Language
Jun 1	Xu et al., ICML 2015 Neural Image Caption Generation with ...	Steven Hill	Review (05/31)
Jun 1	Ortiz et al., NAACL 2015 Learning to Interpret and Describe Abstract Scenes	Yuting Wang
Jun 6	Venugopalan et al., CVPR 2017 Captioning Images with Diverse Objects	Dustin Wright	Review (05/04)
Jun 6	Mei et al., AAAI 2016 Mapping of Navigational Instructions to Action Sequences	Vraj Shah
Jun 8	Project proposal peer review