CSE 291 G00: Statistical Natural Language Processing

Term: Spring Qtr 2018
Instructor: Ndapa Nakashole, EBU3B 4108
Lecture: Tuesday and Thursday 3:30pm-4:50pm, EBU3B 4140
Credits: 4
Office hours: Tuesday 4:50 - 5.30pm and Thursday 2:30 - 3:30pm

TA: Vishal Thanvantri Vasudevan (office hours: Wednesday 9am-10:00 am, room: CSE 260A)



Course Description

This is a graduate course on statistical natural language processing (NLP). The course will start off with instructor-led presentations. The rest of the course will focus on understanding state-of-the-art machine learning algorithms for a series of important topics in NLP by discussing reseach papers. Participants take turns as discussion leaders.



Assignments and Project

The learning objective of the assignments and project is to gain experience doing NLP research.

Assignment and final report details:
Assignment and project reports should be 4-6 pages long (not including references), written by you alone
Formatting: Please use the ACL style files. Reports longer than 6 pages of the ACL format will not be considered. Suggested tasks:
Below is a list of suggested tasks for the project. You are welcome to work on any NLP task, however please confirm with the instructor before you start assignment 1.
Syllabus (Tentative, subject to substantial change!)
Date Topic/Readings Discussion Leader Homework
Apr 3 Introduction
Goldberg, Morgan & Claypool, 2017 Neural Network Methods for NLP. Ch, 1-4 Ndapa Nakashole
Hirschberg & Manning, Science 2015 Advances in NLP
Representation Learning (Words)
Apr 5 Mikolov et al., NIPS 2013 Distributed Representations of Words and Phrases and their Compositionality Ndapa Nakashole Review (04/04)
Mikolov et al., 2013 Efficient Estimation of Word Representations in Vector Space
Goldberg, Morgan & Claypool, 2017 Neural Network Methods for NLP. Ch, 10-11
Machine Reading
Apr 10 Sukhbaatar et al., NIPS 2015 End-To-End Memory Networks Ndapa Nakashole Review (09/04)
Herman et al., NIPS 2015 Teaching Machines to Read and Comprehend
Goldberg, Morgan & Claypool, 2017 Neural Network Methods for NLP. Ch, 14-15
Apr 12 Carlson et al., AAAI 2010 Toward an Architecture for Never-Ending Language Learning Ndapa Nakashole Review (11/04)
Mitchell et al., AAAI 2015 Never-Ending Learning
Sentence Representation
Apr 17 Kiros et al., NIPS 2015 Skip-Thought Vectors Chester Holtz Review (16/04)
Hill et al., NAACL 2016 Learning Distributed Representations of Sentences from Unlabelled Data Bala Srinivasan
Apr 19 Bowman, et al., ACL 2016 A Fast Unified Model for Parsing and Sentence Understanding Prahal Arora Review (18/04)
Yogatoma et al., ICLR 2017 Learning to Compose Words into Sentences using Reinforcement Learning Xiaochen Liu
Sequence-to-Sequence Models
Apr 24 Intro
Sutskever, et al., NIPS 2014 Sequence to Sequence Learning with Neural Networks Techit Limtiyakul Review (23/04)
Bahdanau et al., ICLR 2015 Neural Machine Translation by Jointly Learning to Align and Translate Yifan Zhou
Apr 26 Seq2Seq improvements
Gu et al. ACL 2016 Incorporating Copying Mechanism in Sequence-to-Sequence Learning Zhi Zhuang Review (25/04)
Luong et al. EMNLP 2015 Effective Approaches to Attention-based Neural Machine Translation
Unsupervised Learning
May 1 Unsupervised Word Translation
Conneau, et al., ICLR 2018 Word Translation Without Parallel Data Dewal Gupta review
Artetxe et al ACL 2017 Learning bilingual word embeddings with (almost) no bilingual data
May 3 Unsupervised Sentence Translation
Lample, et al., ICLR 2018 Unsupervised Machine Translation Using Monolingual Corpora Only Ziyang Wang review
Artetxe et al ICLR 2018 Unsupervised Neural Machine Translation Trehan, Shobhit
Part of Speech Tagging
May 8 Intro
Ling et al., EMNLP 2015 Finding Function in Form: Compositional Character Models for ... Rishabh Misra review
Lin et al NAACL 2017 Unsupervised POS Induction with Word Embeddings
Documents & Discourse Analysis
May 10 Coreference
Lee et al EMNLP 2017 End-to-end Neural Coreference Resolution Chaoguang Lin review
Ng ACL 2010 Supervised Noun Phrase Coreference Research: The First Fifteen Years
May 15 Dialog and Chatbots
Sordoni et al NAACL 2015 A Neural Network Approach to Context-Sensitive Generation of Conversational Responses Zisheng Zhang review
Serban et al. 2016 AAAI 2016 Building End-to-End Dialogue Systems Using Generative Hierarchical Neural Network Models
May 17 Chatbot Personality
Li et al ACL 2016 A Persona-Based Neural Conversation Model Yi-An Lai review
Mairesse and Walker ACL 2007 PERSONAGE: Personality Generation for Dialogue
May 22 Sentiment Analyis and Document Classification
Socher et al., EMNLP 2013 Recursive Deep Models for Semantic Compositionality ... Mengting Wan review
Yang et al., NAACL 2016 Hierarchical Attention Networks for Document Classification Xin Jin
Structured Prediction
May 24 Lample, et al 2016 Neural Architectures for Named Entity Recognition Nitesh Sekhar
Dong and Lapata, ACL 2016 Language to Logical Form with Neural Attention
Multi-relational Data
May 29 Inference
Bordes et al., NIPS 2013 Translating Embeddings for Modeling Multi-relational Data Zining Wang review
Socher et al., NIPS 2013 Reasoning With Neural Tensor Networks for Knowledge Base Completion
Multimodal Learning
May 31 Learning with Images + Text
Lazaridou et al., ICLR 2017 Multi-Agent Cooperation and the Emergence of (Natural) Language Lihui Lu review
Hu et al., CVPR 2017 Modeling Relationships in Referential Expressions with Compositional Modular Networks Asmitha Rathis
Jun 5 & 7 Project