CS 188: Natural Language Processing — Fall 2022
Prof. Nanyun (Violet) Peng

Announcements | Course Information | Schedule

Announcements

Course Information

Course objectives: Welcome! This course is designed to introduce you to some of the problems and solutions of NLP, and their relation to machine learning, statistics, linguistics, and social sciences. You need to know how to program and use common data structures.
It might also be nice—though it's not required—to have some previous familiarity with linear algebra and probabilities.
At the end you should agree (I hope!) that language is subtle and interesting, feel some ownership over some of NLP's techniques, and be able to understand research papers in the field.

Lectures:M/W 12:00 - 1:50pm
Location: 4760 Boelter Hall.
Prof:Nanyun (Violet) Peng Email: violetpeng@cs.ucla.edu
TAs: Zi-Yi Dou Email: zdou@cs.ucla.edu
Office hrs: Prof: Mon. 11:00am at Eng VI 397A; or zoom: link
TA: Wed. 11:00am Eng VI 389; or zoom: link
TA sessions: Friday 12:00 - 1:50pm, KAPLAN 169
Discussion site: Piazza https://piazza.com/class/l8av3ac3t0e36a
... public questions, discussion, announcements
Web page:https://vnpeng.net/cs188_win22.html
Textbook: Jurafsky & Martin, 3rd ed. (recommended)
Manning & Schütze (recommended)
Policies: Grading: homework 35%, project 15%, midterm 20%, final 25%, participation 5%
Honesty: UCLA Student Conduct Code

Schedule

Warning: The schedule below may change. Links to future lectures and assignments are just placeholders and will not be available until shortly before or after the actual lecture.


Week Monday Wednesday Friday (TA sessions) Suggested Reading
9/26 Introduction
  • Why is NLP hard? What's important?
  • Levels of language
  • NLP applications
    		• Project description out
    Text classification and lexical semantics
  • Text classification
  • Naive Bayes classifier
  • Logistic Regression
  • Review of linear algebra and calculus
  • Intro: J&M chapter 1
  • Chomsky hierarchy: J&M 16
  • Prob/Bayes: M&S 2
    • 10/3 Assignment 1 release
    Lexical semantics
  • Semantic phenomena and representations
  • WordNet
  • Thesaurus-based semantic similarity
    		• Distributional semantics
  • Word-Document Matrix
  • LSA
  • Semantic Similarity
  • Word Vectors
  • Data preparation and ML practice
  • Overview of ML system components
  • Language models: J&M 3
    • 10/10 N-gram language models
  • How to model language?
  • What's wrong with n-grams?
  • What do language models model?
    		• Smoothing n-grams
  • Add-one or add-λ smoothing
  • Cross-validation
  • Smoothing with backoff
  • Intro to google cloud computing
  • Intro to colab
  • Smoothing: J&M 3; Rosenfeld (2000)
    • 10/17 Assignment 1 due
    Log-linear models and neural language models
  • Log-linear models
  • Neural network basics (recap)
  • Feedforward neural language Models
    		• Assignment 2 release
    RNN language models
  • Recurrent neural networks (RNNs)
  • Long short-term memory networks (LSTMs)
    		• Assignment 1 answer keys release
  • Deep learning workshop
  • Intro to PyTorch
  • Neural language models: J&M 7
  • OpenAI blog post GPT-2 (with paper)
    • 10/24 Project midterm report due
    Transformers and Masked Language Models
  • Long-short term memory networks (LSTMs)
  • The transformer model
  • Masked languge models -- BERT
    		• Masked Language Models (cont.)
  • Neural network basics
  • PyTorch tutorial
  • Transformer paper ; BERT paper
    • 10/31 Midterm exam
    (12:00-1:50pm in class)
    Return assignment 1 gradings
    		 Syntax
  • Part-of-speech tagging
  • NP Chunking
  • Shallow Parsing
    		• Assignment 2 due
    Return project feedbacks
  • More Pytorch tutorial
  • Intro to Huggingface
  • The Viterbi Algorithm: J&M 8
  • Hidden Markov Models: J&M Appendix A;
    • 11/7 Sequence tagging models
  • POS-tagging leftovers
  • Hidden Markov Models (HMMs)
  • The Viterbi Algorithm
    		• Sequence tagging models (cont.)
  • The Viterbi Algorithm leftovers
  • Maximum Entropy Markov Models (MEMMs)
    		• Return midterm gradings
  • Deep dive of Huggingface tools
  • John Lafferty's paper on CRF
    • 11/14 Assignment 3 release
    Named Entity Recognition
  • MEMM leftovers
  • Intro to NER
  • Nested NERs
    		• Probabilistic parsing
  • What is parsing?
  • Why is it useful?
  • Brute-force algorithm
  • CKY algorithms
  • PCFG parsing
  • Review midterm exam
  • Attributes: J&M 12
  • Parsing: J&M 13
    • 11/21 Dependency Parser
  • Dependency grammar
  • Dependency trees
    		• Dependency Parser (Cont.)
  • Shift-reduce parser
  • Final exam recitation
  • CCG: Steedman & Baldridge; more
  • TAG/TSG: Van Noord, Guo, Zhang 1/2/3
  • Prob. parsing: J&M 14
    • 11/28 Assignment 3 due
    [Guest Lecture]     		[Guest Lecture] Project final report due
  • Final exam recitation
  • MT: J&M 25, M&S 13, statmt.org; tutorial (2003), workbook (1999), introductory essay (1997), technical paper (1993); tutorial (2006) focusing on more recent developments (slides, 3-hour video part 1, part 2)