Instructor	Susan (Xueqing) Liu (xueqing.liu AT stevens DOT edu)
Meeting	Monday 6:30-9:00PM, Babbio 122
Susan's Office Hour	Wed: 11-1, GS 250
TA	Guanqun Yang Fri: 4-6

Class Description

This course is a graduate-level course on fundamental techniques in information retrieval and text mining. By taking this course, students learn how to crawl, clean, process, mine, and infer knowledge from a massive amount of text data; how to build a search engine from scratch, including indexing, building retrieval models, and evaluating the performance of a search engine; they will also learn important machine learning and deep learning techniques for text data, including topic model, LSTM and BERT; finally, they will learn state-of-the-art research topics in text mining and information retrieval, and get research experience in these topics by working on the final project.

Learning Goals: Upon successful completion of this course, students should be able to:

• Perform basic text processing tasks such as crawling data from the web, pre-processing and cleaning natural language sentences;
• Evaluate ranking algorithms by using information retrieval evaluation techniques, and implement text retrieval models such as TF-IDF and BM25;
• Use Elastic search to implement a prototypical search engine on Twitter data;
• Derive inference algorithms for the maximum likelihood estimation (MLE), implement the expectation maximization (EM) algorithm;
• Use state-of-the-art tools such as LSTM/Bert for text classification tasks;

Recommended reading:

• Text Data Management and Analysis: A Practical Introduction to Information Retrieval and Text Mining . ChengXiang Zhai and Sean Massung
• Introduction to information retrieval . Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze

Schedule

Note: some slides require password, you can find the password from the home page in Canvas

Date	Slides/Readings	Homework/Exams
Week 1 of Sept 12	History of IR, vector space model slides pptx Reading: overview.ipynb Scoring, term weighting and the vector space model (lecture from Stanford IR course) The history of information retrieval research Pivoted length normalization A Formal Study of Information Retrieval Heuristics
Week 2 of Sept 19	Probabilistic ranking principle, Probabilistic/LM-based retrieval slides ppt Reading: RSJ model without relevance judgment A Study of Retrieval Models for Long Documents and Queries in Information Retrieval (WWW 2016) A language model approach to information retrieval Notes on the KL-divergence retrieval formula and Dirichlet prior smoothing Retrieval Models for Question and Answer Archives Two-stage language models for information retrieval Model-based feedback in the language modeling approach to information retrieval
Week 3 of Sept 26	IR evaluation, relevance feedback slides ppt Reading: A Comparative Study of Methods for Estimating Query Language Models with Pseudo Feedback An experimental comparison of click position-bias models	HW1 due Implementing IR models, evaluations to retrievel Stack Overflow posts
Week 4 of Oct 3	IR infrastructure: inverted index slides ppt ElasticSearch slides Reading Introduction to information retrieval. Chapter 2-5. The PageRank Citation Ranking: Bringing Order to the Web
Week 5 of Oct 10	Web search; learning to rank slides ppt Reading An Experimental Comparison of Click Position-Bias Models Ranking with Large Margin Principle: Two Approaches Learning to Rank Using an Ensemble of Lambda-Gradient Models From RankNet to LambdaRank to LambdaMART: An Overview XGBoost: A Scalable Tree Boosting System	HW2 due Using ElasticSearch to retrieve StackOverflow posts
Week 6 of Oct 17	Expectation maximization (EM) algorithm, PLSA slides ppt Reading Automatic Labeling of Multinomial Topic Models Notes on Probabilistic Latent Semantic Analysis Mining Causal Topics in Text Data: Iterative Topic Modeling with Time Series Feedback
Week 7 of Oct 24	RNN/LSTM, word embedding slides ppt Reading wordembedding.ipynb A Neural Probabilistic Language Model Distributed Representations of Words and Phrases and their Compositionality GloVe: Global Vectors for Word Representation On the difficulty of training recurrent neural networks	Project paper review due
Week 8 of Oct 31	midterm	Midterm
Week 9 of Nov 7	Attention; Transformers slides ppt Reading Introduction of Self-Attention Layer in Transformer Effective Approaches to Attention-based Neural Machine Translation NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE Deep contextualized word representations Attention Is All You Need BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Self-Attention with Relative Position Representations	HW3 due deriving/implementing EM algorithm for StackOverflow word tagging
Week 10 of Nov 14	Frontier topic 1slides ppt Reading RoBERTa: A Robustly Optimized BERT Pretraining Approach XLNet: Generalized Autoregressive Pretraining for Language Understanding Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Longformer: The Long-Document Transformer CodeBERT: A Pre-Trained Model for Programming and Natural Languages SciBERT: A Pretrained Language Model for Scientific Text BioBERT: a pre-trained biomedical language representation model for biomedical text mining	Project proposal due
Week 11 of Nov 21	Frontier topic 2 slides ppt Reading Derivation for cross entropy gradient Distilling the Knowledge in a Neural Network ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Reformer: The Efficient Transformer	HW4 due StackOverflow posts tag prediction
Week 12 of Nov 28	Frontier topic 3 Reading
Week 13 of Dec 5	Project presentation Reading
Week 14 of Dec 12	Project presentation

Final Grade Calculator

Homework	40%
Midterm	30%
Project	30%

Policies

Late Policy: submit within 24 hours of deadline - 90%, within 48 hours - 70%, over 48 hours - 0 point, 0 if code not compile.

Academic Integrity: Students must follow the instructions from the (Stevens Honor system). This course will have a zero-tolerance policy regarding plagiarism. You should complete all the assignments and quizzes on your own. You can help your classmates with questions such as how to use the programming language, what the library classes or methods do, what the errors mean, and how to interpret the assignment instructions. You are encouraged to come to both the instructor and the CAs' office hours regarding any questions you have, or email your questions to both the instructors or the CAs. However, you may not give or receive help from others (except the CA) with the actual implementation or answers for any of the assignments or tests. Do not show or share your code with others, and do not view or copy source code from others. For the same reason, you are not allowed to copy and paste a code snippet you found online in the assignments. All electronic work submitted for this course will be archived and subjected to automatic plagiarism detection. Whenever in doubt, please seek clarifications from the instructor. Students who violate the academic intergrity principle of Stevens will be immediately reported to the department and the school (which could leave a permanent mark on the transcript).

If you need special accommodations because of a disability, please contact the instructor through emails.