NLP course notes and research notebooks

Natural Language Processing

Author

Oren Bochman

Published

Tuesday, February 11, 2025

 

Domination Games

Establishing hierarchies
Game Theory
Agent Based Models
Social Structures
Hierarchies
In a previous post titled Samurai’s world, I considered the complex states and a framing game of a Lewis signaling game for which the requisite linguistics aspects of…
Tuesday, February 18, 2025
Oren Bochman

Floating Constraints in Lexical Choice

review
This is one of the three paper mentioned by Kathleen McKeown in her heroes of NLP interview by Andrew NG. The paper is about the floating constraints in lexical choice.
Tuesday, February 18, 2025
Oren Bochman

 

A Samurai’s World

Inducing politeness, honor, into an Emergent language
Emergent Language
Game Theory
Social Structure
In this post, we will explore the emergence of face, politeness, and formality via multi-agent signaling language game. Specifically I consider the complex states and a framing game for a signaling game for which the requisite linguistics aspects of politeness and formality would emerge out of language evolution
Monday, February 17, 2025
Oren Bochman

 

Knights and Knaves world

Inducing logic into an Emergent language
“if proof theory is about the sacred, then model theory is about the profane” – (Dalen 2012, 3:1)
Monday, February 17, 2025
Oren Bochman

 

Further Desiderata for Emergent Language

Some additional desiderata for emergent language come from:
Monday, February 17, 2025
Oren Bochman

A Convolutional Attention Network for Extreme Summarization of Source Code

review
NLP
Paper
Attention
Deep learning
Review
Stub
Thursday, February 13, 2025
Oren Bochman

Neural Morphological Analysis: Encoding-Decoding Canonical Segments

Review
Paper
Review
Canonical morphological segmentation aims to divide words into a sequence of standardized segments. In this work, we propose a character-based neural encoder-decoder model…
Wednesday, February 12, 2025
Oren Bochman

Coverage Embedding Models for Neural Machine Translation

Review
Paper
Review
In this paper, we enhance the attention-based neural machine translation (NMT) by adding explicit coverage embedding models to alleviate issues of repeating and dropping…
Wednesday, February 12, 2025
Oren Bochman

 

Summarization Task

NLP
Notes
Literature review
Summarization task
Relevance
Thought on engineering an Automatic text Summarizer
Monday, February 10, 2025
Oren Bochman

Morphological Word Embeddings

Review
Paper
Review
Podcast
NLP
Morphology
This is a mentioned in the tokenization lab in course four week 3
Friday, February 7, 2025
Oren Bochman

Practical and Optimal LSH for Angular Distance

Review
Paper
Review
In the NLP specialization we have covered and used LSH a number of times in at least two courses. In one sense I have an understanding of what is going on when we implement…
Wednesday, February 5, 2025
Oren Bochman

xLSTM: Extended Long Short-Term Memory

Review
Paper
Review
NLP
LSTM
Seq2Seqs
RNN
Podcast
Whate’er the theme, the Maiden sang
As if her song could have no ending;
I saw her singing at her work,
And o’er the sickle bending;—
I listened, motionless and still;
And…
Monday, February 3, 2025
Oren Bochman

cover

The Secret Life of Pronouns What Our Words Say About Us

Review
Review
Book
NLP
Sentiment Analysis
Sentiment Analysis
Podcast
To grunt and sweat under a weary life, But that the dread of something after death, The undiscovered country from whose bourn No traveler returns, puzzles the will, And…
Thursday, January 30, 2025
Oren Bochman

When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?

Review
Attention
Bidirectional LSTM
Deep learning
Embeddings
LLM
NLP
Paper
Review
Podcast
Sunday, February 11, 2024
Oren Bochman

Neural Machine Translation by Jointly Learning to Align and Translate

Review
Attention
Bidirectional LSTM
Deep learning
Embeddings
LLM
NLP
NMT
Paper
Review
Podcast
Translation task
the paper that introduced the attention for NMT
Sunday, February 11, 2024
Oren Bochman

The LORELEI Project

CMU CS11-737: Multilingual NLP
Active learning
Multilingual NLP
NLP
Notes
This week we’re delve into the LORELEI project, a DARPA initiative to rapidly develop low-resource information extraction in new languages.
Tuesday, April 12, 2022
Oren Bochman

Active Learning

CMU CS11-737: Multilingual NLP
Active learning
Multilingual NLP
NLP
Notes
This week we’re delve into active learning, a tool that allows us to efficiently generate examples for low resource languages.
Tuesday, April 5, 2022
Oren Bochman

Syntax and Parsing

CMU CS11-737: Multilingual NLP
Syntax
Dependency parsing
Multilingual NLP
NLP
Notes
This week we’re delve into syntax in general but dependency parsing in particular. We’ll also discuss the Universal Dependencies treebank and the differences between syntactic and semantic dependencies.
Tuesday, March 29, 2022
Oren Bochman

Text-to-speech

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes

course banner
Thursday, March 17, 2022
Oren Bochman

Automatic Speech Recognition

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
Thursday, March 3, 2022
Oren Bochman

Speech

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
Tuesday, March 1, 2022
Oren Bochman

Multilingual ASR and TTS

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
Today’s lecture uh it is about multilingual speech processing I will mainly uh explain about the speech recognition but it also covers most of the techniques that are quite…
Tuesday, March 1, 2022
Oren Bochman

Morphology

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
Tuesday, March 1, 2022
Oren Bochman

Speech

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
This week we will be discussing speech recognition and the various methods used to achieve it.
Tuesday, March 1, 2022
Oren Bochman

Typology: The Space of Languages

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
This week we will cover Typology and how to structure the space of languages. We will discuss how to quantify similarity between languages, language families, linguistic typology, and typological databases.
Friday, February 25, 2022
Oren Bochman

Code Switching, Pidgins, Creoles

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
This week we will explores code-switching, pidgins, and creoles as linguistic phenomena. Code-switching involves fluent speakers mixing languages, often driven by convenience, social signaling, or semantic nuance. Pidgins emerge as simplified lingua francas between groups with different native languages, while creoles develop when pidgins acquire native speakers.
Thursday, February 24, 2022
Oren Bochman

Unsupervised Machine Translation

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
This week we will cover text classification and sequence labeling. We will start with the basics of text classification, and then move on to more advanced topics like sequence labeling.
Thursday, February 17, 2022
Oren Bochman

Assignment 1: Multilingual POS Tagging

CMU CS11-737: Multilingual NLP
Multilingual NLP
NLP
Notes
This week we will cover text classification and sequence labeling. We will start with the basics of text classification, and then move on to more advanced topics like sequence labeling.
Tuesday, February 15, 2022
Oren Bochman

Translation and Translation Data

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
This week we will cover text classification and sequence labeling. We will start with the basics of text classification, and then move on to more advanced topics like sequence labeling.
Tuesday, February 15, 2022
Oren Bochman

Multilingual Q&A

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
This week we will cover multilingual question answering.
Tuesday, February 15, 2022
Oren Bochman

Language Contact and Change

CMU CS11-737: Multilingual NLP
Multilingual NLP
NLP
Notes
Language Contact
Language Change
This week we will cover language contact and change, including dialects, nonnative writing, cognates, and borrowing. We will also discuss computational models of transliteration, cognates, and borrowing, as well as bilingual embedding and lexicon induction.
Thursday, February 10, 2022
Oren Bochman

Data-driven Strategies for NMT

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
NMT
This week we will cover …
Tuesday, February 8, 2022
Oren Bochman

Translation Models

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
seq2seq
NMT
This week cover MT and seq2seq models, including language models that calculate the probability of text and conditional language models that generate text based on specifications. We will delve into calculating probabilities of a sentence using autoregressive models, implemented with recurrent neural networks, and how to generate sentences using methods such as sampling and argmax. We also introduce attention mechanisms and the transformer model, which uses self-attention and other techniques like positional encodings, layer normalization and specialized training schedules.
Thursday, February 3, 2022
Oren Bochman

Translation and Translation Data

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
This week we will cover the practice of translation, machine translation, translation evaluation metrics, translation data sources, and bi-text extraction & eoderedfiltering.
Tuesday, February 1, 2022
Oren Bochman

Words, Parts of Speech, Morphology

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
This week we will cover text classification and sequence labeling. We will start with the basics of text classification, and then move on to more advanced topics like sequence labeling.
Monday, January 24, 2022
Oren Bochman

Sequence Labeling

CMU CS11-737: Multilingual NLP
Multilingual NLP
NLP
Notes
This week we will cover text classification and sequence labeling. We will start with the basics of text classification, and then move on to more advanced topics like sequence labeling.
Thursday, January 20, 2022
Oren Bochman

Text Classification

CMU CS11-737: Multilingual NLP
Attention
Multilingual NLP
NLP
Notes
This week we will cover text classification and sequence labeling. We will start with the basics of text classification, and then move on to more advanced topics like sequence labeling.
Tuesday, January 18, 2022
Oren Bochman

Data Augmentation for Low-Resource Neural Machine Translation

review
NLP
Paper
NMT
Deep learning
Review
Stub
The quality of a Neural Machine Translation system depends substantially on the availability of sizable parallel corpora. For low-resource language pairs this is not the…
Friday, May 14, 2021
Oren Bochman

Restoring Hebrew Diacritics Without a Dictionary

review
NLP
Paper
LSTM
Hebrew
Morphology
Deep learning
Review
Stub
We demonstrate that it is feasible to accurately diacritize Hebrew script without any human-curated resources other than plain diacritized text. We present Nakdimon, a…
Thursday, May 13, 2021
Oren Bochman

It’s MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk

Review
NLP
Paper
Review
MBR
Minimum Bayes Risk
ASR task
Automatic speech recognition
MT task
Syntactic Parsing task
AMR parsing task
Question answering task
Summarization task
Minimum Bayes Risk (MBR) decoding is a method for choosing the outputs of a machine learning system based not on the output with the highest probability, but the output with…
Monday, May 10, 2021
Oren Bochman

Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decoding

Review
NLP
Paper
Review
MBR
Minimum Bayes Risk
Text Generation
Decoding Algorithms
Encoder-Decoder Models
Language Models
Machine Translation
Image Captioning
Question Generation
Common Sense Reasoning
Text Summarization
One of the most important challenges in text generation systems is to produce outputs that are not only correct but also diverse. Recently, Minimum Bayes-Risk (MBR) decoding…
Monday, May 10, 2021
Oren Bochman

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Review
Attention
Bidirectional LSTM
Deep learning
Embeddings
LLM
NLP
Paper
Review
Transformer
Podcast
review of the paper 2018 paper introducing BERT, a pre-trained language model
Sunday, May 9, 2021
Oren Bochman

ELMo - Deep contextualized word representations

Review
Attention
Bidirectional LSTM
Deep learning
Embeddings
LLM
NLP
Paper
Review
Stub
Podcast
review of the paper “Deep contextualized word representations” on dot product attention
Sunday, May 9, 2021
Oren Bochman

Exposing Attention Glitches with Flip-Flop Language Modeling

Review
Attention
LSTM
Deep learning
LLM
NLP
Paper
Podcast
Review
review
Sunday, May 9, 2021
Oren Bochman

Attention Is All You Need

review
NLP
Paper
Attention
Deep learning
Review
Stub
Review of the 2017 paper “Attention Is All You Need” on the transformer architecture.
Saturday, May 8, 2021
Oren Bochman

Effective Approaches to Attention-based Neural Machine Translation

Review
NLP
Paper
Attention
Deep learning
Review
Podcast
Translation task
Review of the landmark 2015 paper innovating attention-based mechanisms for Neural Machine Translation
Saturday, May 8, 2021
Oren Bochman

Putting the “Re” in Reformer: Ungraded Lab

This ungraded lab will explore Reversible Residual Networks. You will use these networks in this week’s assignment that utilizes the Reformer model. It is based on on the…
Thursday, April 29, 2021
Oren Bochman

Reformer Efficient Attention: Ungraded Lab

The videos describe two ‘reforms’ made to the Transformer to make it more memory and compute efficient. The Reversible Layers reduce memory and Locality Sensitive Hashing(LSH)…
Wednesday, April 28, 2021
Oren Bochman

Chat Bots

NLP with Attention Models
Attention
Chat bot development
Coursera
Intelligent agents
Locality sensitive hashing
NLP with Attention Models
Neural Machine Translation
NLP
Notes
Positional encoding
Question answering task
Reversible layers
Teacher forcing
Transformer
This week of the NLP Specialization, we explore Chatbot. We will be building Reformer model, an efficient transformer to create intelligent conversational agents.We will learn how to train this model on dialogue datasets and generate responses that mimic human-like conversations. Through hands-on exercises using JAX, we will gain practical experience in building chatbot that can understand and respond to user queries. We will master the skills to develop sophisticated chatbot applications using state-of-the-art NLP techniques
Tuesday, April 27, 2021
Oren Bochman

notes

Automatic Summarization Task

NLP
Notes
Literature review
Summarization task
Relevance
Conference talk
Concepts, slide commentaries and Lecture notes on Automatic text Summarization by Masa Nekic
Saturday, April 24, 2021
Oren Bochman

Assignment 3 Ungraded Sections - Part 2: T5 SQuAD Model

NLP with Attention Models
NLP
Coursera
Lab
NLP with Attention Models
Welcome to the part 2 of testing the models for this week’s assignment. This time we will perform decoding using the T5 SQuAD model. In this notebook we’ll perform Question…
Tuesday, April 13, 2021
Oren Bochman

Assignment 3 Ungraded Sections - Part 1: BERT Loss Model

NLP with Attention Models
NLP
Coursera
Lab
NLP with Attention Models
Welcome to the part 1 of testing the models for this week’s assignment. We will perform decoding using the BERT Loss model. In this notebook we’ll use an input, mask (hide)…
Monday, April 12, 2021
Oren Bochman

SentencePiece and Byte Pair Encoding

NLP with Attention Models
NLP
Coursera
Lab
NLP with Attention Models
In order to process text in neural network models, it is first required to encode text as numbers with ids (such as the embedding vectors we’ve been using in the previous…
Sunday, April 11, 2021
Oren Bochman

tl-features-vs-fine-tuning

Question Answering

NLP with Attention Models
Attention
Coursera
Deep Learning Algorithms
NLP
Notes
NLP with Attention Models
Neural Machine Translation
Transformer
Teacher forcing
Positional encoding
Question answering task
This week we will dive into Neural Question Answering. We will build advanced models like T5 and BERT to accurately answer questions based on given contexts. We will fine-tune these models to optimize their performance. We will gain practical experience in building question-answering systems.
Saturday, April 10, 2021
Oren Bochman

The Transformer Decoder: Ungraded Lab Notebook

In this notebook, you’ll explore the transformer decoder and how to implement it with Trax.
Saturday, April 3, 2021
Oren Bochman

The Three Ways of Attention and Dot Product Attention: Ungraded Lab Notebook

In this notebook you’ll explore the three ways of attention (encoder-decoder attention, causal attention, and bi-directional self attention) and how to implement the latter…
Friday, April 2, 2021
Oren Bochman

Week 2 - Text Summarization

NLP with Attention Models
NLP with Attention Models
Neural Machine Translation
Coursera
Notes
Deep Learning Algorithms
Transformer
Teacher forcing
Positional encoding
GPT2
Transformer decoder
Attention
Dot product attention
Self attention
Causal attention
Multi-head attention
Summarization task
This week we unlock the secrets of Neural Text Summarization. We will building a powerful Transformer model to extract crucial information and create concise summaries. Through hands-on exercises using JAX, we will learn techniques like beam search and length normalization to enhance the quality of our summaries. Through hands-on exercises we will train our model on a dataset of articles.
Thursday, April 1, 2021
Oren Bochman

Calculating the Bilingual Evaluation Understudy (BLEU) score: Ungraded Lab

Sequence Models
NLP
Coursera
Lab
Sequence Models
In this ungraded lab, we will implement a popular metric for evaluating the quality of machine-translated text: the BLEU score proposed by Kishore Papineni, et al. In their…
Tuesday, March 23, 2021
Oren Bochman

Stack Semantics in Trax: Ungraded Lab

Sequence Models
NLP
Coursera
Lab
Sequence Models
In this ungraded lab, we will explain the stack semantics in Trax. This will help in understanding how to use layers like Select and Residual which gets . If you’ve taken a…
Monday, March 22, 2021
Oren Bochman

Neural Machine Translation

NLP with Attention Models
Attention
Beam search
BLEU
ROUGE
Coursera
NLP with Attention Models
Notes
Machine translation task
MBR
NLP
Positional encoding
Seq2Seq
Transformer
Teacher forcing
Translation task
Word alignment
This week we dive deep into the Neural Machine Translation. We’ll learn about the encoder-decoder architecture, explore attention mechanisms that enable the model to focus on different parts of the input sequence during translation. In the hands-on exercises, we’ll implement an attention model for English to German translation, train it on a dataset of sentence pairs, and evaluate its performance.
Saturday, March 20, 2021
Oren Bochman

 

Machine Translation Course 2020 - Lecture 1 - Introduction

NMT
BIU
NLP
Notes
Monday, March 1, 2021
Oren Bochman

Evaluate a Siamese model: Ungraded Lecture Notebook

Sequence Models
NLP
Coursera
Lab
Sequence Models
In this lecture notebook you will learn how to evaluate a Siamese model using the accuracy metric. Because there are many steps before evaluating a Siamese network (as you…
Saturday, November 21, 2020
Oren Bochman

Modified Triplet Loss

Sequence Models
NLP
Coursera
Lab
Sequence Models
In this notebook you’ll see how to calculate the full triplet loss, step by step, including the mean negative and the closest negative. You’ll also calculate the matrix of…
Friday, November 20, 2020
Oren Bochman

Creating a Siamese model using Trax: Ungraded Lecture Notebook

Sequence Models
NLP
Coursera
Lab
Sequence Models
Before building the model you will need to define a function that applies L2 normalization to a tensor. This is very important because in this week’s assignment you will…
Thursday, November 19, 2020
Oren Bochman

course banner

Siamese Networks

Sequence Models
NLP
Coursera
notes
Sequence Models
LSTM
Siamese networks
One shot learning
Triplet loss
we cover Neural networks for deep learning, then build a tweet classifier that places tweets into positive or negative sentiment categories, using a DNN.
Wednesday, November 18, 2020
Oren Bochman

Vanishing Gradients

Sequence Models
NLP
Coursera
Lab
Sequence Models
In this notebook you’ll take another look at vanishing gradients, from an intuitive standpoint.
Tuesday, November 17, 2020
Oren Bochman

LSTMs and Named Entity Recognition

Sequence Models
NLP
Coursera
Notes
Sequence Models
we cover Neural networks for deep learning, then build a tweet classifier that places tweets into positive or negative sentiment categories, using a DNN.
Monday, November 16, 2020
Oren Bochman

Creating a GRU model using Trax: Ungraded Lecture Notebook

Sequence Models
NLP
Coursera
Lab
Sequence Models
For this lecture notebook you will be using Trax’s layers. These are the building blocks for creating neural networks with Trax.
Saturday, November 14, 2020
Oren Bochman

Vanilla RNNs, GRUs and the scan function

Sequence Models
NLP
Coursera
Lab
Sequence Models
In this notebook, you will learn how to define the forward method for vanilla RNNs and GRUs. Additionally, you will see how to define and use the function scan to compute…
Friday, November 13, 2020
Oren Bochman

Working with JAX numpy and calculating perplexity: Ungraded Lecture Notebook

Sequence Models
NLP
Coursera
Lab
Sequence Models
Normally you would import numpy and rename it as np.
Thursday, November 12, 2020
Oren Bochman

Hidden State Activation : Ungraded Lecture Notebook

Sequence Models
NLP
Coursera
Lab
Sequence Models
In this notebook you’ll take another look at the hidden state activation function. It can be written in two different ways.
Wednesday, November 11, 2020
Oren Bochman

Recurrent Neural Networks for Language Modeling

Sequence Models
NLP
Coursera
Notes
Sequence Models
Tuesday, November 10, 2020
Oren Bochman

Data generators

Sequence Models
NLP
Coursera
Lab
Sequence Models
In Python, a generator is a function that behaves like an iterator. It will return the next item. Here is a link to review python generators. In many AI applications, it is…
Monday, November 9, 2020
Oren Bochman

Classes and subclasses

Sequence Models
NLP
Coursera
Lab
Sequence Models
In this notebook, I will show you the basics of classes and subclasses in Python. As you’ve seen in the lectures from this week, Trax uses layer classes as building blocks…
Sunday, November 8, 2020
Oren Bochman

Trax : Ungraded Lecture Notebook

Sequence Models
NLP
Coursera
Lab
Sequence Models
In this notebook you’ll get to know about the Trax framework and learn about some of its basic building blocks.
Saturday, November 7, 2020
Oren Bochman

Neural Networks for Sentiment Analysis

Sequence Models
NLP
Coursera
Notes
Sequence Models
Sentiment analysis task
we cover Neural networks for deep learning, then build a tweet classifier that places tweets into positive or negative sentiment categories, using a DNN.
Friday, November 6, 2020
Oren Bochman

course banner

Word Embeddings: Hands On

NLP
Coursera
Lab
Probabilistic Models
In previous lecture notebooks we saw all the steps needed to train the CBOW model. This notebook will walk we through how to extract the word embedding vectors from a model.
Wednesday, November 4, 2020
Oren Bochman

course banner

Word Embeddings: Ungraded Practice Notebook

NLP
Coursera
Lab
Probabilistic Models
In this ungraded notebook, you’ll try out all the individual techniques that we learned about in the lecture. Practicing on small examples will prepare we for the graded…
Tuesday, November 3, 2020
Oren Bochman

course banner

Word Embeddings: Training the CBOW model

NLP
Coursera
Lab
Probabilistic Models
In previous lecture notebooks we saw
Monday, November 2, 2020
Oren Bochman

course banner

Word Embeddings: Intro to CBOW model, activation functions and working with Numpy

NLP
Coursera
Lab
Probabilistic Models
In this lecture notebook we will be given an introduction to the continuous bag-of-words model, its activation functions and some considerations when working with Numpy.
Sunday, November 1, 2020
Oren Bochman

course banner

Word Embeddings First Steps: Data Preparation

NLP
Coursera
Lab
Probabilistic Models
In this series of ungraded notebooks, you’ll try out all the individual techniques that we learned about in the lectures. Practicing on small examples will prepare we for…
Friday, October 30, 2020
Oren Bochman

Word embeddings with neural networks

Probabilistic Models
NLP
Coursera
notes
Probabilistic Models
Extract features from text into numerical vectors, then build a binary classifier for tweets using logistic regression!
Thursday, October 29, 2020
Oren Bochman

course banner

Out of vocabulary words (OOV)

Probabilistic Models
NLP
Coursera
Notes
Probabilistic Models
In the video about the out of vocabulary words, we saw that the first step in dealing with the unknown words is to decide which words belong to the vocabulary.
Tuesday, October 27, 2020
Oren Bochman

course banner

Building the language model

Probabilistic Models
NLP
Coursera
Notes
Probabilistic Models
To calculate the n-gram probability, we will need to count frequencies of n-grams and n-gram prefixes in the training dataset. In some of the code assignment exercises, we…
Monday, October 26, 2020
Oren Bochman

course banner

N-grams Corpus preprocessing

Probabilistic Models
NLP
Coursera
Notes
Probabilistic Models
The input corpus in this week’s assignment is a continuous text that needs some preprocessing so that we can start calculating the n-gram probabilities.
Sunday, October 25, 2020
Oren Bochman

course banner

Autocomplete and Language Models

Probabilistic Models
NLP
Coursera
Notes
Probabilistic Models
Extract features from text into numerical vectors, then build a binary classifier for tweets using logistic regression!
Saturday, October 24, 2020
Oren Bochman

Parts-of-Speech Tagging - Working with tags and Numpy

Probabilistic Models
NLP
Coursera
lab
Probabilistic Models
In this lab we will create a matrix using some tag information and then modify it using different approaches. This will serve as hands-on experience working with Numpy
Thursday, October 22, 2020
Oren Bochman

Parts-of-Speech Tagging - First Steps: Working with text files, Creating a Vocabulary and Handling Unknown Words

Probabilistic Models
NLP
Coursera
lab
Probabilistic Models
Extract features from text into numerical vectors, then build a binary classifier for tweets using logistic regression!
Wednesday, October 21, 2020
Oren Bochman

Part of Speech Tagging and Hidden Markov Models

Probabilistic Models
NLP
Coursera
Notes
Probabilistic Models
Extract features from text into numerical vectors, then build a binary classifier for tweets using logistic regression!
Tuesday, October 20, 2020
Oren Bochman

Assignment 1: Auto Correct

Welcome to the first assignment of Course 2. This assignment will give we a chance to brush up on your python and probability skills. In doing so, we will implement an…
Sunday, October 18, 2020
Oren Bochman

course banner

Candidates from String Edits

NLP
Coursera
Lab
Logistic regression
Sentiment analysis task
Classification & Vector Spaces
Estimated Time: 20 minutes
Saturday, October 17, 2020
Oren Bochman

course banner

Building the vocabulary

NLP
Coursera
Lab
Logistic regression
Sentiment analysis task
Classification & Vector Spaces
Estimated Time: 10 minutes
Friday, October 16, 2020
Oren Bochman

course banner

Autocorrect and minimum edit distance

Probabilistic Models
NLP
Coursera
Notes
Probabilistic Models
Auto-correct text with minimum edit distances
Thursday, October 15, 2020
Oren Bochman

Hash functions and multiplanes

NLP
Coursera
Lab
Word Embeddings
Translation task
Search Task
In this lab, we are going to practice the most important concepts related to the hash functions explained in the videos. We will be using these in this week’s assignment.
Tuesday, October 13, 2020
Oren Bochman

Vector manipulation in Python

NLP
Coursera
Lab
Word Embeddings
Translation task
Search Task
In this lab, we will have the opportunity to practice once again with the NumPy library. This time, we will explore some advanced operations with arrays and matrices.
Monday, October 12, 2020
Oren Bochman

Machine Translation and Document Search

Classification & Vector Spaces
NLP
Coursera
notes
Word Embeddings
Translation task
Search Task
Concepts, code snippets, and slide commentaries for this week’s lesson of the Course notes from the deeplearning.ai natural language programming specialization.
Sunday, October 11, 2020
Oren Bochman

Manipulating word embeddings

NLP
Coursera
Lab
Logistic regression
Sentiment analysis task
Classification & Vector Spaces
In this week’s assignment, we are going to use a pre-trained word embedding for finding word analogies and equivalence. This exercise can be used as an Intrinsic Evaluation…
Friday, October 9, 2020
Oren Bochman

Linear algebra in Python with NumPy

NLP
Coursera
Lab
Logistic regression
Sentiment analysis task
Classification & Vector Spaces
In this lab, we will have the opportunity to remember some basic concepts about linear algebra and how to use them in Python.
Thursday, October 8, 2020
Oren Bochman

Vector Space Models

Classification & Vector Spaces
NLP
Coursera
Notes
Classification & Vector Spaces
Vector space models capture semantic meaning and relationships between words. You’ll learn how to create word vectors that capture dependencies between words, then visualize their relationships in two dimensions using PCA.
Wednesday, October 7, 2020
Oren Bochman

Visualizing Naive Bayes

NLP
Coursera
Lab
Logistic regression
Sentiment analysis task
Classification & Vector Spaces
In this lab, we will cover an essential part of data analysis that has not been included in the lecture videos. As we stated in the previous module, data visualization gives…
Tuesday, October 6, 2020
Oren Bochman

Probability and Bayes Rule

Classification & Vector Spaces
NLP
Coursera
Code
Conditional Probability
Bayes rule
Naïve Bayes
Laplace smoothing
Log-likelihood
Classification
Sentiment analysis task
Concepts, code snippets, and slide commentaries for this week’s lesson of the Course notes from the deeplearning.ai natural language programming specialization.
Tuesday, October 6, 2020
Oren Bochman

Probability and Bayes Rule

Classification & Vector Spaces
NLP
Coursera
Classification & Vector Spaces
Notes
Naïve Bayes
Sentiment analysis task
The theory behind Bayes’ rule for conditional probabilities, and its application toward building a Naive Bayes tweet classifier
Monday, October 5, 2020
Oren Bochman

Lab: Visualizing tweets & the Logistic Regression model

NLP with Classification & Vector Spaces
NLP
Coursera
Lab
Logistic regression
Sentiment analysis task
Classification & Vector Spaces
NLTK
Objectives: Visualize and interpret the logistic regression model
Sunday, October 4, 2020
Oren Bochman

Lab: Building and Visualizing word frequencies

NLP with Classification & Vector Spaces
Classification & Vector Spaces
Coursera
Lab
NLP
NLTK
Sentiment analysis task
Word frequencies
In this lab, we will focus on the build_freqs() helper function and visualizing a dataset fed into it. In our goal of tweet sentiment analysis, this function will build a…
Saturday, October 3, 2020
Oren Bochman

Lab: Preprocessing

NLP with Classification & Vector Spaces
NLP
Coursera
Lab
Logistic regression
Sentiment analysis task
Classification & Vector Spaces
NLTK
In this lab, we will be exploring how to preprocess tweets for sentiment analysis. We will provide a function for preprocessing tweets during this week’s assignment, but it…
Friday, October 2, 2020
Oren Bochman

Sentiment Analysis with Logistic Regression

NLP with Classification & Vector Spaces
Classification & Vector Spaces
Coursera
Logistic regression
NLP
Notes
Sentiment analysis task
Extract features from text into numerical vectors, then build a binary classifier for tweets using logistic regression!
Thursday, October 1, 2020
Oren Bochman
No matching items

Citation

BibTeX citation:
@online{bochman2025,
  author = {Bochman, Oren},
  title = {NLP Course Notes and Research Notebooks},
  date = {2025-02-11},
  url = {https://orenbochman.github.io/notes-nlp/},
  langid = {en}
}
For attribution, please cite this work as:
Bochman, Oren. 2025. “NLP Course Notes and Research Notebooks .” February 11, 2025. https://orenbochman.github.io/notes-nlp/.