rl-notes

Published

Saturday, January 18, 2025

Training Classifiers with Natural Language Explanations

Paper Review
Paper
Review
NLP
Classification
Named Entity Recognition
A review of the paper ‘Training Classifiers with Natural Language Explanations’ by Hancock et al.
Tuesday, January 28, 2025

Off-Policy Learning-to-Bid with AuctionGym

Paper Review
Paper
Review
Advertising
Additional work with AuctionGym
Saturday, January 18, 2025

Learning to Bid with AuctionGym

Paper Review
Paper
Review
Advertising
Introducing (AuctionGym)
Saturday, January 18, 2025

Dynamic collaborative filtering Thompson Sampling for cross-domain advertisements recommendation

Paper Review
Paper
Review
Bandit
Advertising
Collaborative Filtering
Introducing (DCTS) The Dynamic Collaborative Filtering Thompson Sampling algorithm for cross-domain advertisements recommendation.
Saturday, January 18, 2025

On Learning To Become a Successful Loser: A Comparison of Alternative Abstractions of Learning Processes in the Loss Domain

Paper Review
Reinforcement Learning
Paper
Review
A comparison of five mathematical models that predict student behavior in repeated decision-making tasks involving gains and losses.
Thursday, January 2, 2025

Temporal Abstraction in Reinforcement Learning with the Successor Representation

Paper Review
Reinforcement Learning
Paper
Review
Temporal Abstraction
Options
This paper review is an extended introduction to temporal abstraction using options. It covers lots of advanced concepts in reinforcement learning that were introduced in Doina’s Precup’s talk in the Coursera Specialization on Reinforcement Learning by Martha White and Adam White. The paper is a deep dive into the topic of options and the successor representation. It is a long paper with lots of advanced concepts and algorithms. The paper is a great resource for anyone interested in reinforcement learning and temporal abstraction.
Sunday, November 10, 2024

Emergent Communication of Generalizations

paper review
signaling games
stub
I think this is an amazing paper. I read it critically and made copious notes to see what I could learn from it. The paper point out some limitations of Lewis referential…
Wednesday, October 9, 2024

The Evolution of Language

Paper Review
Paper
Review
Signaling Games
Emergent Languages
Evolutionary game theory
In this paper the authors model the evolutionof Languages through its adaptation to resist errors in communication.
Thursday, April 18, 2024

Policy Gradient

Prediction and Control with Function Approximation
Coursera
Prediction and Control with Function Approximation
Reinforcement Learning
Sometimes, the behavior codified in the policy is much simpler than the action value function. Thus, learning the policy directly can be more efficient. Learning policies is…
Thursday, April 4, 2024
Oren Bochman

Control with Approximation

Prediction and Control with Function Approximation
Coursera
Prediction and Control with Function Approximation
Reinforcement Learning
In this video, Adam White, discusses the algorithm for “Episodic SARSA with function approximation”. He explains how it can be used to solve reinforcement learning problems…
Wednesday, April 3, 2024
Oren Bochman

Constructing Features for Prediction

Prediction and Control with Function Approximation
Coursera
Prediction and Control with Function Approximation
Reinforcement Learning
We discussed methods for representing large, an possibly continuous state spaces. Ways to construct features. A representation is an agent’s internal encoding of the state…
Tuesday, April 2, 2024
Oren Bochman

Constructing Features for Prediction

Prediction and Control with Function Approximation
Coursera
Prediction and Control with Function Approximation
Reinforcement Learning
This is not a video lecture or notes for a learning goal. This is however my attempts to cover some material from the readings from chapter 9 of [@sutton2018reinforcement] me…
Tuesday, April 2, 2024
Oren Bochman

On-Policy Prediction with Approximation

Prediction and Control with Function Approximation
Coursera
Prediction and Control with Function Approximation
Reinforcement Learning
I did not find the derivation of the SGD alg particularly enlightening and I have seen it several times. However the online setting is the best motivation for the use of SGD…
Monday, April 1, 2024
Oren Bochman

On-Policy Prediction with Approximation

Prediction and Control with Function Approximation
Coursera
Prediction and Control with Function Approximation
Reinforcement Learning
Some of the notes I made in this course became a bit too long. Rather than break the flow of the lesson I decided to move them to a separate file. This is one of those notes.
Monday, April 1, 2024
Oren Bochman

Sample-based Learning Methods

Planning, Learning & Acting
Coursera
Sample-based Learning Methods
Reinforcement Learning
In these module we define cover model based RL sampling. We start with the Dyna architecture. Then we consider tabular Q-planning algorithm, the Tabular Dyna-Q and Dyna-Q+ algorithms
Monday, March 4, 2024
Oren Bochman

Temporal Difference Learning Methods for Control

Sample-based Learning Methods
Coursera
Sample-based Learning Methods
Reinforcement Learning
This week, we will learn to using TD learning for control, as a generalized policy iteration strategy. We will see three different algorithms based on bootstrapping and Bellman equations for control: Sarsa, Q-learning and Expected Sarsa. We will see some of the differences between the methods for on-policy and off-policy control, and that Expected Sarsa is a unified algorithm for both.
Sunday, March 3, 2024

Temporal Difference Learning Methods for Prediction

Sample-based Learning Methods
Coursera
Sample-based Learning Methods
Reinforcement Learning
In these unit we define some key terms like rewards, states, action, value functions, action values functions. Then we consider at the the multi-armed bandit problem leading to exploration explotation dillema, the epsilon greedy algorithm.
Saturday, March 2, 2024

Monte-Carlo Methods for Prediction & Control

Sample-based Learning Methods
Coursera
Sample-based Learning Methods
Reinforcement Learning
In this module we learn about Sample based MC methods that allow learning from sampled episodes. We revise our initial algorithm to better handle exploration. In off policy RL we see methods to learn a policy using samples from another policy, corrected using importance sampleing.
Friday, March 1, 2024

Dynamic Programming

RL Fundamentals
Coursera
Fundamentals
Reinforcement Learning
In week 4 we learn how to compute value functions and optimal policies, assuming you have the MDP model. You will implement dynamic programming to compute value functions and optimal policies and understand the utility of dynamic programming.
Thursday, May 5, 2022
Oren Bochman

Value Functions & Bellman Equations

RL Fundamentals
Coursera
Fundamentals
Reinforcement Learning
In week 3 we learn about Value Functions and Bellman Equations, which are the key technology behind all the algorithms we will learn. We learn the definition of policies and value functions, as well as Bellman equations.
Wednesday, May 4, 2022
Oren Bochman

Markov Decision Processes

RL Fundamentals
Coursera
Fundamentals
Reinforcement Learning
In week 2 we learn about Markov Decision Processes (MDP) and how to compute value functions and optimal policies, assuming you have the MDP model. We implement dynamic programming to compute value functions and optimal policies.
Tuesday, May 3, 2022
Oren Bochman

The K-Armed Bandit Problem

RL Fundamentals
Coursera
Fundamentals
Reinforcement Learning
In week 1 we define some key concepts like rewards, states, action, value functions, action values functions. We consider the the multi-armed bandit problem, leading to exploration explotation dillema, and the epsilon greedy algorithm.
Monday, May 2, 2022
Oren Bochman

Course Introduction

RL Fundamentals
Coursera
Fundamentals
Reinforcement Learning
In week 1 we define some key concepts like rewards, states, action, value functions, action values functions. We consider the the multi-armed bandit problem, leading to exploration explotation dillema, and the epsilon greedy algorithm.
Sunday, May 1, 2022
Oren Bochman
No matching items