Lesson 1: Feature Construction for Linear Methods

Learning Objectives

Define the difference between coarse coding and tabular representations #
Explain the trade-off when designing representations between discrimination and generalization #
Understand how different coarse coding schemes affect the functions that can be represented #
Explain how tile coding is a (computationally?) convenient case of coarse coding #
Describe how designing the tilings affects the resultant representation #
Understand that tile coding is a computationally efficient implementation of coarse coding #

Lesson 2: Neural Networks

Learning Objectives

Define a neural network #
Define activation functions #
Define a feed-forward architecture #
Understand how neural networks are doing feature construction #
Understand how neural networks are a non-linear function of state #
Understand how deep networks are a composition of layers #
Understand the tradeoff between learning capacity and challenges presented by deeper networks #

Lesson 3: Training Neural Networks

Learning Objectives

Compute the gradient for a single hidden layer neural network #
Understand how to compute the gradient for arbitrarily deep networks #
Understand the importance of initialization for neural networks #
Describe strategies for initializing neural networks #
Describe optimization techniques for training neural networks #

Discussion prompt

What properties of the representation are important for our online setting? This contrasts the offline, batch setting.

References

Sutton, R. S., and A. G. Barto. 2018. Reinforcement Learning, Second Edition: An Introduction. Adaptive Computation and Machine Learning Series. MIT Press. http://incompleteideas.net/book/RLbook2020.pdf.

Reuse

CC SA BY-NC-ND

Citation

BibTeX citation:

@online{bochman2024,
  author = {Bochman, Oren},
  title = {Constructing {Features} for {Prediction}},
  date = {2024-04-02},
  url = {https://orenbochman.github.io/notes/RL/c3-w2.html},
  langid = {en}
}

For attribution, please cite this work as:

Bochman, Oren. 2024. “Constructing Features for Prediction.” April 2, 2024. https://orenbochman.github.io/notes/RL/c3-w2.html.