Intro and modular design

Replay Memory: Add and use experiences, Benefits

DQNNet: Online, Target and Architecture

DQNAgent: How to act, exploration vs exploitation, Epsilon-greedy

DQNAgent: How to learn, Q-value recap, updating the online and target networks

Walk through of the algorithm presented in the paper

See ya

Intro

Deep Q Network Architecture

Why Deep Q Learning?

Conclusion

Summary

Creating the Model

Dropout layer

Updating the Target Model

Hyperparameters

Why use one log file?

DQN wins at Breakout

Hello there

Say hello to Garfield

Garfield Gridworld and the need for DQN

But we don’t have a labelled dataset here…

What is a Replay Memory?

What do Q-values represent: Intuition

What do Q-values represent: Formalisation

Wait, Q-values are recursive

NN estimates Q-value

We finally have labels…

Erm, but the targets are estimates… (Bootstrapping)

DQN Paper: 2013 vs 2015 (Introducing the Target Network)

Updated loss function

Don’t forget to explore during training

Let’s summarise this information overload

Don’t forget to check out the blog posts

Seeeeee ya

Continuous Domain

Learning Representation

Motivation for Deep Reinforcement Learning

How does it work?

Learning the Q-Network

Replay Buffer and Shuffling

Buffer Size

Sample Selection

Topology-guided sampling

Conclusion

What Is Reinforced Algorithm

What Is off Policy

The Mountain Car

Dq N Is a Shallow Neural Network

Constants

The Training Loop

Bellman Equation

Experience Replay

Results

Introduction

Recap: Reinforcement Learning

Recap: Markov Decision Process

Tabular Q-Learning to Deep Q-Learning

Neural Network Process for DQN

Intuition of Q-learning

Q-Learning as a form of Temporal Difference Learning

How to generate experience: Epsilon-greedy actions

How to learn from experience: Replay buffer

Q-Value

Learning Q-Values

Policy & Target Network

Visualizing Q-Values

Choosing the Optimal Action

Improving DQN

Jupyter Notebook on Cart Pole

High-level overview of the paper

Experience replay buffer

Difficulties with RL (correlations, non-stationary distributions)

DQN is very general

MDP formalism and optimal Q function

Function approximators

The loss function explained

The deadly triad

Algorithm walk-through

Preprocessing and architecture details

Additional details - normalizing score, schedule, etc.

Agent training metrics

Results

Deep Q-Network Visual Example

Advantage Function

Deep Q-Network implementation

Deep Q-Network architecture

Deep Q-Network training

Deep Q-Network Results

Intro, my other deep learning projects

Patreon

Tensorboard walk-through

Code walk-through - understand DQN arguments

Main loop (collecting experience and learning from it)

Main actor-learner class

Visualizations (matplolib, tensorboard)

Analyzing other people's projects