Asked about How the algorithms actually work
Why is RL less stable than supervised learning?
An answer that works
The data distribution depends on the current policy so nothing is stationary, rewards are sparse and delayed making credit assignment ambiguous, exploration is unsolved, and variance is high enough that results need many seeds.
Then they push further
Put a number on the seed variance.
On the cart-pole run in this topic, identical code at three seeds finished at 293, 500 and 14 steps per episode. The third had reached the 500-step cap earlier in training and collapsed. Any one of those could have been reported as the result of the method.
And on sample efficiency?
On a thirteen-state gridworld, value iteration with the transition probabilities takes 1,872 backups; Q-learning from samples needs around 54,000 environment steps to get within one and a half per cent of the same policy. Roughly thirty times the work on a problem small enough to print, and the gap grows with the state space.
What gets marked up
Strong answers give numbers rather than adjectives, and treat a single learning curve as not being evidence.
Reading an answer is not being able to give one. The section that teaches this is How the algorithms actually work, 4 hours of Reinforcement learning.
Learn this properlyAlso asked about Reinforcement learning