Asked about The simplest case: bandits and exploration
What is the exploration-exploitation trade-off?
An answer that works
Choosing between the action that looks best so far and trying others that might be better. Exploit too early and you may lock in a worse option; explore too much and you waste reward on options you already know are poor.
Then they push further
How does Thompson sampling handle it?
It keeps a probability distribution over each action's value and acts on a random draw from each. Uncertain actions sometimes draw high and get tried; as evidence accumulates, the distributions narrow and it naturally exploits.
Where would you use a bandit instead of an A/B test?
When the cost of showing the worse option is high and you care about reward during the experiment, such as choosing among many headlines. An A/B test gives cleaner estimates of each option; a bandit gives more reward while learning.
What gets marked up
Marked up for describing a concrete algorithm and knowing when a bandit beats a fixed split test.
Reading an answer is not being able to give one. The section that teaches this is The simplest case: bandits and exploration, 4 hours of Reinforcement learning.
Learn this properlyAlso asked about Reinforcement learning