Asked about Learning by acting
What is the credit assignment problem?
An answer that works
A reward arrives after a long sequence of actions and it is unclear which actions caused it. It is the central difficulty in reinforcement learning and why it needs far more experience than supervised learning.
Then they push further
How do the two families of method attack it?
Value methods propagate reward backwards one step at a time through the Bellman recursion, so information about the goal reaches earlier states gradually. Policy methods credit every action in an episode with the return that followed it and rely on averaging over many episodes for the coincidences to cancel.
Where is it hardest?
When the reward is a single number at the end of a long sequence, which is exactly the language model case: one score for a response of a thousand tokens. That is why post-training leans so heavily on variance reduction and on limiting how far each update may move the policy.
What gets marked up
Passable answers define it. Strong ones say what each family does about it and name the sparse end-of-episode reward as the hard case.
Reading an answer is not being able to give one. The section that teaches this is Learning by acting, 4 hours of Reinforcement learning.
Learn this properlyAlso asked about Reinforcement learning