Asked about Sequence models before transformers
What problems did recurrent networks have?
An answer that works
Long-range dependencies degraded, because information had to survive repeated compression into a fixed-size hidden state and gradients vanished across many steps. And they could not be parallelised across a sequence, since each step depended on the previous one, which capped training scale.
Reading an answer is not being able to give one. The section that teaches this is Sequence models before transformers, 5 hours of Sequence models before transformers.
Learn this properlyAlso asked about Sequence models before transformers