Asked about Autograd, custom pieces and performance
Why might a tensor's gradient be None after backward?
An answer that works
The operation that produced it was not recorded in the autograd graph, usually because it was detached, created from a Python number, or computed inside no_grad. Autograd can only backpropagate through what it recorded.
Reading an answer is not being able to give one. The section that teaches this is Autograd, custom pieces and performance, 5 hours of PyTorch.
Learn this properlyAlso asked about PyTorch
What does model.eval() do?Why wrap validation in torch.no_grad()?Your GPU utilisation is 20 percent during training. What is wrong?How would you speed up a training run?A regression model's loss decreases slowly but predictions are nonsense. What would you check first?Walk me through a minimal GPT-style model.Your model's training loss is flat from the first step. How do you debug it?What do you need to save to resume a training run exactly?