Asked about PyTorch in practice
Why wrap validation in torch.no_grad()?
An answer that works
It stops the framework recording operations for gradient computation, which is unnecessary during evaluation. It saves memory and time, and without it a large validation pass can run out of memory.
Reading an answer is not being able to give one. The section that teaches this is PyTorch in practice, 6 hours of PyTorch.
Learn this properlyAlso asked about PyTorch
What does model.eval() do?Your GPU utilisation is 20 percent during training. What is wrong?Why might a tensor's gradient be None after backward?How would you speed up a training run?A regression model's loss decreases slowly but predictions are nonsense. What would you check first?Walk me through a minimal GPT-style model.Your model's training loss is flat from the first step. How do you debug it?What do you need to save to resume a training run exactly?