Asked about PyTorch in practice
What does model.eval() do?
An answer that works
Switches layers that behave differently at inference, notably dropout and batch normalisation, into evaluation mode. Forgetting it makes validation numbers noisy and pessimistic, and forgetting to switch back disables dropout for the rest of training.
Reading an answer is not being able to give one. The section that teaches this is PyTorch in practice, 6 hours of PyTorch.
Learn this properlyAlso asked about PyTorch
Why wrap validation in torch.no_grad()?Your GPU utilisation is 20 percent during training. What is wrong?Why might a tensor's gradient be None after backward?How would you speed up a training run?A regression model's loss decreases slowly but predictions are nonsense. What would you check first?Walk me through a minimal GPT-style model.Your model's training loss is flat from the first step. How do you debug it?What do you need to save to resume a training run exactly?