Asked about What a neural network is
Why is ReLU the default rather than sigmoid?
An answer that works
Sigmoid saturates at both ends, so gradients approach zero and early layers stop learning. ReLU does not saturate for positive inputs, keeping gradients alive, and it is cheaper to compute.
Reading an answer is not being able to give one. The section that teaches this is What a neural network is, 5 hours of Neural networks.
Learn this properlyAlso asked about Neural networks
Why do neural networks need activation functions?What is the dying ReLU problem?Which loss would you use for multi-class classification?Why do residual connections help deep networks?Why do transformers use layer norm rather than batch norm?Walk me through backpropagation for a two-layer network.Your validation loss starts rising while training loss keeps falling. What is happening and what do you do?