Asked about What a neural network is
Why do neural networks need activation functions?
An answer that works
Without them, stacked linear layers collapse into a single linear transformation, since a matrix multiply of matrix multiplies is one matrix multiply. The non-linearity is what makes depth add expressive power.
Reading an answer is not being able to give one. The section that teaches this is What a neural network is, 5 hours of Neural networks.
Learn this properlyAlso asked about Neural networks
Why is ReLU the default rather than sigmoid?What is the dying ReLU problem?Which loss would you use for multi-class classification?Why do residual connections help deep networks?Why do transformers use layer norm rather than batch norm?Walk me through backpropagation for a two-layer network.Your validation loss starts rising while training loss keeps falling. What is happening and what do you do?