Asked about How the main algorithms actually work
Why does gradient boosting work?
An answer that works
Each tree is fitted to the gradient of the loss with respect to the current ensemble's predictions, so adding it moves the ensemble towards lower loss. It is gradient descent over functions, which is why a small learning rate with more trees generally wins.
Reading an answer is not being able to give one. The section that teaches this is How the main algorithms actually work, 5 hours of Supervised learning.
Learn this properlyAlso asked about Supervised learning
You have a tabular dataset and a classification task. What do you try first?Why do gradient boosted trees still beat neural networks on tabular data?What is the difference between bagging and boosting?How does a decision tree choose where to split?Can you trust linear regression coefficients as feature importance?Walk me through how you would approach a new tabular classification problem.Explain the bias-variance trade-off.Your dataset has 20% missing values in an important column. What do you do?
All 12 questions on Supervised learning