XGBoost: How Deep Learning Can Replace Gradient Boosting and Decision Trees — Part 2: Training | by Saupin Guillaume | Sep, 2023
Photo by Simon Wilkes on Unsplash In a previous article: you have learned about rewriting decision trees using a Differentiable Programming approach, as suggested by the NODE paper. The idea of this paper is to replace XGBoost by a Neural Network. More specifically, after explaining why the process of building Decision Trees is not differentiable, it introduced the necessary mathematical tools to regularize the two main elements associated with a decision node: Feature Selection Branch detection The NODE paper shows that both can be handled using the entmax function. To summarize, we have show...
Source: huewire.com