GRANDE: Gradient-Based Decision Tree Ensembles for tabular data


Marton, Sascha ; Lüdtke, Stefan ; Bartelt, Christian ; Stuckenschmidt, Heiner



URL: https://iclr.cc/virtual/2024/poster/18457
Additional URL: https://openreview.net/forum?id=XEFWBxi075
Document Type: Conference or workshop publication
Year of publication: 2024
Book title: International Conference on Learning Representations
Page range: 1-27
Conference title: TheTwelfth International Conference on Learning Representations, ICLR 2024
Location of the conference venue: Wien, Austria
Date of the conference: 07.-11.05.2024
Publishing house: OpenReview.net
Related URLs:
Publication language: English
Institution: School of Business Informatics and Mathematics > Practical Computer Science II: Artificial Intelligence (Stuckenschmidt 2009-)
Außerfakultäre Einrichtungen > Institut für Enterprise Systems (InES)
Subject: 004 Computer science, internet
Keywords (English): tabular data , architectures . ensembles , decision trees , gradient descent
Abstract: Despite the success of deep learning for text and image data, tree-based ensemble models are still state-of-the-art for machine learning with heterogeneous tabular data. However, there is a significant need for tabular-specific gradient-based methods due to their high flexibility. In this paper, we propose GRANDE, GRAdieNt-Based Decision Tree Ensembles, a novel approach for learning hard, axis-aligned decision tree ensembles using end-to-end gradient descent. GRANDE is based on a dense representation of tree ensembles, which affords to use backpropagation with a straight-through operator to jointly optimize all model parameters. Our method combines axis-aligned splits, which is a useful inductive bias for tabular data, with the flexibility of gradient-based optimization. Furthermore, we introduce an advanced instance-wise weighting that facilitates learning representations for both, simple and complex relations, within a single model. We conducted an extensive evaluation on a predefined benchmark with 19 classification datasets and demonstrate that our method outperforms existing gradient-boosting and deep learning frameworks on most datasets. The method is available under: https://github.com/s-marton/GRANDE




Dieser Eintrag ist Teil der Universitätsbibliographie.




Metadata export


Citation


+ Search Authors in

+ Page Views

Hits per month over past year

Detailed information



You have found an error? Please let us know about your desired correction here: E-Mail


Actions (login required)

Show item Show item