A Comparison of Feature Selection Using SHAP-Values for Tree and Non-tree-Based Models

Federico Santavicca; Harald Rietdijk; Patricia Conde-Cespedes; Maria Trocan

doi:10.1007/978-981-92-0068-9_22

A Comparison of Feature Selection Using SHAP-Values for Tree and Non-tree-Based Models

Vertaalde titel van de bijdrage: Een vergelijking van kenmerkselectie met behulp van SHAP-waarden voor boomgebaseerde en niet-boomgebaseerde modellen

Federico Santavicca
, Harald Rietdijk
, Patricia Conde-Cespedes
, Maria Trocan

Digitale Transformatie

University of Rome La Sapienza
Isep

Onderzoeksoutput: Paper › Academic

Samenvatting

In de gezondheidszorg zijn datasets van klinische onderzoeken vaak hoogdimensionaal en hebben ze een kleine steekproefomvang, wat het risico op overfitting vergroot. Bovendien richt onderzoek in deze context zich doorgaans niet alleen op de nauwkeurigheid van de resulterende modellen, maar ook op de verklaarbaarheid ervan. Kenmerkselectie kan overfitting tegengaan en waardevolle inzichten bieden in relevante kenmerken. Het gebruik van SHAP-waarden (SHapley Additive exPlanation) is een nuttig hulpmiddel gebleken om significante kenmerken te identificeren. In dit artikel stellen we voor om Kernel SHAP te implementeren in de shap-select-module om de functionaliteit ervan uit te breiden naar niet-boomgebaseerde modellen. Dit gebeurt door de introductie van een masker dat achtergrondwaarden genereert die SHAP gebruikt om het verwijderen van een kenmerk te simuleren. De prestaties worden vervolgens geëvalueerd op een kleine, hoogdimensionale dataset, en de resulterende kenmerkselectie wordt beoordeeld. De resultaten tonen aan dat, ondanks een grotere instabiliteit in de prestatiemaatstaven, het vermogen van de methode om significante kenmerken te identificeren veelbelovend blijft. Deze bevindingen kunnen dienen als uitgangspunt voor verder onderzoek naar de relatie tussen kenmerkkenmerken en methoden voor kenmerkselectie, evenals voor het verbeteren van het vermogen om relevante kenmerken te identificeren met behulp van SHAP-waarden voor kenmerkselectie.

Vertaalde titel van de bijdrage	Een vergelijking van kenmerkselectie met behulp van SHAP-waarden voor boomgebaseerde en niet-boomgebaseerde modellen
Originele taal-2	English
Pagina's	324-337
Aantal pagina's	14
DOI's	https://doi.org/10.1007/978-981-92-0068-9_22
Status	E-pub ahead of print - 1 jun. 2026
Evenement	18th Asian Conference on Intelligent Information and Database Systems - Howard Plaza Hotel Kaohsiung, Kaohsiung, Taiwan, Province of China Duur: 13 apr. 2026 → 15 apr. 2026 https://aciids.pwr.edu.pl/2026/

Conference

Conference	18th Asian Conference on Intelligent Information and Database Systems
Verkorte titel	ACIID 2026
Land/Regio	Taiwan, Province of China
Stad	Kaohsiung
Periode	13/04/26 → 15/04/26
Internet adres	https://aciids.pwr.edu.pl/2026/

Keywords

boommodellen
machinaal leren
uitlegbaarheid
Shapley-waarden
SHAP-waarden
featureselectie

Research Focus Areas Hanze University of Applied Sciences

Ondernemerschap
Energie
Healthy Ageing

Research Focus Areas Research Centre or Centre of Expertise

Digitale Transformatie

Publinova thema's

Taal, Cultuur & Kunsten
Economie en Management
Recht
ICT & Media
Gezondheid
Mens en Maatschappij
Techniek

Toegang tot document

10.1007/978-981-92-0068-9_22

Citeer dit

Santavicca, F., Rietdijk, H., Conde-Cespedes, P., & Trocan, M. (2026). A Comparison of Feature Selection Using SHAP-Values for Tree and Non-tree-Based Models. 324-337. Paper gepresenteerd op 18th Asian Conference on Intelligent Information and Database Systems, Kaohsiung, Taiwan, Province of China. Online publicatie vooraf. https://doi.org/10.1007/978-981-92-0068-9_22

@conference{318c5a9b5e664c919690b9307a8634dd,

title = "A Comparison of Feature Selection Using SHAP-Values for Tree and Non-tree-Based Models",

abstract = "In healthcare, clinical trial datasets are often high-dimensional and have small sample sizes, increasing the risk of overfitting. Furthermore, research in this context usually focuses not only on the accuracy of the resulting models but also on their explainability. Feature selection can mitigate overfitting and provide valuable insights into relevant features. Using SHAP (SHapley Additive exPlanation) values has proved to be a useful tool to identify significant features. In this paper, we propose implementing Kernel SHAP in the shap-select module to extend its functionality to non-tree-based models. This is done by introducing a masker that generates background values that SHAP uses to simulate the removal of a feature. The performance is then evaluated on a small high-dimensional dataset, and the resulting feature selection is evaluated. The results show that, despite higher instability in the performance metrics, the method{\textquoteright}s capacity to identify significant features remains promising. These findings can serve as a starting point for further research into the relationship between feature characteristics and feature selection methods, as well as for improving the capacity to identify relevant features using SHAP values for feature selection.",

keywords = "boommodellen, machinaal leren, uitlegbaarheid, Shapley-waarden, SHAP-waarden, featureselectie, Tree models, Machine learning, Explainability, Shapley-values, SHAP-values, Feature selection",

author = "Federico Santavicca and Harald Rietdijk and Patricia Conde-Cespedes and Maria Trocan",

year = "2026",

month = jun,

day = "1",

doi = "10.1007/978-981-92-0068-9\_22",

language = "English",

pages = "324--337",

note = "18th Asian Conference on Intelligent Information and Database Systems ; Conference date: 13-04-2026 Through 15-04-2026",

url = "https://aciids.pwr.edu.pl/2026/",

}

TY - CONF

T1 - A Comparison of Feature Selection Using SHAP-Values for Tree and Non-tree-Based Models

AU - Santavicca, Federico

AU - Rietdijk, Harald

AU - Conde-Cespedes, Patricia

AU - Trocan, Maria

PY - 2026/6/1

Y1 - 2026/6/1

N2 - In healthcare, clinical trial datasets are often high-dimensional and have small sample sizes, increasing the risk of overfitting. Furthermore, research in this context usually focuses not only on the accuracy of the resulting models but also on their explainability. Feature selection can mitigate overfitting and provide valuable insights into relevant features. Using SHAP (SHapley Additive exPlanation) values has proved to be a useful tool to identify significant features. In this paper, we propose implementing Kernel SHAP in the shap-select module to extend its functionality to non-tree-based models. This is done by introducing a masker that generates background values that SHAP uses to simulate the removal of a feature. The performance is then evaluated on a small high-dimensional dataset, and the resulting feature selection is evaluated. The results show that, despite higher instability in the performance metrics, the method’s capacity to identify significant features remains promising. These findings can serve as a starting point for further research into the relationship between feature characteristics and feature selection methods, as well as for improving the capacity to identify relevant features using SHAP values for feature selection.

AB - In healthcare, clinical trial datasets are often high-dimensional and have small sample sizes, increasing the risk of overfitting. Furthermore, research in this context usually focuses not only on the accuracy of the resulting models but also on their explainability. Feature selection can mitigate overfitting and provide valuable insights into relevant features. Using SHAP (SHapley Additive exPlanation) values has proved to be a useful tool to identify significant features. In this paper, we propose implementing Kernel SHAP in the shap-select module to extend its functionality to non-tree-based models. This is done by introducing a masker that generates background values that SHAP uses to simulate the removal of a feature. The performance is then evaluated on a small high-dimensional dataset, and the resulting feature selection is evaluated. The results show that, despite higher instability in the performance metrics, the method’s capacity to identify significant features remains promising. These findings can serve as a starting point for further research into the relationship between feature characteristics and feature selection methods, as well as for improving the capacity to identify relevant features using SHAP values for feature selection.

KW - boommodellen

KW - machinaal leren

KW - uitlegbaarheid

KW - Shapley-waarden

KW - SHAP-waarden

KW - featureselectie

KW - Tree models

KW - Machine learning

KW - Explainability

KW - Shapley-values

KW - SHAP-values

KW - Feature selection

U2 - 10.1007/978-981-92-0068-9_22

DO - 10.1007/978-981-92-0068-9_22

M3 - Paper

SP - 324

EP - 337

T2 - 18th Asian Conference on Intelligent Information and Database Systems

Y2 - 13 April 2026 through 15 April 2026

ER -