•  
  •  
 

Corresponding Author ORCID Identifer

Choo-Yee Ting https://orcid.org/0000-0001-5667-2816
Thanh-Trung Nguyen https://orcid.org/0009-0000-9948-1753

Section

Regular Issue

Abstract

Feature Section (FS) is a fundamental step in developing accurate, robust, and interpretable Machine Learning (ML) models, especially when working with high-dimensional structured datasets commonly found in healthcare, finance, and other dataintensive domains. Selecting the most relevant subset of features not only improves predictive performance but also enhances computational efficiency and model transparency. This paper presents a comprehensive and systematic study of widely adopted feature selection strategies, including filter, wrapper, embedded, model-agnostic methods, and dimensionality reduction techniques. Each category is examined through experimental evaluations across multiple classifiers and benchmark datasets to understand how different feature selection mechanisms affect model performance, interpretability, and stability. The results show that while filter methods, such as mutual information and chi-square, are computationally efficient and suitable for large-scale screening, wrapper and embedded methods achieve higher accuracy at the cost of increased computational demand. The model-agnostic approach, represented by Permutation Feature Importance (PFI), was employed to assess the relative contribution of each feature to model performance, thereby providing an interpretable and unbiased measure of feature relevance. Dimensionality reduction techniques, such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), further assist in managing multicollinearity, improving class separability, and enhancing model generalization by transforming features into more discriminative lowerdimensional spaces. Moreover, the study captures the trade-offs between accuracy, speed, and explainability, offering practical recommendations for researchers and practitioners to select suitable methods based on their specific context and computational resources. The findings underscore the importance of adaptive, context-aware feature selection pipelines, particularly in healthcare analytics, where interpretable and trustworthy models can improve diagnostic confidence and support effective decision-making.

Download

Share

COinS