Sparse and robust feature selection in SVM classification through ℓ_p-quasi-norms

While numerous support vector machine (SVM) approaches have been proposed to improve robustness, interpretability (understood here in terms of feature-level sparsity and stability), or scalability in high-dimensional settings, few frameworks successfully integrate all three aspects within a unified and computationally efficient formulation. This article addresses this gap by introducing a novel framework for robust and interpretable classification in high-dimensional spaces. Leveraging the sparsity-inducing properties of ℓ_p-quasi-norms (0<p<1), we develop new SVM-based models that incorporate ℓ_p-regularization directly into the optimization problem, thereby embedding feature selection and robustness within a single mathematical formulation. These models are inspired by the Minimum Error Minimax Probability Machine (MEMPM) and the Cobb–Douglas Learning Machine (CD-LeMa), but introduce key innovations to effectively handle nonconvex sparsity penalties and improve numerical tractability. To efficiently solve the resulting nonsmooth and nonconvex problems, we propose a Diagonal Two-Step Algorithm that alternates between reweighted convex subproblems and auxiliary-parameter updates. The theoretical analysis establishes a monotonic decrease of the regularized objective values, boundedness of the generated sequence, and KKT stationarity of its accumulation points for the regularized approximation problem. The algorithm exhibits polynomial per-iteration complexity in the number of features. Extensive experiments on biomedical benchmark datasets systematically evaluate the effect of the norm parameter p, revealing clear trade-offs between sparsity, robustness, and balanced accuracy. A complementary statistical study based on Friedman and Nemenyi tests further supports the significance of the observed performance differences among models. Compared with recent robust SVM frameworks such as RoBoSS-SVM and Wave-SVM, the proposed approach achieves a favorable empirical balance between classification accuracy and stability under feature perturbations, while maintaining efficient training times. Overall, the ℓ_p-regularized models provide a scalable and interpretable solution for high-dimensional data. Model interpretability is enhanced not only through sparsity, which yields compact and easily examinable weight vectors, but also through improved stability of the learned classifier with respect to feature perturbations, thereby increasing the reliability of feature relevance assessments.