Sparse feature selection via ℓ_p-quasi-norm second-order cone programming

Most feature selection methods achieve either sparsity or distributional robustness with respect to uncertainty in data moments, but rarely both, often resulting in models that retain redundant or noisy features. We propose a novel second-order cone programming (SOCP)-based classification model that integrates the nonconvex ℓ_p quasi-norm (0<p<1) with second-order cone constraints on the first and second moments of the data, yielding classifiers that are simultaneously sparse and robust in the minimax moment-based sense. A key theoretical contribution of this work is the derivation of explicit positive lower bounds on the magnitude of every nonzero component of any local minimizer, which depend on the regularization parameter, the quasi-norm exponent, and moment-based quantities associated with class separation and within-class variability, providing an analytical sparsity guarantee for ℓ_p-SOCP models. We also propose an iteratively reweighted ℓ₁-algorithm tailored to this structure and establish rigorous convergence results, including monotonic descent, boundedness of iterates, and convergence to first-order stationary points of the original nonconvex problem. Extensive experiments on ten benchmark datasets, including two UCI collections, two image-based datasets, and six high-dimensional microarrays, show that the proposed method achieves competitive balanced accuracy while selecting only 0.1%–1% of the features, which, in datasets with several thousand variables corresponds to only a few tens of selected genes. (e.g., 8.2 features on average for both Colorectal and Lymphoma). Friedman–Holm statistical tests further confirm that the proposed method attains the best average ranking and is statistically comparable to the top-performing models. These finding indicate that the ℓ_p-SOCP model yields compact, stable, and accurate classifiers with theoretical guarantees of sparsity and convergence, making it a promising tool for feature selection in high-dimensional pattern recognition.