Feature selection in small-sample high-dimensional datasets enhances classification accuracy and reduces computational time for model training. This paper introduces the filter Classification Error Impurity (CEI) as a frequency-based ranker that improves upon existing methods by better identifying non-linear patterns. According to this, the top features identified by the ensemble of CEI, along with Mutual Information (MI) and Fisher Ratio (FR), form the feature space utilized by the Adaptive Genetic Algorithm with External Repository (AGAwER) to identify the optimal feature combination in a wrapper approach. AGAwER leverages an external repository, incorporating the best solutions to enrich the Genetic Algorithm’s (GA) population, thus promoting diversity and enhancing exploration. As a result, a hybrid method called CMF-AGAwER is proposed, which surpasses existing modern feature selection methods. The implementation and data are accessible on GitHub at https://github.com/KhaosResearch/CMF-AGAwER.
