Feature selection algorithms in machine learning are techniques for choosing the most relevant and useful features for building predictive models.
Many typical applications in machine learning, from customer segmentation to medical diagnosis, arise from complex interactions between all types of variables.
However, analyzing multiple internal and external variables is very complicated. Data scientists might have huge data sets and need innovative methods to select relevant variables.
In this post, we formulate the model selection problem and describe the algorithms most used in practice.
Contents
Neural Designer implements all the model selection algorithms described here. To use these algorithms in practice, you can download the Neural Designer.
Problem formulation
In this way, model selection is applied to find the topology of a neural network that minimizes the error on new data.
Input selection algorithms are responsible for finding the optimal subset of inputs.
We define the selection error as the error of a neural network for new data. It measures the ability of the model to predict the result in a new case.
Input selection is a method to improve the quality of the predictions. It consists of extracting the subset of inputs that influence a particular physical, biological, social, etc. process more.
Growing inputs
The growing inputs method starts by calculating every input’s correlation with every output in the neural network.
The growing inputs method starts with the most correlated input and keeps adding well-correlated variables until the selection error starts increasing.
Pruning inputs
The pruning input method also starts by calculating the correlations among every input and output in the neural network.
The pruning inputs algorithm starts with all the variables of the data set and removes the inputs with little correlation with the outputs.
Genetic algorithm
A different class of inputs selection method is the genetic algorithm.
This stochastic method is based on the mechanics of natural genetics and biological evolution.
The genetic algorithm implemented includes several methods to perform fitness assignment, selection, crossover, and mutation operators.
The following figure shows a simplified flow diagram of the genetic algorithm.
The genetic algorithm starts with a population of different subsets of variables.
In every generation, the fitness of every individual in the population is computed as the selection error for that subset of inputs.
Then, the method evolves the population by selecting some individuals to generate the new population, performing a crossover with the selected population, and mutating the offspring generated during the crossover.
Conclusions
But model selection algorithms are very expensive in computational terms, so a big drawback here is the performance.
Neural Designer includes an advanced model selection framework capable of representing very complex data sets.
This system procures high added value to data scientists, providing them with results in a way previously unachievable.
