Contents<\/h3>\n\n- Application type<\/a>.<\/li>\n
- Data set<\/a>.<\/li>\n
- Neural network<\/a>.<\/li>\n
- Training strategy<\/a>.<\/li>\n
- Model selection<\/a>.<\/li>\n
- Testing analysis<\/a>.<\/li>\n
- Model deployment<\/a>.<\/li>\n<\/ol>\n
This example is solved with Neural Designer<\/a>. To follow it step by step, you can use the free trial<\/a>.<\/p>\n<\/section>\n\n1. Application type<\/h2>\n
We will predict between two types of raisins, so the variable to be predicted is binary (Kecimen or Besni). Therefore, this is a classification<\/a> project.<\/p>\nThe goal is to model the type of raisin based on its features for subsequent use.<\/p>\n<\/section>\n\n2. Data set<\/h2>\n
The first step is to prepare the data set<\/a>, which is the source of information for the problem. We need to configure three things here:<\/p>\n\n- Data source.<\/li>\n
- Variables.<\/li>\n
- Instances.<\/li>\n<\/ul>\n
Data source<\/h3>\n
The data file raisin_dataset.csv<\/a> contains the data for this example. 900 raisins were used, including 450 from both varieties and seven features were extracted. Here, the number of variables (columns) is 8, and the number of instances (rows) is 900.<\/p>\nVariables<\/h3>\n
The features or variables<\/a> are the following:<\/p>\n\n- Area<\/b>: Gives the number of pixels within the boundaries of the raisin.<\/li>\n
- Perimeter<\/b>: It measures the environment by calculating the distance between the boundaries of the raisin and the pixels around it.<\/li>\n
- MajorAxisLenght<\/b>: Specifies the length of the major axis, denoting the longest line drawable on the raisin.<\/li>\n
- MinorAxisLenght<\/b>: Indicates the length of the small axis, representing the shortest line drawable on the raisin.<\/li>\n
- Eccentricity<\/b>: It measures the ellipse’s eccentricity, which has the same moments as raisins.<\/li>\n
- ConvexArea<\/b>: Gives the number of pixels of the smallest convex shell of the region formed by the raisin.<\/li>\n
- Extent<\/b>: It gives the ratio of the region the raisin forms to the total pixels in the bounding box.<\/li>\n
- Class<\/b>: Kecimen and Besni raisin.<\/li>\n<\/ul>\n
All variables in the study are inputs, except ‘Class’, the output we want to extract for this machine learning study. Note that ‘Class’ is binary and only takes Kecimen and Besni’s values.<\/p>\n
Instances<\/h3>\n
The instances<\/a> are divided into training, selection, and testing subsets. They account for 540 (60%), 180 (20%), and 180 (20%) of the original instances, respectively, distributed randomly across the dataset.<\/p>\nWe can perform a few related analytics once the data set has been set. First, we check the provided information and ensure that the data is of good quality.<\/p>\n
Variables distributions<\/h3>\n
The data distributions<\/a> tell us the percentages of Kecimen and Besni raisins.<\/p>\n![](\"https:\/\/www.neuraldesigner.com\/images\/raisin_class_pie_chart.webp\")
<\/p>\n
This data set has the same number of samples of both categories (Kecimen and Besni).<\/p>\n
Inputs-targets correlations<\/h3>\n
The\u00a0input-target correlations<\/a> might indicate to us which factors are more influential.<\/span><\/p>\n![](\"https:\/\/www.neuraldesigner.com\/images\/raisin_class_pearon_correlations_chart.webp\")
<\/p>\n
From the above chart, we can see that all the features have a significant influence except the ‘Extent’, compared to the others.<\/p>\n<\/section>\n\n3. Neural network<\/h2>\n
The second step is to choose a neural network<\/a> to represent the classification function. Classification problems it consist of:<\/p>\n\n- A scaling layer<\/a>.<\/li>\n
- A perceptron layer<\/a>.<\/li>\n
- A probabilistic layer<\/a>.<\/li>\n<\/ul>\n
The scaling layer<\/a> contains the statistics on the input calculated from the data file and the method for scaling the input variables. Here, the\u00a0minimum and maximum scaling method<\/a> is set, but the mean and standard deviation scaling method<\/a> produces similar results.<\/p>\nWe set one perceptron layer, with 3 neurons as a first guess, having the hyperbolic tangent (tanh) as the activation function<\/a>.<\/p>\nThe following figure is a diagram of the neural network used in this example.<\/p>\n
![](\"https:\/\/www.neuraldesigner.com\/images\/raisin_network_architecture.webp\")
<\/p>\n
The yellow circles represent scaling neurons, the blue circles perceptron neurons, and the red circles probabilistic neurons. The number of inputs is 7, and the number of outputs is 1.<\/p>\n<\/section>\n\n4. Training strategy<\/h2>\n
The training strategy<\/a> is applied to the neural network to obtain the best possible performance. It comprises two components:<\/p>\n\n- A loss index.<\/li>\n
- An optimization algorithm.<\/li>\n<\/ul>\n
The selected loss index<\/a> is the normalized squared error (NSE)<\/a> with L2 regularization<\/a>. In situations with balanced targets, like in this case, the normalized squared error provides valuable assistance.<\/p>\nThe error term<\/a> fits the neural network to the training instances of the data set. The regularization term<\/a> makes the model more stable and improves generalization, so our model will be more predictive.<\/p>\nThe selected optimization algorithm<\/a> that minimizes the loss index is the quasi-Newton method<\/a>.<\/p>\nThe following chart shows how the training (blue) and selection (orange) errors decrease with the training epochs.<\/p>\n
![](\"https:\/\/www.neuraldesigner.com\/images\/raisin_Quasi-Newton_method_errors_history.webp\")
<\/p>\n
The final training and selection errors are training error = 0.353 NSE<\/b> (blue) and selection error = 0.419 NSE<\/b> (orange). The following section will improve the generalization performance by reducing the selection error.<\/p>\n<\/section>\n\n5. Model selection<\/h2>\n
The objective of model selection<\/a> is to find the network architecture with the best generalization properties. We want to improve the final selection error obtained before (0.419 NSE).<\/p>\nThe best selection error is achieved using a model whose complexity is the most appropriate to produce a better data fit. Order selection<\/a> algorithms are responsible for finding the optimal number of perceptron neurons in the neural networks.<\/p>\nThe following chart shows the error history for different numbers of perceptron neurons. We conclude that for two neurons, the selection error is the minimum.<\/p>\n
![](\"https:\/\/www.neuraldesigner.com\/images\/raisin_selection_errors_plot)
<\/p>\n
The chart shows that the optimal number of neurons is 2, with selection error = 0.398 NSE<\/b>.<\/p>\n
The following figure shows the final network architecture for this application.<\/p>\n
![](\"https:\/\/www.neuraldesigner.com\/images\/raisin_new_network_architecture.webp\")
<\/p>\n<\/section>\n\n6. Testing analysis<\/h2>\n
The next step is to perform an exhaustive testing analysis<\/a> to validate the neural network’s predictive capabilities. The testing compares the values provided by this technique to the observed values.<\/p>\nA good measure of the precision of a binary classification model is the ROC curve<\/a>. Furthermore, the ROC curve analysis returns the optimal value for the decision threshold = 0.63.<\/p>\n![](\"https:\/\/www.neuraldesigner.com\/images\/raisin_ROC_chart.webp\")
<\/p>\n
We are interested in the area under the curve (AUC). A perfect classifier would have an AUC=1, and a random one would have an AUC=0.5.<\/p>\n
Our model has an AUC\u00a0<\/b>of\u00a00.942<\/strong>, which is a good indicator of our model’s performance<\/span>.<\/p>\nWe can also look at the confusion matrix<\/a>.\u00a0Next, we show the elements of this matrix for a decision threshold = 0.63<\/b>.<\/p>\n\n\n\n<\/th>\n Predicted positive<\/th>\n Predicted negative<\/th>\n<\/tr>\n \nReal positive<\/th>\n 87 (48.3%)<\/td>\n 4 (2.2%)<\/td>\n<\/tr>\n \nReal negative<\/th>\n 14 (7.8%)<\/td>\n 75 (41.7%)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
This example is solved with Neural Designer<\/a>. To follow it step by step, you can use the free trial<\/a>.<\/p>\n<\/section>\n We will predict between two types of raisins, so the variable to be predicted is binary (Kecimen or Besni). Therefore, this is a classification<\/a> project.<\/p>\n The goal is to model the type of raisin based on its features for subsequent use.<\/p>\n<\/section>\n The first step is to prepare the data set<\/a>, which is the source of information for the problem. We need to configure three things here:<\/p>\n The data file raisin_dataset.csv<\/a> contains the data for this example. 900 raisins were used, including 450 from both varieties and seven features were extracted. Here, the number of variables (columns) is 8, and the number of instances (rows) is 900.<\/p>\n The features or variables<\/a> are the following:<\/p>\n All variables in the study are inputs, except ‘Class’, the output we want to extract for this machine learning study. Note that ‘Class’ is binary and only takes Kecimen and Besni’s values.<\/p>\n The instances<\/a> are divided into training, selection, and testing subsets. They account for 540 (60%), 180 (20%), and 180 (20%) of the original instances, respectively, distributed randomly across the dataset.<\/p>\n We can perform a few related analytics once the data set has been set. First, we check the provided information and ensure that the data is of good quality.<\/p>\n The data distributions<\/a> tell us the percentages of Kecimen and Besni raisins.<\/p>\n This data set has the same number of samples of both categories (Kecimen and Besni).<\/p>\n The\u00a0input-target correlations<\/a> might indicate to us which factors are more influential.<\/span><\/p>\n From the above chart, we can see that all the features have a significant influence except the ‘Extent’, compared to the others.<\/p>\n<\/section>\n The second step is to choose a neural network<\/a> to represent the classification function. Classification problems it consist of:<\/p>\n The scaling layer<\/a> contains the statistics on the input calculated from the data file and the method for scaling the input variables. Here, the\u00a0minimum and maximum scaling method<\/a> is set, but the mean and standard deviation scaling method<\/a> produces similar results.<\/p>\n We set one perceptron layer, with 3 neurons as a first guess, having the hyperbolic tangent (tanh) as the activation function<\/a>.<\/p>\n The following figure is a diagram of the neural network used in this example.<\/p>\n The yellow circles represent scaling neurons, the blue circles perceptron neurons, and the red circles probabilistic neurons. The number of inputs is 7, and the number of outputs is 1.<\/p>\n<\/section>\n The training strategy<\/a> is applied to the neural network to obtain the best possible performance. It comprises two components:<\/p>\n The selected loss index<\/a> is the normalized squared error (NSE)<\/a> with L2 regularization<\/a>. In situations with balanced targets, like in this case, the normalized squared error provides valuable assistance.<\/p>\n The error term<\/a> fits the neural network to the training instances of the data set. The regularization term<\/a> makes the model more stable and improves generalization, so our model will be more predictive.<\/p>\n The selected optimization algorithm<\/a> that minimizes the loss index is the quasi-Newton method<\/a>.<\/p>\n The following chart shows how the training (blue) and selection (orange) errors decrease with the training epochs.<\/p>\n The final training and selection errors are training error = 0.353 NSE<\/b> (blue) and selection error = 0.419 NSE<\/b> (orange). The following section will improve the generalization performance by reducing the selection error.<\/p>\n<\/section>\n The objective of model selection<\/a> is to find the network architecture with the best generalization properties. We want to improve the final selection error obtained before (0.419 NSE).<\/p>\n The best selection error is achieved using a model whose complexity is the most appropriate to produce a better data fit. Order selection<\/a> algorithms are responsible for finding the optimal number of perceptron neurons in the neural networks.<\/p>\n The following chart shows the error history for different numbers of perceptron neurons. We conclude that for two neurons, the selection error is the minimum.<\/p>\n The chart shows that the optimal number of neurons is 2, with selection error = 0.398 NSE<\/b>.<\/p>\n The following figure shows the final network architecture for this application.<\/p>\n The next step is to perform an exhaustive testing analysis<\/a> to validate the neural network’s predictive capabilities. The testing compares the values provided by this technique to the observed values.<\/p>\n A good measure of the precision of a binary classification model is the ROC curve<\/a>. Furthermore, the ROC curve analysis returns the optimal value for the decision threshold = 0.63.<\/p>\n We are interested in the area under the curve (AUC). A perfect classifier would have an AUC=1, and a random one would have an AUC=0.5.<\/p>\n Our model has an AUC\u00a0<\/b>of\u00a00.942<\/strong>, which is a good indicator of our model’s performance<\/span>.<\/p>\n We can also look at the confusion matrix<\/a>.\u00a0Next, we show the elements of this matrix for a decision threshold = 0.63<\/b>.<\/p>\n1. Application type<\/h2>\n
2. Data set<\/h2>\n
\n
Data source<\/h3>\n
Variables<\/h3>\n
\n
Instances<\/h3>\n
Variables distributions<\/h3>\n
<\/p>\nInputs-targets correlations<\/h3>\n
<\/p>\n3. Neural network<\/h2>\n
\n
<\/p>\n4. Training strategy<\/h2>\n
\n
<\/p>\n5. Model selection<\/h2>\n
<\/p>\n
<\/p>\n<\/section>\n6. Testing analysis<\/h2>\n
<\/p>\n\n\n
\n <\/th>\n Predicted positive<\/th>\n Predicted negative<\/th>\n<\/tr>\n \n Real positive<\/th>\n 87 (48.3%)<\/td>\n 4 (2.2%)<\/td>\n<\/tr>\n \n Real negative<\/th>\n 14 (7.8%)<\/td>\n 75 (41.7%)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n