Contents<\/h2>\n\n- Application type<\/a>.<\/li>\n
- Data set<\/a>.<\/li>\n
- Neural network<\/a>.<\/li>\n
- Training strategy<\/a>.<\/li>\n
- Model selection<\/a>.<\/li>\n
- Testing analysis<\/a>.<\/li>\n
- Model deployment<\/a>.<\/li>\n<\/ol>\n<\/section>\n
\n1. Application type<\/h2>\n
The variable to be predicted is continuous (power). Therefore, this is an approximation<\/a> project.<\/p>\nThe primary goal here is to model energy production as a function of wind speed.<\/p>\n<\/section>\n\n2. Data set<\/h2>\n
The first step is to prepare the data set<\/a>, which is the source of information for the approximation problem.<\/p>\nIt consists of:<\/p>\n
\n- Data source.<\/li>\n
- Variables.<\/li>\n
- Instances.<\/li>\n<\/ul>\n
The file wind-turbine.csv<\/a> contains the data for this example.<\/p>\nHere, the number of variables (columns) is 2, and the number of instances (rows) is 48007.<\/p>\n
We have the following variables for this analysis:<\/p>\n
\n- wind-speed<\/b>, the wind speed at the hub height of the turbine, in meters per second.<\/li>\n
- power<\/b>, the power generated by the turbine for that moment, in kilowatts.<\/li>\n<\/ul>\n
<\/p>\n
Our target variable<\/a> will be the last one, power.<\/p>\nThe instances<\/a> are divided into training, selection, and testing subsets. They represent 60%, 20%, and 20% of the original instances, respectively, and are split at random.<\/p>\nCalculating the data distributions<\/a> helps us verify the accuracy of the available information and identify anomalies. The following chart shows the histogram for the power-generated variable:<\/p>\n![](\"https:\/\/www.neuraldesigner.com\/images\/wind-turbine-power-distribution.webp\")
<\/p>\n
It is also interesting to look for dependencies between input and target variables. To do that, we can calculate an\u00a0input-targets correlation<\/a> table.<\/p>\n![](\"https:\/\/www.neuraldesigner.com\/images\/wind-turbine-correlations.webp\")
<\/p>\n
As expected, there is a high correlation between wind speed and the energy produced by the turbines.<\/p>\n
Next, we plot a scatter chart<\/a>.<\/p>\n![](\"https:\/\/www.neuraldesigner.com\/images\/wind-turbine-power-vs-wind-speed.webp\")
<\/p>\n
Here, we observe that the power generated increases linearly once a threshold wind level is surpassed until reaching an upper bound. Beyond this point, regardless of how much the wind speed exceeds the threshold, the power generated remains constant.<\/p>\n<\/section>\n\n3. Neural network<\/h3>\n
The second step is to build a neural network that represents the approximation function. Approximation problems are typically composed of:<\/p>\n
\n- Scaling layer.<\/li>\n
- Perceptron layers.<\/li>\n
- Unscaling layer.<\/li>\n<\/ul>\n
The neural network has 1 input (wind speed) and 1 output (power).<\/p>\n
The scaling layer<\/a> contains the statistics of the inputs. We use the automatic setting for this layer to accommodate the best scaling technique for our data.<\/p>\nWe use 2 perceptron layers here:<\/p>\n
\n- The first perceptron layer has 1 input, 3 neurons, and a hyperbolic tangent activation function<\/a>.<\/li>\n
- The second perceptron layer has 3 inputs, 1 neuron, and a linear activation function<\/a>.<\/li>\n<\/ul>\n
<\/p>\n
The unscaling layer<\/a> contains the statistics of the outputs. We use the automatic method as before.<\/p>\nThe next graph represents the neural network for this example.<\/p>\n
![](\"https:\/\/www.neuraldesigner.com\/images\/wind-turbine-initial-neural-network.webp\")
<\/p>\n<\/section>\n\n4. Training strategy<\/h2>\n
The fourth step is to select an appropriate training strategy<\/a>. It comprises two parameters:<\/p>\n\n- Loss index.<\/li>\n
- Optimization algorithm.<\/li>\n<\/ul>\n
<\/p>\n
The loss index<\/a> defines what the neural network will learn. It comprises an error term<\/a> and a regularization term<\/a>.<\/p>\nWe choose the normalized squared<\/a> error as the error term. It divides the squared error between the neural network’s outputs and the targets in the data set by its normalization coefficient. If the normalized squared error has a value of 1, then the neural network is predicting the data ‘in the mean’, while a value of zero means a perfect prediction of the data. This error term does not have any parameters to set.<\/p>\nThe regularization term is the L2 regularization<\/a>. To control the neural network’s complexity, we adjust its parameters by reducing their values. We use a weak weight for this regularization term.<\/p>\nThe optimization algorithm<\/a> is responsible for searching for the neural network parameters that minimize the loss index. Here, we chose the quasi-Newton method<\/a> as an optimization algorithm.<\/p>\n![](\"https:\/\/www.neuraldesigner.com\/images\/wind-turbine-training-history.webp\")
<\/p>\n
The following chart illustrates how the training (blue) and selection (orange) errors decrease with the number of epochs during the training process. The final values are training error = 0.00857 NSE<\/b> and selection error = 0.00869 NSE<\/b>, respectively.<\/p>\n<\/section>\n\n5. Model selection<\/h2>\n
Model selection<\/a> algorithms\u00a0enhance the generalization performance of the neural network. As the selection error we have achieved so far is minimal (0.00869 NSE), this algorithm is unnecessary here.<\/p>\n<\/section>\n\n6. Testing analysis<\/h2>\n
The purpose of the testing analysis<\/a> is to validate the generalization capabilities of the neural network. We use the testing instances<\/a> in the data set, which have never been used before.<\/p>\nA standard testing method in approximation applications is to perform a linear regression analysis<\/a> between the predicted and the real pollutant level values.<\/p>\n![](\"https:\/\/www.neuraldesigner.com\/images\/wind-turbine-linear-regression-analysis.webp\")
<\/p>\n
For a perfect fit, the correlation coefficient R2 would be 1.\u00a0With\u00a0R2 = 0.996<\/strong>, the neural network accurately predicts the testing data<\/span>.<\/p>\n<\/section>\n\n7. Model deployment<\/h2>\n
In the model deployment<\/a> phase, the neural network is used to predict outputs for inputs that it has not seen before.<\/p>\nWe can calculate the neural network outputs<\/a> for a given set of inputs:<\/p>\n\n- wind-speed: 7.465 meters per second.<\/li>\n
- power: 1166.580 kilowatts.<\/li>\n<\/ul>\n
Directional outputs<\/a> plot the neural network outputs through some reference points.<\/p>\nThe reference point for the plot is the one we show next.<\/p>\n
\n- wind-speed: 7.465 meters per second.<\/li>\n<\/ul>\n
We can see here how the wind speed to solar noon affects the power generated:<\/p>\n
![](\"https:\/\/www.neuraldesigner.com\/images\/wind-turbine-power-wind-speed-directional-output.webp\")
<\/p>\n
This application finds its true meaning when comparing the mean quadratic error of the fabricant’s calculations, using the theoretical turbine power equation (1.95%) versus our model’s error (1.41%), resulting in a 27.69% relative improvement.
\nOur model performs better because it considers the actual empirical data instead of a physical approximation.<\/p>\n
![](\"https:\/\/www.neuraldesigner.com\/images\/wind-turbine-mse-improvement.webp\")
<\/p>\n
The mathematical expression<\/a> represented by the predictive model is displayed next:<\/p>\nscaled_wind-speed = wind-speed*(1+1)\/(25.20599937-(0))-0*(1+1)\/(25.20599937-0)-1; \r\nperceptron_layer_output_0 = tanh[ -0.0837533 + (scaled_wind-speed*-0.297207) ];\r\nperceptron_layer_output_1 = tanh[ -0.0199511 + (scaled_wind-speed*0.654869) ];\r\nperceptron_layer_output_2 = tanh[ 0.811123 + (scaled_wind-speed*2.76357) ];\r\nperceptron_layer_output_0 = [ -0.128215 + (perceptron_layer_output_0*0.290953)+ (perceptron_layer_output_1*-0.775739)+ (perceptron_layer_output_2*1.48548) ];\r\nunscaling_layer_output_0 = perceptron_layer_output_0*(3618.72998+2.471410036)\/(1+1)-2.471410036+1*(3618.72998+2.471410036)\/(1+1);\r\n<\/pre>\n<\/section>\n\n8. Video tutorial<\/h2>\n
You can watch the step-by-step tutorial video below to help you complete this Machine Learning example
\nfor free using the powerful machine learning software Neural Designer<\/a>.<\/p>\n
1. Application type<\/h2>\n
The variable to be predicted is continuous (power). Therefore, this is an approximation<\/a> project.<\/p>\n The primary goal here is to model energy production as a function of wind speed.<\/p>\n<\/section>\n The first step is to prepare the data set<\/a>, which is the source of information for the approximation problem.<\/p>\n It consists of:<\/p>\n The file wind-turbine.csv<\/a> contains the data for this example.<\/p>\n Here, the number of variables (columns) is 2, and the number of instances (rows) is 48007.<\/p>\n We have the following variables for this analysis:<\/p>\n <\/p>\n Our target variable<\/a> will be the last one, power.<\/p>\n The instances<\/a> are divided into training, selection, and testing subsets. They represent 60%, 20%, and 20% of the original instances, respectively, and are split at random.<\/p>\n Calculating the data distributions<\/a> helps us verify the accuracy of the available information and identify anomalies. The following chart shows the histogram for the power-generated variable:<\/p>\n It is also interesting to look for dependencies between input and target variables. To do that, we can calculate an\u00a0input-targets correlation<\/a> table.<\/p>\n As expected, there is a high correlation between wind speed and the energy produced by the turbines.<\/p>\n Next, we plot a scatter chart<\/a>.<\/p>\n Here, we observe that the power generated increases linearly once a threshold wind level is surpassed until reaching an upper bound. Beyond this point, regardless of how much the wind speed exceeds the threshold, the power generated remains constant.<\/p>\n<\/section>\n The second step is to build a neural network that represents the approximation function. Approximation problems are typically composed of:<\/p>\n The neural network has 1 input (wind speed) and 1 output (power).<\/p>\n The scaling layer<\/a> contains the statistics of the inputs. We use the automatic setting for this layer to accommodate the best scaling technique for our data.<\/p>\n We use 2 perceptron layers here:<\/p>\n <\/p>\n The unscaling layer<\/a> contains the statistics of the outputs. We use the automatic method as before.<\/p>\n The next graph represents the neural network for this example.<\/p>\n The fourth step is to select an appropriate training strategy<\/a>. It comprises two parameters:<\/p>\n <\/p>\n The loss index<\/a> defines what the neural network will learn. It comprises an error term<\/a> and a regularization term<\/a>.<\/p>\n We choose the normalized squared<\/a> error as the error term. It divides the squared error between the neural network’s outputs and the targets in the data set by its normalization coefficient. If the normalized squared error has a value of 1, then the neural network is predicting the data ‘in the mean’, while a value of zero means a perfect prediction of the data. This error term does not have any parameters to set.<\/p>\n The regularization term is the L2 regularization<\/a>. To control the neural network’s complexity, we adjust its parameters by reducing their values. We use a weak weight for this regularization term.<\/p>\n The optimization algorithm<\/a> is responsible for searching for the neural network parameters that minimize the loss index. Here, we chose the quasi-Newton method<\/a> as an optimization algorithm.<\/p>\n The following chart illustrates how the training (blue) and selection (orange) errors decrease with the number of epochs during the training process. The final values are training error = 0.00857 NSE<\/b> and selection error = 0.00869 NSE<\/b>, respectively.<\/p>\n<\/section>\n Model selection<\/a> algorithms\u00a0enhance the generalization performance of the neural network. As the selection error we have achieved so far is minimal (0.00869 NSE), this algorithm is unnecessary here.<\/p>\n<\/section>\n The purpose of the testing analysis<\/a> is to validate the generalization capabilities of the neural network. We use the testing instances<\/a> in the data set, which have never been used before.<\/p>\n A standard testing method in approximation applications is to perform a linear regression analysis<\/a> between the predicted and the real pollutant level values.<\/p>\n For a perfect fit, the correlation coefficient R2 would be 1.\u00a0With\u00a0R2 = 0.996<\/strong>, the neural network accurately predicts the testing data<\/span>.<\/p>\n<\/section>\n In the model deployment<\/a> phase, the neural network is used to predict outputs for inputs that it has not seen before.<\/p>\n We can calculate the neural network outputs<\/a> for a given set of inputs:<\/p>\n Directional outputs<\/a> plot the neural network outputs through some reference points.<\/p>\n The reference point for the plot is the one we show next.<\/p>\n We can see here how the wind speed to solar noon affects the power generated:<\/p>\n This application finds its true meaning when comparing the mean quadratic error of the fabricant’s calculations, using the theoretical turbine power equation (1.95%) versus our model’s error (1.41%), resulting in a 27.69% relative improvement. The mathematical expression<\/a> represented by the predictive model is displayed next:<\/p>\n You can watch the step-by-step tutorial video below to help you complete this Machine Learning example2. Data set<\/h2>\n
\n
\n
<\/p>\n<\/p>\n<\/p>\n3. Neural network<\/h3>\n
\n
\n
<\/p>\n<\/section>\n4. Training strategy<\/h2>\n
\n
<\/p>\n5. Model selection<\/h2>\n
6. Testing analysis<\/h2>\n
<\/p>\n7. Model deployment<\/h2>\n
\n
\n
<\/p>\n
\nOur model performs better because it considers the actual empirical data instead of a physical approximation.<\/p>\n<\/p>\nscaled_wind-speed = wind-speed*(1+1)\/(25.20599937-(0))-0*(1+1)\/(25.20599937-0)-1; \r\nperceptron_layer_output_0 = tanh[ -0.0837533 + (scaled_wind-speed*-0.297207) ];\r\nperceptron_layer_output_1 = tanh[ -0.0199511 + (scaled_wind-speed*0.654869) ];\r\nperceptron_layer_output_2 = tanh[ 0.811123 + (scaled_wind-speed*2.76357) ];\r\nperceptron_layer_output_0 = [ -0.128215 + (perceptron_layer_output_0*0.290953)+ (perceptron_layer_output_1*-0.775739)+ (perceptron_layer_output_2*1.48548) ];\r\nunscaling_layer_output_0 = perceptron_layer_output_0*(3618.72998+2.471410036)\/(1+1)-2.471410036+1*(3618.72998+2.471410036)\/(1+1);\r\n<\/pre>\n<\/section>\n
8. Video tutorial<\/h2>\n
\nfor free using the powerful machine learning software Neural Designer<\/a>.<\/p>\n