The data set used here is related to the direct marketing campaigns of a Portuguese bank institution.<\/p>\n
This example is solved with Neural Designer<\/a>. To follow it step by step, you can use the free trial<\/a>.<\/p>\n The variable to be predicted is binary (i.e., whether to buy or not). Thus, this is a classification<\/a> project.<\/p>\n The goal here is to model the probability of buying as a function of the customer features.<\/p>\n<\/section>\n In general, a data set<\/a> contains the following concepts:<\/p>\n The data file bank_marketing.csv<\/a> contains the information used to create the model. It consists of 1522 rows and 19 columns. Each row represents a different customer, while each column represents a distinct feature for each customer.<\/p>\n The variables<\/a> are:<\/p>\n<\/section>\n Conversion<\/strong>: 1 if the client subscribed to a term deposit, 0 otherwise.<\/p>\n<\/li>\n<\/ul>\n There are 1,522\u00a0instances<\/a> in the dataset<\/span>. 60% are used for training, 20% for selection, and 20% for testing.<\/p>\n We can also calculate the input-target correlations<\/a> between the conversion rate and all the customer features to see which variables might influence the buying process.<\/p>\n The second step is to configure the neural network<\/a> parameters. For classification problems, it is composed of:<\/p>\n The following figure is a graphical representation of the neural network used for this problem.<\/p>\n The fourth step is to configure the training strategy<\/a>, which is composed of two concepts:<\/p>\n The loss index<\/a> chosen is the weighted squared error<\/a> with L2 regularization<\/a>.<\/p>\n The optimization algorithm<\/a> is applied to the neural network to get the minimum loss.<\/p>\n The chosen algorithm here is the quasi-Newton method<\/a>. We leave the default training parameters, stopping criteria, and training history settings.<\/p>\n The following chart illustrates how the training and selection errors decrease with the number of epochs during the training process.<\/p>\n The final values are training error = 0.821 WSE<\/strong> and selection error = 0.889 WSE<\/strong>, respectively.<\/p>\n<\/section>\n The objective of\u00a0model selection<\/a>\u00a0is to find a network architecture with the best generalization properties, that is, one that minimizes the error on the\u00a0selected instances<\/a> of the data set.<\/span><\/p>\n More specifically, we\u00a0aim to find a neural network with a selection error of less than\u00a00.889 WSE<\/strong>, which is the current best value we have achieved<\/span>.<\/p>\n Order selection<\/a> algorithms train several network architectures with different numbers of neurons and select the one with the smallest selection error.<\/p>\n The incremental order<\/a> method starts with a small number of neurons and increases the complexity at each iteration. The following chart shows the training error (blue) and the selection error (orange) as a function of the number of neurons.<\/p>\n The objective of the testing analysis<\/a> is to evaluate the generalization performance of the neural network.<\/p>\n The standard way to do this is to compare the neural network outputs against data that it has never seen before, the testing instances<\/a>.<\/p>\n A commonly used method to test a neural network is the ROC curve<\/a>.<\/p>\n One of the parameters obtained from this chart is the area under the curve (AUC). The closer to 1 is the area under the curve, the better is the classifier. In this case, the area under the curve takes a high value: AUC = 0.80<\/strong>.<\/p>\n The binary classification tests<\/a> provide us with helpful information for testing the performance of a binary classification problem:<\/p>\n The classification accuracy achieves a high value, indicating that the prediction is suitable for many cases.<\/p>\n The second is another visual aid that shows the advantage of using a predictive model against randomness.<\/p>\n The following picture depicts the cumulative gain<\/a> for the current example.<\/p>\n As we can see, this chart shows that by calling only half of the clients, we can achieve more than 80% of the positive responses.<\/p>\n The conversion rates<\/a> for this problem are depicted in the following chart.<\/p>\n In the model deployment<\/a> phase, the neural network can be used for different techniques.<\/p>\n We can predict which clients have a higher probability of buying the product by calculating the neural network outputs<\/a>.<\/p>\n We need to know the input variables for each new client.<\/p>\n<\/section>\nContents<\/h3>\n
\n
1. Application type<\/h2>\n
2. Data set<\/h2>\n
\n
Demographics<\/strong><\/h4>\n
\n
Financial Information<\/strong><\/h4>\n
\n
Campaign Information<\/strong><\/h4>\n
\n
Target Variable<\/strong><\/h4>\n
\n
Instances<\/h3>\n
Distributions<\/h3>\n
![\"\"](\"https:\/\/www.neuraldesigner.com\/images\/bank-marketing-data-distribution.webp\")
As expected, the number of calls without conversion is much greater than the number of calls with conversion.<\/p>\nInput-target correlations<\/h3>\n
![\"\"](\"https:\/\/www.neuraldesigner.com\/images\/bank-marketing-inputs-targets-correlations.webp\")
<\/p>\n<\/section>\n3. Neural network<\/h2>\n
\n
Neural network graph<\/h3>\n
![\"\"](\"https:\/\/www.neuraldesigner.com\/images\/bank-marketing-initial-neural-network.webp\")
<\/p>\n<\/section>\n4. Training strategy<\/h2>\n
\n
Loss index<\/h3>\n
Optimization algorithm<\/h3>\n
Training<\/h3>\n
![\"\"](\"https:\/\/www.neuraldesigner.com\/images\/bank-marketing-training-history.webp\")
<\/p>\n5. Model selection<\/h2>\n
![\"\"](\"https:\/\/www.neuraldesigner.com\/images\/bank-marketing-final-neural-network.webp\")
<\/p>\n<\/section>\n6. Testing analysis<\/h2>\n
ROC curve<\/h3>\n
![\"\"](\"https:\/\/www.neuraldesigner.com\/images\/bank-marketing-roc-curve.webp\")
<\/p>\nBinary classification metrics<\/h3>\n
\n
Cumulative gain<\/h3>\n
Positive rates<\/h3>\n
![\"\"](\"https:\/\/www.neuraldesigner.com\/images\/bank-marketing-positives-rates.webp\")
<\/p>\n<\/section>\n7. Model deployment<\/h2>\n
References:<\/h2>\n