The central goal here is to design a model that makes useful detection of gamma rays and discriminates the background.
This example is solved with Neural Designer. To follow it step by step, you can use the free trial.
This is a classification project, since the variable to be predicted is binary: gamma (signal) or hadron (background).
The first step is to prepare the data set, which is the source of information for the classification problem. For that, we need to configure the following concepts:
The data source is the file gamma.csv. It contains the data for this example in comma-separated values (CSV) format. The number of columns is 11, and the number of rows is 19020.
The data set was generated by the Corsika simulation program: a Monte Carlo code to simulate extensive air showers. It has been used to simulate the registration of high-energy gamma particles from the Cherenkov gamma telescope using the imaging technique.
The Cherenkov gamma telescope uses the atmosphere as a means of detection. It records the image of the radiation Cherenkov that produces an extensive air shower generated in turn by the gamma-ray.
Cherenkov photons get collected, in patterns (called the shower image), allowing to discriminate statistically those caused by primary gammas (signal) from the images of hadronic showers initiated by cosmic rays in the upper atmosphere (background).
The image recorded is in the shape of an ellipse. The characteristic parameters of this ellipse (Hillas parameters) are among the image variables that can be used for discrimination. There are, in addition, further discriminating characteristics, like asymmetric energy depositions along the major axis, the extent of the cluster in the image plane, or the total sum of depositions.
The variables are:
The instances are divided into training, selection, and testing subsets. They represent 60% (11412), 20% (3804), and 20% (3804) of the original instances, respectively, and are split at random.
We can calculate the distributions of all variables. The next figure is the pie chart for the gamma or hadron cases.
As we can see, the most os the samples are gamma signal. In the real data, the hadron class (background) represents the majority of the events.
Finally, the inputs-targets correlations might indicate to us what factors most influence.
Here, the most correlated variables with the classification are fAlpha and fDist, which refers to ellipse variables. Also are not many correlated variables like fConc1 and fM3Trans.
The second step is to choose a neural network. For classification problems, it is usually composed by:
The scaling layer contains the statistics on the inputs calculated from the data file and the method for scaling the input variables. Here the minimum and maximum method has been set. Nevertheless, the mean and standard deviation method would produce very similar results.
The number of perceptron layers is 1. This perceptron layer has 10 inputs and 10 neurons.
Finally, we will set the binary probabilistic method for the probabilistic layer as we want the predicted target variable to be binary.
The next figure is a graphical representation of this classification neural network:
Here, the yellow circles represent scaling neurons, the blue circles perceptron neurons and the red circles probabilistic neurons. The number of inputs is 10, and the number of outputs is 1.
The fourth step is to set the training strategy, which is composed of:
The loss index chosen for this application is the normalized squared error with L2 regularization.
The error term fits the neural network to the training instances of the data set. The regularization term makes the model more stable and improves generalization.
The optimization algorithm searches for the neural network parameters which minimize the loss index. The quasi-Newton method is chosen here.
The following chart shows how the training and selection errors decrease with the epochs during the training process.
The final values are training error = 0.547 NSE (blue), and selection error = 0.550 NSE (orange).
The objective of model selection is to find the network architecture with the best generalization properties, that is, that which minimizes the error on the selected instances of the data set.
More specifically, we want to find a neural network with a selection error of less than 0.550 WSE, which is the value that we have achieved so far.
Order selection algorithms train several network architectures with a different number of neurons and select that with the smallest selection error.
The incremental order method starts with a small number of neurons and increases the complexity at each iteration.
The last step is to test the generalization performance of the trained neural network.
The objective of the testing analysis is to validate the generalization performance of the trained neural network. To validate a classification technique, we need to compare the values provided by this technique to the observed values. We can use the ROC curve as it is the standard testing method for binary classification projects.
In the confusion matrix, the rows represent the targets (or real values) and the columns the corresponding outputs (or predictive values). The diagonal cells show the cases that are correctly classified, and the off-diagonal cells show the misclassified cases.
|Predicted positive (gamma)||Predicted negative (background)|
|Real positive (gamma)||2120 (55.8%)||283 (7.4%)|
|Real negative (background)||407 (10.7%)||993 (26.1%)|
As we can see, the number of instances that the model can correctly predict is 3114 (81.9%) while it misclassifies is 690 (18.1%). This shows that our predictive model has a great classification accuracy.
The next list depicts the binary classification tests for this application:
The neural network is now ready to predict outputs for inputs that it has never seen. This process is called model deployment.
To classify a given signal, we calculate the neural network outputs from the differents variables. For instance:
For this particular case, the neural network would classify the signal as a gamma-ray signal.
The mathematical expression of the trained neural network is listed below.
We have just built a predictive model to determine if the measured data come from gamma rays or from the hadron shower that we have considered as background.