Diagnose faults in ultrasonic flow meters using Neural Designer

Flow meters are devices used to measure the volumetric or mass flow of a fluid with ultrasound. However, these devices present serious engineering problems that give rise to a defective meter function and cause errors in reading the flow velocity.

In this study, we analyze a database with the characteristics of a liquid ultrasonic flow meter and its state (healthy or unhealthy) to know the relevant factors that cause failures in the system.

Therefore, a solution that provides a balance between accurate measurement and the necessary costs of recalibration is needed.

To carry out the analysis, we work with the most powerful method of analysis, Advanced Analytics.


  1. Application type.
  2. Data set.
  3. Neural network.
  4. Training strategy.
  5. Model selection.
  6. Testing analysis.
  7. Model deployment.

This example is solved with Neural Designer. To follow it step by step, you can use the free trial.

1. Application type

This is a classification project, since the variable to be predicted is binary (healthy or unhealthy).

The goal is to model the probability that an ultrasonic flow meter fails, conditioned on the process characteristics.

2. Data set

The first step is to prepare the data set, which is the source of information for the classification problem. It is composed of:

The file fault_detection.csv contains 87 instances of 37 diagnostic parameters corresponding to an 8-path liquid ultrasonic flow meter. These 37 variables are all continuous except for the state of health.

The variables of the problem are:

Since neural networks work with numbers, the class attribute (health state of the meter) has been transformed into two numerical values, 0 when the meter is faulty and 1 when it works properly.

Once we have the prepared data, we will make a descriptive analysis of the problem to detect the factors to be taken into account in the training to provide a logical and adequate solution to the problem. We modify the most critical aspects during the development, shaping the solution until obtaining the most precise and convenient one.

It is essential to know the ratio of negative and positive instances that we have in the data set in the first place.

The chart shows that the number of negative instances(40.23%) and positive instances(59.77%) are similar. This information will later be used to design the predictive model properly.

The inputs-targets correlations shown in the next chart, analyze the dependencies between each input variable and the target.

The flatness ratio (0.506) and some of the gains are the variables most relevant to the target. On the other hand, flow velocity in the paths contributes much less, so they are less determinant when deciding the state of health of the device.

This data set contains 87 instances. From these, 53 instances are used for training (60%), 17 for generalization (20%), and 17 for testing (20%).

3. Neural network

The next step is to choose a neural network. For classification problems, it is usually composed by:

The scaling layer contains the statistics on the inputs calculated from the data file and the method for scaling the input variables.

A perceptron layer with a hidden logistic layer is used. The neural network must have 36 inputs since these are all the inputs we have in the dataset. As an initial guess, we use three neurons in the hidden layer.

The probabilistic layer only contains the method for interpreting the outputs as probabilities. For this project, we set the binary probabilistic method as the target variable's possible values 1 or 0.

The following figure shows the structure of the neural network that has been set for this data set.

4. Training strategy

The next step is to select an appropriate training strategy, which defines what the neural network will learn. A general training strategy is composed of two concepts:

The data set is slightly unbalanced. Nevertheless, we use the normalized squared error with L2 regularization as the loss index.

Now, the model is ready to be trained. As the optimization algorithm, we use the default option for this type of case, the quasi-Newton method.

The following chart shows how the training and selection errors decrease with the epochs during the training process.

The final values are training error = 0.0161 NSE and selection error = 0.954 NSE, respectively.

This selection error is not a reliable value. Therefore to improve it, we must perform a model selection process.

5. Model selection

The objective of the model selection is to find the network architecture with the best generalization properties, that is, that which minimizes the error on the selected instances of the data set. More specifically, we want to find a neural network with a selection error of less than 0.954 NSE, which is the value we have achieved so far.

The number of inputs is too high. Thus, we will use the inputs selection to set some of the input variables to unused variables. The growing inputs method starts with a small number of inputs and adds more at each iteration. The following chart represents the training error (blue) and the selection error (red) as a function of the number of inputs in each iteration.

Order selection algorithms train several network architectures with a different number of neurons and select that with the smallest selection error. The incremental order method starts with a small number of neurons and increases the complexity at each iteration. The following chart shows the training error (blue) and the selection error (yellow) as a function of the number of neurons that have been set for each iteration.

The new network architecture is represented below.

After the model selection, the results are that the optimal network architecture consists of 14 inputs and two neurons in the perceptron layer. The selection error for this configuration is 0.326 NSE, which is far better than before.

6. Testing analysis

A good measure to check the precision of a binary classification model is the ROC curve shown below.

The parameter of importance here is the area under the curve (AUC). For this model, AUC = 1, which is the best it can be. This means that the classifier predicts well all the testing instances.

Another valid option to test the accuracy of a binary classification model is the confusion matrix. This matrix shows the number of negative and positive predicted values versus the real ones.

Predicted positive Predicted negative
Real positive 11 (64.7%) 1 (5.9%)
Real negative 0 (0.0%) 5 (29.4%)

From the confusion matrix, we can obtain the following binary classification tests:

According to the results, our predictive model is very accurate (94.12%). The probability of success in positive cases is 91.67%, and in negative cases, it is 100%. Therefore, the classifier is better for faulty meters than healthy ones. This is good because our goal is to find these faulty flow meters.

7. Model deployment

Once we know that our model is reliable and accurate, we can use it to check the health state of a meter, given the input variables' required values. This is called model deployment.

The predictive model also provides the most significant factors when determining the health state of the device. This fact allows us to act fast, thus avoiding the inadequate functioning of the system for a long time. On the other hand, we also save significant economic resources by predicting that the system works correctly.

The predictive model provides us with a mathematical expression to integrate the function to other meters with the same variables. This mathematical expression is shown below:

scaled_flatness_ratio = (flatness_ratio-0.8239070177)/0.01825289987;
scaled_symmetry = (symmetry-1.010969996)/0.00580408005;
scaled_crossflow = (crossflow-0.997435987)/0.002026459901;
scaled_speed_of_sound_path_4 = (speed_of_sound_path_4-1484.880005)/0.813575983;
scaled_gain_2 = (gain_2-33.39569855)/0.04274249822;
scaled_gain_3 = (gain_3-36.69630051)/0.02147400007;
scaled_gain_4 = (gain_4-36.8946991)/0.1071270034;
scaled_gain_5 = (gain_5-35.1576004)/0.04292529821;
scaled_gain_6 = (gain_6-35.38410187)/0.09594769776;
scaled_gain_8 = (gain_8-31.50309944)/0.1031560004;
scaled_gain_10 = (gain_10-33.18560028)/0.1018899977;
scaled_gain_11 = (gain_11-36.66719818)/0.01935099997;
scaled_gain_13 = (gain_13-35.94530106)/0.05782689899;
scaled_gain_14 = (gain_14-35.92010117)/0.04493260011;

perceptron_layer_1_output_00 = logistic( -0.522177 + (scaled_flatness_ratio*-1.85383) + (scaled_symmetry*-0.151413) + (scaled_crossflow*0.0967363) + (scaled_speed_of_sound_path_4*0.384233) + (scaled_gain_2*-1.95048) + (scaled_gain_3*0.965418) + (scaled_gain_4*-0.643104) + (scaled_gain_5*0.736049) + (scaled_gain_6*-0.495829) + (scaled_gain_8*-0.784475) + (scaled_gain_10*-1.76817) + (scaled_gain_11*-0.430356) + (scaled_gain_13*-0.511694) + (scaled_gain_14*-1.25687) );
perceptron_layer_1_output_11 = logistic( -4.34864 + (scaled_flatness_ratio*3.1918) + (scaled_symmetry*0.135014) + (scaled_crossflow*2.39612) + (scaled_speed_of_sound_path_4*1.35508) + (scaled_gain_2*-2.66284) + (scaled_gain_3*0.628813) + (scaled_gain_4*-2.35691) + (scaled_gain_5*-1.20798) + (scaled_gain_6*2.22323) + (scaled_gain_8*1.34993) + (scaled_gain_10*-0.382694) + (scaled_gain_11*2.0298) + (scaled_gain_13*0.123624) + (scaled_gain_14*1.35976) );

probabilistic_layer_combinations_0 = -3.6438 +7.00984*perceptron_layer_1_output_0 +7.36045*perceptron_layer_1_output_1 
fault = 1.0/(1.0 + exp(-probabilistic_layer_combinations_0);

    return 1/(1+exp(-x))

This expression can be exported elsewhere, for instance, a dedicated engineering software used in the industry.


Related examples:

Related solutions: