Diagnose faults in ultrasonic flowmeters

End-to-end machine learning examples

A significant problem for engineers is finding the faults and inconveniences of the facilities created. On a large scale, these can cause important inconveniences in the process and commit unacceptable malfunctions.

But detection is complicated, because these defects depend on so many factors that sometimes it seems impossible to identify them. In addition, the fault must be precisely located, since the objective is to correct its cause.

In this study, we analyze a database with the characteristics of a liquid ultrasonic flowmeter and their state (healthy or unhealthy), so we can know which are the relevant factors that cause failures in the system. To carry out the analysis, we work with the most powerful method of analysis, Advanced Analytics.

Flow meters are devices used for the purpose of measuring the volumetric or mass flow of a fluid with ultrasound. They are commonly used in the oil and gas industry. However, they present serious engineering problems that entail enormous costs. Among such problems are the failure of the wax deposit and the transducer due to high temperatures and pressures. These system failures give rise to a defective meter function and cause errors in the reading of the flow velocity.

In addition, according to the studies, the errors associated with the measurement of the flow can increase and overflow the margin of error allowed. Then it is usual that the meters are taken to accredited flow facilities to be recalibrated after one year of operation. Nevertheless, having the recalibration established after one year does not take into account two important factors: the device can fail before the year is completed and provides erroneous measurements or otherwise, that it is healthy and that recalibration is not necessary, leading to a totally unnecessary important expense.

Therefore, a new solution to this problem is needed, which provides a balance between accurate measurement and the necessary costs of recalibration. The solution of the advanced analytics is the management by means of the precise prediction of the state of health of the device based on its condition, using the diagnostic parameters. These allow a real-time estimation of the error in the measurement.

Contents:

  1. Application type
  2. Data set
  3. Neural network
  4. Training strategy
  5. Model selection
  6. Testing analysis
  7. Model deployment

1. Application type

This is a classification project, since the variable to be predicted is binary (healthy or unhealthy).

The goal here is to model the probability that an ultrasonic flowmeter fails, conditioned on the process characteristics.

2. Data set

The first step is to prepare the data set, which is the source of information for the approximation problem. It is composed of:

The file Meter A

contains 87 instances of 37 diagnostic parameters corresponding to an 8-path liquid ultrasonic flow meter. It has 37 variables or attributes of the flow meters, all continuous except for the state of health, which can be healthy or unhealthy.

The variables of the problem are:

In total, the data set contains 87 instances. From that, 53 instances are used for training (60%), 17 for generalization (20%), and 17 for testing (20%).

Once we have the prepared data we will make a descriptive analysis of the problem to detect the factors to be taken into account in the training to provide a logical and adequate solution to the problem. During the development, we modify the most important aspects, shaping the solution until obtaining the most precise and convenient one.

A resource that provides us with relevant information are the logistic correlations of each variable with the target. The next chart illustrates the dependency of the target Class_attribute with all the input variables.

Flatness ratio (0.574) and some of the gains are the variable most relevant to the target. On the other hand, flow velocity in the paths contribute much less, so they are less determinant when deciding the state of health of the device. At this point we could ask ourselves if the diagnostic parameters of the 8 routes would be useful or if the average would be sufficient. As we can see it is important to keep the data separately, because the logistic correlations can be very different and some contribute much more than others. In addition, none of the correlation is greater than 0.6, so they are all important. In short, you can not disregard the exact data of a path because it can be determinant. Before starting the predictive analysis, it is also essential to know the target distribution instances that we have in the data set.

The chart shows that the number of instances with negative Class_attribute (blue) is 35, and the number of instances with positive Class_attribute (purple) is 52. We will use this information later to design correctly our model.

3. Neural network

The second step is to configure the neural network stuff.

For this classification example, the neural network is composed by:

The number of inputs, in this case, is 36 and the number of outputs is 1. The number of hidden perceptrons or complexity is 3, so this neural network can be denoted as 36:3:1.

The neural network will take all the diagnostic parameters of the flow meter and it will transform them into a probability of health status. The scaling and unscaling layer, which will be respectively found between the input and the hidden layers and between the hidden and the output layers, will both use the minimum-maximum method.

4. Training strategy

The next step is to select an appropriate training strategy, which defines what the neural network will learn. A general training strategy is composed of two concepts:

The data set is slightly unbalanced. Therefore, we use the weighted squared error with L2 regularization as loss index. The weight for the positive and negative weights.

Now, the model is ready to be trained. We use the quasi-Newton method as optimization algorithm.

The following chart shows how the training and selection errors decrease with the epochs during the training process. The final values are training error = 0.001 WSE and selection error = 0.407 WSE, respectively.

XXX

Generally, to study whether during the training process over-fitting has appeared, we also plot the selection loss history. But in our case we do not allocate data to the selection section. This is because the number of instances we have is small and we must take full advantage of them to carry out a better training with more data and to analyze the predictive capacity of our model more effectively. Therefore, to carry out the over-fitting check we have plotted the functions of each of the variables with respect to the target. Here we show the ones that are most relevant in the analysis.

As we can see they are soft approximations, so there is no over-fitting.

5. Model selection

The objective of model selection is to find the network architecture with best generalization properties, that is, that which minimizes the error on the selection instances of the data set.

More specifically, we want to find a neural network with a selection error less than XXX, which is the value that we have achieved so far.

Order selection algorithms train several network architectures with different number of neurons and select that with the smallest selection error.

The incremental order method starts with a small number of neurons and increases the complexity at each iteration. The following chart shows the training error (blue) and the selection error (orange) as a function of the number of neurons.

6. Testing analysis

Testing analysis is in charge of examining the performance of our predictive model. To do this, the data used for testing is used, which corresponds to almost 20% (17 instances). The results of the analysis will indicate the precision of our predictive model and therefore if the health status of a given flowmeter is valid for later. Of the 17 instances destined to testing, we have obtained the following results:

Now we study the ROC curve, which is a graphical illustration of how well the classifier discriminates between the two different classes. This capacity of discrimination is measured by calculating area under curve (AUC).

As a perfect classifier passes through the upper left corner, i.e., the point (0,1), the area under curve for it would be 1. A random classifier, represented by the base line, has an area under curve of 0.5. If the area under curve takes a value lower than 0.5, means that it is worse than randomness. In practice, this measure should take a value between 0.5 and 1.0. The closer to 1 area under curve, the better the classifier.

As we can see our model is very close to the upper left corner, which indicates that it is almost perfect. However, keep in mind that if you have a larger database to train this accuracy may be somewhat lower.

The following table contains the elements of the confusion matrix.

Predicted positive Predicted negative
Real positive 103 0
Real negative 0 171

From the confusion matrix, we can obtain the following binary classification tests:

According to the results our predictive model is quite accurate (96.1%) because the error is 0.038%. The probability of success in positive cases is 100% and in the negative cases it is 91%.

7. Model deployment

Once we know that our model is reliable and accurate, we can use it in new installations. This is called model deployment. The predictive model also provides information on the most significant factors when determining the health status of the device, so knowing the values of the most influential factors we can have a slight idea of the situation. This fact allows us to act fast thus avoiding the inadequate functioning of the system for a long time. On the other hand, we also save important economic resources by predicting that the system works correctly.

The predictive model provides us with a mathematical expression to integrate the function to other meters with the same variables.

References:

Related examples:

Related solutions: