Recognize human activity from smartphone signals using Neural Designer

This example aims to identify human activity from data collected by a smartphone.

For instance, activity recognition allows the development of health apps that track users' behavior and provide recommendations.

The experiments have been carried out with 30 volunteers within an age bracket of 19-48 years. Each person performed six activities (walking, walking_upstairs, walking_downstairs, sitting, standing, laying) wearing a smartphone on the waist.

Using its embedded accelerometer and gyroscope, we captured 3-axial linear acceleration and 3-axial angular velocity at a constant rate of 50Hz. The experiments have been video-recorded to label the data manually.

Contents:

  1. Application type.
  2. Data set.
  3. Neural network.
  4. Training strategy.
  5. Model selection.
  6. Testing analysis.
  7. Model deployment.

This example is solved with Neural Designer. To follow it step by step, you can use the free trial.

1. Application type

This is a classification project, since the variable to be predicted is categorical (walking, walking_upstairs, walking_downstairs, sitting, standing, or laying).

The goal here is to model the probabilities of class membership, conditioned on the input variables.

2. Data set

The file activity_recognition.csv contains 10299 samples, each of them with 561 inputs and one categorical target.

The input variables include time and frequency domain signals obtained from the smartphone sensors:

The target variable has six different classes, each corresponding to one of the previously mentioned activities.

The instances are split at random into training (60%), selection (20%), and testing (20%) subsets.

By calculating the data distribution, we can see the number of instances belonging to each class in the data set.

As we can see, the number of instances belonging to each category is similar. Therefore this data set is well balanced.

We can also calculate the inputs-targets correlations to see which signals better define each activity. The following chart shows the 20 variables most correlated with the activity "standing".

3. Neural network

The person's activity model will be a neural network composed of:

The scaling layer uses the minimum and maximum scaling method.

The number of perceptron layers is 2:

The probabilistic layer uses the softmax probabilistic method.

4. Training strategy

The procedure used to carry out the learning process is called a training strategy. The training strategy is applied to the neural network to obtain the best possible performance. The type of training is determined by how the adjustment of the parameters in the neural network takes place.

We set the normalized squared error with L2 regularization as the loss index.

On the other hand, we use the quasi-Newton method as optimization algorithm.

The following chart shows how the training and selection errors decrease with the quasi-Newton method's epochs during the training process.

As we can see, both curves' behavior is similar along with the iterations, which means that no over-fitting has appeared. The final training and selection errors are training error = 0.008 NSE and selection error = 0.048 NSE. That indicates that the neural network has good generalization capabilities.

5. Model selection

The objective of model selection is to find the network architecture with the best generalization properties.

Since the final selection error that we have so far is minimal (0.048 NSE), there is no need to use these kinds of algorithms here.

6. Testing analysis

Once the model is trained, we perform a testing analysis to validate its prediction capacity. We use a subset of data that has not been used before, the testing instances.

The next table shows the confusion matrix for our problem. In the confusion matrix, the rows represent the real classes and the predicted classes' columns for the testing data.

Predicted STANDING Predicted SITTING Predicted LAYING Predicted WALKING Predicted WALKING_DOWNSTAIRS Predicted WALKING_UPSTAIRS
Real STANDING 376 (18.3%) 18 (0.874%) 0 0 0 0
Real SITTING 18 (0.874%) 330 (16%) 1 (0.0486%) 0 0 0
Real LAYING 0 2 (0.0971%) 402 (19.5%) 0 0 0
Real WALKING 1 (0.0486%) 0 0 307 (14.9%) 0 0
Real WALKING_DOWNSTAIRS 0 0 0 0 279 (13.6%) 1 (0.0486%)
Real WALKING_UPSTAIRS 1 (0.0486%) 0 0 0 5 (0.243%) 318 (15.4%)

As we can see, the number of instances that the model can correctly predict is 2012 while it misclassifies only 47. This shows that our predictive model has a great classification accuracy.

7. Model deployment

The neural network is now ready to predict the activity of new people in the so-called model deployment phase.

The file activity_recognition.py implements the mathematical expression of the neural network in Python. This piece of software can be embedded in any tool to make predictions on new data.

References

Related solutions: