Although all those frameworks are based on neural networks, they present essential differences in functionality, usability, performance, consumption, etc.
This post compares the energy consumption of TensorFlow and Neural Designer using the GPU for an approximation benchmark.
As we will see, Neural Designer consumes 42% less than its competitor machine learning platform.
In this article, we provide all the steps you need to reproduce the result using the free trial of Neural Designer.
Two of the most essential features of machine learning platforms are their training speed and the total amount of energy consumed during this process.
In most cases, modeling huge data sets is very expensive in computational terms, which leads to a high economic cost of neural network training and a high environmental impact.
Thus, this article aims to measure the GPU energy consumption of TensorFlow and Neural Designer for a benchmark application. Also, a couple of instructions are given to enable anyone to repeat this one or a similar benchmark and check on their own the fantastic results obtained when Neural Designer is used.
The following table summarizes the technical features of these tools that might impact their GPU performance.
|Written in||C++, CUDA, Python||C++, CUDA|
|Interface||Python||Graphical User Interface|
The above table shows that TensorFlow is programmed in C++ and Python, whereas Neural Designer is entirely programmed in C++.
Interpreted languages like Python have advantages over compiled languages like C ++, such as their ease of use.
However, the performance of Python is generally lower than that of C++. Indeed, Python takes significant time to interpret sentences during the program’s execution.
On the other hand, TensorFlow uses automatic differentiation, while Neural Designer uses analytical differentiation.
As before, automatic differentiation has some advantages over analytical differentiation. Indeed, it simplifies obtaining the gradient for new architectures or loss indices.
However, the performance of automatic differentiation is, in general, lower than that of analytical differentiation:
The first derives the gradient during the program’s execution, while the second has that formula pre-calculated.
Next, we use TensorFlow and Neural Designer to measure the energy consumption for a benchmark problem on a reference computer. The results produced by these platforms are then compared.
The first step is to choose a benchmark application that is general enough to conclude the performance of the machine learning platforms. As previously stated, we will train a neural network that approximates a set of input-target samples.
In this regard, an approximation application is defined by a data set, a neural network, and an associated training strategy.
The following table uniquely defines these three components.
Once the TensorFlow and Neural Designer applications have been created, we must run them.
The next step is to choose the computer in which the neural network will be trained with TensorFlow and Neural Designer.
The table below shows the features of the computer used for this instance.
|Operating system:||Windows 11 Home 64-bit|
|Processor:||Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz, 3192 Mhz, 6 Core(s), 12 Logical Processor(s)|
|Physical RAM:||31.9 GB|
|Device (GPU):||NVIDIA GeForce GTX 1050 Ti|
Once the computer has been selected, we install TensorFlow (2.1.0) and Neural Designer (5.9.9) on it.
Below, the TensorFlow code used is shown.
Reference electricity consumption meter
This section describes the device used for the energy consumption measurements so that the reader can reproduce the results obtained in the following section with maximum accuracy.
|Device model||Perel E305EM5-G|
The last step is to run the benchmark application on the selected machine with TensorFlow and Neural Designer and compare the energy consumed by those platforms during training.
The following figure shows the training time with TensorFlow.
As we can see, TensorFlow takes 30:14:30 to train the neural network for 20000 epochs (5.44 seconds/epoch).
The final mean squared error is 0.0003. The overall energy consumption of the training process is 4.5 kWh, as shown below.
Finally, the following figure shows the training time with Neural Designer.
Neural Designer takes 21:03:43 to train the neural network for 20000 epochs (3.79 seconds/epoch). During that time, it reaches a mean squared error of 0.023. The overall energy consumption of the training process is 2.6 kWh, as shown below.
The following table summarizes the the most important metrics the two machine learning platforms yielded.
|Epoch time||5.44 seconds/epoch||3.79 seconds/epoch|
|Training speed||183,824 samples/second||263.852 samples/second|
|Total energy consumed||4.5kWh||2.6 kWh|
The following chart depicts the energy consumed using TensorFlow and Neural Designer graphically in this case.
Electric energy consumption
As we can see, the energy consumption of Neural Designer for this application is 42 % lower than that of TensorFlow.
Neural Designer is entirely written in C ++, uses analytical differentiation, and has been optimized to minimize the number of operations during training.
As a result, its energy consumption during the training process using Neural Designer is 42 % lower than that using TensorFlow.
To reproduce these results, download the free trial of Neural Designer and follow the steps described in this article.