Deployment in machine learning is the process of applying a trained model to new data.
Neural Designer can calculate predictions, explain input influence, optimize responses and export models for use in other applications.
Contents
- Neural network outputs
- Output data
- Directional outputs
- Sample input importances
- Model input importances
- Response optimization
- Mathematical expression
- Programming language expressions
- Deployment package
Neural network outputs
A neural network produces outputs for every set of inputs.
The input and output formats depend on the model type:
| Model | Input | Output |
|---|---|---|
| Approximation and classification | Variable values | Predictions or probabilities |
| Forecasting | Time series values | Future values |
| Image classification | Image file | Predicted class and probabilities |
| Text classification | Sentence | Predicted class and probabilities |
The following table shows the output of a model that estimates a car’s fuel consumption.
| Cylinders | Displacement | Horsepower | Weight | Acceleration | Model year | Fuel consumption |
|---|---|---|---|---|---|---|
| 8 | 307 | 130 | 3504 | 12 | 1980 | 17 mpg |
Output data
This task applies the model to many new cases and exports their outputs.
| Model | Source | Result |
|---|---|---|
| Numerical and forecasting | Data file | File with predictions |
| Image classification | Image folder | Predictions for every image |
| Text classification | Text file | Prediction for every line |
For example, a marketing model can export conversion probabilities for every customer in an input file.
| Recency | Frequency | Monetary | Conversion |
|---|---|---|---|
| 2 months | 5 times | 125 USD | 70% |
| 5 months | 2 times | 20 USD | 8% |
| 3 months | 9 times | 225 USD | 85% |
Directional outputs
Directional outputs show how predictions change when one input varies and all other inputs remain fixed.
They help to understand the model and improve designs or processes.
The following example estimates the residuary resistance of a sailing yacht. The chart varies the Froude number while keeping the other design variables fixed.
| Reference input | Value |
|---|---|
| Center of buoyancy | -2.38 |
| Prismatic coefficient | 0.56 |
| Length displacement | 4.79 |
| Beam draught ratio | 3.94 |
| Length beam ratio | 3.21 |
The residuary resistance increases rapidly for high Froude numbers, while the remaining inputs stay at the reference point.
Sample input importances
Sample input importances explain one prediction by calculating how every output changes with respect to the inputs at a selected sample.
- Positive values increase the output.
- Negative values decrease the output.
- Values close to zero have little local influence.
Model input importances
Model input importances measure the overall influence of every input on the model outputs.
Neural Designer perturbs up to 50 data samples, accumulates the changes and normalizes the results for comparison.
Response optimization
For approximation models, response optimization searches for input values that minimize or maximize selected outputs while satisfying the imposed conditions.
With one objective, it returns the best feasible solution. With several objectives, it returns a Pareto front and highlights an advised trade-off.
Mathematical expression
Numerical models can be written as a mathematical function that maps inputs to outputs.
The expression includes scaling, dense layers, activation functions, unscaling and clamping when those operations are present.
scaled_shear_rate = 2*(shear_rate-50)/(90-50)-1; scaled_particle_diameter = 2*(particle_diameter-0.72)/(6.596-0.72)-1; dense_1_1 = tanh(-1.06007 + 0.448487*scaled_shear_rate - 0.861393*scaled_particle_diameter); dense_1_2 = tanh(0.756922 + 2.00716*scaled_shear_rate + 0.391539*scaled_particle_diameter); scaled_particles_adhering = -1.30536 - 1.0244*dense_1_1 + 0.56055*dense_1_2; particles_adhering = 0.5*(scaled_particles_adhering+1.0)*(74.75-13.22)+13.22;
Image and text models do not have a readable closed-form expression. They use a deployment package instead.
Programming language expressions
Neural Designer exports numerical model expressions to C, Python, JavaScript and PHP.
The generated code reproduces the complete prediction pipeline and can be embedded in another application.
Python expression
def neural_network(inputs):
scaled_inputs = scaling_layer(inputs)
outputs = dense_layer_0(scaled_inputs)
outputs = dense_layer_1(outputs)
outputs = unscaling_layer(outputs)
return clamping_layer(outputs)C expression
void neural_network(const double inputs[], double outputs[])
{
double scaled_inputs[2];
double dense_outputs[3];
scaling_layer(inputs, scaled_inputs);
dense_layer_0(scaled_inputs, dense_outputs);
dense_layer_1(dense_outputs, outputs);
unscaling_layer(outputs, outputs);
clamping_layer(outputs, outputs);
}Deployment package
Image and text classification models can be exported as a complete deployment package.
The package contains everything needed by the model:
| Item | Contents |
|---|---|
| Model | Architecture, trained parameters and class labels |
| Engine | Neural Designer Engine and runtime dependencies |
| Wrapper | C program or Python script |
| Documentation | README and command-line examples |
The exported folder can be copied to another Windows computer and run without installing Neural Designer or a machine learning framework.
Image packages classify an image or a complete folder. Text packages classify a sentence or every line in a file. Both can return the predicted class, probabilities or percentages.
The Python wrapper requires Python 3. The C wrapper must be compiled once with a C compiler.
Inference uses the CPU by default. GPU execution is available when the bundled engine and runtime support CUDA.