Deep learning neural network
Since R2019b
expand all in page
Description
A dlnetwork object specifies a deep learning neural network architecture
Tip
For most deep learning tasks, you can use a pretrained neural network and adapt it to your own data. For an example showing how to use transfer learning to retrain a convolutional neural network to classify a new set of images, see Retrain Neural Network to Classify New Images. Alternatively, you can create and train neural networks from scratch using the trainnet and trainingOptions functions.
If the trainingOptions function does not provide the training options that you need for your task, then you can create a custom training loop using automatic differentiation. To learn more, see Train Network Using Custom Training Loop.
If the trainnet function does not provide the loss function that you need for your task, then you can specify a custom loss function to the trainnet as a function handle. For loss functions that require more inputs than the predictions and targets (for example, loss functions that require access to the neural network or additional inputs), train the model using a custom training loop. To learn more, see Train Network Using Custom Training Loop.
If Deep Learning Toolbox™ does not provide the layers you need for your task, then you can create a custom layer. To learn more, see Define Custom Deep Learning Layers. For models that cannot be specified as networks of layers, you can define the model as a function. To learn more, see Train Network Using Model Function.
For more information about which training method to use for which task, see Train Deep Learning Model in MATLAB.
Creation
Syntax
net = dlnetwork
net = dlnetwork(layers)
net = dlnetwork(layers,OutputNames=names)
net = dlnetwork(layers,Initialize=tf)
net = dlnetwork(layers,X1,...,XN)
net = dlnetwork(layers,X1,...,XN,OutputNames=names)
net = dlnetwork(prunableNet)
Description
Empty Network
example
net = dlnetworkdlnetwork object with no layers. Use this syntax to create a neural network from scratch. (since R2024a)
Network with Input Layers
example
net = dlnetwork(layers)layers to determine the size and format of the learnable and state parameters of the neural network.
Use this syntax when layers defines a complete single-input neural network, has layers arranged in series, and has an input layer.
net = dlnetwork(layers,OutputNames=names)OutputNames property specifies the layers or layer outputs that correspond to network outputs.
Use this syntax when layers defines a complete single-input multi-output neural network, has layers arranged in series, and has an input layer.
net = dlnetwork(layers,Initialize=tf)tf is 1, (true), this syntax is equivalent to net = dlnetwork(layers). When tf is 0 (false), this syntax is equivalent to creating an empty network and then adding layers using the addLayers function.
Network With Unconnected Inputs
net = dlnetwork(layers,X1,...,XN)X1,...,XN to determine the size and format of the learnable parameters and state values of the neural network, where N is the number of network inputs.
Use this syntax when layers defines a complete neural network, has layers arranged in series, and has inputs that are not connected to input layers.
net = dlnetwork(layers,X1,...,XN,OutputNames=names)
Use this syntax when layers defines a complete neural network, has multiple outputs, has layers arranged in series, and has inputs that are not connected to input layers.
Prunable Network
net = dlnetwork(prunableNet) TaylorPrunableNetwork to dlnetwork object by removing filters selected for pruning from the convolution layers of prunableNet and returns a compressed dlnetwork object that has fewer learnable parameters and is smaller in size.
Input Arguments
expand all
layers — Network layers
Layer array
Network layers, specified as a Layer array.
The software connects the layers in series.
For a list of supported layers, see List of Deep Learning Layers.
X1,...,XN — Example network inputs or data layouts
formatted dlarray object | formatted networkDataLayout object
Example data or data layouts to use to determine the size and formats of learnable and state parameters, specified as formatted dlarray objects or formatted networkDataLayout objects. The software propagates X1,...XN through the network to determine the appropriate sizes and formats of the learnable and state parameters of the dlnetwork object and initializes any unset learnable or state parameters.
The order of X1,...,XN must match the order of the layers that require inputs in layers.
Note
Automatic initialization uses only the size and format information of the input data. For initialization that depends on the values on the input data, you must initialize the learnable parameters manually.
tf — Flag to initialize learnable and state parameters
1 (true) (default) | 0 (false)
Flag to initialize learnable and state parameters, specified as one of these values:
1(true) — Initialize the learnable and state parameters. The software uses the input layer inlayersto determine the sizes of the learnable and state parameters.0(false) — Do not initialize the learnable and state parameters. Use this option when:You expect to make further edits to the neural network. For example, when you expect to add or remove layers and connections.
You use the network in a custom layer and you want to use a custom initialize function.
Neural network prediction and custom training loops requires an initialized network. To initialize an uninitialized network, use the initialize function.
prunableNet — Network for pruning by using first-order Taylor approximation
TaylorPrunableNetwork object
Network for pruning by using first-order Taylor approximation, specified as a TaylorPrunableNetwork object.
Pruning a deep neural network requires the Deep Learning Toolbox Model Quantization Library support package. This support package is a free add-on that you can download using the Add-On Explorer. Alternatively, see Deep Learning Toolbox Model Quantization Library.
Properties
expand all
Layers — Network layers
Layer array
Network layers, specified as a Layer array.
Connections — Layer connections
table
Layer connections, specified as a table with two columns.
Each table row represents a connection in the neural network. The first column, Source, specifies the source of each connection. The second column, Destination, specifies the destination of each connection. The connection sources and destinations are either layer names or have the form "layerName/IOName", where "IOName" is the name of the layer input or output.
Data Types: table
Learnables — Network learnable parameters
table
Network learnable parameters, specified as a table with three columns:
Layer— Layer name, specified as a string scalar.Parameter— Parameter name, specified as a string scalar.Value— Value of parameter, specified as adlarrayobject.
The network learnable parameters contain the features learned by the network. For example, the weights of convolution and fully connected layers.
The learnable parameter values can be complex-valued. (since R2024a)
Data Types: table
State — Network state
table
Network state, specified as a table.
The network state is a table with three columns:
Layer– Layer name, specified as a string scalar.Parameter– State parameter name, specified as a string scalar.Value– Value of state parameter, specified as adlarrayobject.
Layer states contain information calculated during the layer operation to be retained for use in subsequent forward passes of the layer. For example, the cell state and hidden state of LSTM layers, or running statistics in batch normalization layers.
For recurrent layers, such as LSTM layers, with the HasStateInputs property set to 1 (true), the state table does not contain entries for the states of that layer.
During training or inference, you can update the network state using the output of the forward and predict functions.
The state values can be complex-valued. (since R2024a)
Data Types: table
InputNames — Names of network inputs
cell array of character vectors
This property is read-only.
Names of the network inputs, specified as a cell array of character vectors.
Network inputs are the input layers and the unconnected inputs of layers.
For input layers and layers with a single input, the input name is the name of the layer. For layers with multiple inputs, the input name is "layerName/inputName", where layerName is the name of the layer and inputName is the name of the layer input.
Data Types: cell
OutputNames — Names of network outputs
cell array of character vectors
Names of the network outputs, specified as a cell array of character vectors.
For layers with a single output, the output name is the name of the layer. For layers with multiple outputs, the output name is "layerName/outputName", where layerName is the name of the layer and outputName is the name of the layer output.
If you do not specify the output names, then the software sets the OutputNames property to the layers with unconnected outputs.
The predict and forward functions, by default, return the data output by the layers given by the OutputNames property.
Data Types: cell
Initialized — Flag for initialized network
0 (false) | 1 (true)
This property is read-only.
Flag for initialized network, specified as one of these values:
1(true) — Network is initialized and is ready for prediction and custom training loops. If you change the values of the learnable or state parameters, then the network remains initialized.0(false) — Network is not initialized and is not ready prediction or custom training loops. To initialize an uninitialized network, use the initialize function.
Data Types: logical
Object Functions
| addInputLayer | Add input layer to network |
| addLayers | Add layers to neural network |
| removeLayers | Remove layers from neural network |
| connectLayers | Connect layers in neural network |
| disconnectLayers | Disconnect layers in neural network |
| replaceLayer | Replace layer in neural network |
| getLayer | Look up a layer by name or path |
| expandLayers | Expand network layers |
| groupLayers | Group layers into network layers |
| summary | Print network summary |
| plot | Plot neural network architecture |
| initialize | Initialize learnable and state parameters of a dlnetwork |
| predict | Compute deep learning network output for inference |
| forward | Compute deep learning network output for training |
| resetState | Reset state parameters of neural network |
| setL2Factor | Set L2 regularization factor of layer learnable parameter |
| setLearnRateFactor | Set learn rate factor of layer learnable parameter |
| getLearnRateFactor | Get learn rate factor of layer learnable parameter |
| getL2Factor | Get L2 regularization factor of layer learnable parameter |
Examples
collapse all
Create Neural Network from Scratch
Open Live Script
Define a two-output neural network that predicts both categorical labels and numeric values given 2-D images as input.
Specify the number of classes and responses.
numClasses = 10;numResponses = 1;
Create an empty neural network.
net = dlnetwork;
Define the layers of the main branch of the network and the softmax output.
layers = [ imageInputLayer([28 28 1],Normalization="none") convolution2dLayer(5,16,Padding="same") batchNormalizationLayer reluLayer(Name="relu_1") convolution2dLayer(3,32,Padding="same",Stride=2) batchNormalizationLayer reluLayer convolution2dLayer(3,32,Padding="same") batchNormalizationLayer reluLayer additionLayer(2,Name="add") fullyConnectedLayer(numClasses) softmaxLayer(Name="softmax")];net = addLayers(net,layers);
Add the skip connection.
layers = [ convolution2dLayer(1,32,Stride=2,Name="conv_skip") batchNormalizationLayer reluLayer(Name="relu_skip")];net = addLayers(net,layers);net = connectLayers(net,"relu_1","conv_skip");net = connectLayers(net,"relu_skip","add/in2");
Add the fully connected layer for the regression output.
layers = fullyConnectedLayer(numResponses,Name="fc_2");net = addLayers(net,layers);net = connectLayers(net,"add","fc_2");
View the neural network in a plot.
figureplot(net)
Convert Layer Array to Neural Network
Open Live Script
If you have a layer that defines a complete single-input neural network, has layers arranged in series, and has an input layer, then you can convert the layer array to a dlnetwork object directly.
Specify an LSTM network as a layer array.
layers = [ sequenceInputLayer(12) lstmLayer(100) fullyConnectedLayer(9) softmaxLayer];
Convert the layer array to a dlnetwork object. Because the layer array has an input layer and no other inputs, the software initializes the neural network.
net = dlnetwork(layers)
net = dlnetwork with properties: Layers: [4x1 nnet.cnn.layer.Layer] Connections: [3x2 table] Learnables: [5x3 table] State: [2x3 table] InputNames: {'sequenceinput'} OutputNames: {'softmax'} Initialized: 1 View summary with summary.Freeze Learnable Parameters
Open Live Script
Load a pretrained network.
net = imagePretrainedNetwork;
The Learnables property of the dlnetwork object is a table that contains the learnable parameters of the network. The table includes parameters of nested layers in separate rows. View the first few rows of the learnables table.
learnables = net.Learnables;head(learnables)
Layer Parameter Value __________________ _________ ___________________ "conv1" "Weights" {3x3x3x64 dlarray} "conv1" "Bias" {1x1x64 dlarray} "fire2-squeeze1x1" "Weights" {1x1x64x16 dlarray} "fire2-squeeze1x1" "Bias" {1x1x16 dlarray} "fire2-expand1x1" "Weights" {1x1x16x64 dlarray} "fire2-expand1x1" "Bias" {1x1x64 dlarray} "fire2-expand3x3" "Weights" {3x3x16x64 dlarray} "fire2-expand3x3" "Bias" {1x1x64 dlarray}To freeze the learnable parameters of the network, loop over the learnable parameters and set the learn rate to 0 using the setLearnRateFactor function.
factor = 0;numLearnables = size(learnables,1);for i = 1:numLearnables layerName = learnables.Layer(i); parameterName = learnables.Parameter(i); net = setLearnRateFactor(net,layerName,parameterName,factor);end
To use the updated learn rate factors when training, you must pass the dlnetwork object to the update function in the custom training loop. For example, use the command
[net,velocity] = sgdmupdate(net,gradients,velocity);
Extended Capabilities
C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.
Usage notes and limitations:
Code generation supports only the
InputNamesandOutputNamesproperties.The
Initializedproperty of thedlnetworkobject must be1(true).Code generation supports tuning the variable
Valueof theStateproperty. Code generation does not support modifying variablesLayerandParameterof theStateproperty.Code generation supports these functions for the
Stateproperty:height
isempty
istable
istabular
ndims
numel
size
width
For Simulink simulation, code generation does not support extracting and updating the
Stateof adlnetworkin a MATLAB Function Block. Instead, use a Stateful Predict or a Stateful Classify block.Code generation supports multi-input multi-output
dlnetworkobjects with heterogeneous input layers.For recurrent neural networks, multiple inputs are not supported.
For Intel® MKL-DNN, all input layers must be sequence input layers.
For ARM® Compute, the
dlnetworkcan have sequence and non-sequence input layers.
Code generation supports
dlarrayinputs with these limitations for data formats.For generic C/C++ that does not depend on any third-party libraries, the
dlnetworkcan have input layers with any number of spatial dimensions.For ARM Compute and Intel MKL-DNN, code generation supports
dlarrayobjects containing zero or two spatial dimensions. For example, code generation supportsVector sequences that have
"CT"or"CBT"data formatsImage sequences that have
"SSCT"or"SSCBT"data formats
For code generation, only the "T" (time) dimension can be variable-sized, all other dimensions must have fixed sizes.
Code generation supports only the
predictobject function. Thedlarrayinput to thepredictmethod must be asingledatatype.To create a
dlnetworkobject for code generation, see Load Pretrained Networks for Code Generation (MATLAB Coder).
GPU Code Generation
Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.
Usage notes and limitations:
Code generation supports only the
InputNamesandOutputNamesproperties.For code generation, the
Initializedproperty of thedlnetworkobject must betrue.Code generation supports tuning the variable
Valueof theStateproperty. Code generation does not support modifying variablesLayerandParameterof theStateproperty.Code generation supports these functions for the
Stateproperty:height
isempty
istable
istabular
ndims
numel
size
width
For Simulink simulation, code generation does not support extracting and updating the
Stateof adlnetworkin a MATLAB Function Block. Instead, use a Stateful Predict or a Stateful Classify block.Code generation supports multi-input multi-output
dlnetworkobjects with heterogeneous input layers. For recurrent neural networks, multiple inputs are not supported.Code generation supports
dlnetworkobjects for cuDNN and TensorRT targets.Code generation supports
dlarrayinputs with these limitations for data formats.For plain CUDA that does not depend on any third-party libraries, the
dlnetworkcan have input layers with any number of spatial dimensions.For cuDNN and TensorRT targets, code generation supports
dlarrayobjects containing zero or two spatial dimensions. For example, code generation supportsVector sequences that have
"CT"or"CBT"data formatsImage sequences that have
"SSCT"or"SSCBT"data formats
For code generation, only the "T" (time) dimension can be variable-sized, all other dimensions must have fixed sizes.
Code generation supports only the
predictobject function. You must passsingledata typedlarrayinput to thepredictmethod.When targeting TensorRT with
INT8precision, the last layer(s) of the network must be asoftmaxLayerlayer.To create a
dlnetworkobject for code generation, see Load Pretrained Networks for Code Generation (GPU Coder).
Version History
Introduced in R2019b
expand all
R2024a: Create empty neural network
Create an empty neural network using the dlnetwork function with no input arguments. Use empty neural networks as a starting point for building neural networks from scratch.
R2024a: Complex-valued learnables and state
The values in the Learnables and State properties can be complex-valued.
R2024a: Layer graphs are not recommended
Starting in R2024a, LayerGraph objects are not recommended, use dlnetwork objects instead.
This recommendation means that these syntaxes are also not recommend for layer graph input:
net = dlnetwork(lgraph)net = dlnetwork(lgraph,X1,...,XN)net = dlnetwork(__,Initialize=tf)net = dlnetwork(__,OutputNames=names)
To specify a neural network with a graph structure, create a dlnetwork object and add and connect layers using the addLayers and connectLayers functions, respectively. To update your code, in most cases, you can update your code using these replacements:
| Layer Graph Syntax | Replacement |
|---|---|
lgraph = layerGraph; | net = dlnetwork; |
lgraph = layerGraph(layers); | net = dlnetwork(layers,Initialize=false); |
Most LayerGraph object functions such as addLayers and connectLayers also support dlnetwork and should not need updating.
Depending on the neural network architecture, the resulting dlnetwork object can be uninitialized. Neural network prediction and custom training loops require initialized neural networks. To initialize an uninitialized neural network, use the initialize function.
Existing code using LayerGraph objects still runs. You can still convert LayerGraph objects to dlnetwork objects by using net = dlnetwork(lgraph) and specifying the Initialize or X1,...,XN arguments if necessary.
R2023b: Initialize networks containing input layers with unset normalization statistics
Input layers such as imageInputLayer and sequenceInputLayer contain properties that networks use for data normalization. These properties are Mean, StandardDeviation, Min, and Max. The software uses these properties to apply the data normalization method defined by the Normalization property of the layer.
Starting in R2023b, when you initialize a network by creating an initialized dlnetwork or by using the initialize function, the software initializes the Mean, StandardDeviation, Min, and Max properties of input layers if you do not set them when you create the layer and if the normalization method requires them. For normalization methods that use two properties, for example, zscore, the software initializes those properties only if you do not set either property when you create the layer.
For
zerocenternormalization,Meanis initialized to0.For
zscorenormalization,Meanis initialized to0andStandardDeviationis initialized to1.For
rescale-symmetricnormalization,Minis initialized to-1andMaxis initialized to1.For
rescale-zero-onenormalization,Minis initialized to0andMaxis initialized to1.
By default, the software automatically calculates the normalization statistics during training. To customize the normalization, set the Mean, StandardDeviation, Min, and Max properties of input layers manually.
In previous releases, the software errors when you initialize a network containing an input layer that uses a normalization method requiring properties that you do not specify when you create the layer.
R2021a: dlnetwork state values are dlarray objects
The State of a dlnetwork object is a table containing the state parameter names and values for each layer in the network.
Starting in R2021a, the state values are dlarray objects. This change enables better support when using AcceleratedFunction objects. To accelerate deep learning functions that have frequently changing input values, for example, an input containing the network state, the frequently changing values must be specified as dlarray objects.
In previous versions, the state values are numeric arrays.
In most cases, you will not need to update your code. If you have code that requires the state values to be numeric arrays, then to reproduce the previous behavior, extract the data from the state values manually using the extractdata function with the dlupdate function.
state = dlupdate(@extractdata,net.State);
See Also
trainnet | trainingOptions | dlarray | dlgradient | dlfeval | forward | predict | initialize | TaylorPrunableNetwork
Topics
- Retrain Neural Network to Classify New Images
- Train Neural Network with Tabular Data
- Train Network Using Custom Training Loop
- Train Generative Adversarial Network (GAN)
- Define Custom Training Loops, Loss Functions, and Networks
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- Deutsch
- English
- Français
- United Kingdom (English)
Contact your local office
