Deep learning neural network
Since R2019b
expand all in page
Description
A dlnetwork
object specifies a deep learning neural network architecture
Tip
For most deep learning tasks, you can use a pretrained neural network and adapt it to your own data. For an example showing how to use transfer learning to retrain a convolutional neural network to classify a new set of images, see Retrain Neural Network to Classify New Images. Alternatively, you can create and train neural networks from scratch using the trainnet and trainingOptions functions.
If the trainingOptions function does not provide the training options that you need for your task, then you can create a custom training loop using automatic differentiation. To learn more, see Train Network Using Custom Training Loop.
If the trainnet function does not provide the loss function that you need for your task, then you can specify a custom loss function to the trainnet
as a function handle. For loss functions that require more inputs than the predictions and targets (for example, loss functions that require access to the neural network or additional inputs), train the model using a custom training loop. To learn more, see Train Network Using Custom Training Loop.
If Deep Learning Toolbox™ does not provide the layers you need for your task, then you can create a custom layer. To learn more, see Define Custom Deep Learning Layers. For models that cannot be specified as networks of layers, you can define the model as a function. To learn more, see Train Network Using Model Function.
For more information about which training method to use for which task, see Train Deep Learning Model in MATLAB.
Creation
Syntax
net = dlnetwork
net = dlnetwork(layers)
net = dlnetwork(layers,OutputNames=names)
net = dlnetwork(layers,Initialize=tf)
net = dlnetwork(layers,X1,...,XN)
net = dlnetwork(layers,X1,...,XN,OutputNames=names)
net = dlnetwork(prunableNet)
Description
Empty Network
example
creates a net
= dlnetworkdlnetwork
object with no layers. Use this syntax to create a neural network from scratch. (since R2024a)
Network with Input Layers
example
creates neural network using the specified layers and initializes any unset learnable and state parameters. This syntax uses the input layer in net
= dlnetwork(layers)layers
to determine the size and format of the learnable and state parameters of the neural network.
Use this syntax when layers
defines a complete single-input neural network, has layers arranged in series, and has an input layer.
also sets the OutputNames property. The net
= dlnetwork(layers,OutputNames=names)OutputNames
property specifies the layers or layer outputs that correspond to network outputs.
Use this syntax when layers
defines a complete single-input multi-output neural network, has layers arranged in series, and has an input layer.
specifies whether to initialize the learnable and state parameters of the neural network. When net
= dlnetwork(layers,Initialize=tf)tf
is 1
, (true
), this syntax is equivalent to net = dlnetwork(layers)
. When tf
is 0
(false
), this syntax is equivalent to creating an empty network and then adding layers
using the addLayers
function.
Network With Unconnected Inputs
creates a neural network using the specified layers and initializes any unset learnable and state parameters. This syntax uses the network data layout objects or example inputs net
= dlnetwork(layers,X1,...,XN)X1,...,XN
to determine the size and format of the learnable parameters and state values of the neural network, where N
is the number of network inputs.
Use this syntax when layers
defines a complete neural network, has layers arranged in series, and has inputs that are not connected to input layers.
also sets the OutputNames property.net
= dlnetwork(layers,X1,...,XN,OutputNames=names)
Use this syntax when layers
defines a complete neural network, has multiple outputs, has layers arranged in series, and has inputs that are not connected to input layers.
Prunable Network
converts a net
= dlnetwork(prunableNet) TaylorPrunableNetwork
to dlnetwork
object by removing filters selected for pruning from the convolution layers of prunableNet
and returns a compressed dlnetwork
object that has fewer learnable parameters and is smaller in size.
Input Arguments
expand all
layers
— Network layers
Layer
array
Network layers, specified as a Layer
array.
The software connects the layers in series.
For a list of supported layers, see List of Deep Learning Layers.
X1,...,XN
— Example network inputs or data layouts
formatted dlarray
object | formatted networkDataLayout
object
Example data or data layouts to use to determine the size and formats of learnable and state parameters, specified as formatted dlarray objects or formatted networkDataLayout objects. The software propagates X1,...XN
through the network to determine the appropriate sizes and formats of the learnable and state parameters of the dlnetwork
object and initializes any unset learnable or state parameters.
The order of X1,...,XN
must match the order of the layers that require inputs in layers
.
Note
Automatic initialization uses only the size and format information of the input data. For initialization that depends on the values on the input data, you must initialize the learnable parameters manually.
tf
— Flag to initialize learnable and state parameters
1
(true
) (default) | 0
(false
)
Flag to initialize learnable and state parameters, specified as one of these values:
1
(true
) — Initialize the learnable and state parameters. The software uses the input layer inlayers
to determine the sizes of the learnable and state parameters.0
(false
) — Do not initialize the learnable and state parameters. Use this option when:You expect to make further edits to the neural network. For example, when you expect to add or remove layers and connections.
You use the network in a custom layer and you want to use a custom initialize function.
Neural network prediction and custom training loops requires an initialized network. To initialize an uninitialized network, use the initialize function.
prunableNet
— Network for pruning by using first-order Taylor approximation
TaylorPrunableNetwork
object
Network for pruning by using first-order Taylor approximation, specified as a TaylorPrunableNetwork object.
Pruning a deep neural network requires the Deep Learning Toolbox Model Quantization Library support package. This support package is a free add-on that you can download using the Add-On Explorer. Alternatively, see Deep Learning Toolbox Model Quantization Library.
Properties
expand all
Layers
— Network layers
Layer
array
Network layers, specified as a Layer
array.
Connections
— Layer connections
table
Layer connections, specified as a table with two columns.
Each table row represents a connection in the neural network. The first column, Source
, specifies the source of each connection. The second column, Destination
, specifies the destination of each connection. The connection sources and destinations are either layer names or have the form "layerName/IOName"
, where "IOName"
is the name of the layer input or output.
Data Types: table
Learnables
— Network learnable parameters
table
Network learnable parameters, specified as a table with three columns:
Layer
— Layer name, specified as a string scalar.Parameter
— Parameter name, specified as a string scalar.Value
— Value of parameter, specified as adlarray
object.
The network learnable parameters contain the features learned by the network. For example, the weights of convolution and fully connected layers.
The learnable parameter values can be complex-valued. (since R2024a)
Data Types: table
State
— Network state
table
Network state, specified as a table.
The network state is a table with three columns:
Layer
– Layer name, specified as a string scalar.Parameter
– State parameter name, specified as a string scalar.Value
– Value of state parameter, specified as adlarray
object.
Layer states contain information calculated during the layer operation to be retained for use in subsequent forward passes of the layer. For example, the cell state and hidden state of LSTM layers, or running statistics in batch normalization layers.
For recurrent layers, such as LSTM layers, with the HasStateInputs
property set to 1
(true
), the state table does not contain entries for the states of that layer.
During training or inference, you can update the network state using the output of the forward and predict functions.
The state values can be complex-valued. (since R2024a)
Data Types: table
InputNames
— Names of network inputs
cell array of character vectors
This property is read-only.
Names of the network inputs, specified as a cell array of character vectors.
Network inputs are the input layers and the unconnected inputs of layers.
For input layers and layers with a single input, the input name is the name of the layer. For layers with multiple inputs, the input name is "layerName/inputName"
, where layerName
is the name of the layer and inputName
is the name of the layer input.
Data Types: cell
OutputNames
— Names of network outputs
cell array of character vectors
Names of the network outputs, specified as a cell array of character vectors.
For layers with a single output, the output name is the name of the layer. For layers with multiple outputs, the output name is "layerName/outputName"
, where layerName
is the name of the layer and outputName
is the name of the layer output.
If you do not specify the output names, then the software sets the OutputNames
property to the layers with unconnected outputs.
The predict and forward functions, by default, return the data output by the layers given by the OutputNames
property.
Data Types: cell
Initialized
— Flag for initialized network
0
(false
) | 1
(true
)
This property is read-only.
Flag for initialized network, specified as one of these values:
1
(true
) — Network is initialized and is ready for prediction and custom training loops. If you change the values of the learnable or state parameters, then the network remains initialized.0
(false
) — Network is not initialized and is not ready prediction or custom training loops. To initialize an uninitialized network, use the initialize function.
Data Types: logical
Object Functions
addInputLayer | Add input layer to network |
addLayers | Add layers to neural network |
removeLayers | Remove layers from neural network |
connectLayers | Connect layers in neural network |
disconnectLayers | Disconnect layers in neural network |
replaceLayer | Replace layer in neural network |
getLayer | Look up a layer by name or path |
expandLayers | Expand network layers |
groupLayers | Group layers into network layers |
summary | Print network summary |
plot | Plot neural network architecture |
initialize | Initialize learnable and state parameters of a dlnetwork |
predict | Compute deep learning network output for inference |
forward | Compute deep learning network output for training |
resetState | Reset state parameters of neural network |
setL2Factor | Set L2 regularization factor of layer learnable parameter |
setLearnRateFactor | Set learn rate factor of layer learnable parameter |
getLearnRateFactor | Get learn rate factor of layer learnable parameter |
getL2Factor | Get L2 regularization factor of layer learnable parameter |
Examples
collapse all
Create Neural Network from Scratch
Open Live Script
Define a two-output neural network that predicts both categorical labels and numeric values given 2-D images as input.
Specify the number of classes and responses.
numClasses = 10;numResponses = 1;
Create an empty neural network.
net = dlnetwork;
Define the layers of the main branch of the network and the softmax output.
layers = [ imageInputLayer([28 28 1],Normalization="none") convolution2dLayer(5,16,Padding="same") batchNormalizationLayer reluLayer(Name="relu_1") convolution2dLayer(3,32,Padding="same",Stride=2) batchNormalizationLayer reluLayer convolution2dLayer(3,32,Padding="same") batchNormalizationLayer reluLayer additionLayer(2,Name="add") fullyConnectedLayer(numClasses) softmaxLayer(Name="softmax")];net = addLayers(net,layers);
Add the skip connection.
layers = [ convolution2dLayer(1,32,Stride=2,Name="conv_skip") batchNormalizationLayer reluLayer(Name="relu_skip")];net = addLayers(net,layers);net = connectLayers(net,"relu_1","conv_skip");net = connectLayers(net,"relu_skip","add/in2");
Add the fully connected layer for the regression output.
layers = fullyConnectedLayer(numResponses,Name="fc_2");net = addLayers(net,layers);net = connectLayers(net,"add","fc_2");
View the neural network in a plot.
figureplot(net)
Convert Layer Array to Neural Network
Open Live Script
If you have a layer that defines a complete single-input neural network, has layers arranged in series, and has an input layer, then you can convert the layer array to a dlnetwork
object directly.
Specify an LSTM network as a layer array.
layers = [ sequenceInputLayer(12) lstmLayer(100) fullyConnectedLayer(9) softmaxLayer];
Convert the layer array to a dlnetwork
object. Because the layer array has an input layer and no other inputs, the software initializes the neural network.
net = dlnetwork(layers)
net = dlnetwork with properties: Layers: [4x1 nnet.cnn.layer.Layer] Connections: [3x2 table] Learnables: [5x3 table] State: [2x3 table] InputNames: {'sequenceinput'} OutputNames: {'softmax'} Initialized: 1 View summary with summary.
Freeze Learnable Parameters
Open Live Script
Load a pretrained network.
net = imagePretrainedNetwork;
The Learnables
property of the dlnetwork
object is a table that contains the learnable parameters of the network. The table includes parameters of nested layers in separate rows. View the first few rows of the learnables table.
learnables = net.Learnables;head(learnables)
Layer Parameter Value __________________ _________ ___________________ "conv1" "Weights" {3x3x3x64 dlarray} "conv1" "Bias" {1x1x64 dlarray} "fire2-squeeze1x1" "Weights" {1x1x64x16 dlarray} "fire2-squeeze1x1" "Bias" {1x1x16 dlarray} "fire2-expand1x1" "Weights" {1x1x16x64 dlarray} "fire2-expand1x1" "Bias" {1x1x64 dlarray} "fire2-expand3x3" "Weights" {3x3x16x64 dlarray} "fire2-expand3x3" "Bias" {1x1x64 dlarray}
To freeze the learnable parameters of the network, loop over the learnable parameters and set the learn rate to 0 using the setLearnRateFactor
function.
factor = 0;numLearnables = size(learnables,1);for i = 1:numLearnables layerName = learnables.Layer(i); parameterName = learnables.Parameter(i); net = setLearnRateFactor(net,layerName,parameterName,factor);end
To use the updated learn rate factors when training, you must pass the dlnetwork object to the update function in the custom training loop. For example, use the command
[net,velocity] = sgdmupdate(net,gradients,velocity);
Extended Capabilities
C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.
Usage notes and limitations:
Code generation supports only the
InputNames
andOutputNames
properties.The
Initialized
property of thedlnetwork
object must be1
(true
).Code generation supports tuning the variable
Value
of theState
property. Code generation does not support modifying variablesLayer
andParameter
of theState
property.Code generation supports these functions for the
State
property:height
isempty
istable
istabular
ndims
numel
size
width
For Simulink simulation, code generation does not support extracting and updating the
State
of adlnetwork
in a MATLAB Function Block. Instead, use a Stateful Predict or a Stateful Classify block.Code generation supports multi-input multi-output
dlnetwork
objects with heterogeneous input layers.For recurrent neural networks, multiple inputs are not supported.
For Intel® MKL-DNN, all input layers must be sequence input layers.
For ARM® Compute, the
dlnetwork
can have sequence and non-sequence input layers.
Code generation supports
dlarray
inputs with these limitations for data formats.For generic C/C++ that does not depend on any third-party libraries, the
dlnetwork
can have input layers with any number of spatial dimensions.For ARM Compute and Intel MKL-DNN, code generation supports
dlarray
objects containing zero or two spatial dimensions. For example, code generation supportsVector sequences that have
"CT"
or"CBT"
data formatsImage sequences that have
"SSCT"
or"SSCBT"
data formats
For code generation, only the "T" (time) dimension can be variable-sized, all other dimensions must have fixed sizes.
Code generation supports only the
predict
object function. Thedlarray
input to thepredict
method must be asingle
datatype.To create a
dlnetwork
object for code generation, see Load Pretrained Networks for Code Generation (MATLAB Coder).
GPU Code Generation
Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.
Usage notes and limitations:
Code generation supports only the
InputNames
andOutputNames
properties.For code generation, the
Initialized
property of thedlnetwork
object must betrue
.Code generation supports tuning the variable
Value
of theState
property. Code generation does not support modifying variablesLayer
andParameter
of theState
property.Code generation supports these functions for the
State
property:height
isempty
istable
istabular
ndims
numel
size
width
For Simulink simulation, code generation does not support extracting and updating the
State
of adlnetwork
in a MATLAB Function Block. Instead, use a Stateful Predict or a Stateful Classify block.Code generation supports multi-input multi-output
dlnetwork
objects with heterogeneous input layers. For recurrent neural networks, multiple inputs are not supported.Code generation supports
dlnetwork
objects for cuDNN and TensorRT targets.Code generation supports
dlarray
inputs with these limitations for data formats.For plain CUDA that does not depend on any third-party libraries, the
dlnetwork
can have input layers with any number of spatial dimensions.For cuDNN and TensorRT targets, code generation supports
dlarray
objects containing zero or two spatial dimensions. For example, code generation supportsVector sequences that have
"CT"
or"CBT"
data formatsImage sequences that have
"SSCT"
or"SSCBT"
data formats
For code generation, only the "T" (time) dimension can be variable-sized, all other dimensions must have fixed sizes.
Code generation supports only the
predict
object function. You must passsingle
data typedlarray
input to thepredict
method.When targeting TensorRT with
INT8
precision, the last layer(s) of the network must be asoftmaxLayer
layer.To create a
dlnetwork
object for code generation, see Load Pretrained Networks for Code Generation (GPU Coder).
Version History
Introduced in R2019b
expand all
R2024a: Create empty neural network
Create an empty neural network using the dlnetwork
function with no input arguments. Use empty neural networks as a starting point for building neural networks from scratch.
R2024a: Complex-valued learnables and state
The values in the Learnables and State properties can be complex-valued.
R2024a: Layer graphs are not recommended
Starting in R2024a, LayerGraph objects are not recommended, use dlnetwork objects instead.
This recommendation means that these syntaxes are also not recommend for layer graph input:
net = dlnetwork(lgraph)
net = dlnetwork(lgraph,X1,...,XN)
net = dlnetwork(__,Initialize=tf)
net = dlnetwork(__,OutputNames=names)
To specify a neural network with a graph structure, create a dlnetwork
object and add and connect layers using the addLayers and connectLayers functions, respectively. To update your code, in most cases, you can update your code using these replacements:
Layer Graph Syntax | Replacement |
---|---|
lgraph = layerGraph; | net = dlnetwork; |
lgraph = layerGraph(layers); | net = dlnetwork(layers,Initialize=false); |
Most LayerGraph
object functions such as addLayers and connectLayers also support dlnetwork
and should not need updating.
Depending on the neural network architecture, the resulting dlnetwork
object can be uninitialized. Neural network prediction and custom training loops require initialized neural networks. To initialize an uninitialized neural network, use the initialize function.
Existing code using LayerGraph
objects still runs. You can still convert LayerGraph
objects to dlnetwork
objects by using net = dlnetwork(lgraph)
and specifying the Initialize
or X1,...,XN
arguments if necessary.
R2023b: Initialize networks containing input layers with unset normalization statistics
Input layers such as imageInputLayer and sequenceInputLayer contain properties that networks use for data normalization. These properties are Mean
, StandardDeviation
, Min
, and Max
. The software uses these properties to apply the data normalization method defined by the Normalization
property of the layer.
Starting in R2023b, when you initialize a network by creating an initialized dlnetwork or by using the initialize function, the software initializes the Mean
, StandardDeviation
, Min
, and Max
properties of input layers if you do not set them when you create the layer and if the normalization method requires them. For normalization methods that use two properties, for example, zscore
, the software initializes those properties only if you do not set either property when you create the layer.
For
zerocenter
normalization,Mean
is initialized to0
.For
zscore
normalization,Mean
is initialized to0
andStandardDeviation
is initialized to1
.For
rescale-symmetric
normalization,Min
is initialized to-1
andMax
is initialized to1
.For
rescale-zero-one
normalization,Min
is initialized to0
andMax
is initialized to1
.
By default, the software automatically calculates the normalization statistics during training. To customize the normalization, set the Mean
, StandardDeviation
, Min
, and Max
properties of input layers manually.
In previous releases, the software errors when you initialize a network containing an input layer that uses a normalization method requiring properties that you do not specify when you create the layer.
R2021a: dlnetwork
state values are dlarray
objects
The State of a dlnetwork
object is a table containing the state parameter names and values for each layer in the network.
Starting in R2021a, the state values are dlarray objects. This change enables better support when using AcceleratedFunction objects. To accelerate deep learning functions that have frequently changing input values, for example, an input containing the network state, the frequently changing values must be specified as dlarray
objects.
In previous versions, the state values are numeric arrays.
In most cases, you will not need to update your code. If you have code that requires the state values to be numeric arrays, then to reproduce the previous behavior, extract the data from the state values manually using the extractdata function with the dlupdate function.
state = dlupdate(@extractdata,net.State);
See Also
trainnet | trainingOptions | dlarray | dlgradient | dlfeval | forward | predict | initialize | TaylorPrunableNetwork
Topics
- Retrain Neural Network to Classify New Images
- Train Neural Network with Tabular Data
- Train Network Using Custom Training Loop
- Train Generative Adversarial Network (GAN)
- Define Custom Training Loops, Loss Functions, and Networks
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- Deutsch
- English
- Français
- United Kingdom (English)
Contact your local office