Layer (deep learning)

A layer in a deep learning model is a structure or network topology in the model's architecture, which takes information from the previous layers and then passes it to the next layer.

Layer Types

The first type of layer is the Dense layer, also called the fully-connected layer,^[1]^[2]^[3]^[4] and is used for abstract representations of input data. In this layer, neurons connect to every neuron in the preceding layer. In multilayer perceptron networks, these layers are stacked together.

The Convolutional layer^[5] is typically used for image analysis tasks. In this layer, the network detects edges, textures, and patterns. The outputs from this layer are then fed into a fully-connected layer for further processing. See also: CNN model.

The Pooling layer^[6]^[7] is used to reduce the size of data input.

The Recurrent Layer is used for text processing with a memory function. Similar to the Convolutional layer, the output of recurrent layers are usually fed into a fully-connected layer for further processing. See also: RNN model.^[8]^[9]^[10]

The Normalization layer adjusts the output data from previous layers to achieve a regular distribution. This results in improved scalability and model training.

Differences with layers of the neocortex

There is an intrinsic difference between deep learning layering and neocortical layering: deep learning layering depends on network topology, while neocortical layering depends on intra-layers homogeneity.

References

↑ "CS231n Convolutional Neural Networks for Visual Recognition". 10 May 2016. https://cs231n.github.io/convolutional-networks/. "Fully-connected layer: Neurons in a fully connected layer have full connections to all activations in the previous layer, as seen in regular Neural Networks."
↑ "Convolutional Neural Network. In this article, we will see what are… - by Arc". 26 Dec 2018. https://towardsdatascience.com/convolutional-neural-network-17fb77e76c05. "Fully Connected Layer is simply, feed forward neural networks."
↑ "Fully connected layer". 1 Mar 2021. https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.fullyconnectedlayer.html;jsessionid=138f36fdee803f96d4303a42aaa1. "A fully connected layer multiplies the input by a weight matrix and then adds a bias vector."
↑ Géron, Aurélien (2019). Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow : concepts, tools, and techniques to build intelligent systems. Sebastopol, CA: O'Reilly Media, Inc. pp. 322–323. ISBN 978-1-4920-3264-9. OCLC 1124925244.
↑ Habibi, Aghdam, Hamed (2017-05-30). Guide to convolutional neural networks : a practical application to traffic-sign detection and classification. Heravi, Elnaz Jahani. Cham, Switzerland. ISBN 9783319575490. OCLC 987790957.
↑ Yamaguchi, Kouichi; Sakamoto, Kenji; Akabane, Toshio; Fujimoto, Yoshiji (November 1990). "A Neural Network for Speaker-Independent Isolated Word Recognition". First International Conference on Spoken Language Processing (ICSLP 90). Kobe, Japan. https://www.isca-speech.org/archive/icslp_1990/i90_1077.html.
↑ Ciresan, Dan; Meier, Ueli; Schmidhuber, Jürgen (June 2012). "Multi-column deep neural networks for image classification". 2012 IEEE Conference on Computer Vision and Pattern Recognition. New York, NY: Institute of Electrical and Electronics Engineers (IEEE). pp. 3642–3649. doi:10.1109/CVPR.2012.6248110. ISBN 978-1-4673-1226-4. OCLC 812295155.
↑ Dupond, Samuel (2019). "A thorough review on the current advance of neural network structures.". Annual Reviews in Control 14: 200–230. https://www.sciencedirect.com/journal/annual-reviews-in-control.
↑ Abiodun, Oludare Isaac; Jantan, Aman; Omolara, Abiodun Esther; Dada, Kemi Victoria; Mohamed, Nachaat Abdelatif; Arshad, Humaira (2018-11-01). "State-of-the-art in artificial neural network applications: A survey" (in en). Heliyon 4 (11): e00938. doi:10.1016/j.heliyon.2018.e00938. ISSN 2405-8440. PMID 30519653. Bibcode: 2018Heliy...400938A.
↑ Tealab, Ahmed (2018-12-01). "Time series forecasting using artificial neural networks methodologies: A systematic review" (in en). Future Computing and Informatics Journal 3 (2): 334–340. doi:10.1016/j.fcij.2018.10.003. ISSN 2314-7288.

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Layer (deep learning). Read more

[CS231n_Convolutional_Neural_Networks_for_Visual_Recognition_2016-1] "CS231n Convolutional Neural Networks for Visual Recognition". 10 May 2016. https://cs231n.github.io/convolutional-networks/. "Fully-connected layer: Neurons in a fully connected layer have full connections to all activations in the previous layer, as seen in regular Neural Networks."

[Medium_2018-2] "Convolutional Neural Network. In this article, we will see what are… - by Arc". 26 Dec 2018. https://towardsdatascience.com/convolutional-neural-network-17fb77e76c05. "Fully Connected Layer is simply, feed forward neural networks."

[MATLAB_2021-3] "Fully connected layer". 1 Mar 2021. https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.fullyconnectedlayer.html;jsessionid=138f36fdee803f96d4303a42aaa1. "A fully connected layer multiplies the input by a weight matrix and then adds a bias vector."

[Géron_2019_p.-4] Géron, Aurélien (2019). Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow : concepts, tools, and techniques to build intelligent systems. Sebastopol, CA: O'Reilly Media, Inc. pp. 322–323. ISBN 978-1-4920-3264-9. OCLC 1124925244.

[5] Habibi, Aghdam, Hamed (2017-05-30). Guide to convolutional neural networks : a practical application to traffic-sign detection and classification. Heravi, Elnaz Jahani. Cham, Switzerland. ISBN 9783319575490. OCLC 987790957.

[Yamaguchi111990-6] Yamaguchi, Kouichi; Sakamoto, Kenji; Akabane, Toshio; Fujimoto, Yoshiji (November 1990). "A Neural Network for Speaker-Independent Isolated Word Recognition". First International Conference on Spoken Language Processing (ICSLP 90). Kobe, Japan. https://www.isca-speech.org/archive/icslp_1990/i90_1077.html.

[mcdns-7] Ciresan, Dan; Meier, Ueli; Schmidhuber, Jürgen (June 2012). "Multi-column deep neural networks for image classification". 2012 IEEE Conference on Computer Vision and Pattern Recognition. New York, NY: Institute of Electrical and Electronics Engineers (IEEE). pp. 3642–3649. doi:10.1109/CVPR.2012.6248110. ISBN 978-1-4673-1226-4. OCLC 812295155.

[8] Dupond, Samuel (2019). "A thorough review on the current advance of neural network structures.". Annual Reviews in Control 14: 200–230. https://www.sciencedirect.com/journal/annual-reviews-in-control.

[9] Abiodun, Oludare Isaac; Jantan, Aman; Omolara, Abiodun Esther; Dada, Kemi Victoria; Mohamed, Nachaat Abdelatif; Arshad, Humaira (2018-11-01). "State-of-the-art in artificial neural network applications: A survey" (in en). Heliyon 4 (11): e00938. doi:10.1016/j.heliyon.2018.e00938. ISSN 2405-8440. PMID 30519653. Bibcode: 2018Heliy...400938A.

[10] Tealab, Ahmed (2018-12-01). "Time series forecasting using artificial neural networks methodologies: A systematic review" (in en). Future Computing and Informatics Journal 3 (2): 334–340. doi:10.1016/j.fcij.2018.10.003. ISSN 2314-7288.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]