Delta rule and backpropagation

Leonardo Vanneschi; Mauro Castelli

doi:10.1016/B978-0-12-809633-8.20340-3

Delta rule and backpropagation

Leonardo Vanneschi, Mauro Castelli

Research output: Chapter in Book/Report/Conference proceeding › Entry for encyclopedia/dictionary › peer-review

5 Citations (Scopus)

Abstract

Assuming that the reader is already familiar with the general concept of Artificial Neural Network and with the Perceptron learning rule, this paper introduces the Delta learning rule, as a basis for the Backpropagation learning rule. After discussing the necessity of using multi-layer Artificial Neural Networks for solving non-linearly separable problems, the paper describes all the mathematical steps that allow us to pass from the simple gradient descent formulation to the Backpropagation algorithm, still nowadays one of the most used methods to train feed-forward multi-layer Artificial Neural Networks. The paper is concluded by discussing issues related to overfitting in feed-forward multi-layer Artificial Neural Networks, and by presenting some heuristics and ideas for an appropriate parameter setting.

Original language	English
Title of host publication	Encyclopedia of Bioinformatics and Computational Biology
Subtitle of host publication	ABC of Bioinformatics
Editors	Shoba Ranganathan, Michael Gribskov, Kenta Nakai, Christian Schönbach
Publisher	Elsevier
Pages	621-633
Number of pages	13
Volume	1-3
ISBN (Electronic)	9780128114322
ISBN (Print)	9780128114148
DOIs	https://doi.org/10.1016/B978-0-12-809633-8.20340-3
Publication status	Published - 2019

Keywords

Artificial neural networks
Backpropagation learning rule
Delta learning rule
Multi-layer neural networks
Non-linearly separable problems

Access to Document

10.1016/B978-0-12-809633-8.20340-3

Cite this

@inbook{ff10ea166f1e457c8766a869ff63565b,

title = "Delta rule and backpropagation",

abstract = "Assuming that the reader is already familiar with the general concept of Artificial Neural Network and with the Perceptron learning rule, this paper introduces the Delta learning rule, as a basis for the Backpropagation learning rule. After discussing the necessity of using multi-layer Artificial Neural Networks for solving non-linearly separable problems, the paper describes all the mathematical steps that allow us to pass from the simple gradient descent formulation to the Backpropagation algorithm, still nowadays one of the most used methods to train feed-forward multi-layer Artificial Neural Networks. The paper is concluded by discussing issues related to overfitting in feed-forward multi-layer Artificial Neural Networks, and by presenting some heuristics and ideas for an appropriate parameter setting.",

keywords = "Artificial neural networks, Backpropagation learning rule, Delta learning rule, Multi-layer neural networks, Non-linearly separable problems",

author = "Leonardo Vanneschi and Mauro Castelli",

note = "Vanneschi, L., & Castelli, M. (2019). Delta rule and backpropagation. In S. Ranganathan, M. Gribskov, K. Nakai, & C. Sch{\"o}nbach (Eds.), Encyclopedia of Bioinformatics and Computational Biology: ABC of Bioinformatics (Vol. 1-3, pp. 621-633). Elsevier. https://doi.org/10.1016/B978-0-12-809633-8.20340-3",

year = "2019",

doi = "10.1016/B978-0-12-809633-8.20340-3",

language = "English",

isbn = "9780128114148",

volume = "1-3",

pages = "621--633",

editor = "Shoba Ranganathan and Michael Gribskov and Kenta Nakai and Christian Sch{\"o}nbach",

booktitle = "Encyclopedia of Bioinformatics and Computational Biology",

publisher = "Elsevier",

address = "Netherlands",

}

Delta rule and backpropagation. / Vanneschi, Leonardo ; Castelli, Mauro.
Encyclopedia of Bioinformatics and Computational Biology: ABC of Bioinformatics. ed. / Shoba Ranganathan; Michael Gribskov; Kenta Nakai; Christian Schönbach. Vol. 1-3 Elsevier, 2019. p. 621-633.

Research output: Chapter in Book/Report/Conference proceeding › Entry for encyclopedia/dictionary › peer-review

TY - CHAP

T1 - Delta rule and backpropagation

AU - Vanneschi, Leonardo

AU - Castelli, Mauro

N1 - Vanneschi, L., & Castelli, M. (2019). Delta rule and backpropagation. In S. Ranganathan, M. Gribskov, K. Nakai, & C. Schönbach (Eds.), Encyclopedia of Bioinformatics and Computational Biology: ABC of Bioinformatics (Vol. 1-3, pp. 621-633). Elsevier. https://doi.org/10.1016/B978-0-12-809633-8.20340-3

PY - 2019

Y1 - 2019

N2 - Assuming that the reader is already familiar with the general concept of Artificial Neural Network and with the Perceptron learning rule, this paper introduces the Delta learning rule, as a basis for the Backpropagation learning rule. After discussing the necessity of using multi-layer Artificial Neural Networks for solving non-linearly separable problems, the paper describes all the mathematical steps that allow us to pass from the simple gradient descent formulation to the Backpropagation algorithm, still nowadays one of the most used methods to train feed-forward multi-layer Artificial Neural Networks. The paper is concluded by discussing issues related to overfitting in feed-forward multi-layer Artificial Neural Networks, and by presenting some heuristics and ideas for an appropriate parameter setting.

AB - Assuming that the reader is already familiar with the general concept of Artificial Neural Network and with the Perceptron learning rule, this paper introduces the Delta learning rule, as a basis for the Backpropagation learning rule. After discussing the necessity of using multi-layer Artificial Neural Networks for solving non-linearly separable problems, the paper describes all the mathematical steps that allow us to pass from the simple gradient descent formulation to the Backpropagation algorithm, still nowadays one of the most used methods to train feed-forward multi-layer Artificial Neural Networks. The paper is concluded by discussing issues related to overfitting in feed-forward multi-layer Artificial Neural Networks, and by presenting some heuristics and ideas for an appropriate parameter setting.

KW - Artificial neural networks

KW - Backpropagation learning rule

KW - Delta learning rule

KW - Multi-layer neural networks

KW - Non-linearly separable problems

UR - http://www.scopus.com/inward/record.url?scp=85055798605&partnerID=8YFLogxK

U2 - 10.1016/B978-0-12-809633-8.20340-3

DO - 10.1016/B978-0-12-809633-8.20340-3

M3 - Entry for encyclopedia/dictionary

SN - 9780128114148

VL - 1-3

SP - 621

EP - 633

BT - Encyclopedia of Bioinformatics and Computational Biology

A2 - Ranganathan, Shoba

A2 - Gribskov, Michael

A2 - Nakai, Kenta

A2 - Schönbach, Christian

PB - Elsevier

ER -

Delta rule and backpropagation

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this