On the Information Content of Some Stochastic Algorithms

Manuel L. Esquível; Nélio Machado; Nadezhda P. Krasii; Pedro P. Mota

doi:10.1007/978-3-030-83266-7_5

On the Information Content of Some Stochastic Algorithms

Manuel L. Esquível, Nélio Machado, Nadezhda P. Krasii, Pedro P. Mota

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

We formulate an optimization stochastic algorithm convergence theorem, of Solis and Wets type, and we show several instances of its application to concrete algorithms. In this convergence theorem the algorithm is a sequence of random variables and, in order to describe the increasing flow of information associated to this sequence we define a filtration – or flow of σ -algebras – on the probability space, depending on the sequence of random variables and on the function being optimized. We compare the flow of information of two convergent algorithms by comparing the associated filtrations by means of the Cotter distance of σ -algebras. The main result is that two convergent optimization algorithms have the same information content if both their limit minimization functions generate the full σ -algebra of the probability space.

Original language	English
Title of host publication	Recent Developments in Stochastic Methods and Applications - ICSM-5, Selected Contributions
Editors	Albert N. Shiryaev, Konstantin E. Samouylov, Dmitry V. Kozyrev
Place of Publication	Cham
Publisher	Springer
Pages	57-75
Number of pages	19
ISBN (Electronic)	978-3-030-83266-7
ISBN (Print)	978-3-030-83265-0
DOIs	https://doi.org/10.1007/978-3-030-83266-7_5
Publication status	Published - 2021
Event	5th International Conference on Stochastic Methods, ICSM-5 2020 - Moscow, Russian Federation Duration: 23 Nov 2020 → 27 Nov 2020

Publication series

Name	Springer Proceedings in Mathematics and Statistics
Publisher	Springer
Volume	371
ISSN (Print)	2194-1009
ISSN (Electronic)	2194-1017

Conference

Conference	5th International Conference on Stochastic Methods, ICSM-5 2020
Country/Territory	Russian Federation
City	Moscow
Period	23/11/20 → 27/11/20

Keywords

Convergence of information σ -fields
Global optimization
Stochastic algorithms

Access to Document

10.1007/978-3-030-83266-7_5

Cite this

Esquível, M. L., Machado, N., Krasii, N. P., & Mota, P. P. (2021). On the Information Content of Some Stochastic Algorithms. In A. N. Shiryaev, K. E. Samouylov, & D. V. Kozyrev (Eds.), Recent Developments in Stochastic Methods and Applications - ICSM-5, Selected Contributions (pp. 57-75). (Springer Proceedings in Mathematics and Statistics; Vol. 371). Springer. https://doi.org/10.1007/978-3-030-83266-7_5

Esquível, Manuel L. ; Machado, Nélio ; Krasii, Nadezhda P. et al. / On the Information Content of Some Stochastic Algorithms. Recent Developments in Stochastic Methods and Applications - ICSM-5, Selected Contributions. editor / Albert N. Shiryaev ; Konstantin E. Samouylov ; Dmitry V. Kozyrev. Cham : Springer, 2021. pp. 57-75 (Springer Proceedings in Mathematics and Statistics).

@inproceedings{d0b02d34a3f6489b830f528bd5d2e4b7,

title = "On the Information Content of Some Stochastic Algorithms",

abstract = "We formulate an optimization stochastic algorithm convergence theorem, of Solis and Wets type, and we show several instances of its application to concrete algorithms. In this convergence theorem the algorithm is a sequence of random variables and, in order to describe the increasing flow of information associated to this sequence we define a filtration – or flow of σ -algebras – on the probability space, depending on the sequence of random variables and on the function being optimized. We compare the flow of information of two convergent algorithms by comparing the associated filtrations by means of the Cotter distance of σ -algebras. The main result is that two convergent optimization algorithms have the same information content if both their limit minimization functions generate the full σ -algebra of the probability space.",

keywords = "Convergence of information σ -fields, Global optimization, Stochastic algorithms",

author = "Esqu{\'i}vel, {Manuel L.} and N{\'e}lio Machado and Krasii, {Nadezhda P.} and Mota, {Pedro P.}",

note = "Funding Information: info:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UIDB%2F00297%2F2020/PT# Acknowledgements. This work was partially supported, for the first and fourth authors, (Portuguese Foundation for Science and Technology) and, for the third author, through RFBR (Grant n. 19-01-00451). Publisher Copyright: {\textcopyright} 2021, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 5th International Conference on Stochastic Methods, ICSM-5 2020 ; Conference date: 23-11-2020 Through 27-11-2020",

year = "2021",

doi = "10.1007/978-3-030-83266-7_5",

language = "English",

isbn = "978-3-030-83265-0",

series = "Springer Proceedings in Mathematics and Statistics",

publisher = "Springer",

pages = "57--75",

editor = "Shiryaev, {Albert N.} and Samouylov, {Konstantin E.} and Kozyrev, {Dmitry V.}",

booktitle = "Recent Developments in Stochastic Methods and Applications - ICSM-5, Selected Contributions",

address = "Netherlands",

}

Esquível, ML , Machado, N, Krasii, NP & Mota, PP 2021, On the Information Content of Some Stochastic Algorithms. in AN Shiryaev, KE Samouylov & DV Kozyrev (eds), Recent Developments in Stochastic Methods and Applications - ICSM-5, Selected Contributions. Springer Proceedings in Mathematics and Statistics, vol. 371, Springer, Cham, pp. 57-75, 5th International Conference on Stochastic Methods, ICSM-5 2020, Moscow, Russian Federation, 23/11/20. https://doi.org/10.1007/978-3-030-83266-7_5

On the Information Content of Some Stochastic Algorithms. / Esquível, Manuel L.; Machado, Nélio; Krasii, Nadezhda P. et al.
Recent Developments in Stochastic Methods and Applications - ICSM-5, Selected Contributions. ed. / Albert N. Shiryaev; Konstantin E. Samouylov; Dmitry V. Kozyrev. Cham: Springer, 2021. p. 57-75 (Springer Proceedings in Mathematics and Statistics; Vol. 371).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - On the Information Content of Some Stochastic Algorithms

AU - Esquível, Manuel L.

AU - Machado, Nélio

AU - Krasii, Nadezhda P.

AU - Mota, Pedro P.

N1 - Funding Information: info:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UIDB%2F00297%2F2020/PT# Acknowledgements. This work was partially supported, for the first and fourth authors, (Portuguese Foundation for Science and Technology) and, for the third author, through RFBR (Grant n. 19-01-00451). Publisher Copyright: © 2021, The Author(s), under exclusive license to Springer Nature Switzerland AG.

PY - 2021

Y1 - 2021

N2 - We formulate an optimization stochastic algorithm convergence theorem, of Solis and Wets type, and we show several instances of its application to concrete algorithms. In this convergence theorem the algorithm is a sequence of random variables and, in order to describe the increasing flow of information associated to this sequence we define a filtration – or flow of σ -algebras – on the probability space, depending on the sequence of random variables and on the function being optimized. We compare the flow of information of two convergent algorithms by comparing the associated filtrations by means of the Cotter distance of σ -algebras. The main result is that two convergent optimization algorithms have the same information content if both their limit minimization functions generate the full σ -algebra of the probability space.

AB - We formulate an optimization stochastic algorithm convergence theorem, of Solis and Wets type, and we show several instances of its application to concrete algorithms. In this convergence theorem the algorithm is a sequence of random variables and, in order to describe the increasing flow of information associated to this sequence we define a filtration – or flow of σ -algebras – on the probability space, depending on the sequence of random variables and on the function being optimized. We compare the flow of information of two convergent algorithms by comparing the associated filtrations by means of the Cotter distance of σ -algebras. The main result is that two convergent optimization algorithms have the same information content if both their limit minimization functions generate the full σ -algebra of the probability space.

KW - Convergence of information σ -fields

KW - Global optimization

KW - Stochastic algorithms

UR - http://www.scopus.com/inward/record.url?scp=85113715262&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-83266-7_5

DO - 10.1007/978-3-030-83266-7_5

M3 - Conference contribution

AN - SCOPUS:85113715262

SN - 978-3-030-83265-0

T3 - Springer Proceedings in Mathematics and Statistics

SP - 57

EP - 75

BT - Recent Developments in Stochastic Methods and Applications - ICSM-5, Selected Contributions

A2 - Shiryaev, Albert N.

A2 - Samouylov, Konstantin E.

A2 - Kozyrev, Dmitry V.

PB - Springer

CY - Cham

T2 - 5th International Conference on Stochastic Methods, ICSM-5 2020

Y2 - 23 November 2020 through 27 November 2020

ER -

On the Information Content of Some Stochastic Algorithms

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this