Qualitative and quantitative assessment of step size adaptation rules

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt

Standard

Qualitative and quantitative assessment of step size adaptation rules. / Krause, Oswin; Glasmachers, Tobias; Igel, Christian.

Proceedings of the 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms. Association for Computing Machinery, 2017. s. 139-148.

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt

Harvard

Krause, O, Glasmachers, T & Igel, C 2017, Qualitative and quantitative assessment of step size adaptation rules. i Proceedings of the 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms. Association for Computing Machinery, s. 139-148, 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms, København, Danmark, 12/01/2017. https://doi.org/10.1145/3040718.3040725

APA

Krause, O., Glasmachers, T., & Igel, C. (2017). Qualitative and quantitative assessment of step size adaptation rules. I Proceedings of the 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms (s. 139-148). Association for Computing Machinery. https://doi.org/10.1145/3040718.3040725

Vancouver

Krause O, Glasmachers T, Igel C. Qualitative and quantitative assessment of step size adaptation rules. I Proceedings of the 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms. Association for Computing Machinery. 2017. s. 139-148 https://doi.org/10.1145/3040718.3040725

Author

Krause, Oswin ; Glasmachers, Tobias ; Igel, Christian. / Qualitative and quantitative assessment of step size adaptation rules. Proceedings of the 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms. Association for Computing Machinery, 2017. s. 139-148

Bibtex

@inproceedings{1ce0cde698184b779de1baded422035e,

title = "Qualitative and quantitative assessment of step size adaptation rules",

abstract = "We present a comparison of step size adaptation methods for evolution strategies, covering recent developments in the field. Following recent work by Hansen et al. we formulate a concise list of performance criteria: a) fast convergence of the mean, b) near-optimal fixed point of the normalized step size dynamics, and c) invariance to adding constant dimensions of the objective function. Our results show that algorithms violating these principles tend to underestimate the step size or are unreliable when the function does not fit to the algorithm's tuned hyperparameters. In contrast, we find that cumulative step size adaptation (CSA) and twopoint adaptation (TPA) provide reliable estimates of the optimal step size. We further find that removing the evolution path of CSA still leads to a reliable algorithm without the computational requirements of CSA.",

keywords = "Comparison, Evolution strategies, Step size adaptation",

author = "Oswin Krause and Tobias Glasmachers and Christian Igel",

year = "2017",

doi = "10.1145/3040718.3040725",

language = "English",

pages = "139--148",

booktitle = "Proceedings of the 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms",

publisher = "Association for Computing Machinery",

note = "null ; Conference date: 12-01-2017 Through 15-01-2017",

}

RIS

TY - GEN

T1 - Qualitative and quantitative assessment of step size adaptation rules

AU - Krause, Oswin

AU - Glasmachers, Tobias

AU - Igel, Christian

N1 - Conference code: 14

PY - 2017

Y1 - 2017

N2 - We present a comparison of step size adaptation methods for evolution strategies, covering recent developments in the field. Following recent work by Hansen et al. we formulate a concise list of performance criteria: a) fast convergence of the mean, b) near-optimal fixed point of the normalized step size dynamics, and c) invariance to adding constant dimensions of the objective function. Our results show that algorithms violating these principles tend to underestimate the step size or are unreliable when the function does not fit to the algorithm's tuned hyperparameters. In contrast, we find that cumulative step size adaptation (CSA) and twopoint adaptation (TPA) provide reliable estimates of the optimal step size. We further find that removing the evolution path of CSA still leads to a reliable algorithm without the computational requirements of CSA.

AB - We present a comparison of step size adaptation methods for evolution strategies, covering recent developments in the field. Following recent work by Hansen et al. we formulate a concise list of performance criteria: a) fast convergence of the mean, b) near-optimal fixed point of the normalized step size dynamics, and c) invariance to adding constant dimensions of the objective function. Our results show that algorithms violating these principles tend to underestimate the step size or are unreliable when the function does not fit to the algorithm's tuned hyperparameters. In contrast, we find that cumulative step size adaptation (CSA) and twopoint adaptation (TPA) provide reliable estimates of the optimal step size. We further find that removing the evolution path of CSA still leads to a reliable algorithm without the computational requirements of CSA.

KW - Comparison

KW - Evolution strategies

KW - Step size adaptation

UR - http://www.scopus.com/inward/record.url?scp=85018963553&partnerID=8YFLogxK

U2 - 10.1145/3040718.3040725

DO - 10.1145/3040718.3040725

M3 - Article in proceedings

AN - SCOPUS:85018963553

SP - 139

EP - 148

BT - Proceedings of the 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms

PB - Association for Computing Machinery

Y2 - 12 January 2017 through 15 January 2017

ER -

ID: 179557726