Standard
On the initialization of long short-term memory networks. / Mehdipour Ghazi, Mostafa; Nielsen, Mads; Pai, Akshay; Modat, Marc; Cardoso, M. Jorge; Ourselin, Sébastien; Sørensen, Lauge.
Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings. red. / Tom Gedeon; Kok Wai Wong; Minho Lee. Springer VS, 2019. s. 275-286 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Bind 11953 LNCS).
Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
Harvard
Mehdipour Ghazi, M, Nielsen, M, Pai, A, Modat, M, Cardoso, MJ, Ourselin, S & Sørensen, L 2019,
On the initialization of long short-term memory networks. i T Gedeon, KW Wong & M Lee (red),
Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings. Springer VS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), bind 11953 LNCS, s. 275-286, 26th International Conference on Neural Information Processing, ICONIP 2019, Sydney, Australien,
12/12/2019.
https://doi.org/10.1007/978-3-030-36708-4_23
APA
Mehdipour Ghazi, M., Nielsen, M., Pai, A., Modat, M., Cardoso, M. J., Ourselin, S., & Sørensen, L. (2019).
On the initialization of long short-term memory networks. I T. Gedeon, K. W. Wong, & M. Lee (red.),
Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings (s. 275-286). Springer VS. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) Bind 11953 LNCS
https://doi.org/10.1007/978-3-030-36708-4_23
Vancouver
Mehdipour Ghazi M, Nielsen M, Pai A, Modat M, Cardoso MJ, Ourselin S o.a.
On the initialization of long short-term memory networks. I Gedeon T, Wong KW, Lee M, red., Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings. Springer VS. 2019. s. 275-286. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Bind 11953 LNCS).
https://doi.org/10.1007/978-3-030-36708-4_23
Author
Mehdipour Ghazi, Mostafa ; Nielsen, Mads ; Pai, Akshay ; Modat, Marc ; Cardoso, M. Jorge ; Ourselin, Sébastien ; Sørensen, Lauge. / On the initialization of long short-term memory networks. Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings. red. / Tom Gedeon ; Kok Wai Wong ; Minho Lee. Springer VS, 2019. s. 275-286 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Bind 11953 LNCS).
Bibtex
@inproceedings{9361a5045c914564ab1548a8d4158aaf,
title = "On the initialization of long short-term memory networks",
abstract = "Weight initialization is important for faster convergence and stability of deep neural networks training. In this paper, a robust initialization method is developed to address the training instability in long short-term memory (LSTM) networks. It is based on a normalized random initialization of the network weights that aims at preserving the variance of the network input and output in the same range. The method is applied to standard LSTMs for univariate time series regression and to LSTMs robust to missing values for multivariate disease progression modeling. The results show that in all cases, the proposed initialization method outperforms the state-of-the-art initialization techniques in terms of training convergence and generalization performance of the obtained solution.",
keywords = "Deep neural networks, Disease progression modeling, Initialization, Long short-term memory, Time series regression",
author = "{Mehdipour Ghazi}, Mostafa and Mads Nielsen and Akshay Pai and Marc Modat and Cardoso, {M. Jorge} and S{\'e}bastien Ourselin and Lauge S{\o}rensen",
year = "2019",
doi = "10.1007/978-3-030-36708-4_23",
language = "English",
isbn = "9783030367077",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer VS",
pages = "275--286",
editor = "Tom Gedeon and Wong, {Kok Wai} and Minho Lee",
booktitle = "Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings",
note = "26th International Conference on Neural Information Processing, ICONIP 2019 ; Conference date: 12-12-2019 Through 15-12-2019",
}
RIS
TY - GEN
T1 - On the initialization of long short-term memory networks
AU - Mehdipour Ghazi, Mostafa
AU - Nielsen, Mads
AU - Pai, Akshay
AU - Modat, Marc
AU - Cardoso, M. Jorge
AU - Ourselin, Sébastien
AU - Sørensen, Lauge
PY - 2019
Y1 - 2019
N2 - Weight initialization is important for faster convergence and stability of deep neural networks training. In this paper, a robust initialization method is developed to address the training instability in long short-term memory (LSTM) networks. It is based on a normalized random initialization of the network weights that aims at preserving the variance of the network input and output in the same range. The method is applied to standard LSTMs for univariate time series regression and to LSTMs robust to missing values for multivariate disease progression modeling. The results show that in all cases, the proposed initialization method outperforms the state-of-the-art initialization techniques in terms of training convergence and generalization performance of the obtained solution.
AB - Weight initialization is important for faster convergence and stability of deep neural networks training. In this paper, a robust initialization method is developed to address the training instability in long short-term memory (LSTM) networks. It is based on a normalized random initialization of the network weights that aims at preserving the variance of the network input and output in the same range. The method is applied to standard LSTMs for univariate time series regression and to LSTMs robust to missing values for multivariate disease progression modeling. The results show that in all cases, the proposed initialization method outperforms the state-of-the-art initialization techniques in terms of training convergence and generalization performance of the obtained solution.
KW - Deep neural networks
KW - Disease progression modeling
KW - Initialization
KW - Long short-term memory
KW - Time series regression
U2 - 10.1007/978-3-030-36708-4_23
DO - 10.1007/978-3-030-36708-4_23
M3 - Article in proceedings
AN - SCOPUS:85077503119
SN - 9783030367077
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 275
EP - 286
BT - Neural Information Processing - 26th International Conference, ICONIP 2019, Proceedings
A2 - Gedeon, Tom
A2 - Wong, Kok Wai
A2 - Lee, Minho
PB - Springer VS
T2 - 26th International Conference on Neural Information Processing, ICONIP 2019
Y2 - 12 December 2019 through 15 December 2019
ER -
