Label-Similarity Curriculum Learning

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningfagfællebedømt

Standard

Label-Similarity Curriculum Learning. / Dogan, Ürün; Deshmukh, Aniket Anand; Machura, Marcin Bronislaw; Igel, Christian.

Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. red. / Andrea Vedaldi; Horst Bischof; Thomas Brox; Jan-Michael Frahm. Springer VS, 2020. s. 174-190 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Bind 12374 LNCS).

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningfagfællebedømt

Harvard

Dogan, Ü, Deshmukh, AA, Machura, MB & Igel, C 2020, Label-Similarity Curriculum Learning. i A Vedaldi, H Bischof, T Brox & J-M Frahm (red), Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. Springer VS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), bind 12374 LNCS, s. 174-190, 16th European Conference on Computer Vision, ECCV 2020, Glasgow, Storbritannien, 23/08/2020. https://doi.org/10.1007/978-3-030-58526-6_11

APA

Dogan, Ü., Deshmukh, A. A., Machura, M. B., & Igel, C. (2020). Label-Similarity Curriculum Learning. I A. Vedaldi, H. Bischof, T. Brox, & J-M. Frahm (red.), Computer Vision – ECCV 2020 - 16th European Conference, Proceedings (s. 174-190). Springer VS. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Bind. 12374 LNCS https://doi.org/10.1007/978-3-030-58526-6_11

Vancouver

Dogan Ü, Deshmukh AA, Machura MB, Igel C. Label-Similarity Curriculum Learning. I Vedaldi A, Bischof H, Brox T, Frahm J-M, red., Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. Springer VS. 2020. s. 174-190. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Bind 12374 LNCS). https://doi.org/10.1007/978-3-030-58526-6_11

Author

Dogan, Ürün ; Deshmukh, Aniket Anand ; Machura, Marcin Bronislaw ; Igel, Christian. / Label-Similarity Curriculum Learning. Computer Vision – ECCV 2020 - 16th European Conference, Proceedings. red. / Andrea Vedaldi ; Horst Bischof ; Thomas Brox ; Jan-Michael Frahm. Springer VS, 2020. s. 174-190 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Bind 12374 LNCS).

Bibtex

@inproceedings{162a16e8591841cf9fad482257d0c55a,
title = "Label-Similarity Curriculum Learning",
abstract = "Curriculum learning can improve neural network training by guiding the optimization to desirable optima. We propose a novel curriculum learning approach for image classification that adapts the loss function by changing the label representation. The idea is to use a probability distribution over classes as target label, where the class probabilities reflect the similarity to the true class. Gradually, this label representation is shifted towards the standard one-hot-encoding. That is, in the beginning minor mistakes are corrected less than large mistakes, resembling a teaching process in which broad concepts are explained first before subtle differences are taught. The class similarity can be based on prior knowledge. For the special case of the labels being natural words, we propose a generic way to automatically compute the similarities. The natural words are embedded into Euclidean space using a standard word embedding. The probability of each class is then a function of the cosine similarity between the vector representations of the class and the true label. The proposed label-similarity curriculum learning (LCL) approach was empirically evaluated using several popular deep learning architectures for image classification tasks applied to five datasets including ImageNet, CIFAR100, and AWA2. In all scenarios, LCL was able to improve the classification accuracy on the test data compared to standard training. Code to reproduce results is available at https://github.com/speedystream/LCL.",
keywords = "Classification, Curriculum learning, Deep learning, Multi-modal learning",
author = "{\"U}r{\"u}n Dogan and Deshmukh, {Aniket Anand} and Machura, {Marcin Bronislaw} and Christian Igel",
year = "2020",
doi = "10.1007/978-3-030-58526-6_11",
language = "English",
isbn = "9783030585259",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer VS",
pages = "174--190",
editor = "Andrea Vedaldi and Horst Bischof and Thomas Brox and Jan-Michael Frahm",
booktitle = "Computer Vision – ECCV 2020 - 16th European Conference, Proceedings",
note = "16th European Conference on Computer Vision, ECCV 2020 ; Conference date: 23-08-2020 Through 28-08-2020",

}

RIS

TY - GEN

T1 - Label-Similarity Curriculum Learning

AU - Dogan, Ürün

AU - Deshmukh, Aniket Anand

AU - Machura, Marcin Bronislaw

AU - Igel, Christian

PY - 2020

Y1 - 2020

N2 - Curriculum learning can improve neural network training by guiding the optimization to desirable optima. We propose a novel curriculum learning approach for image classification that adapts the loss function by changing the label representation. The idea is to use a probability distribution over classes as target label, where the class probabilities reflect the similarity to the true class. Gradually, this label representation is shifted towards the standard one-hot-encoding. That is, in the beginning minor mistakes are corrected less than large mistakes, resembling a teaching process in which broad concepts are explained first before subtle differences are taught. The class similarity can be based on prior knowledge. For the special case of the labels being natural words, we propose a generic way to automatically compute the similarities. The natural words are embedded into Euclidean space using a standard word embedding. The probability of each class is then a function of the cosine similarity between the vector representations of the class and the true label. The proposed label-similarity curriculum learning (LCL) approach was empirically evaluated using several popular deep learning architectures for image classification tasks applied to five datasets including ImageNet, CIFAR100, and AWA2. In all scenarios, LCL was able to improve the classification accuracy on the test data compared to standard training. Code to reproduce results is available at https://github.com/speedystream/LCL.

AB - Curriculum learning can improve neural network training by guiding the optimization to desirable optima. We propose a novel curriculum learning approach for image classification that adapts the loss function by changing the label representation. The idea is to use a probability distribution over classes as target label, where the class probabilities reflect the similarity to the true class. Gradually, this label representation is shifted towards the standard one-hot-encoding. That is, in the beginning minor mistakes are corrected less than large mistakes, resembling a teaching process in which broad concepts are explained first before subtle differences are taught. The class similarity can be based on prior knowledge. For the special case of the labels being natural words, we propose a generic way to automatically compute the similarities. The natural words are embedded into Euclidean space using a standard word embedding. The probability of each class is then a function of the cosine similarity between the vector representations of the class and the true label. The proposed label-similarity curriculum learning (LCL) approach was empirically evaluated using several popular deep learning architectures for image classification tasks applied to five datasets including ImageNet, CIFAR100, and AWA2. In all scenarios, LCL was able to improve the classification accuracy on the test data compared to standard training. Code to reproduce results is available at https://github.com/speedystream/LCL.

KW - Classification

KW - Curriculum learning

KW - Deep learning

KW - Multi-modal learning

UR - http://www.scopus.com/inward/record.url?scp=85093077930&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-58526-6_11

DO - 10.1007/978-3-030-58526-6_11

M3 - Article in proceedings

AN - SCOPUS:85093077930

SN - 9783030585259

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 174

EP - 190

BT - Computer Vision – ECCV 2020 - 16th European Conference, Proceedings

A2 - Vedaldi, Andrea

A2 - Bischof, Horst

A2 - Brox, Thomas

A2 - Frahm, Jan-Michael

PB - Springer VS

T2 - 16th European Conference on Computer Vision, ECCV 2020

Y2 - 23 August 2020 through 28 August 2020

ER -

ID: 250554869