Jointly Learning to Label Sentences and Tokens
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Standard
Jointly Learning to Label Sentences and Tokens. / Rei, Marek; Søgaard, Anders.
Proceedings of 33nd AAAI Conference on Artificial Intelligence, AAAI 2019. AAAI Press, 2019. p. 6916-6923.Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Jointly Learning to Label Sentences and Tokens
AU - Rei, Marek
AU - Søgaard, Anders
PY - 2019
Y1 - 2019
N2 - Learning to construct text representations in end-to-end systems can be difficult, as natural languages are highly compositional and task-specific annotated datasets are often limited in size. Methods for directly supervising language composition can allow us to guide the models based on existing knowledge, regularizing them towards more robust and interpretable representations. In this paper, we investigate how objectives at different granularities can be used to learn better language representations and we propose an architecture for jointly learning to label sentences and tokens. The predictions at each level are combined together using an attention mechanism, with token-level labels also acting as explicit supervision for composing sentence-level representations. Our experiments show that by learning to perform these tasks jointly on multiple levels, the model achieves substantial improvements for both sentence classification and sequence labeling.
AB - Learning to construct text representations in end-to-end systems can be difficult, as natural languages are highly compositional and task-specific annotated datasets are often limited in size. Methods for directly supervising language composition can allow us to guide the models based on existing knowledge, regularizing them towards more robust and interpretable representations. In this paper, we investigate how objectives at different granularities can be used to learn better language representations and we propose an architecture for jointly learning to label sentences and tokens. The predictions at each level are combined together using an attention mechanism, with token-level labels also acting as explicit supervision for composing sentence-level representations. Our experiments show that by learning to perform these tasks jointly on multiple levels, the model achieves substantial improvements for both sentence classification and sequence labeling.
U2 - 10.1609/aaai.v33i01.33016916
DO - 10.1609/aaai.v33i01.33016916
M3 - Article in proceedings
SN - 978-1-57735-809-1
SP - 6916
EP - 6923
BT - Proceedings of 33nd AAAI Conference on Artificial Intelligence, AAAI 2019
PB - AAAI Press
T2 - 33rd AAAI Conference on Artificial Intelligence - AAAI 2019
Y2 - 27 January 2019 through 1 February 2019
ER -
ID: 240420866