Linguistically debatable or just plain wrong?
Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
Standard
Linguistically debatable or just plain wrong? / Plank, Barbara; Hovy, Dirk; Søgaard, Anders.
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Bind volume 2 Baltimore, Maryland : Association for Computational Linguistics, 2014. s. 507-511.Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Linguistically debatable or just plain wrong?
AU - Plank, Barbara
AU - Hovy, Dirk
AU - Søgaard, Anders
PY - 2014
Y1 - 2014
N2 - In linguistic annotation projects, we typically develop annotation guidelines to maximize inter-annotator agreement and learnability. However, in this position paper we question whether we should actually limit the disagreements between annotators, rather than embrace them. We present an empirical analysis of part-of-speech annotated data sets that suggests that certain disagreements are systematic across domains and languages. This points to an underlying ambiguity rather than random errors. Moreover, a quantitative analysis of disagreements reveals that the majority of them are due to linguistically debatable cases, rather than to actual annotation errors. Specifically, we show that even in the absence of annotation guidelines, only 2% of annotator choices are linguistically unmotivated.
AB - In linguistic annotation projects, we typically develop annotation guidelines to maximize inter-annotator agreement and learnability. However, in this position paper we question whether we should actually limit the disagreements between annotators, rather than embrace them. We present an empirical analysis of part-of-speech annotated data sets that suggests that certain disagreements are systematic across domains and languages. This points to an underlying ambiguity rather than random errors. Moreover, a quantitative analysis of disagreements reveals that the majority of them are due to linguistically debatable cases, rather than to actual annotation errors. Specifically, we show that even in the absence of annotation guidelines, only 2% of annotator choices are linguistically unmotivated.
M3 - Article in proceedings
VL - volume 2
SP - 507
EP - 511
BT - Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
PB - Association for Computational Linguistics
CY - Baltimore, Maryland
ER -
ID: 107673308