Predicting word sense annotation agreement
Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
High agreement is a common objective when annotating data for word senses.
However, a number of factors make perfect agreement impossible, e.g. the limitations of the sense inventories, the difficulty of the examples or the interpretation preferences of the annotations.
Estimating potential agreement is thus a relevant task to supplement the evaluation of sense annotations.
In this article we propose two methods to predict agreement on word-annotation instances. We experiment with a continuous representation and a three-way discretization of observed agreement. In spite of the difficulty of the task, we find that different levels of agreement can be identified---in particular, low-agreement examples are easier to identify.
However, a number of factors make perfect agreement impossible, e.g. the limitations of the sense inventories, the difficulty of the examples or the interpretation preferences of the annotations.
Estimating potential agreement is thus a relevant task to supplement the evaluation of sense annotations.
In this article we propose two methods to predict agreement on word-annotation instances. We experiment with a continuous representation and a three-way discretization of observed agreement. In spite of the difficulty of the task, we find that different levels of agreement can be identified---in particular, low-agreement examples are easier to identify.
Originalsprog | Engelsk |
---|---|
Titel | LSDSem 2015 : Linking Models of Lexical, Sentential and Discourse-level Semantics |
Antal sider | 6 |
Forlag | Association for Computational Linguistics |
Publikationsdato | 2015 |
Sider | 89-94 |
ISBN (Trykt) | 978-1-941643-32-7 |
Status | Udgivet - 2015 |
Links
- http://aclweb.org/anthology/W15-27
Forlagets udgivne version
ID: 141768705