Are Multilingual Sentiment Models Equally Right for the Right Reasons?

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningfagfællebedømt

Dokumenter

  • Fulltext

    Forlagets udgivne version, 238 KB, PDF-dokument

Multilingual NLP models provide potential solutions to the digital language divide, i.e., cross-language performance disparities. Early analyses of such models have indicated good performance across training languages and good generalization to unseen, related languages. This work examines whether, between related languages, multilingual models are equally right for the right reasons, i.e., if interpretability methods reveal that the models put emphasis on the same words as humans. To this end, we provide a new trilingual, parallel corpus of rationale annotations for English, Danish, and Italian sentiment analysis models and use it to benchmark models and interpretability methods. We propose rank-biased overlap as a better metric for comparing input token attributions to human rationale annotations. Our results show: (i) models generally perform well on the languages they are trained on, and align best with human rationales in these languages; (ii) performance is higher on English, even when not a source language, but this performance is not accompanied by higher alignment with human rationales, which suggests that language models favor English, but do not facilitate successful transfer of rationales.
OriginalsprogEngelsk
TitelProceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
ForlagAssociation for Computational Linguistics (ACL)
Publikationsdato2022
Sider131–141
StatusUdgivet - 2022
BegivenhedFifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP - Abu Dhabi
Varighed: 8 dec. 20228 dec. 2022

Workshop

WorkshopFifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
ByAbu Dhabi
Periode08/12/202208/12/2022

Antal downloads er baseret på statistik fra Google Scholar og www.ku.dk


Ingen data tilgængelig

ID: 338603346