BERT Busters: Outlier Dimensions That Disrupt Transformers
Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
Multiple studies have shown that Transformers are remarkably robust to pruning. Contrary to this received wisdom, we demonstrate that pre-trained Transformer encoders are surprisingly fragile to the removal of a very small number of features in the layer outputs ($
Originalsprog | Engelsk |
---|---|
Titel | Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 |
Antal sider | 14 |
Udgivelsessted | Online |
Forlag | Association for Computational Linguistics (ACL) |
Publikationsdato | 1 aug. 2021 |
Sider | 3392-3405 |
DOI | |
Status | Udgivet - 1 aug. 2021 |
ID: 285387504