Unsupervised Domain Adaptation: A Reality Check
Research output: Working paper › Preprint › Research
Standard
Unsupervised Domain Adaptation: A Reality Check. / Musgrave, Kevin; Belongie, Serge; Lim, Ser-Nam.
arXiv.org, 2022.Research output: Working paper › Preprint › Research
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - UNPB
T1 - Unsupervised Domain Adaptation: A Reality Check
AU - Musgrave, Kevin
AU - Belongie, Serge
AU - Lim, Ser-Nam
PY - 2022
Y1 - 2022
N2 - Interest in unsupervised domain adaptation (UDA) has surged in recent years, resulting in a plethora of new algorithms. However, as is often the case in fast-moving fields, baseline algorithms are not tested to the extent that they should be. Furthermore, little attention has been paid to validation methods, i.e. the methods for estimating the accuracy of a model in the absence of target domain labels. This is despite the fact that validation methods are a crucial component of any UDA train/val pipeline. In this paper, we show via large-scale experimentation that 1) in the oracle setting, the difference in accuracy between UDA algorithms is smaller than previously thought, 2) state-of-the-art validation methods are not well-correlated with accuracy, and 3) differences between UDA algorithms are dwarfed by the drop in accuracy caused by validation methods.
AB - Interest in unsupervised domain adaptation (UDA) has surged in recent years, resulting in a plethora of new algorithms. However, as is often the case in fast-moving fields, baseline algorithms are not tested to the extent that they should be. Furthermore, little attention has been paid to validation methods, i.e. the methods for estimating the accuracy of a model in the absence of target domain labels. This is despite the fact that validation methods are a crucial component of any UDA train/val pipeline. In this paper, we show via large-scale experimentation that 1) in the oracle setting, the difference in accuracy between UDA algorithms is smaller than previously thought, 2) state-of-the-art validation methods are not well-correlated with accuracy, and 3) differences between UDA algorithms are dwarfed by the drop in accuracy caused by validation methods.
UR - https://arxiv.org/abs/2111.15672
M3 - Preprint
BT - Unsupervised Domain Adaptation: A Reality Check
PB - arXiv.org
ER -
ID: 303688653