Exploring Visual Engagement Signals for Representation Learning
Publikation: Bidrag til tidsskrift › Konferenceartikel › Forskning › fagfællebedømt
Standard
Exploring Visual Engagement Signals for Representation Learning. / Belongie, Serge; Wu, Zuxuan; Cardie, Claire; Jia, Menglin; Reiter, Austin; Lim, Ser-Nam.
I: IEEE Xplore Digital Library, Bind 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 28.02.2022, s. 4186-4197.Publikation: Bidrag til tidsskrift › Konferenceartikel › Forskning › fagfællebedømt
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Exploring Visual Engagement Signals for Representation Learning
AU - Belongie, Serge
AU - Wu, Zuxuan
AU - Cardie, Claire
AU - Jia, Menglin
AU - Reiter, Austin
AU - Lim, Ser-Nam
PY - 2022/2/28
Y1 - 2022/2/28
N2 - Visual engagement in social media platforms comprises interactions with photo posts including comments, shares, and likes. In this paper, we leverage such visual engagement clues as supervisory signals for representation learning. However, learning from engagement signals is non-trivial as it is not clear how to bridge the gap between low-level visual information and high-level social interaction. We present VisE,, a weakly supervised learning approach, which maps social images to pseudo labels derived by clustered engagement signals. We then study how models trained in this way benefit subjective downstream computer vision tasks such as emotion recognition or political bias detection. Through extensive studies, we empirically demonstrate the effectiveness of VisE across a diverse set of classification tasks beyond the scope of conventional recognition.
AB - Visual engagement in social media platforms comprises interactions with photo posts including comments, shares, and likes. In this paper, we leverage such visual engagement clues as supervisory signals for representation learning. However, learning from engagement signals is non-trivial as it is not clear how to bridge the gap between low-level visual information and high-level social interaction. We present VisE,, a weakly supervised learning approach, which maps social images to pseudo labels derived by clustered engagement signals. We then study how models trained in this way benefit subjective downstream computer vision tasks such as emotion recognition or political bias detection. Through extensive studies, we empirically demonstrate the effectiveness of VisE across a diverse set of classification tasks beyond the scope of conventional recognition.
UR - https://openaccess.thecvf.com/content/ICCV2021/html/Jia_Exploring_Visual_Engagement_Signals_for_Representation_Learning_ICCV_2021_paper.html
U2 - 10.1109/ICCV48922.2021.00417
DO - 10.1109/ICCV48922.2021.00417
M3 - Conference article
VL - 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
SP - 4186
EP - 4197
JO - IEEE Xplore Digital Library
JF - IEEE Xplore Digital Library
ER -
ID: 303806039