Brain-Supervised Image Editing
Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
Standard
Brain-Supervised Image Editing. / Davis, Keith M.; De La Torre-Ortiz, Carlos; Ruotsalo, Tuukka.
Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022. IEEE Computer Society Press, 2022. s. 18459-18468 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Bind 2022-June).Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Brain-Supervised Image Editing
AU - Davis, Keith M.
AU - De La Torre-Ortiz, Carlos
AU - Ruotsalo, Tuukka
N1 - Publisher Copyright: © 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Despite recent advances in deep neural models for semantic image editing, present approaches are dependent on explicit human input. Previous work assumes the availability of manually curated datasets for supervised learning, while for unsupervised approaches the human inspection of discovered components is required to identify those which modify worthwhile semantic features. Here, we present a novel alternative: the utilization of brain responses as a supervision signal for learning semantic feature representations. Participants $(N=30)$ in a neurophysiological experiment were shown artificially generated faces and instructed to look for a particular semantic feature, such as 'old' or 'smiling', while their brain responses were recorded via electroencephalography (EEG). Using supervision signals inferred from these responses, semantic features within the latent space of a generative adversarial network (GAN) were learned and then used to edit semantic features of new images. We show that implicit brain supervision achieves comparable semantic image editing performance to explicit manual labeling. This work demonstrates the feasibility of utilizing implicit human reactions recorded via brain-computer interfaces for semantic image editing and interpretation.
AB - Despite recent advances in deep neural models for semantic image editing, present approaches are dependent on explicit human input. Previous work assumes the availability of manually curated datasets for supervised learning, while for unsupervised approaches the human inspection of discovered components is required to identify those which modify worthwhile semantic features. Here, we present a novel alternative: the utilization of brain responses as a supervision signal for learning semantic feature representations. Participants $(N=30)$ in a neurophysiological experiment were shown artificially generated faces and instructed to look for a particular semantic feature, such as 'old' or 'smiling', while their brain responses were recorded via electroencephalography (EEG). Using supervision signals inferred from these responses, semantic features within the latent space of a generative adversarial network (GAN) were learned and then used to edit semantic features of new images. We show that implicit brain supervision achieves comparable semantic image editing performance to explicit manual labeling. This work demonstrates the feasibility of utilizing implicit human reactions recorded via brain-computer interfaces for semantic image editing and interpretation.
KW - Image and video synthesis and generation
KW - Vision + X
UR - http://www.scopus.com/inward/record.url?scp=85141304372&partnerID=8YFLogxK
U2 - 10.1109/CVPR52688.2022.01793
DO - 10.1109/CVPR52688.2022.01793
M3 - Article in proceedings
AN - SCOPUS:85141304372
T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
SP - 18459
EP - 18468
BT - Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
PB - IEEE Computer Society Press
T2 - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
Y2 - 19 June 2022 through 24 June 2022
ER -
ID: 339144873