Cross-Subject EEG Feedback for Implicit Image Generation
Publikation: Bidrag til tidsskrift › Tidsskriftartikel › Forskning › fagfællebedømt
Standard
Cross-Subject EEG Feedback for Implicit Image Generation. / Torre-Ortiz, Carlos de la; Spape, Michiel M.; Ravaja, Niklas; Ruotsalo, Tuukka.
I: IEEE Transactions on Cybernetics, 2024, s. 1-0.Publikation: Bidrag til tidsskrift › Tidsskriftartikel › Forskning › fagfællebedømt
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - Cross-Subject EEG Feedback for Implicit Image Generation
AU - Torre-Ortiz, Carlos de la
AU - Spape, Michiel M.
AU - Ravaja, Niklas
AU - Ruotsalo, Tuukka
N1 - Publisher Copyright: Authors
PY - 2024
Y1 - 2024
N2 - Generative models are powerful tools for producing novel information by learning from example data. However, the current approaches require explicit manual input to steer generative models to match human goals. Furthermore, how these models would integrate implicit, diverse feedback and goals of multiple users remains largely unexplored. Here, we present a first-of-its-kind system that produces novel images of faces by inferring human goals directly from cross-subject brain signals while study subjects are looking at example images. We report on an experiment where brain responses to images of faces were recorded using electroencephalography in 30 subjects, focusing on specific salient visual features (VFs). Preferences toward VFs were decoded from subjects’ brain responses and used as implicit feedback for a generative adversarial network (GAN), which generated new images of faces. The results from a follow-up user study evaluating the presence of the target salient VFs show that the images generated from brain feedback represent the goal of the study subjects and are comparable to images generated with manual feedback. The methodology provides a stepping stone toward humans-in-the-loop image generation.
AB - Generative models are powerful tools for producing novel information by learning from example data. However, the current approaches require explicit manual input to steer generative models to match human goals. Furthermore, how these models would integrate implicit, diverse feedback and goals of multiple users remains largely unexplored. Here, we present a first-of-its-kind system that produces novel images of faces by inferring human goals directly from cross-subject brain signals while study subjects are looking at example images. We report on an experiment where brain responses to images of faces were recorded using electroencephalography in 30 subjects, focusing on specific salient visual features (VFs). Preferences toward VFs were decoded from subjects’ brain responses and used as implicit feedback for a generative adversarial network (GAN), which generated new images of faces. The results from a follow-up user study evaluating the presence of the target salient VFs show that the images generated from brain feedback represent the goal of the study subjects and are comparable to images generated with manual feedback. The methodology provides a stepping stone toward humans-in-the-loop image generation.
KW - Brain modeling
KW - Brain–computer interfaces
KW - Electroencephalography
KW - electroencephalography (EEG)
KW - Faces
KW - generative models
KW - image generation
KW - Image synthesis
KW - Manuals
KW - Task analysis
KW - Visualization
UR - http://www.scopus.com/inward/record.url?scp=85196722868&partnerID=8YFLogxK
U2 - 10.1109/TCYB.2024.3406159
DO - 10.1109/TCYB.2024.3406159
M3 - Journal article
C2 - 38889044
AN - SCOPUS:85196722868
SP - 1
EP - 0
JO - IEEE Transactions on Cybernetics
JF - IEEE Transactions on Cybernetics
SN - 2168-2267
ER -
ID: 397029959