Flatfish have the most extreme asymmetric body morphology of vertebrates. During metamorphosis, one eye migrates to the contralateral side of the skull, and this migration is accompanied by extensive craniofacial transformations and simultaneous development of lopsided body pigmentation. The evolution of this developmental and physiological innovation remains enigmatic. Comparative genomics of two flatfish and transcriptomic analyses during metamorphosis point to a role for thyroid hormone and retinoic acid signaling, as well as phototransduction pathways. We demonstrate that retinoic acid is critical in establishing asymmetric pigmentation and, via cross-talk with thyroid hormones, in modulating eye migration. The unexpected expression of the visual opsins from the phototransduction pathway in the skin translates illumination differences and generates retinoic acid gradients that underlie the generation of asymmetry. Identifying the genetic underpinning of this unique developmental process answers long-standing questions about the evolutionary origin of asymmetry, but it also provides insight into the mechanisms that control body shape in vertebrates.