Breast milk has neurodevelopmental advantages compared to infant formula, especially in low-birth-weight infants, which may in part relate to the fat source. This study compared neurodevelopmental outcomes in three-day-old normal birth weight (NBW) and intrauterine growth restricted (IUGR) piglets fed a formula diet with either vegetable oil (VEG) or bovine milk fat sources (MILK) for three weeks in a 2 × 2 factorial design. Behavioural tests, lipidomics, MRI and RNA sequencing analyses of plasma and brain tissue were conducted. The absolute levels of 82% and 11% of lipid molecules were different between dietary groups in plasma and hippocampus, respectively. Of the lipid molecules with differential abundance in the hippocampus, the majority were upregulated in MILK versus VEG, and they mainly belonged to the group of glycerophospholipids. Lower absolute brain weights, absolute grey and white matter volumes and behaviour and motor function scores, and higher relative total brain weights were present in IUGR compared to NBW with minor influence of diet. Cognitive function and cerebellar gene expression profiles were similar for dietary and weight groups, and overall only minor interactive effects between diet and birth weight were observed. Overall, we show that the dietary fat source influences the plasma and to a lesser degree the hippocampal lipidome and is unable to improve on IUGR-induced brain structural and functional impairments.