Trait-based approaches are increasingly used to understand crop yield improvement, although they have not been widely applied to anatomical traits. Little is known about the relationships between root and leaf anatomy and yield in wheat. We selected 20 genotypes that have been widely planted in Luoyang, in the major wheat-producing area of China, to explore these relationships. A field study was performed to measure the yields and yield components of the genotypes. Root and leaf samples were collected at anthesis to measure the anatomical traits relevant to carbon allocation and water transport. Yield was negatively correlated with cross-sectional root cortex area, indicating that reduced root cortical tissue and therefore reduced carbon investment have contributed to yield improvement in this region. Yield was positively correlated with root xylem area, suggesting that a higher water transport capacity has also contributed to increased yields in this study. The area of the leaf veins did not significantly correlate with yield, showing that the high-yield genotypes did not have larger veins, but they may have had a conservative water use strategy, with tight regulation of water loss from the leaves. This study demonstrates that breeding for higher yields in this region has changed wheat’s anatomical traits, reducing the roots’ cortical tissue and increasing the roots’ xylem investment.