Semiconductor quantum dots (QDs) provide useful means to couple light and matter in applications such as light-harvesting1, 2 and all-solid-state quantum information processing3, 4. This coupling can be increased by placing QDs in nanostructured optical environments such as photonic crystals or metallic nanostructures that enable strong confinement of light and thereby enhance the light–matter interaction. It has thus far been assumed that QDs can be described in the same way as atomic photon emitters—as point sources with wavefunctions whose spatial extent can be disregarded. Here we demonstrate that this description breaks down for QDs near plasmonic nanostructures. We observe an eightfold enhancement of the plasmon excitation rate, depending on QD orientation as a result of their mesoscopic character. Moreover, we show that the interaction can be enhanced or suppressed, determined by the geometry of the plasmonic nanostructure, consistent with a newly developed theory that takes mesoscopic effects into account. This behaviour has no equivalence in atomic systems and offers new opportunities to exploit the unique mesoscopic characteristics of QDs in the development of nanophotonic devices that use the increased light–matter interaction.