Visual crowding is a phenomenon that impairs object recognition when the features of an object are positioned too closely together. Crowding limits recognition in normal peripheral vision and it has been suggested to be the core deficit in visual agnosia, leading to a domain-general deficit in object recognition. Using a recently developed tool, we test whether crowding is the underlying deficit in four patients with category specific agnosias: Two with pure alexia and two with acquired prosopagnosia. We expected all patients to show abnormal crowding. We find that the two patients with acquired prosopagnosia show abnormal crowding effects in foveal vision, while the pure alexic patients do not, and that this constitutes a significant dissociation. Thus, abnormal crowding cannot explain all cases of visual agnosia. Much recent work has focused on similarities between pure alexia and acquired prosopagnosia. Here we show a difference in a basic visual mechanism—visual crowding.