Empathy is a complex social behaviour mediated by a network of brain structures. Recently, several functional imaging studies have investigated the neural basis of empathy, but few corroborative human lesion studies exist. Severe empathy loss is a common feature of frontotemporal lobar degeneration (FTLD), and is also seen in other neurodegenerative diseases. In this study, the neuroanatomic basis of empathy was investigated in 123 patients with FTLD, Alzheimer's disease, corticobasal degeneration and progressive supranuclear palsy using the Interpersonal Reactivity Index (IRI). IRI Empathic Concern and Perspective taking scores were correlated with structural MRI brain volume using voxel-based morphometry. Voxels in the right temporal pole, the right fusiform gyrus, the right caudate and right subcallosal gyrus correlated significantly with total empathy score (P < 0.05 after whole-brain correction for multiple comparisons). Empathy score correlated positively with the volume of right temporal structures in semantic dementia, and with subcallosal gyrus volume in frontotemporal dementia. These findings are consistent with previous research suggesting that a primarily right frontotemporal network of brain regions is involved in emotion processing, and highlights the roles of the right temporal pole and inferior frontal/striatal regions in regulating complex social interactions. This is the first large-scale lesion study to investigate the neural basis of empathy using correlational analytic methods. The results suggest that the right anterior temporal and medial frontal regions are essential for real-life empathic behaviour.