Human vision has striking radial asymmetries, with performance on many tasks varying sharply with stimulus polar angle. Performance is generally better on the horizontal than vertical meridian, and on the lower than upper vertical meridian, and these asymmetries decrease gradually with deviation from the vertical meridian. Here, we report cortical magnification at a fine angular resolution around the visual field. This precision enables comparisons between cortical magnification and behavior, between cortical magnification and retinal cell densities, and between cortical magnification in twin pairs. We show that cortical magnification in the human primary visual cortex, measured in 163 subjects, varies substantially around the visual field, with a pattern similar to behavior. These radial asymmetries in the cortex are larger than those found in the retina, and they are correlated between monozygotic twin pairs. These findings indicate a tight link between cortical topography and behavior, and suggest that visual field asymmetries are partly heritable.