The face inversion effect (FIE) is a reduction in recognition performance for inverted faces compared to upright faces that is greater than that typically observed with other stimulus types (e.g. houses; Yin, 1969). Nevertheless, the demonstration that the inversion effect in recognition memory can be as strong with images of dogs as with faces when the subjects are experts in specific dog breeds (Diamond & Carey, 1986), suggests that there may be other factors, such as expertise, which give rise to the FIE. Event-related potentials (ERPs) were recorded while subjects performed an Old/New recognition study on normal and scrambled faces presented in upright and inverted orientations. We obtained the standard result for normal faces: The electrophysiological activity corresponding to the N170 was larger and delayed for normal inverted faces as compared to normal upright ones. On the other hand, the ERPs for scrambled inverted faces were not significantly larger or delayed as compared to scrambled upright stimuli. These results, in combination, show how the effect of inversion on the N170 is reliably greater when the faces are normal compared to scrambled, which suggests the disruption of configural information affects the FIE.