The N170 is an occipito-temporal visual event-related potential that is larger in response to faces than other nonface object categories and has been associated with the early activation of visual face representations in the human brain. It has been recently showed that spatially misaligning the bottom and top halves of a face stimulus-a manipulation used in behavioural studies to disrupt the processing of the whole face configuration (i.e. holistic processing)-increases the latency and the amplitude of the N170, particularly in the right hemisphere. Here we show that these observations cannot be accounted for by a general effect of spatial misalignment of visual patterns. A first experiment showed that the effect of misalignment on the N170 was larger when both top and bottom halves of the stimulus were made of face parts (i.e. a full face) compared to when one of the two halves was made of visual noise. The N170 delay and increase for misaligned faces is similar to the effect of upside-down face inversion, a manipulation thought to disrupt holistic processing. Supporting this view, in a second experiment in which we presented upside-down versions of the stimuli used in the first experiment, we did not observe such a larger effect of misalignment on the N170 for a full face compared to the control conditions. These observations support the view that the early face representation activated in the human brain at the level of the N170, is that of a global upright face pattern. Specifically, when the global configuration of a face is modified by inverting it or spatially misaligning its top and bottom halves, the activation of its representation is delayed in cortical areas coding preferentially for faces. Moreover, when the global face configuration is disrupted, the face stimulus appears to recruit additional neural processes compared to normal face processing, leading to an increase of the N170 amplitude on the scalp.