Face perception

Face perception is an individual's understanding and interpretation of the face, particularly the human face, especially in relation to the associated information processing in the brain. Facial features play an important role in human development, carrying a wealth of social information. Infants as young as two days old are capable of mimicking the facial expressions of an adult, displaying their capacity to note details like mouth and eye shape as well as to move their own muscles in a way that produces similar patterns in their faces. At this stage, newborns are not yet aware of the emotional content encoded within facial expressions. But by the age of seven months the child is able to recognize an angry or fearful facial expression, perhaps because of the threat-salient nature of the emotion.

Face perceptions are very complex, as the recognition of facial expressions involves extensive and diverse areas in the brain. Sometimes, damaged parts of the brain can cause specific impairments in understanding faces, called prosopagnosia. Brain imaging studies typically show a great deal of activity in an area of the temporal lobe known as the fusiform gyrus, an area also known to cause prosopagnosia when damaged (particularly when damage occurs on both sides). This evidence has led to a particular interest in this area and it is sometimes referred to as the fusiform face area (FFA) for that reason. It is important to note that while certain areas of the brain respond selectively to faces, facial processing involves many neural networks which include visual and emotional processing systems.

Development

From birth, infants possess rudimentary facial processing capacities and show heightened interest in faces.^[1][2] For example, newborns (1–3 days) have been shown to be able to recognize faces even when they are rotated up to 45 degrees.^[3] However, interest in faces is not continuously present in infancy and shows increases and decreases over time as the child grows older. Specifically, while newborns show a preference for faces, this behavior is reduced between one- to four months of age.^[4] Around three months of age, a preference for faces re-emerges and interest in faces seems to peak late during the first year but then declines again slowly over the next two years of life.^[5] The re-emergence of a preference for faces at three months of age may be influenced by the child's own motor abilities and experiences.^[6][7] Infants as young as two days of age are capable of mimicking the facial expressions of an adult, displaying their capacity to note details like mouth and eye shape as well as to move their own muscles in a way that produces similar patterns in their faces.^[8][9] However, despite this ability, newborns are not yet aware of the emotional content encoded within facial expressions.

Five-month-olds, when presented with an image of a person making a fearful expression and a person making a happy expression, pay the same amount of attention to and exhibit similar event-related potentials (ERPs) for both. However, when seven-month-olds are given the same treatment, they focus more on the fearful face, and their event-related potential for the scared face shows a stronger initial negative central component than that for the happy face. This result indicates an increased attentional and cognitive focus toward fear that reflects the threat-salient nature of the emotion.^[10] In addition, infants' negative central components were not different for new faces that varied in the intensity of an emotional expression but portrayed the same emotion as a face they had been habituated to but were stronger for different-emotion faces, showing that seven-month-olds regarded happy and sad faces as distinct emotive categories.^[11] While seven-month-olds have been found to focus more on fearful faces, another study by Jessen, Altvater-Mackensen, and Grossmann found that "happy expressions elicit enhanced sympathetic arousal in infants" both when facial expressions were presented subliminally and when they were presented supraliminally, or in a way that the infants were consciously aware of the stimulus.^[12] These results show that conscious awareness of a stimulus is not connected to an infant's reaction to that stimulus.^[12]

The recognition of faces is an important neurological mechanism that individuals in society use every day. Jeffrey and Rhodes^[13] write that faces "convey a wealth of information that we use to guide our social interactions".^[14] Emotions play a large role in our social interactions. The perception of a positive or negative emotion on a face affects the way that an individual perceives and processes that face. For example, a face that is perceived to have a negative emotion is processed in a less holistic manner than a face displaying a positive emotion.^[15] The ability of face recognition is apparent even in early childhood. The neurological mechanisms responsible for face recognition are present by age five. Research shows that the way children process faces is similar to that of adults, but adults process faces more efficiently. The reason for this may be because of advancements in memory and cognitive functioning that occur with age.^[14]

Infants are able to comprehend facial expressions as social cues representing the feelings of other people before they are a year old. At seven months, the object of an observed face's apparent emotional reaction is relevant in processing the face. Infants at this age show greater negative central components to angry faces that are looking directly at them than elsewhere, although the direction of fearful faces' gaze produces no difference. In addition, two ERP components in the posterior part of the brain are differently aroused by the two negative expressions tested. These results indicate that infants at this age can at least partially understand the higher level of threat from anger directed at them as compared to anger directed elsewhere.^[16] By at least seven months of age, infants are also able to use facial expressions to understand others' behavior. Seven-month-olds will look to facial cues to understand the motives of other people in ambiguous situations, as shown by a study in which they watched an experimenter's face longer if she took a toy from them and maintained a neutral expression than if she made a happy expression.^[17] Interest in the social world is increased by interaction with the physical environment. Training three-month-old infants to reach for objects with Velcro-covered "sticky mitts" increases the amount of attention that they pay to faces as compared to passively moving objects through their hands and non-trained control groups.^[18]

In following with the notion that seven-month-olds have categorical understandings of emotion, they are also capable of associating emotional prosodies with corresponding facial expressions. When presented with a happy or angry face, shortly followed by an emotionally neutral word read in a happy or angry tone, their ERPs follow different patterns. Happy faces followed by angry vocal tones produce more changes than the other incongruous pairing, while there was no such difference between happy and angry congruous pairings, with the greater reaction implying that infants held greater expectations of a happy vocal tone after seeing a happy face than an angry tone following an angry face. Considering an infant's relative immobility and thus their decreased capacity to elicit negative reactions from their parents, this result implies that experience has a role in building comprehension of facial expressions.^[19]

Several other studies indicate that early perceptual experience is crucial to the development of capacities characteristic of adult visual perception, including the ability to identify familiar people and to recognize and comprehend facial expressions.^[20] The capacity to discern between faces, much like language, appears to have a broad potential in early life that is whittled down to kinds of faces that are experienced in early life.^[20] Infants can discern between macaque faces at six months of age, but, without continued exposure, cannot at nine months of age. Being shown photographs of macaques during this three-month period gave nine-month-olds the ability to reliably distinguish between unfamiliar macaque faces.^[21]

The neural substrates of face perception in infants are likely similar to those of adults, but the limits of imaging technology that are feasible for use with infants currently prevent very specific localization of function as well as specific information from subcortical areas^[22] like the amygdala, which is active in the perception of facial expression in adults.^[20] In a study on healthy adults, it was shown that faces are likely to be processed, in part, via a retinotectal (subcortical) pathway.^[23]

However, there is activity near the fusiform gyrus,^[22] as well as in occipital areas.^[16] when infants are exposed to faces, and it varies depending on factors including facial expression and eye gaze direction.^[11][16]

Adult

Recognizing and perceiving faces are vital abilities needed to coexist in society. Faces can tell things such as identity, mood, age, sex, race, and the direction that someone is looking.^[24][25][26] Studies based on neuropsychology, behavior, electrophysiology, and neuro-imaging have supported the notion of a specialized mechanism for perceiving faces.^[26] Prosopagnosia patients demonstrate neuropsychological support for a specialized face perception mechanism as these people, due to brain damage, have deficits in facial perception, but their cognitive perception of objects remains intact. The face inversion effect provides behavioral support of a specialized mechanism as people tend to have greater deficits in task performance when prompted to react to an inverted face than to an inverted object. Electrophysiological support comes from the finding that the N170 and M170 responses tend to be face-specific. Neuro-imaging studies such as PET and fMRI studies have shown support for a specialized facial processing mechanism as they have identified regions of the fusiform gyrus that have higher activation during face perception tasks than other visual perception tasks.^[26] Theories about the processes involved in adult face perception have largely come from two sources: research on normal adult face perception and the study of impairments in face perception that are caused by brain injury or neurological illness. Novel optical illusions such as the Flashed Face Distortion Effect, in which scientific phenomenology outpaces neurological theory, also provide areas for research.

One of the most widely accepted theories of face perception argues that understanding faces involves several stages:^[27] from basic perceptual manipulations on the sensory information to derive details about the person (such as age, gender or attractiveness), to being able to recall meaningful details such as their name and any relevant past experiences of the individual.

This model (developed by psychologists Vicki Bruce and Andrew Young) argues that face perception might involve several independent sub-processes working in unison. A "view centered description" is derived from the perceptual input. Simple physical aspects of the face are used to work out age, gender or basic facial expressions. Most analysis at this stage is on feature-by-feature basis. That initial information is used to create a structural model of the face, which allows it to be compared to other faces in memory, and across views. After several exposures to a face this structural code allows us to recognize that face in different contexts.^[28] This explains why the same person seen from a novel angle can still be recognized. This structural encoding can be seen to be specific for upright faces as demonstrated by the Thatcher effect. The structurally encoded representation is transferred to notional "face recognition units" that are used with "personal identity nodes" to identify a person through information from semantic memory. The natural ability to produce someone's name when presented with their face has been shown in experimental research to be damaged in some cases of brain injury, suggesting that naming may be a separate process from the memory of other information about a person.

The study of prosopagnosia (an impairment in recognizing faces which is usually caused by brain injury) has been particularly helpful in understanding how normal face perception might work. Individuals with prosopagnosia may differ in their abilities to understand faces, and it has been the investigation of these differences which has suggested that several stage theories might be correct.

Face perception is an ability that involves many areas of the brain; however, some areas have been shown to be particularly important. Brain imaging studies typically show a great deal of activity in an area of the temporal lobe known as the fusiform gyrus, an area also known to cause prosopagnosia when damaged (particularly when damage occurs on both sides). This evidence has led to a particular interest in this area and it is sometimes referred to as the fusiform face area (FFA) for that reason.^[29]

Neuroanatomy of facial processing

There are several parts of the brain that play a role in face perception. Rossion, Hanseeuw, and Dricot^[30] used BOLD fMRI mapping to identify activation in the brain when subjects viewed both cars and faces. The majority of BOLD fMRI studies use blood oxygen level dependent (BOLD) contrast to determine which areas of the brain are activated by various cognitive functions.^[31] They found that the occipital face area, located in the occipital lobe, the fusiform face area, the superior temporal sulcus, the amygdala, and the anterior/inferior cortex of the temporal lobe, all played roles in contrasting the faces from the cars, with the initial face perception beginning in the fusiform face area and occipital face areas. This entire region links to form a network that acts to distinguish faces. The processing of faces in the brain is known as a "sum of parts" perception.^[32] However, the individual parts of the face must be processed first in order to put all of the pieces together. In early processing, the occipital face area contributes to face perception by recognizing the eyes, nose, and mouth as individual pieces.^[33] Furthermore, Arcurio, Gold, and James^[34] used BOLD fMRI mapping to determine the patterns of activation in the brain when parts of the face were presented in combination and when they were presented singly. The occipital face area is activated by the visual perception of single features of the face, for example, the nose and mouth, and preferred combination of two-eyes over other combinations. This research supports that the occipital face area recognizes the parts of the face at the early stages of recognition. On the contrary, the fusiform face area shows no preference for single features, because the fusiform face area is responsible for "holistic/configural" information,^[35] meaning that it puts all of the processed pieces of the face together in later processing. This theory is supported by the work of Gold et al.^[32] who found that regardless of the orientation of a face, subjects were impacted by the configuration of the individual facial features. Subjects were also impacted by the coding of the relationships between those features. This shows that processing is done by a summation of the parts in the later stages of recognition.

Facial perception has well identified, neuroanatomical correlates in the brain. During the perception of faces, major activations occur in the extrastriate areas bilaterally, particularly in the fusiform face area, the occipital face area (OFA), and the superior temporal sulcus (fSTS).^[36][37] Perceiving an inverted human face involves increased activity in the inferior temporal cortex, while perceiving a misaligned face involves increased activity in the occipital cortex. However, none of these results were found when perceiving a dog face, suggesting that this process may be specific to perception of human faces.^[38]

The fusiform face area is located in the lateral fusiform gyrus. It is thought that this area is involved in holistic processing of faces and it is sensitive to the presence of facial parts as well as the configuration of these parts. The fusiform face area is also necessary for successful face detection and identification. This is supported by fMRI activation and studies on prosopagnosia, which involves lesions in the fusiform face area.^[36][37][39]

The OFA is located in the inferior occipital gyrus.^[37] Similar to the FFA, this area is also active during successful face detection and identification, a finding that is supported by fMRI activation.^[36] The OFA is involved and necessary in the analysis of facial parts but not in the spacing or configuration of facial parts. This suggests that the OFA may be involved in a facial processing step that occurs prior to the FFA processing.^[36]

The fSTS is involved in recognition of facial parts and is not sensitive to the configuration of these parts. It is also thought that this area is involved in gaze perception.^[40] The fSTS has demonstrated increased activation when attending to gaze direction.^{[36] [41]}

Bilateral activation is generally shown in all of these specialized facial areas.^{[42][43][44][45][46][47]} However, there are some studies that include increased activation in one side over the other. For instance McCarthy (1997) has shown that the right fusiform gyrus is more important for facial processing in complex situations.^[39]

Gorno-Tempini and Price have shown that the fusiform gyri are preferentially responsive to faces, whereas the parahippocampal/lingual gyri are responsive to buildings.^[48]

It is important to note that while certain areas respond selectively to faces, facial processing involves many neural networks. These networks include visual and emotional processing systems as well. Emotional face processing research has demonstrated that there are some of the other functions at work. While looking at faces displaying emotions (especially those with fear facial expressions) compared to neutral faces there is increased activity in the right fusiform gyrus. This increased activity also correlates with increased amygdala activity in the same situations.^[49] The emotional processing effects observed in the fusiform gyrus are decreased in patients with amygdala lesions.^[49] This demonstrates possible connections between the amygdala and facial processing areas.^[49]

Another aspect that affects both the fusiform gyrus and the amygdala activation is the familiarity of faces. Having multiple regions that can be activated by similar face components indicates that facial processing is a complex process.^[49] Platek and Kemp (2009) further showed increased brain activation in precuneus and cuneus when differentiation of two faces are easy (e.g., kin and familiar non-kin faces) and the role of posterior medial substrates for visual processing of faces with familiar features (e.g., faces that are averaged with the face of a sibling).^[50]

Ishai and colleagues have proposed the object form topology hypothesis, which posits that there is a topological organization of neural substrates for object and facial processing.^[51] However, Gauthier disagrees and suggests that the category-specific and process-map models could accommodate most other proposed models for the neural underpinnings of facial processing.^[52]

Most neuroanatomical substrates for facial processing are perfused by the middle cerebral artery (MCA). Therefore, facial processing has been studied using measurements of mean cerebral blood flow velocity in the middle cerebral arteries bilaterally. During facial recognition tasks, greater changes in the right middle cerebral artery (RMCA) than the left (LMCA) have been observed.^[53][54] It has been demonstrated that men were right lateralized and women left lateralized during facial processing tasks.^[55]

Just as memory and cognitive function separate the abilities of children and adults to recognize faces, the familiarity of a face may also play a role in the perception of faces.^[32] Zheng, Mondloch, and Segalowitz recorded event-related potentials in the brain to determine the timing of recognition of faces in the brain.^[56] The results of the study showed that familiar faces are indicated and recognized by a stronger N250,^[56] a specific wavelength response that plays a role in the visual memory of faces.^[57] Similarly, Moulson et al.^[58] found that all faces elicit the N170 response in the brain.

Using fMRI with single-unit electrophysiological recordings, Doris Tsao's group revealed^[59] a code that brain uses to process faces in macaques. The brain conceptually needs only ~50 neurons to encode any human face, with facial features projected on individual axes (neurons) in a 50-dimensional "Face Space".

Hemispheric asymmetries in facial processing capability

The mechanisms underlying gender-related differences in facial processing have not been studied extensively.

Studies using electrophysiological techniques have demonstrated gender-related differences during a face recognition memory (FRM) task and a facial affect identification task (FAIT). The male subjects used a right, while the female subjects used a left, hemisphere neural activation system in the processing of faces and facial affect.^[60] Moreover, in facial perception there was no association to estimated intelligence, suggesting that face recognition performance in women is unrelated to several basic cognitive processes.^[61] Gender-related differences^[62] may suggest a role for sex hormones. In females there may be variability for psychological functions^[63] related to differences in hormonal levels during different phases of the menstrual cycle.^[64]

Data obtained in norm and in pathology support asymmetric face processing.^[65][66][67] Gorno-Tempini and others in 2001, suggested that the left inferior frontal cortex and the bilateral occipitotemporal junction respond equally to all face conditions. Some neuroscientists contend that both the left inferior frontal cortex (Brodmann area 47) and the occipitotemporal junction are implicated in facial memory.^[68][69][70] The right inferior temporal/fusiform gyrus responds selectively to faces but not to non-faces. The right temporal pole is activated during the discrimination of familiar faces and scenes from unfamiliar ones.^[71] Right asymmetry in the mid temporal lobe for faces has also been shown using 133-Xenon measured cerebral blood flow (CBF).^[72] Other investigators have observed right lateralization for facial recognition in previous electrophysiological and imaging studies.^[73]

The implication of the observation of asymmetry for facial perception would be that different hemispheric strategies would be implemented. The right hemisphere would be expected to employ a holistic strategy, and the left an analytic strategy.^{[74][75][76][77]} In 2007, Philip Njemanze, using a novel functional transcranial Doppler (fTCD) technique called functional transcranial Doppler spectroscopy (fTCDS), demonstrated that men were right lateralized for object and facial perception, while women were left lateralized for facial tasks but showed a right tendency or no lateralization for object perception.^[78] Njemanze demonstrated using fTCDS, summation of responses related to facial stimulus complexity, which could be presumed as evidence for topological organization of these cortical areas in men. It may suggest that the latter extends from the area implicated in object perception to a much greater area involved in facial perception.

This agrees with the object form topology hypothesis proposed by Ishai and colleagues in 1999. However, the relatedness of object and facial perception was process-based, and appears to be associated with their common holistic processing strategy in the right hemisphere. Moreover, when the same men were presented with facial paradigm requiring analytic processing, the left hemisphere was activated. This agrees with the suggestion made by Gauthier in 2000, that the extrastriate cortex contains areas that are best suited for different computations, and described as the process-map model. Therefore, the proposed models are not mutually exclusive, and this underscores the fact that facial processing does not impose any new constraints on the brain other than those used for other stimuli.

It may be suggested that each stimulus was mapped by category into face or non-face, and by process into holistic or analytic. Therefore, a unified category-specific process-mapping system was implemented for either right or left cognitive styles. Njemanze in 2007, concluded that, for facial perception, men used a category-specific process-mapping system for right cognitive style, but women used same for the left.

Cognitive neuroscience

Cognitive neuroscientists Isabel Gauthier and Michael Tarr are two of the major proponents of the view that face recognition involves expert discrimination of similar objects (See the Perceptual Expertise Network). Other scientists, in particular Nancy Kanwisher and her colleagues, argue that face recognition involves processes that are face-specific and that are not recruited by expert discriminations in other object classes (see the domain specificity).

Studies by Gauthier have shown that an area of the brain known as the fusiform gyrus (sometimes called the fusiform face area because it is active during face recognition) is also active when study participants are asked to discriminate between different types of birds and cars,^[79] and even when participants become expert at distinguishing computer generated nonsense shapes known as greebles.^[80] This suggests that the fusiform gyrus may have a general role in the recognition of similar visual objects. Yaoda Xu, then a post doctoral fellow with Nancy Kanwisher, replicated the car and bird expertise study using an improved fMRI design that was less susceptible to attentional accounts.

The activity found by Gauthier when participants viewed non-face objects was not as strong as when participants were viewing faces, however this could be because we have much more expertise for faces than for most other objects. Furthermore, not all findings of this research have been successfully replicated, for example, other research groups using different study designs have found that the fusiform gyrus is specific to faces and other nearby regions deal with non-face objects.^[81]

However, these failures to replicate are difficult to interpret, because studies vary on too many aspects of the method. It has been argued that some studies test experts with objects that are slightly outside of their domain of expertise. More to the point, failures to replicate are null effects and can occur for many different reasons. In contrast, each replication adds a great deal of weight to a particular argument. With regard to "face specific" effects in neuroimaging, there are now multiple replications with Greebles, with birds and cars,^[82] and two unpublished studies with chess experts.^[83][84]

Although it is sometimes found that expertise recruits the FFA (e.g. as hypothesized by a proponent of this view in the preceding paragraph), a more common and less controversial finding is that expertise leads to focal category-selectivity in the fusiform gyrus—a pattern similar in terms of antecedent factors and neural specificity to that seen for faces. As such, it remains an open question as to whether face recognition and expert-level object recognition recruit similar neural mechanisms across different subregions of the fusiform or whether the two domains literally share the same neural substrates. Moreover, at least one study argues that the issue as to whether expertise-predicated category-selective areas overlap with the FFA is nonsensical in that multiple measurements of the FFA within an individual person often overlap no more with each other than do measurements of FFA and expertise-predicated regions.^[85] At the same time, numerous studies have failed to replicate them altogether.^{[citation needed]} For example, four published fMRI studies have asked whether expertise has any specific connection to the FFA in particular, by testing for expertise effects in both the FFA and a nearby but not face-selective region called LOC (Rhodes et al., JOCN 2004; Op de Beeck et al., JN 2006; Moore et al., JN 2006; Yue et al. VR 2006). In all four studies, expertise effects are significantly stronger in the LOC than in the FFA, and indeed expertise effects were only borderline significant in the FFA in two of the studies, while the effects were robust and significant in the LOC in all four studies.^{[citation needed]}

Therefore, it is still not clear in exactly which situations the fusiform gyrus becomes active, although it is certain that face recognition relies heavily on this area and damage to it can lead to severe face recognition impairment.

Self-face perception

Studies regarding face perception have also looked specifically at self-face perception. One study found that the perception/recognition of one's own face was unaffected by changing contexts, while the perception/recognition of familiar and unfamiliar faces was adversely affected.^[86] Another study that focused on older adults found that they had self-face advantage in configural processing but not featural processing.^[87]

Face advantage in memory recall

During face perception, neural networks make connections with the brain to recall memories.^[88] According to the Seminal Model of face perception, there are three stages of face processing including recognition of the face, the recall of memories and information that are linked with that face, and finally name recall.^[27][88] There are, however, exceptions to this order. For example, names are recalled faster than semantic information in cases of highly familiar stimuli.^[89] While the face is powerful identifier of individuals, the voice also helps in the recognition of people and is an identifier for important information.^[90][91]

Research has been conducted to see if faces or voices make it easier to identify individuals and recall semantic memory and episodic memory.^[92] These experiments look at all three stages of face processing. The experiment method was to show two groups celebrity and familiar faces or voices with a between-group design and ask the participants to recall information about them.^[92] The participants are first asked if the stimulus is familiar. If they answer yes then they are asked to information (semantic memory) and memories they have of the person (episodic memory) that fits the face or voice presented. These experiments all demonstrate the strong phenomenon of the face advantage and how it persists through different follow-up studies with different experimental controls and variables.^[92]

Recognition-performance issue

After the first experiments on the advantage of faces over voices in memory recall, errors and gaps were found in the methods used.^[92] For one, there was not a clear face advantage for the recognition stage of face processing. Participants showed a familiarity-only response to voices more often than faces.^[93] In other words, when voices were recognized (about 60-70% of the time) they were much harder to recall biographical information but very good at being recognized.^[92] The results were looked at as remember versus know judgements. A lot more remember results (or familiarity) occurred with voices, and more know (or memory recall) responses happened with faces.^[91] This phenomenon persists through experiments dealing with criminal line-ups in prisons. Witnesses are more likely to say that a suspect's voice sounded familiar than his/her face even though they cannot remember anything about the suspect.^[94] This discrepancy is due to a larger amount of guesswork and false alarms that occur with voices.^[91]

To give faces a similar ambiguity to that of voices, the face stimuli were blurred in the follow-up experiment.^[93] This experiment followed the same procedures as the first, presenting two groups with sets of stimuli made up of half celebrity faces and half unfamiliar faces.^[92] The only difference was that the face stimuli were blurred so that detailed features could not be seen. Participants were then asked to say if they recognized the person, if they could recall specific biographical information about them, and finally if they knew the person's name. The results were completely different from those of the original experiment, supporting the view that there were problems in the first experiment's methods.^[92] According to the results of the followup, the same amount of information and memory could be recalled through voices and faces, dismantling the face advantage. However, these results are flawed and premature because other methodological issues in the experiment still needed to be fixed.^[92]

Content of speech

The process of controlling the content of speech extract has proven to be more difficult than the elimination of non facial cues in photographs.^[92] Thus the findings of experiments that did not control this factor lead to misleading conclusions regarding the voice recognition over the face recognition.^[92] For example, in an experiment it was found that 40% of the time participants could easily pair the celebrity-voice with their occupation just by guessing.^[93] In order to eliminate these errors, experimenters removed parts of the voice samples that could possibly give clues to the identity of the target, such as catchphrases.^[95] Even after controlling the voice samples as well as the face samples (using blurred faces), studies have shown that semantic information can be more accessible to retrieve when individuals are recognizing faces than voices.^[96]

Another technique to control the content of the speech extracts is to present the faces and voices of personally familiar individuals, like the participant's teachers or neighbors, instead of the faces and voices of celebrities.^[92] In this way alike words are used for the speech extracts.^[92] For example, the familiar targets are asked to read exactly the same scripted speech for their voice extracts. The results showed again that semantic information is easier to retrieve when individuals are recognizing faces than voices.^[92]

Frequency-of-exposure issue

Another factor that has to be controlled in order for the results to be reliable is the frequency of exposure.^[92] If we take the example of celebrities, people are exposed to celebrities' faces more often than their voices because of the mass media.^[92] Through magazines, newspapers and the Internet, individuals are exposed to celebrities' faces without their voices on an everyday basis rather than their voices without their faces.^[92] Thus, someone could argue that for all of the experiments that were done until now the findings were a result of the frequency of exposure to the faces of celebrities rather than their voices.^[97]

To overcome this problem researchers decided to use personally familiar individuals as stimuli instead of celebrities.^[92] Personally familiar individuals, such as participant's teachers, are for the most part heard as well as seen.^[98] Studies that used this type of control also demonstrated the face advantage.^[98] Students were able to retrieve semantic information more readily when recognizing their teachers faces (both normal and blurred) rather than their voices.^[96]

However, researchers over the years have found an even more effective way to control not only the frequency of exposure but also the content of the speech extracts, the associative learning paradigm.^[92] Participants are asked to link semantic information as well as names with pre-experimentally unknown voices and faces.^[99][100] In a current experiment that used this paradigm, a name and a profession were given together with, accordingly, a voice, a face or both to three participant groups.^[99] The associations described above were repeated four times.^[99] The next step was a cued recall task in which every stimulus that was learned in the previous phase was introduced and participants were asked to tell the profession and the name for every stimulus.^[99][101] Again, the results showed that semantic information can be more accessible to retrieve when individuals are recognizing faces than voices even when the frequency of exposure was controlled.^[92][99]

Extension to episodic memory and explanation for existence

Episodic memory is our ability to remember specific, previously experienced events.^[102] In recognition of faces as it pertains to episodic memory, there has been shown to be activation in the left lateral prefrontal cortex, parietal lobe, and the left medial frontal/anterior cingulate cortex.^[103][104] It was also found that a left lateralization during episodic memory retrieval in the parietal cortex correlated strongly with success in retrieval.^[103] This may possibly be due to the hypothesis that the link between face recognition and episodic memory were stronger than those of voice and episodic memory.^[93] This hypothesis can also be supported by the existence of specialized face recognition devices thought to be located in the temporal lobes.^[103][105] There is also evidence of the existence of two separate neural systems for face recognition: one for familiar faces and another for newly learned faces.^[103] One explanation for this link between face recognition and episodic memory is that since face recognition is a major part of human existence, the brain creates a link between the two in order to be better able to communicate with others.^[106]

Ethnicity

Main page: Cross-race effect

Differences in own- versus other-race face recognition and perceptual discrimination was first researched in 1914.^[107] Humans tend to perceive people of other races than their own to all look alike:

Other things being equal, individuals of a given race are distinguishable from each other in proportion to our familiarity, to our contact with the race as whole. Thus, to the uninitiated American all Asiatics look alike, while to the Asiatics, all White men look alike.^[107]

This phenomenon is known mainly as the cross-race effect, but is also called the own-race effect, other-race effect, own race bias, or interracial face-recognition deficit.^[108]

In 1990, Mullen reported finding evidence that the other-race effect is larger among White subjects than among African American subjects, whereas Brigham and Williamson (1979, cited in Shepherd, 1981) obtained the opposite pattern.^[109] However, it is difficult to measure the true influence of the cross-race effect. D. Stephen Lindsay and colleagues note that results in these studies could be due to intrinsic difficulty in recognizing the faces presented, an actual difference in the size of cross-race effect between the two test groups, or some combination of these two factors.^[109] Shepherd reviewed studies that found better performance on both African American^[110] and White faces,^[111] and yet Shepherd also reviewed other studies in which no difference was found.^[112] Overall, Shepherd reported a reliable positive correlation between the size of the effect and the amount of interaction subjects had with members of the other race. This correlation reflects the fact that African American subjects, who performed equally well on faces of both races in Shepherd's study, almost always responded with the highest possible self-rating of amount of interaction with white people (M = 4.75; with 5 being the most interaction with people of that race, 1 being the least), whereas their white counterparts both displayed a larger other-race effect and reported less other-race interaction (M = 2.13). This difference in rating was found statistically reliable, £(30) = 7.86, p < .01 .^[109]

The cross-race effect seems to appear in humans around 6 months of age.^[113] Cross-race effects can be changed from early childhood through adulthood through interaction with people of other races.^[114] Other-race experience in own- versus other-race face processing is a major influence on the cross-race effect (O'Toole et al., 1991; Slone et al., 2000; Walker & Tanaka, 2003). In a series of studies, Walker and colleagues revealed that participants with greater other-race experience were consistently more accurate at discriminating between other-race faces than were participants with less other-race experience (Walker & Tanaka, 2003; Walker & Hewstone, 2006a,b; 2007). Many current models of the cross-race effect assume that holistic face processing mechanisms are more fully engaged when viewing own-race faces compared to other-race faces.^[115]

The own-race effect appears to be related to increased ability to extract information about the spatial relationships between different facial features.^[116] Levin (2000)^[117] writes that a deficit occurs when viewing people of another race because visual information specifying race takes up mental attention at the expense of individuating information when recognizing faces of other races.^[117] Further research using perceptual tasks could shed light on the specific cognitive processes involved in the other-race effect.^[109] Bernstein et al. (2007) demonstrate that the own-race effect likely extends beyond racial membership into in-group versus out-group concepts. It was shown that categorizing somebody by the university he or she attends results in similar results compared to studies about the own-race effect.^[118] Hugenberg, Miller, and Claypool (2007) shed light on overcoming the own-race effect. They performed a study in which they introduced people to the concept of the own-race effect before presenting them faces and found that if people were made aware of the own-race effect prior to the experiment, the test subjects showed significantly less if any own-race effect.^[119]

Studies on adults have also shown sex differences in face recognition: Men tend to recognize fewer faces of women than women do, whereas there are no sex differences with regard to male faces.^[120]

In individuals with autism spectrum disorder

Autism spectrum disorder (ASD) is a comprehensive neural developmental disorder that produces many deficits including social, communicative,^[121] and perceptual deficits.^[122] Of specific interest, individuals with autism exhibit difficulties in various aspects of facial perception, including facial identity recognition and recognition of emotional expressions.^[123][124] These deficits are suspected to be a product of abnormalities occurring in both the early and late stages of facial processing.^[125]

Speed and methods

People with ASD process face and non-face stimuli with the same speed.^[125][126] In typically developing individuals, there is a preference for face processing, thus resulting in a faster processing speed in comparison to non-face stimuli.^[125][126] These individuals primarily utilize holistic processing when perceiving faces.^[122] Contrastingly, individuals with ASD employ part-based processing or bottom-up processing, focusing on individual features rather than the face as a whole.^[127][128] When focusing on the individual parts of the face, persons with ASD direct their gaze primarily to the lower half of the face, specifically the mouth, varying from the eye trained gaze of typically developing people.^{[127][128][129][130][131]} This deviation from holistic face processing does not employ the use of facial prototypes, which are templates stored in memory that make for easy retrieval.^[124][132]

Additionally, individuals with ASD display difficulty with recognition memory, specifically memory that aids in identifying faces. The memory deficit is selective for faces and does not extend to other objects or visual inputs.^[124] Some evidence lends support to the theory that these face-memory deficits are products of interference between connections of face processing regions.^[124]

Associated difficulties

The atypical facial processing style of people with ASD often manifests in constrained social ability, due to decreased eye contact, joint attention, interpretation of emotional expression, and communicative skills.^[133] These deficiencies can be seen in infants as young as 9 months; specifically in terms of poor eye contact and difficulties engaging in joint attention.^[125] Some experts have even used the term 'face avoidance' to describe the phenomena where infants who are later diagnosed with ASD preferentially attend to non-face objects over faces.^[121] Furthermore, some have proposed that the demonstrated impairment in children with ASD's ability to grasp emotional content of faces is not a reflection of the incapacity to process emotional information, but rather, the result of a general inattentiveness to facial expression.^[121] The constraints of these processes that are essential to the development of communicative and social-cognitive abilities are viewed to be the cause of impaired social engagement and responsivity.^[134] Furthermore, research suggests that there exists a link between decreased face processing abilities in individuals with ASD and later deficits in Theory of Mind; for example, while typically developing individuals are able to relate others' emotional expressions to their actions, individuals with ASD do not demonstrate this skill to the same extent.^[135]

There is some contention about this causation however, resembling the chicken or the egg dispute. Others theorize that social impairment leads to perceptual problems rather than vice versa.^[127] In this perspective, a biological lack of social interest inherent to ASD inhibits developments of facial recognition and perception processes due to under-use.^[127] Continued research is necessary to determine which theory is best supported.

Neurology

Many of the obstacles that individuals with ASD face in terms of facial processing may be derived from abnormalities in the fusiform face area and amygdala, which have been shown to be important in face perception as discussed above. Typically, the fusiform face area in individuals with ASD has reduced volume compared to normally developed persons.^[136] This volume reduction has been attributed to deviant amygdala activity that does not flag faces as emotionally salient and thus decreases activation levels of the fusiform face area. This hypoactivity in the fusiform face area has been found in several studies.^[127]

Studies are not conclusive as to which brain areas people with ASD use instead. One study found that, when looking at faces, people with ASD exhibit activity in brain regions normally active when typically developing individuals perceive objects.^[127] Another study found that during facial perception, people with ASD use different neural systems, with each one of them using their own unique neural circuitry.^[136]

Traumatic Brain Injury

Difficulties in facial emotion processing can also be seen in individuals with traumatic brain injury, in both diffuse axonal injury and focal brain injury.^[137]

Compensation mechanisms

As ASD individuals age, scores on behavioral tests assessing ability to perform face-emotion recognition increase to levels similar to controls.^[125][138] Yet, it is apparent that the recognition mechanisms of these individuals are still atypical, though often effective.^[138] In terms of face identity-recognition, compensation can take many forms including a more pattern-based strategy which was first seen in face inversion tasks.^[130] Alternatively, evidence suggests that older individuals compensate by using mimicry of other's facial expressions and rely on their motor feedback of facial muscles for face emotion-recognition.^[139] These strategies help overcome the obstacles individuals with ASD face in interacting within social contexts.

In individuals with schizophrenia

Attention, perception, memory, learning, processing, reasoning, and problem solving are known to be affected in individuals with schizophrenia.^[140] Schizophrenia has been linked to impaired face and emotion perception.^{[140][3][141][86]} People with schizophrenia demonstrate worse accuracy and slower response time in face perception tasks in which they are asked to match faces, remember faces, and recognize which emotions are present in a face.^[86] People with schizophrenia have more difficulty matching upright faces than they do with inverted faces.^[140] A reduction in configural processing, using the distance between features of an item for recognition or identification (e.g. features on a face such as eyes or nose), has also been linked to schizophrenia.^[86] Schizophrenia patients are able to easily identify a "happy" affect but struggle to identify faces as "sad" or "fearful".^[141] Impairments in face and emotion perception are linked to impairments in social skills, due to the individual's inability to distinguish facial emotions.^[141][86] People with schizophrenia tend to demonstrate a reduced N170 response, atypical face scanning patterns, and a configural processing dysfunction.^[142] The severity of schizophrenia symptoms has been found to correlate with the severity of impairment in face perception.^[86]

Individuals with diagnosed schizophrenia and antisocial personality disorder have been found to have even more impairment in face and emotion perception than individuals with just schizophrenia. These individuals struggle to identify anger, surprise, and disgust. There is a link between aggression and emotion perception difficulties for people with this dual diagnosis.^[141]

Data from magnetic resonance imaging and functional magnetic resonance imaging has shown that a smaller volume of the fusiform gyrus is linked to greater impairments in face perception.^[3]

There is a positive correlation between self-face recognition and other-face recognition difficulties in individuals with schizophrenia. The degree of schizotypy has also been shown to correlate with self-face difficulties, unusual perception difficulties, and other face recognition difficulties.^[143] Schizophrenia patients report more feelings of strangeness when looking in a mirror than do normal controls. Hallucinations, somatic concerns, and depression have all been found to be associated with self-face perception difficulties.^[144]

Other animals

Neurobiologist Jenny Morton and her team have been able to teach sheep to choose a familiar face over unfamiliar one when presented with two photographs, which has led to the discovery that sheep can recognise human faces.^[145][146] Archerfish (distant relatives of humans) were able to differentiate between forty-four different human faces, which supports the theory that there is no need for a neocortex or a history of discerning human faces in order to do so.^[147] Pigeons were found to use the same parts of the brain as humans do to distinguish between happy and neutral faces or male and female faces.^[147]

Artificial

This section includes a list of references, related reading or external links, but its sources remain unclear because it lacks inline citations. Please help to improve this section by introducing more precise citations. (May 2020) (Learn how and when to remove this template message)

A great deal of effort has been put into developing software that can recognize human faces. Much of the work has been done by a branch of artificial intelligence known as computer vision which uses findings from the psychology of face perception to inform software design. Recent breakthroughs using noninvasive functional transcranial Doppler spectroscopy as demonstrated by Njemanze, 2007, to locate specific responses to facial stimuli have led to improved systems for facial recognition. The new system uses input responses called cortical long-term potentiation (CLTP) derived from Fourier analysis of mean blood flow velocity to trigger target face search from a computerized face database system.^[148][149] Such a system provides for brain-machine interface for facial recognition, and the method has been referred to as cognitive biometrics.

Another application is the estimation of human age from face images. As an important hint for human communication, facial images contain lots of useful information including gender, expression, age, etc. Unfortunately, compared with other cognition problems, age estimation from facial images is still very challenging. This is mainly because the aging process is influenced not only by a person's genes but also many external factors. Physical condition, living style etc. may accelerate or slow the aging process. Besides, since the aging process is slow and with long duration, collecting sufficient data for training is fairly demanding work.^[150]

In the field of face recognition, there have been numerous attempts and studies that try to elucidate the temporal process of individual face recognition, and to do so techniques with superior temporal resolution such as MEG and EEG are frequently used. One of the famous temporal signature during face recognition is well-known N170 ERP component assumed to arise from the fusiform face area (FFA), but there is a controversy around N170 component, whether it is also related to individual face recognition or not. Still, there is a possibility for N170 component’s association that marks the moment of identity related responses during individual face recognition. In addition, some studies claim earlier or later components to be the main temporal landmark for individual face recognition, such as P1, N250, or N400 components. As seen here, consensus is yet to be reached among the studies.

Such discrepancy can be caused by lots of factors, but mostly due to univariate analyses of EEG signals as univariate analyses seem to fail to capture the essence of temporal profile of face recognition. Recent studies have revealed extensive network of cortical regions that contribute to individual face recognition, including face-selective regions such as fusiform face area (FFA). Nemrodov et al. (2016) conducted multivariate analyses of EEG signals that might be involved in identity related information and applied pattern classification to ERP signals both in time and in space. Main target of the study was as follows: 1) evaluating whether previously known ERP components such as N170 and others are involved in individual face recognition or not, 2) locating temporal landmarks of individual level recognition from ERP signals, and 3) figuring out the spatial profile of individual face recognition.  

For the experiment, conventional ERP analyses and pattern classification of ERP signals were conducted given preprocessed EEG signals. Experimental stimuli and detailed procedures can be found in the reference. (Nemrodov et al. 2019)

Result showed that N170 and N250 components are indeed involved in individual face recognition task. However, such sensitivity seen at these components might not have been caused solely by identity discrimination process. Also, this study revealed the temporal profile of identity discrimination through pattern classification to ERP signals. Extensive sensitivity was seen to information that is related to individual identity, and was present not only at the N170 but also at other components in temporal and spatial domain. There were other components that contributed to identity discrimination. P1, for example, mainly dealt with category-level face processing, signifying early onset of individualization. 75 to 100ms of exposure was sufficient to extract necessary information to perform identity discrimination task. Signals related to encoding of individual face identity can also be extracted from pattern of ERP signals during face processing. In terms of spatial profile, occipital lobe showed greatest activation at 100ms, and activation spread out to both anterior and posterior regions. In conclusion, identity related information is widely distributed in temporal and spatial domain.

Since there exists a temporal profile of identity discrimination, Nemrodov et al. (2019) stepped further and conducted a study that reverse-engineers the encoding of individual face recognition information and decodes the neural activities to reconstruct the facial image. The main goal was to uncover the representational basis of individual face recognition by addressing these questions: 1) Can image reconstruction separately recover facial shape and surface information from different modalities? 2) What is the spatiotemporal profile of shape and surface processing? 3) What specific shape/surface features are recovered through reconstruction? 4) Do different modalities reveal similar information about face representations?

Experimental stimuli were images of 54 young adult Caucasian males, both happy and neutral face expressions, similar to the stimuli setting in Nemrodov et al. (2016) Data that are required for reconstruction were behavioral, EEG, fMRI, and Theoretical Observer data, all previously collected by other studies. Further procedural details can be found in the reference. Reconstruction process first started off with creating representational similarity matrices for each data type. Then face space is estimated through application of multidimensional scaling to the similarity matrix. For this study, shape and surface were selected as main features that might show significance in reconstruction process. Thus, shape and surface features are extracted from the face space. Among extracted features, highly correlated and informative features are selected and formed a subset. Then, target face was projected into face space. Reconstruction of shape and surface was done by linear combination of informative features, and amalgamation of average features with classification features leads to fully reconstructed target face.

Shape and surface reconstructions were possible for all data types, although surface had more advantage over shape in terms of face representations. Yet, both surface and shape were considered as informative properties in face recognition. For temporal profile of shape and surface processing, around 150ms seemed to be critical moment, perhaps where integration of shape and surface features happen, leading to successful face recognition, as face reconstruction accuracy also reached significance around that time frame. For spatial profile, result supported the importance of fusiform face area (FFA) and also occipital face area (OFA) in their abilities to integrate and represent the features. Comparing two features, shape was more consistently recovered than surface in all modalities. This might reflect the face that surface information was processed in other cortical areas beforehand.

In conclusion, both studies showed existence of spatiotemporal profile of individual face recognition process and reconstruction of individual face images was possible by utilizing such profile and informative features that contribute to encoding of identity related information. This profile is not yet complete. There will be more distributed cortical areas that contribute to the individual face recognition, perhaps more higher-level features.

Genetic basis

While it has been widely recognized that many cognitive abilities, such as general intelligence, have genetic bases, evidence for the genetic basis of facial recognition abilities specifically is fairly recent. Some of the earliest published research on the relationship between facial recognition and genetics focused on the genetic bases of facial recognition in the context of genetic disorders which impair facial recognition abilities, such as Turner syndrome. In a study by Lawrence, K. et al. in 2003^[87] the authors found significantly poorer facial recognition abilities in individuals with Turner syndrome, a genetic disorder which results in impaired amygdala functioning, suggesting that amygdala functioning may impact face perception.^[87]

Evidence for the genetic basis of facial recognition abilities in the general population, however, comes from studies on face perception in twin participants by Wilmer, J. B. et al. in 2009,^[151] in which the facial recognition scores on the Cambridge Face Memory test were twice as similar for monozygotic twins in comparison to dizygotic twins.^[151] This finding was supported by a twin study on the genetic bases of facial recognition by Zhu, Q. et al. in (2009) which found a similar difference in facial recognition scores when comparing monozygotic and dizygotic twins^[152] and Shakeshaft, N. G. & Plomin, R. (2015),^[153] which determined the heritability of facial recognition to be approximately 61%, using a similar set of twin studies.^[153] There was also no significant relationship identified between facial recognition scores and measures of any other cognitive abilities,^[151] most notably the lack of a correlation with general object recognition abilities. This suggests that facial recognition abilities are not only heritable, but that their genetic basis is independent from the bases of other cognitive abilities and are specialized for face perception.^[151] Research by Cattaneo, Z. et al. (2016)^[154] and suggest that the more extreme examples of facial recognition abilities, specifically hereditary prosopagnosics, are also highly genetically correlated.^[154]

For hereditary prosopagnosics, an autosomal dominant model of inheritance has been proposed by Kennerknecht, I. et al. (2006).^[155] Research by Cattaneo, Z. et al. (2016)^[154] also correlated the probability of hereditary prosopagnosia with the presence of single nucleotide polymorphisms^[154] along the Oxytocin receptor gene (OXTR), specifically at nucleotides rs2254298 and rs53576 on OXTR intron three,^[154] suggesting that these alleles may serve a critical role in normal face perception. Mutation from the wild type allele at these loci has also been found to result in other disorders in which social and facial recognition deficits are common,^[154] such as autism spectrum disorder, which may imply that the genetic bases for general facial recognition are complex and polygenic.^[154] This relationship between the OXTR gene and facial recognition abilities is also supported by studies of individuals who do not suffer from hereditary prosopagnosia by Melchers, M. et al. (2013)^[156] and Westberg, L. et al. (2016)^[157] which correlated general facial recognition abilities with different polymorphisms of the OXTR gene, specifically rs7632287^[157] and rs2268498.^[156]

Further research is needed to confirm the specific mechanisms of these genetic components on face perception; however, current evidence does suggest that facial recognition abilities are highly linked to genetic, rather than environmental, bases.

See also

• Apophenia, seeing meaningful patterns in random data
• Autism
• Capgras delusion
• Cognitive neuropsychology
• Cross-race effect
• Delusional misidentification syndrome
• Facial expression
• Facial recognition system
• Fregoli delusion
• Hollow-Face illusion
• Nonverbal learning disorder
• Pareidolia
• Prosopagnosia
• Social cognition
• Social intelligence
• Super recogniser

References

Further reading

• Bruce, V. and Young, A. (2000) In the Eye of the Beholder: The Science of Face Perception. Oxford: Oxford University Press. ISBN:0-19-852439-0

External links

• Face Recognition Homepage
• Science Aid: Face Recognition
• FaceResearch – Scientific research and online studies on face perception
• Face Blind Prosopagnosia Research Centers at Harvard and University College London
• Face Recognition Tests - online tests for self-assessment of face recognition abilities.
• Perceptual Expertise Network (PEN) Collaborative group of cognitive neuroscientists studying perceptual expertise, including face recognition.
• Face Lab at the University of Western Australia
• Perception Lab at the University of St Andrews, Scotland
• The effect of facial expression and identity information on the processing of own and other race faces by Yoriko Hirose, PhD thesis from the University of Stirling
• Global Emotion Online-Training to overcome Caucasian-Asian other-race effect

0.00 (0 votes)