Individual differences in eyewitness accuracy across multiple lineups of faces

Eyewitness identifications are crucial for police investigations to determine the perpetrators of crime. Many eyewitnesses make perfectly accurate identifications, but many are also prone to error. In criminal investigations, the difficulty therefore arises in differentiating accurate from inaccurate eyewitnesses. Whereas eyewitnesses typically attempt to identify a perpetrator once, theories of face recognition in cognitive psychology stipulate that accurate identification is characterized by the ability to recognize the same face repeatedly. However, such theories are based on the recognition of familiar people, who are well-known to an observer, rather than the recognition of unfamiliar people, which typically gives rise to error in eyewitness scenarios. We therefore tested the ability of observers to identify an unfamiliar person repeatedly in an eyewitness paradigm to determine the extent to which this is possible. We argue that an observer’s ability to identify a target person repeatedly should increase our confidence that an accurate eyewitness identification has been made by this individual. Across three experiments, we find that only a minority of observers can act on a target consistently. These findings suggest that we should measure the degree of familiarity that an eyewitness has gained with a target to better assess their identification accuracy and indicate that this could be achieved at an individual level with multiple lineups of faces.

In criminal investigations, eyewitnesses are routinely required to identify a previously seen perpetrator from a police lineup. In the UK alone, tens of thousands of identity lineups are administered for this purpose every year (e.g. http://www.viper.police.uk). Under these circumstances, many individuals make perfectly accurate eyewitness identifications. However, eyewitness misidentifications are also frequently made, whereby innocent people in a lineup are mistaken for a perpetrator (e.g. Memon, Havard, Clifford, Gabbert, & Watt, 2011; Slater, 1994; Wright & McDaid, 1996). For investigators, the difficulty therefore arises in differentiating individuals that are accurate from those that are inaccurate eyewitnesses.

One approach under investigation in psychology to address this problem is a multiple-lineup method. In this method, eyewitnesses are required to identify a perpetrator repeatedly, but from different person aspects that might have been observed at a crime scene, such as the face, body, voice, clothing, or accessories (Lindsay, Wallbridge, & Drennan, 1987; Pryke, Lindsay, Dysart, & Dupuis, 2004; Sauerland & Sporer, 2008; Sauerland, Stockmar, Sporer, & Broers, 2013). These studies show that the selection of the same identity from different lineup combinations can be used to assess the likelihood that a correct target selection has been made. Sauerland and Sporer (2008) found, for example, that identification of a suspect’s body from a lineup indicated only a .60 probability that the identified person was, in fact, guilty. However, this number rose to .91 when the separate identification of body and face cues, from two different lineups, were considered together.

These findings illustrate the promise that multiple lineups hold for assessing the accuracy of individual eyewitnesses. However, the results of these studies are curtailed by the poor identification accuracy for some person aspects. For example, in studies that have examined multiple-lineup procedures, correct identifications of voices were obtained on only 27% of trials (Pryke et al., 2004) and this number was lower still for bodies and accessories, at 18% and 11%, respectively (Sauerland & Sporer, 2008). In comparison, identification accuracy was much higher for frontal views of faces, at 72% (Pryke et al., 2004) and 61% (Sauerland & Sporer, 2008), and combinations of other person aspects with such frontal face portraits were most useful for diagnosing eyewitness accuracy.

In this study, we sought to investigate this face advantage further, by exploring a new variant of this procedure in which multiple lineups only comprised faces. This manipulation makes good sense given the comparatively high recognition accuracy for faces in previous multiple-lineup studies, but it also has a strong theoretical grounding in the face perception literature. According to cognitive theories of face processing (e.g. Bruce & Young, 1986; Burton, Bruce, & Johnston, 1990; Burton, Jenkins, Hancock, & White, 2005; Haxby, Hoffman, & Gobbini, 2000; Schweinberger & Burton, 2003), the successful recognition of familiar people, such as family, friends, or colleagues, is highly robust and can be triggered by any instance of their face. The ultimate hallmark of accurate person identification is therefore the ability to recognize the same person’s face repeatedly, across many different encounters.

In line with this research, eyewitness identification errors are made rarely when the perpetrator is someone that is already known to a witness (e.g. Memon et al., 2011). Familiarity with a face is, however, a continuum (e.g. Clutterbuck & Johnston, 2002, 2004), which reflects the exposure duration to an identity (e.g. Bornstein, Deffenbacher, Penrod, & McGorty, 2012; Memon, Hope, & Bull, 2003; Roark, O’Toole, Abdi, & Barrett, 2006) as well as the variability in a person’s appearance across different exposures (Andrews, Jenkins, Cursiter, & Burton, 2015; Burton, Kramer, Ritchie, & Jenkins, 2016; Dowsett, Sandford, & Burton, 2016; Murphy, Ipser, Gaigg, & Cook, 2015). Eyewitnesses who are initially unfamiliar with a perpetrator cannot acquire strong familiarity (e.g. as they would in the case of the faces of family, friends, celebrities) with this person’s face at a crime scene. As a consequence, identification of such unfamiliar people can be rather difficult, even under best-possible conditions (e.g. Bruce, Henderson, Greenwood, Hancock, Burton, et al., 1999; Henderson, Bruce, & Burton, 2001; Megreya & Burton, 2006, 2008).

Another factor appears to determine eyewitness performance, as identification accuracy varies even when exposure to a perpetrator is held constant across participants. One possibility is that this reflects individual differences in the ability to recognize unfamiliar faces, whereby some individuals are naturally equipped better than others for this task (e.g. Wilmer, Germine, Chabris, Chatterjee, Williams, et al., 2010; Zhu, Song, Hu, Li, Tian, et al., 2010). Support for this reasoning comes from studies that have revealed correlations of eyewitness accuracy with tests of face recognition (e.g. Geiselman, Tubridy, Blumkin, Schroppel, Turner, et al., 2001; Hosch, 1994; Morgan, Hazlett, Baranoski, Doran, Southwick, et al., 2007). Bindemann, Brown, Koyas, and Russ (2012) showed, for example, that eyewitness identification accuracy for the perpetrator of a staged crime correlated with performance on a laboratory test of face recognition, in which observers had to select newly learned face targets from identity lineups. So far, however, these studies have only examined this link for a single eyewitness identification.

This is an important issue as there is evidence to suggest that, in contrast to familiar face recognition, the repeated recognition of unfamiliar faces might also be particularly difficult. Studies that speak to this issue have focused primarily on unfamiliar face matching tasks, in which observers have to decide whether side-by-side photographs of two unfamiliar faces depict the same person or different people. For example, in these studies observers often decide that a face pair depicts two different people in one block of trials, but classify this pair as depicting the same person in a subsequent block (Alenezi & Bindemann, 2013; Alenezi, Bindemann, Fysh, & Johnston, 2015) or on a different day (Bindemann, Avetisyan, & Rakow, 2012). When these effects are assessed at a group level, by averaging performance across participants, identification accuracy declines with each repetition of the face pairs. However, observers also exhibit broad inter-individual (e.g. Burton, White, & McNeill, 2010) and intra-individual differences (Bindemann et al., 2012).

The differences between familiar and unfamiliar face processing, and the individual differences that are observed in unfamiliar face matching tasks, have important implications for our understanding of eyewitness identification, for two reasons. First, a lineup task is essentially a test of the familiarity that an eyewitness has gained with a perpetrator’s appearance at a crime scene. Following only a brief exposure to a perpetrator (as is often the case at crime scenes), one should therefore expect that eyewitnesses generally have relatively poor memory for a perpetrator’s facial appearance and identification of this person should be taxing. Second, this issue should be compounded by inter- and intra-individual differences in the ability to process faces. Thus, observers at the lower end of face processing ability should be more prone to make errors in a single eyewitness identification and those who tend to be more inconsistent in their identification decisions should also be less capable of identifying a perpetrator repeatedly. In turn, an eyewitness’ ability to correctly recognize an unfamiliar perpetrator across multiple instances should greatly increase one’s confidence in the accuracy of their identification.

Exploring this issue will reveal new information about the robustness of facial representations that eyewitnesses hold of a perpetrator. We therefore assessed the extent to which individual eyewitnesses can identify the same person’s face repeatedly from different identity lineups and, equally, whether they can consistently indicate the absence of a target when he or she is not included in a lineup. To fully explore this question, it is important to compare consistent target identifications with the repeated selection of other, non-target lineup identities. Outside of the laboratory, an identity lineup always includes a suspect, but this person may be the sought-after perpetrator of a crime or an innocent person. The purpose of a lineup is essentially to determine whether a witness selects the suspect, thereby seemingly confirming them as the target, or does not select this individual. The remaining faces act as “fillers” that are known innocents that would not be charged if they were selected by an eyewitness. To determine the extent to which observers might repeatedly identify the same non-target face in a multiple-lineup procedure, one could therefore replace the target with another identity that acts as a designated innocent suspect. By comparing repeated target identifications with selections of the innocent suspect, it would then be possible to determine whether the consistency of the responses of individual observers across multiple lineups can dissociate correct from incorrect eyewitness identifications.

While this approach has obvious applied value, the designation of innocent suspects poses problems in experimentation (Pryke et al., 2004; Sauerland & Sporer, 2008). In police investigations, suspects are arrested on the basis of their similarity to a witness’ description. However, it can be difficult to establish the perceived similarity of targets and suspects in advance. Different strategies for designating innocent suspects and lineup fillers appear to influence eyewitnesses’ identification decisions (Lindsay, Martin, & Webber, 1994; Luus & Wells, 1991; Wells, Rydell, & Seelau, 1993), but the study of such strategies has also yielded inconsistent results (e.g. Darling, Valentine, & Memon, 2008; Tunnicliffe & Clark, 2000). In addition, studies of unfamiliar face matching demonstrate that people vary considerably in how they detect the resemblance of faces in person identification tasks (e.g. Bindemann et al., 2012; Burton et al., 2010).

In light of these problems, we adopted a different approach. Instead of pre-selecting a designated suspect, this identity was defined a posteriori. We provide two contrasting approaches for this analysis. For the first approach, we assess identifications of the non-target identity that was selected first by an eyewitness in the multiple-lineup procedure. This approach minimizes data loss by including all incorrect eyewitnesses in the analysis and provides a “worst case” scenario by comparing consistent target selections with the greatest possible number of the corresponding non-target identifications. With a second approach, we focus on the non-target that was selected most often as the target by all observers during the course of the entire experiment. This “worst non-target” approach provides the highest number of comparison identifications for the target when these are defined by only a single non-target identity (for similar approaches, see, e.g. Pryke et al., 2004; Sauerland & Sporer, 2008). The consistency of these non-target selections across lineups served to contextualize the extent of consistent target selections.

To investigate the extent to which individuals can repeatedly identify a target in a multiple-lineup procedure with faces, we conducted a field experiment in which pedestrians in a city center were approached by a target person under the pretense of requiring route directions to a local landmark. Shortly after this exchange, pedestrians were approached by an experimenter and asked to attempt to identify the just-seen target. For this purpose, six successive identity lineups of faces were shown, comprising a mixture of three target-present and three target-absent lineups. Our aim here was to assess the extent to which individual observers could identify the target person repeatedly.

In this experiment, eyewitness identifications were error-prone for single lineups. For example, correct identifications of the target were made by 53–68% of observers for individual lineups. These identification rates are similar to previous research that has employed six-person lineups (e.g. 61% in Sauerland & Sporer, 2008; 47% in Steblay, Dysart, Fulero, & Lindsay, 2003) and demonstrate the difficulty of this task. However, a single lineup does not provide information about the consistency with which individual eyewitnesses can act on the same target. Experiment 1 shows that, when eyewitness accuracy is assessed across six lineups of faces, many individuals cannot make correct decisions for the same target consistently. In the current experiment, this behavior was such that in comparison to the 98% of observers who made a correct decision to at least one of the lineups, and the 45% of observers who made a correct decision to the first lineup, only 28% of observers acted correctly on all six lineups. The same pattern was evident when responses were broken down by the three target-present and the three target-absent lineups. Taken together, this demonstrates that repeated eyewitness identifications of an unfamiliar target are difficult. Applying theorizing from familiar face recognition (e.g. Bruce & Young, 1986; Burton et al., 2005), this indicates that many eyewitnesses have limited cognitive representations of perpetrators’ faces.

In turn, however, these data also indicate that of the 45% of participants who initially made a correct lineup decision, many also acted consistently on this target across all six lineups, comprising 28% of all observers. These data are contextualized by an analysis of incorrect identification decisions. For example, we found that only 5% of observers based their identification decisions consistently on the same non-target across all six lineups. This indicates that identification decisions across multiple lineups of faces could provide a potential index of eyewitness accuracy, whereby individuals with consistent identification responses are much more likely to have acted on a target than a non-target.

In Experiment 1, consistent target decisions across all six lineups were made by 28% of all observers, but 5% of observers also made similarly consistent decisions to a non-target. However, these results were obtained with same-day, same-camera photographs for each identity. Thus, it is possible that observers were able to repeatedly identify the target and non-targets in part due to similarity in their appearance across the lineups. To explore this issue, Experiment 2 utilized three very different types of photographs for each facial identity, comprising a standardized image, a picture from a photo-identity card, and a completely uncontrolled photograph from the profile of a social networking site. The question of main interest here concerned the extent to which consistent identifications of the target, and of non-targets, remained possible under the additional variation that was introduced by these distinct image categories.

This experiment examined the extent to which consistent target identifications are possible in a multiple-lineup procedure when faces vary more substantially in appearance than in Experiment 1. For this purpose, this experiment included standardized face photographs, as well as images from photo-identity cards and a social networking site. In comparison with Experiment 1, fewer hits were recorded (c.f. Figs. 2a and 4a), which indicates that the variability introduced by the new face images increased general task difficulty (e.g. Burton et al., 2005; Jenkins & Burton, 2011; Jenkins, White, Van Montfort, & Burton, 2011). Crucially, this manipulation also affected observers’ ability to act repeatedly on the target, with only 5% of individual observers capable of making correct eyewitness decisions across all six lineups. Moreover, the proportion of observers who made consistent identifications of the target and non-target identities did not differ. These findings converge with Experiment 1 to demonstrate that repeated identifications of the same unfamiliar person are very difficult, suggesting severe limitations in the facial representations that eyewitnesses can hold of such a perpetrator.

The preceding experiments demonstrate that repeated eyewitness identifications of the same target are difficult but also show that some observers can act more consistently on the target identity than others. This raises the question of what determines successful eyewitness repeat-identification of a target person. One possibility is that this reflects individual differences in the ability to recognize unfamiliar faces. Broad differences in this ability have been reported consistently between people, across a range of paradigms and image sets (e.g. Burton et al., 2010; Fysh & Bindemann, 2018; Megreya & Burton, 2006; Megreya & Bindemann, 2013; Russell, Duchaine, & Nakayama, 2009). Evidence is also accumulating that these individual differences are linked to eyewitness accuracy for a single lineup (see Bindemann et al., 2012; Geiselman et al., 2001; Hosch, 1994; Morgan et al., 2007). If these individual differences also underlie the variation among observers that is observed in Experiments 1 and 2, then performance on laboratory tasks testing the recognition of unfamiliar faces should relate to eyewitness identifications across multiple lineups for the same target.

To investigate this explanation, we created a laboratory version of the multiple-lineup paradigm reported in Experiments 1 and 2. In this task, observers viewed a video of a target person, which was then followed by three target-present and three-target-absent lineups for this person. This eyewitness task was then followed by two further laboratory tests for unfamiliar face recognition, comprising an adaptation of Bruce et al.’s (1999) 1-in-10 task and the Cambridge Face Memory Test (CFMT; see Duchaine & Nakayama, 2006). In the adaptation of the 1-in-10 task, observers were asked to memorize and then recognize a series of unfamiliar faces. On each trial of this test, observers studied a target face in isolation before determining its presence in a subsequent ten-person lineup. This 1-in-10 face-memory test produces large individual differences in identification performance (e.g. Megreya & Burton, 2006, 2008), which relate to eyewitness identification for a single lineup (Bindemann et al., 2012) and other internal factors, such as facets of observers’ personality (Megreya & Bindemann, 2013).

The CFMT also measures recognition memory for newly learned faces, which observers are then required to identify from a three-face array containing a target and two distractor faces. This test has been used widely to assess observers across the spectrum of face processing ability, such as those with impairments in face processing (e.g. Bobak, Parris, Gregory, Bennetts, & Bate, 2016; Ulrich, Wilkinson, Ferguson, Smith, Bindemann, et al., 2017; White, Rivolta, Burton, Al-Janabi, & Palermo, 2017) and those with exceptionally high ability (Bobak, Hancock, & Bate, 2016; Bobak et al., 2016; Russell et al., 2009). The 1-in-10 task and the CFMT therefore provide suitable tests against which performance on the multiple-lineup procedure can be compared.

As an additional aim for Experiment 3, we also sought to confirm that the pattern of results reported in the preceding experiments, which were based on only a single target identity, generalizes to other identities. We therefore examined the identification of two new targets in the multiple-lineup procedure. Both identities were also present in the lineups employed in Experiment 2, allowing us to retain these materials.

This experiment replicates the main findings of the preceding experiments by showing that observers find it difficult to consistently act on a target identity across six lineups of faces. For example, whereas all observers (100%) were accurate in at least one of the lineups, by making a correct identification or lineup rejection, only 24% of observers were able to do so across all six lineups. However, consistent identifications of non-target identities were even less likely across all six lineups, being made by only 1% of all observers.

Experiment 3 extends the findings of Experiments 1 and 2 in two ways. First, this experiment obtained these effects with two new target identities and under more controlled laboratory conditions than the field study paradigm of Experiments 1 and 2, demonstrating generalizability of these effects. Second, this experiment showed correlations of individual performance on the multiple-lineup task with two established tests of unfamiliar face recognition, comprising the 1-in-10 task (Bruce et al., 1999) and CFMT (Duchaine & Nakayama, 2006). These correlations were small to moderate in strength, indicating that one likely source of variation in performance across observers in the multiple-lineup procedure reflects individual differences in the ability to process unfamiliar faces.

These findings converge with other studies reporting correlations between eyewitness accuracy and tests of face recognition ability (e.g. Geiselman et al., 2001; Hosch, 1994; Morgan et al., 2007), including some that have utilized the same 1-in-10 task (Bindemann et al., 2012). As in Bindemann et al. (2012), this correlation was present for lineup identifications but not rejections. One possible explanation for the absence of this latter correlation could be that “target absent” responses may reflect two different response categories in this paradigm, comprising observers’ explicit knowledge that a target is absent from a lineup (i.e. a correct rejection) or rejection of a lineup when observers simply do not know whether a target is present or not (i.e. a “don’t know” response; Sauerland, Sagana, & Sporer, 2012).

This study investigated the ability of individual eyewitnesses to act consistently on a target identity across multiple lineups, comprising three target-present and three target-absent lineups. Performance was error-prone for individual lineups. For example, only 45% of observers made a correct decision to the first lineup in Experiment 1. A more striking pattern emerged when individuals’ ability to consistently identify the presence of a target and to determine his absence from a lineup was explored. When highly similar target images were shown across lineups in Experiment 1, following a face-to-face interaction in a field paradigm, only 28% of observers based their decision correctly on the target identity across all lineups. This substantially exceeded consistent identification errors, which were made by acting on a non-target identity across the six lineups and were committed by 5% of observers. However, the ability to act consistently on a target identity reached similarly low levels when greater within-person variability was introduced in the lineup faces in Experiment 2. Under these conditions, 48% of observers made a correct decision to the first lineup, but only 5% of participants were correct across all six lineups, and none of the observers (0%) acted with such consistency on a non-target.

Experiment 3 extended these findings to a laboratory setting and different target identities. This experiment utilized the same high-variability multiple lineups as Experiment 2, but exposed participants to controlled close-up video of the target faces. In this paradigm, accuracy for individual lineups was improved compared to Experiment 2. For example, 82% of observers now made a correct decision to the first lineup and consistent target decisions, across all six lineups, were made by 24% of participants. These results indicate that consistent correct target decisions need not be as infrequent as Experiment 2 suggests and could reflect the more controlled conditions of the laboratory eyewitness test or enhanced accuracy for the new targets that were employed in this study. Crucially, however, Experiment 2 replicated another key finding of the preceding experiments, by demonstrating that consistent decisions to non-target identities were even less frequent, at 1% across all six lineups.

These data represent an important finding. The ability to recognize a face repeatedly, and across variable images, is a hallmark of the accurate identification of familiar faces (see, e.g. Bruce & Young, 1986; Burton et al., 2005), but the extent to which this is possible with unfamiliar faces is less clear. Recent studies have shown that repeat identifications of unfamiliar faces are taxing in identity-matching tasks (e.g. Alenezi & Bindemann, 2013; Bindemann et al., 2012; Bindemann & Sandford, 2011). The current study demonstrates that this is also the case in an eyewitness scenario that requires the repeated recognition of a face from memory. Indeed, this task appears impossible for almost all individual observers when face images display more realistic within-person variability in conjunction with the naturalistic eyewitness encounter of the field study paradigm of Experiment 2. This indicates that the stored facial representations that observers could develop in the eyewitness scenario here do not necessarily allow for the robust identification of targets. In turn, Experiment 3 shows that identifications can occur more frequently with the same lineup images under more controlled conditions.

A question that arises is what determined successful repeat-identifications in these experiments in some observers but not others. One possibility is that these differences are linked to observers’ inherent ability to process unfamiliar faces (Wilmer et al., 2010; Zhu et al., 2010). Some individuals may be naturally capable of being better eyewitnesses, perhaps akin to “super-recognizers,” who have superior levels of face recognition ability (e.g. Noyes, Phillips, & O’Toole, 2017). In support of this reasoning, eyewitness accuracy in the multiple-lineup task of Experiment 3 correlated with the 1-in-10 task of unfamiliar face recognition as well as the CFMT, which has been employed consistently as a measure of super recognition (e.g. Bobak, Bennetts, Parris, Jansari, & Bate, 2016; Bobak et al., 2016; Russell et al., 2009). However, these correlations were only small to moderate in strength and therefore cannot explain the pattern of results alone.

Another possibility is that the individual variation in the multiple-lineup procedure also reflects qualitative differences in the initial interaction with the target, such as the attention that individual observers paid to the target’s face. It has been shown, for example, that fixations on a perpetrator’s face relate to eyewitness accuracy under laboratory conditions that provide participants with the same crime (video) content (Attard & Bindemann, 2014). Such findings could perhaps also explain why accuracy improved in Experiment 3, which provided all participants with exactly the same video material for initial exposure to the targets, compared with the inevitably more variable exposure that was provided by the social interaction of the field study paradigm in Experiment 2 (cf. Figs. 4 and 5). Future studies are required to further clarify the factors that determine individual differences in multiple-lineup performance.

In summary, the current experiments provide evidence that repeated eyewitness identifications of the same target from multiple lineups are taxing in laboratory and field paradigms. This problem is such that when different images of the target person display realistic within-person variability and are used in conjunction with a field paradigm, repeat identifications appear impossible for almost all eyewitnesses. These findings suggest that most eyewitnesses only develop limited cognitive representations of a target that do not allow for robust identification. In turn, the ability of individual eyewitnesses to repeatedly act on a target should provide insight into the degree of familiarity they have acquired with this person, and could therefore provide a potential index of eyewitness accuracy.