Faulty eyewitness identification (ID) has contributed to the wrongful conviction of hundreds of innocent men and women, playing a role in 72 % of DNA exoneration cases litigated by the Innocence Project (Innocence Project 2015). Understanding the factors that influence these mistaken IDs is of great theoretical and practical interest. One such factor concerns the use of simultaneous or sequential lineups, but our goal here is not to evaluate the empirical evidence marshaled for and against sequential lineups (see Clark, Moreland, & Gronlund 2014; Steblay, Dysart, & Wells 2011; for a review, see Gronlund, Mickes, Wixted, & Clark 2015). Rather, our goal is to formalize and test theoretical conceptualizations that have been proposed to explain the functioning of simultaneous and sequential lineups.
The typical procedure in studies of eyewitness ID involves presenting participants with a mock crime scenario (usually a video) followed by a delay. Participants are then shown either a target present lineup, containing the guilty suspect from the video and five known innocents (fillers), or a target absent lineup, containing a designated innocent suspect and five fillers. If the guilty suspect is identified from a target present lineup, it is counted as a correct ID. If the innocent suspect is identified from a target absent lineup, it is counted as a false ID. In a simultaneous lineup, eyewitnesses view an array of (typically, six) faces presented all at once and are tasked with identifying who they believe to be the suspect. In a sequential lineup, faces are presented one at a time and eyewitnesses are asked to either identify the face as the suspect and terminate the lineup or reject the face and view the next face in the sequence. This continues until the suspect is identified or all faces are rejected.
Lindsay and Wells (1985) were the first to compare the sequential and simultaneous lineup procedures. In their study, participants given a simultaneous lineup had a correct ID rate of 0.58 and a false ID rate of 0.43. Participants shown a sequential lineup had a correct ID rate of 0.50 and false ID rate of 0.17. Lindsay and Wells concluded that sequential lineups were superior to simultaneous lineups. To explain this result, Lindsay and Wells suggested that simultaneous lineups promote the use of relative judgments. Wells first proposed this idea, stating that “the term relative judgment refers to the fact that the witness seems to be choosing the lineup member who most resembles the witnesses’ memory relative to other lineup members” (Wells 1984, p. 92). According to this idea, witnesses who view all the faces in a simultaneous lineup compare the faces of a lineup relative to each other and choose the member that best matches the witness’s memory.Footnote 1 Relative judgments can be contrasted with an absolute judgment strategy, in which witnesses compare faces (typically in a sequential lineup) directly to memory rather than to each other. Lindsay and Wells concluded that a relative judgment strategy is not necessarily harmful in target present lineups, but leads to higher rates of false IDs in target absent lineups. The authors further argued that a sequential lineup would reduce the propensity to use a relative judgment strategy and that this would result in a lowering of the false ID rate but would have little adverse effect on the correct ID rate. This led to their recommendation to use the sequential lineup in real-world settings in order to protect the innocent from being chosen from lineups. This recommendation has been accepted by many policymakers, as has the relative judgment theory that supports it (Innocence Project 2015; Wells et al. 1998).
The goal of this paper is to elucidate how eyewitness memory is mediated through the exploration of formal conceptualizations of recognition memory and, as a result, be better equipped to evaluate theories such as the relative judgment theory advocated by Wells and colleagues. As we shall see, the language of Lindsay and Wells’s (1985) relative judgment theory, and Wells, Steblay, and Dysart’s (2012) update, appear to invoke two different conceptualizations of recognition memory: discrete-state and continuous mediation. We begin with a short description of each. We then outline the tenets of the relative judgment theory that fit into each of these conceptualizations and propose one possible interpretation that is consistent with the language of the theory. Next, we report two studies that empirically evaluate this interpretation. Based on our evidence, we will then re-evaluate the relative judgment theory and the broader implications of our results for theories of eyewitness ID.
Discrete-state and continuous models of recognition
One class of models of recognition memory is consistent with discrete-state mediation conceptualizations of recognition memory (e.g. Rouder & Morey 2009). This conceptualization assumes that items can be in one of two states in memory that affect their probability of being recognized and classified as “Old” or “New.” The simplest model of discrete-state mediation is a single high-threshold (1HT) model. Under this model, Old items can be in either a detect or guess state. In the detect state, Old items are able to be correctly classified as “Old.” If an item is not detected as “Old,” it enters a guess state in which no mnemonic information about the item is available, and it can either be guessed as “Old” or guessed as “New.” According to the 1HT model, New items can only enter a guess state, from whence it can be guessed “Old” (a false alarm) or guessed “New” (a correct rejection).
To extend the conceptualization of discrete-state mediation to the eyewitness memory situation, consider the show-up situation, where only one face is presented to the witness. A witness is tasked with either identifying the face as the suspect or rejecting the face. The Old item in this case would be a guilty suspect. When presented a target present show-up (which includes the guilty suspect), the witness will either detect the guilty suspect as “Old” (and make a correct ID with probability DO) or fail to detect the suspect and guess, either guessing that the suspect is “Old” and making a correct ID (with probability g), or guessing that the suspect is “New” and rejecting the lineup (with probability 1 – g) (see Fig. 1). No mnemonic information is available to guide guessing, only a response bias to endorse “Old” versus “New.” Only guessing governs responses during a target absent show-up.
The second model of interest invokes continuously mediated processes underlying recognition memory. We focus on signal detection theory (SDT; Green & Swets 1966; Macmillan & Creelman 2005) as an exemplar of this class. SDT assumes that all items, both Old and New, possess latent strength values. These strength values vary continuously and are commonly depicted by normal distributions, whereby most items have similar (average) levels of strength, but some items are very strong and some are very weak. When items are studied, the strengths of these items increase, and the distribution of Old item strengths shifts, resulting in two overlapping distributions of strengths that characterize New and Old items. These distributions can vary in degree of overlap depending on how well Old items have been encoded (see Fig. 2). At study, the strengths of tested items are compared to a decision criterion (c). Items that fall above c are classified as “Old” and items that fall below c are classified as “New.”
To illustrate, consider again the show-up. When an eyewitness is presented with a guilty or innocent suspect, SDT posits that a suspect that falls above c will be classified as “Old” and a suspect that falls below c will be classified as “New.” If a guilty suspect in a target present show-up falls above c, it will be classified as “Old” and the witness will make a correct ID. If the guilty suspect falls below c, it will be classified as “New” and the witness will make a false rejection. Conversely, if an innocent suspect from a target absent show-up falls above c, it will be classified as “Old” and the witness will make a false ID; if it falls below c, it will be classified as “New” and the witness will make a correct rejection.
Mediation of relative judgment theory
Because the relative judgment theory is a verbal rather than formal theory, there are no explicit references to the recognition models we just outlined. In fact, past descriptions of the theory imply both discrete and continuous mediation. For example, Wells et al. (2012) argued “the higher rate of hits from the simultaneous lineup is actually just the result of lucky guesses stemming from a higher rate of choosing” (p. 268, emphasis added). For a similar idea, see Penrod, Garcia, and Robertson (2005). This language suggests discrete-state mediation because no mnemonic information is assumed to be attached to these non-detected stimuli. In support of this discrete-state interpretation, Clark (2012) wrote, regarding the relative judgment strategy, that “…it assumes an all or nothing theory of memory, in which the witness makes a recognition decision based on a true memory, or he or she simply guesses…” (p. 281). Moreover, according to Wells et al., relative judgments occur when “…the witness is unable to answer the difficult question (‘Is this the culprit’) and instead shifts to an easier question (‘Which is the closest?’)” (p. 268). One could conceptualize the “shift” from asking who the culprit is to who is closest, as similar to failing to detect the target in a basic recognition task. But Wells (1984) stated that “absolute processing implies that a match (i.e., between a lineup member and one’s recollection of the perpetrator) must exceed some cut-off in order to produce an identification response” (p. 95). This language is reminiscent of a continuously mediated model because absolute judgments appear to involve the comparison of lineup members to memory, much as a strength value is compared to a criterion in SDT. In fact, using SDT, Wixted and Mickes (2014) argued that the relative judgment theory reflected changes in response bias, or the willingness to choose a suspect from a lineup. Specifically, when faces are presented in isolation (sequentially), there is less pressure to choose the face and a more conservative criterion is adopted. However, when faces are presented simultaneously, there is more pressure to choose and the criterion is pushed to a more liberal level, increasing the likelihood that a filler is chosen. Response bias refers to placement of a criterion, and is a basic tenet of SDT; this idea does not fit a discrete-state conceptualization because there is nothing to weigh evidence against in discrete models.
Because different iterations of the relative judgment theory contain facets of discrete-state and continuous mediation, we decided to consider one possible formal implementation of the theory, which would then allow us to develop a hypothesis to test. We chose a dual-process framework, specifically the model proposed by Yonelinas (1994), and present the relative judgment theory in that context. According to the dual-process model, recognition decisions can rely on familiarity or recollection. Recollection is an all-or-none process in which a specific memory of studying an item either succeeds or fails at being retrieved.Footnote 2 The recollection process in the dual-process model is discrete, analogous to the detection process of the 1HT model. We will not test this assumption because our focus is on non-recollected decisions and how they are mediated. There are two possibilities regarding how these decisions are mediated.
One possibility, consistent with Yonelinas (1994), is that items that are not recollected can be recognized based on a continuous, familiarity process that operates under the assumptions of SDT. This seems in line with the weighing of memorial evidence of the lineup members against each other (Lindsay & Wells 1985), which would be a diagnostic process.Footnote 3 Alternatively, if the eyewitness relies on a guessing process (Penrod et al. 2005; Wells et al. 2012), it implies that the relative/familiarity judgment is discrete, like the 1HT model, which means that the guessing process relies on zero mnemonic information (a “lucky guess”). Therefore, relying on a relative judgment strategy in this situation provides no diagnostic information to the witness. The lineup member that the witness believes looks most like the perpetrator is no more likely to be the actual perpetrator than any of the other lineup members. Therefore, the decision process would be completely random, as if rolling a die, and choosing the actual perpetrator would be a lucky guess. In this case, even though the decision process is subjectively continuous to the witness (i.e. he or she is choosing the person that is “most like” the perpetrator), it can nevertheless be modeled by a non-diagnostic, discrete-state process like the 1HT.
To reiterate, our dual-process interpretation is not the only way to instantiate relative judgment theory, but it does allow for a test regarding how the information involving non-recollected stimuli is mediated. According to the dual-process view of relative judgment theory, non-recollected faces in a simultaneous lineup would be subject to a relative judgment process. This process could be continuous, and therefore diagnostic of guilt, or it could be discrete, implying a non-diagnostic guessing process. We will empirically test discrete-state and continuous model predictions using a task similar to eyewitness ID. In doing so, we aim to better understand the recognition evidence that mediates these decisions so as to better inform theory.
Empirical evidence for discrete-state and continuous mediation
Discrete-state and continuous models make different predictions about the shape of receiver operating characteristic (ROC) curves. ROC curves are constructed by plotting the hit rate and false alarm rate at each level of response bias (i.e. willingness to label an item as “Old”). Response bias is often assessed using confidence judgments (Wixted 2007; Yonelinas & Parks 2007), with high confidence signaling a more conservative response bias and low confidence signaling a more liberal response bias. Discrete-state mediation predicts linear ROCs, whereas continuous mediation predicts curvilinear ROCs. However, empirical evidence from the basic recognition memory literature almost always produces curvilinear ROC curves (Wixted 2007), supporting a continuous mediation of recognition memory.
Our dual-process view of relative judgment theory, likewise, makes predictions regarding the shape of eyewitness ROC curves. If non-recollected stimuli possess zero mnemonic information (discrete), the ROC will be linear. However, if the non-recollected stimuli are continuously mediated, a curvilinear ROC will result. Recent studies reporting ROCs of simultaneous lineups appear to show curvilinear ROCs (e.g. Gronlund et al. 2012; Mickes 2015; Wetmore et al., 2015). However, there are two problems with relying on ROC shape to differentiate continuous and discrete mediation accounts.
One problem is that eyewitness ROC curves are truncated because they involve plotting only suspect IDs (see Gronlund, Wixted & Mickes 2014; **Wixted & Mickes, 2012). This truncation can make it challenging to assess the shape of the ROC curve. The second problem is that the predictions made by discrete-state models can mimic curvilinear ROC curves (e.g. Broder & Shutz, 2009; Malmberg, 2002; Province & Rouder 2012; but see Chen, Starns, & Rotello 2015). Province and Rouder refer specifically to the effect of relaxing the certainty assumption (Luce 1963). The certainty assumption posits that under discrete-state mediation, all “detect” items are recognized with high confidence and only “guess” items can be recognized with a range of low to high confidence. This assumption leads to the linear ROCs that the model predicts. However, Province and Rouder showed that if you relax the certainty assumption and allow for the possibility that detected items can be recognized with a broader range of confidence, a discrete-state model can indeed predict curvilinear ROCs. Recently, Kellen, Erdfelder, Malmberg, Dube, and Criss (2016) showed that an alternative discrete model (the low-threshold model, Luce 1963), which assumes that New items can exceed a threshold for detection, also can approximate empirical ROC curves. Therefore, ROC analysis is unable to definitively test between discrete-state and continuous mediation.
If ROC analysis will not distinguish between continuous and discrete mediation, another measure is needed. Kellen and Klauer (2014) provided one such measure. In their study, participants were presented with a list of 270 words; 135 words were presented once (weak encoding, W) and 135 words were presented three times (strong encoding, S). At test, participants were presented with three-word, target present arrays (their Experiment 2) and were told to rank each of the words from “Most likely to have been seen before” to “Least likely to have been seen before.” The critical measure was the conditional probability that the actually-studied target of the array was ranked second, given that it was not chosen as most likely to have been seen before (c2).
The c2 measure requires minimal assumptions (in contrast to ROC analysis). For example, the certainty assumption has no effect on c2. Moreover, c2 evaluates the most fundamental prediction of discrete and continuous models. In a discrete-state model, although strong items would be more likely to be ranked first (DO
S > DO
W), if a strong target was not identified as old, it would have an equal likelihood of being ranked second or third because judgments regarding these items must arise from the guess state. In a guess state, the amount of mnemonic information is zero, regardless of whether the tested item actually was strong or weak. This leads to equal c2 predictions for strong versus weak items, even though the average hit rate of strong items (strong items ranked first) would be greater than the average hit rate of weak items. According to a continuous model, however, strong items, on average, have a greater strength than weak items. Therefore, if a strong target was not ranked first, it would nevertheless have a greater likelihood of being ranked second than a weak target because it would fall higher (on average) in the target distribution (see Kellen & Klauer 2014, for proofs). In sum, the predictions regarding c2 under a discrete-state model are: c2
S = c2
W, but predictions regarding c2 under a continuous model are: c2
S > c2
W. Using words as their stimuli, Kellen and Klauer (2014) found evidence supporting a continuous model (c2
S = 0.63 > c2
W = 0.55, Exp. 2).
In order to evaluate our interpretation of the relative judgment theory, we utilized the same paradigm and c2 measure as Kellen and Klauer (2014) to test whether memory for faces is mediated by continuous or discrete processes. There is theoretical and practical merit in using faces rather than words as our critical stimuli. For example, words are processed and encoded differently than faces. Olivares, Iglesias, and Rodriquez-Holguin (2003) found evidence of separate linguistic and non-linguistic event-related potentials when comparing the N400 component using facial and non-facial stimuli, which suggests that the brain processes these stimuli differently at the neural level. Additionally, separate, specialized modules used in the processing of faces and visual words have been identified using fMRI (Kanwisher 2010). These studies suggest that recognition and processing of faces and words are not homologous at all levels. Additionally, eyewitness tasks, inside and out of the laboratory, require encoding and recognizing novel faces, which differs from encoding and recognizing known words.
Kellen and Klauer (2014) found evidence of continuous mediation for words in a ranking task, adding to the body of ROC evidence suggesting that recognition memory is driven by continuous mediation. Experiment 1 could replicate these findings and extend Kellen and Klauer’s results to faces. Alternatively, the evidence indicating that faces are processed differently from words suggests the possibility that recognition of faces may be mediated differently than words. According to the dual-process view of the relative judgment theory, evidence of continuous mediation would signal that relative judgments (in the absence of recollection) involve the weighing of diagnostic memorial evidence (like SDT), but evidence of discrete-state mediation would be indicative of a reliance on a non-diagnostic guessing process (as in the 1HT). Experiment 1 sought to elucidate the use of discrete-state or continuous mediation using faces as the critical stimuli.
Experiment 1