Much research has demonstrated that people fail to detect visual changes, and some of this work has demonstrated that these failures can occur in important applied settings. In any given setting, however, it is not clear what the implications of these failures are. For example, when someone misses a visual change (such as the substitution of one icon for another) when learning about a graphical computer interface, did this failure occur simply because they failed to look at the changing properties, or did the learner look at the changing properties but fail to create a representation sufficiently durable to see the change? Moreover, when someone misses a change, how broad was their failure? Does a failure to see changes imply a failure to remember the visual features of the interface or, even more broadly, a failure to pay much attention at all to the contents of the lesson? We therefore assessed change blindness in a specific real-world setting: screen-captured instructional videos. We tested the degree to which change blindness in this setting occurs because of failures to look at the changing properties and whether these visual-cognitive failures are associated with failures to remember visual properties and failures to learn instructional contents.
Attention, awareness, and learning in a naturalistic setting
Despite the clear importance of maintaining effective visual awareness, research exploring phenomena such as change blindness and inattentional blindness reveals how people often fail to detect visual changes and unexpected objects in their environments (Jensen, Yao, Street, & Simons, 2011; Levin & Baker, 2015; Simons & Levin, 1997; Varakin, Levin, & Fidler, 2004). These failures of awareness imply that viewers are not representing and remembering what they see, but this seems to conflict with research that documents people’s ability to represent large amounts of visual information and more generally to learn from their experiences. One means of reconciling this apparent conflict is by assuming that change blindness is a very narrow failure that occurs in the face of otherwise effective visual representations. For example, imagine that someone watching an informational video teaching spreadsheet formulas fails to notice that a menu icon suddenly changes color. This could be a sign that they have failed to represent anything about the icons and by extension have not learned much at all about the videos. However, it is also possible that the viewer remembers plenty about the icons—they just didn’t compare the pre- and post-change views, and more generally, their failure to detect the change may say little about the degree to which they have learned the contents of the video.
As we will review below, there is evidence for both of these views. So, some situations produce evidence that change blindness is associated with failures to represent visual properties, while in other settings change blindness seems to occur in spite of otherwise-accessible representations of the changing properties. This makes it important to ask whether visual failures in specific settings are broad or narrow. So, if change blindness is caused by a broad representational failure, it can be used both as a sign that viewers will fail to learn important things (such as the identity of specific icons in the above example) and as a target of possible intervention to improve task performance by highlighting important changes. In this paper we describe two experiments testing change detection in a visual learning setting: screen-captured instructional videos. Our primary question was whether change blindness would reflect failures to represent the changing properties themselves or whether change blindness was caused by a failure to compare otherwise effectively represented properties. We also assessed eye movements to test whether change detection occurs in the face of gaze at both pre- and post-change objects, and whether increased gaze would be associated with increased change detection. Finally, we assessed learning to test whether change blindness would be a sign of a global failure to learn from the videos.
Representation and comparison failures in naturalistic dynamic settings
One might conclude that change detection failures are evidence that merely looking at a scene does not automatically generate extensive visual representations of the things in that scene unless additional processes are invoked (Chen, Swan, & Wyble, 2016; Levin & Baker, 2015; Rensink, 2000). A number of findings support this representation-failure hypothesis. For example, Caplovitz, Fendrich, and Hughes (2008) coined the term “attentive blank stares” to describe the relatively frequent cases of gaze falling upon both pre- and post-change properties in the absence of change detection (see also Fudali-Czyż, Francuz, & Augustynowicz, 2014). Change detection failures in the face of verified looking have been observed in realistic settings such as slight-of-hand magic tricks (O’Regan, Deubel, Clark, & Rensink, 2000; Smith, Lamont, & Henderson, 2012).
Other research has relied upon behavioral paradigms to assess the degree to which change blindness is associated with failures to represent pre- and post-change objects. Levin, Simons, Angelone, and Chabris (2002) found a strong relationship between change detection and subsequent visual recognition of the changing objects. In these experiments, the participants interacted directly with an experimenter who was surreptitiously replaced with another person during the interaction. After reporting whether they saw the change, participants were asked to recognize the experimenter from a forced-choice lineup. A substantial proportion of participants missed the change, and participants who missed the change were often at chance when attempting to recognize both the pre- and post-change experimenters, while participants who reported the change did considerably better. Similar links between change detection and recognition have been observed in participants who viewed videos depicting crimes in which one actor was substituted for another (Davies & Hine, 2007; Nelson et al., 2011). It is even possible to argue that these representational failures may ultimately result in failures to learn about repeatedly presented visual information both in the lab (Wolfe, Klempen, & Dahlen, 2000) and in our everyday visual environment (see, for example, Nickerson, 1965; Nickerson & Adams, 1979).
However, considerable evidence also supports the hypothesis that change blindness can occur even if viewers have represented the pre- and post-change objects but have failed to compare those representations (Scott-Brown, Baker, & Orbach, 2000; Simons, 2000). Also, although fixation is no guarantee that change detection will occur, several studies have demonstrated that change detection is sometimes more likely with fixation (Hollingworth, Schrock, & Henderson, 2001; Hollingworth, Williams, & Henderson, 2001). As many commentators have pointed out, viewers are often able to represent a large amount of visual information with comparatively little effort or control (Olson, Moore, & Drowos, 2008). Classic research demonstrates good picture recognition memory, even for thousands of pictures (Konkle, Brady, Alvarez, & Oliva, 2010; Nickerson, 1965; Shepard, 1967; Standing, 1973). Other work exploring visual statistical learning supports a similar hypothesis by demonstrating that participants learn relationships between a large number of targets and their contexts, or sequential contingencies among serially presented objects (Turk-Browne, Jungé, & Scholl, 2005).
In addition, more naturalistic experiments do sometimes demonstrate that change blindness can be associated with fully effective representations of pre- and post-change objects. For example, Angelone, Levin, and Simons (2003) asked participants to watch a short video showing people conversing. Changes to object included color changes in clothing, or the identity of one of the actors in the video. Not only did participants who missed the change recognize the changing properties at above-chance levels, but they were just as accurate at recognizing the properties as participants who saw the changes. These results dissociate change detection and recognition, suggesting that change blindness may in some cases underestimate the extent of visual representations. Other research demonstrates that people can identify previously seen objects in detail (Hollingworth & Henderson, 2002; Hollingworth, Williams, & Henderson, 2001), even when the recognition test is not expected (Castelhano & Henderson, 2005; Varakin & Levin, 2006; Williams, Henderson, & Zacks, 2005).
In the face of evidence like this, it is important to consider the possibility that participants in real-world change detection experiments are inattentive or overly focused on a social interaction. Thus, poor visual recognition among change-missers may represent an exceptionally low ebb of visual processing (Beck & Levin, 2003; Landman, Spekreijse, & Lamme, 2003). Therefore, it seems critical to explore both the prevalence of change blindness and the relationship between change blindness and visual representation in a variety of settings that characterize important real-world processing. It is especially useful to explore a setting characterized by a rich conceptual framework that participants are motivated to learn from. Participants in this study watched screen-captured instructional videos designed to teach them how to perform specific tasks, then were tested on their memory for the contents of the videos. At some point during the videos, a visual change occurred (for example, a colored region on a banner graphic changed from green to blue; see Additional file 1 for illustrations of all changes), and participants were forewarned of this possibility. Thus, in this setting, participants were both motivated to attend to visual information and aware that they would be asked to detect visual changes.
We asked two basic questions. First, how broad is the representational failure associated with change blindness in a real-world learning setting? We tested whether participants who missed changes also had difficulty recognizing the changing properties, and whether these missers learned less effectively from the videos. Second, we tracked participants’ gaze while viewing the videos. If change blindness is caused by a failure to look at the changing objects, then it should rarely occur when gaze has fallen upon both the pre- and post-change objects, and increased looking at the changing objects should be associated with increased change detection and more accurate visual recognition.