The purpose of this study was to broaden the ongoing investigation on visual search from a regular trial-by-trial variant to a continuous one; both search types are common in real-life scenarios. We began this investigation by designing a continuous visual search task that is easy to manipulate and provides a good abstraction of its real-life counterparts. In our continuous search task, people looked for one or two target features (color, orientation, or both) among items with varying colors and/or orientations over sessions of several minutes. There were no separate trials within each session. We manipulated the target rates (i.e., the occurrence frequency of the target features) and studied whether they affected continuous visual search in similar ways to the LPE in regular, discrete visual search.
In the four experiments we conducted, we found evidence for a target-rate effect in continuous visual search. In Experiment 1, slower hit RTs and higher miss rates were observed when a target was rare. In Experiment 2, when searching for two target features, slower hit RTs and higher miss rates were associated with a relatively rare target feature. In Experiment 3, a larger set-size effect was found for a relatively rare target feature, showing that relative target rate influenced search performance via the effectiveness of attentional guidance. In Experiment 4, it was found that set-size effects were not affected by absolute target-rate manipulations, indicating that only relative target-rate manipulations would influence search efficiency. Furthermore, using a flash detection paradigm, we showed that the absolute target rates did not influence vigilance in continuous visual search.
Our results provided an initial evaluation of the possible explanations for the target-rate effect in continuous visual search. First, the presence of target-rate effects in relative prevalence designs and their absence in flash detection performance rejected a general vigilance explanation. Second, search efficiencies in terms of set-size effects were specifically influenced by relative target-rate manipulations. This indicated that the relative target-rate effect was associated with better attentional guidance to the relatively frequent targets. Third, our data cannot be explained by target exposure history, habitual response, or premature search termination. Taken together, in single-target scenarios, the target-rate effect most likely occurred at a post-search target identification stage. In multiple target scenarios, the target-rate effect was also due to the strategic allocation of attentional resources to each target feature.
Generalization of the LPE to continuous visual search
The main finding of the current study is a target-rate effect in continuous visual search, which is similar to the target-prevalence effect known previously for regular visual search. This finding supports the generalization of knowledge and the associated precautions from regular visual search to continuous visual search. Targets being rare has been raised as a real-life challenge in continuous visual searches like lifeguarding (Lanagan-Leitzel et al., 2015) or face searches in CCTV surveillance (Mileva & Burton, 2019). However, no previous studies have experimentally tested how previous knowledge on regular visual search is generalized to continuous visual search. The current study provides a piece of direct evidence for such a generalization.
The current results also suggest that the explanations for the target-rate effect in continuous visual search are similar to the LPE in regular search. Consistent with previous suggestions that regular search LPE mainly comes from a post-search target identification stage (Godwin et al., 2015; Hout et al., 2015), we reached the same conclusion in this study. Previous visual search and vigilance studies generally suggest that target prevalence influences target identification via a decision bias. In visual search, it is believed that a low target prevalence biases the participant’s response criterion (c) toward target-absent responses without significantly reducing the perceptual sensitivity (d’) to the target (Wolfe & Van Wert, 2010). In vigilance tasks, the rate of a target event (number of targets per unit time) can be decomposed into the event rate (number of events per unit time) and signal probability (number of targets per a fixed number of events). A high event rate is known to cause vigilance decrement (increasing misses over time) by biasing the participant’s criterion toward an absent decision (Mackworth, 1965), unless when the event rate is very high, it also leads to a perceptual challenge (Parasuraman, 1979). A low signal probability is known to increase the miss rate, similar to the LPE in visual search. Colquhoun (1961) found that this effect is due to a decision bias. Therefore, we hypothesize that continuous visual search shares a similar locus to prevalence effects on regular visual search and vigilance tasks, that is, target frequencies seem to cause a decision bias in all cases.
Absolute versus relative target-rate effects
In addition to the basic target-rate effect, another major result of this study is a relative target-rate effect in continuous visual search, which conceptually replicated the relative LPE in regular search (Godwin et al., 2015; Hout et al., 2015; Wolfe et al., 2007). Such a finding was important in demonstrating that the LPE cannot be “cured” by increasing the overall target prevalence with non-critical targets. Instead, the relatively rare targets were still missed more often than the relatively frequent targets (Wolfe et al., 2005, 2007). The current study demonstrated the same phenomenon in continuous search. For instance, although it is not possible to directly compare absolute and relative target-rate conditions among our different experiments due to unmatched stimulus parameters, a rough comparison of the average figures may still be informative. In Experiments 1 and 4, the absolute target-rate produced effects of 7.2 and 8.2 percentage points on miss rates, whereas in Experiments 2 and 3, the relative target-rate effects were 12.1 and 7.6% points on miss rates. In general, the relative target-rate effects were no weaker than the absolute target-rate effects. Therefore, including non-rare targets in continuous visual search does not “cure” and may even worsen the rare target effect.
Another downside of “curing” rare target effects by including non-rare targets is a potential dual-target cost (Menneer et al., 2007, 2010). For instance, top-down guidance must be less optimal when configured for two target representations than one, leading to poorer search performance. We observed such costs in our experiments. Comparing Experiment 1 (single-target) with Experiment 2 (dual-target), the miss rates for the color targets were higher in the latter (35.5%) than the former (12.5%), with \(t\left(67\right)=5.26\) and \(p<.001\), even though Experiment 2 used a smaller set size and a higher contrast from the background, which should theoretically lead to better performance. Comparing Experiment 3 (dual-target) with Experiment 4 (single-target), the average miss rates for the orientation targets were higher in the former (26.2%) than the latter (15.4%), with \(t\left(48\right)=2.46\) and \(p=.02\). The set size and the stimulus parameters were the same across Experiments 3 and 4. Therefore, we observed a dual-target cost in our experiments, that is, dividing attention across two target features leads to poorer overall performance in continuous search.
Attentional guidance and target-rate effects
A theoretically interesting question to ask about the target-rate effects in continuous visual search, as well as LPEs in regular search, is whether target rates influence search performance via the effectiveness of attentional guidance. If better attentional guidance was associated with more frequent targets, we should observe more efficient searches as a result. We found such evidence in Experiment 3 with a relative target-rate manipulation, but not in Experiment 4 with an absolute manipulation. We reasoned that it was because target rate only influences attentional guidance when two targets compete for attentional resources.
Our findings were consistent with previous findings in regular visual search. For example, with absolute designs, Wolfe et al.’s (2005) original study did not observe any LPE in terms of set-size effects in target-present trials. Wolfe et al. (2007) did not observe such effects either. Second, with relative designs, Hout et al.’s (2015) and Godwin et al.’s (2015) eye movement data showed shorter first-landing times for the more frequent targets, indicating better attentional guidance to them.
Theoretically, if attentional guidance is better with more frequent targets in relative prevalence designs, we should also expect a higher search efficiency. However, previous studies that manipulated set size in relative prevalence designs did not observe any LPE in terms of search efficiency in target-present trials (Menneer et al., 2010; Wolfe et al., 2007). One factor that may explain the lack of effects in these studies is the search stimulus. For instance, Menneer et al. (2010) used conjunctively defined colored shapes, while Wolfe et al. (2007) used X-ray-like baggage images; both studies used non-salient stimuli. Such stimulus choices might have rendered attentional guidance ineffective at all prevalence levels, undermining any LPE on search efficiency. However, in their studies, Hout et al. (2015) and Godwin et al. (2015) used simple objects or shapes that allowed the targets to be distinguished from the distractors in terms of basic color and shape. As a result, the participants can develop effective attentional guidance based on basic features, providing enough room for target prevalence to exert its influence.
Two types of continuous search
An aim of the current study was to generalize previous research findings to broader search scenarios. We examined an instance of continuous search, in which observers looked for an occasionally occurring feature among other items with varying non-target features. This laboratory task abstracts the key features of real-life search tasks. Take lifeguarding as an example. Lifeguards have to look for specific movement patterns (drowning) among other less important movement patterns (swimming, playing, diving, etc.). In this type of continuous search, the target item possesses the target feature only temporarily. In our experiments, the color or orientation of a target item only stopped at the target state for 2 s and then reverted to the normal state afterward to become a non-target again. In lifeguarding, the drowning movement may only last for a few minutes.
We believe there is another type of continuous search that should also be examined experimentally. In this type of continuous search, the target item permanently possesses the target feature, but the search items are themselves temporary because they move in and out of the monitoring area. This type of search covers practical scenarios, such as infrared body temperature surveillance, in which the security officer has to detect persons with elevated body temperature (as coded in color) before they leave the screen (the monitoring region). Another example is a police officer who might need to search for a specific person by monitoring a CCTV display, while people enter and leave the monitored area. The police officer has to identify the target person before that person leaves the screen.
In general, we hypothesize that these two types of continuous search are similar in most regards, as they share key common features, such as dynamic displays, multiple items, and temporary targets. In fact, the preliminary data in our laboratory demonstrated an expected target-rate effect on hit RTs and a miss rate in searches of the second kind. Nevertheless, the two types of searches are not without differences. For example, in the second kind of continuous search, the search items typically enter the display from an edge of the monitoring area rather than appearing from nowhere. Therefore, if there is a general item flow direction (such as pedestrians entering the screen mostly from one side), it is intriguing to see whether more attention may be allocated to the entering edge of the screen or not.
Previous studies on visual search with dynamic properties
While most previous studies used static stimuli in visual search, there were attempts to study visual search with dynamic properties. For example, Laxton and Crundall (2018) compared lifeguard and non-lifeguard visual search performances using video clips of confederate swimmers. Meanwhile, Mileva and Burton (2019) studied visual search for faces obtained from photo ID or social media in CCTV surveillance video clips. However, neither study manipulated target prevalence. Furthermore, these studies differed from the current research as they divided the search into 30-s trials, and each trial could only contain zero or one target. Therefore, the decision process involved in such experimental design may be more akin to a regular visual search. For example, by knowing that there would be at most one target in any given trial, you may stop searching upon the detection of a target, you may give up searching when you feel that a target is not going to appear after inspecting most of the video, or when you are unsure about whether a target exists (e.g., you missed part of a trial due to an attention lapse), you may still make an educated guess based on your knowledge of target prevalence (Schwark et al., 2013). Therefore, there is generally more room for an observer to determine their response strategically in a trial-by-trial dynamic search with zero or one target than in a continuous visual search. By contrast, in a continuous search, the number of targets is generally unknown. In such a case, less information is available to support a strategic response. Therefore, the decision process between the two types of dynamic search could be very different.
A more relevant recent study that used dynamic stimuli was conducted by Muhl-Richardson et al. (2018). Not only did they manipulate target prevalence in their study, but they also used a more variable number of targets in their dynamic search trials. They asked the participants to search for a cued target color among an array of 108 color-changing squares. Each trial lasted for 40 s and may contain 0, 1, or 2 targets. The purpose of their study was to investigate whether people detected the target color in a predictive fashion. To achieve this purpose, while most distractors would not get close to the target color, they let some distractors get close to (but not reach) the target color. By recording eye movements, they found that the participants fixated on these target-similar distractors, which is indicative of predictive search behavior.
The third experiment in Muhl-Richardson et al.’s (2018) study was most relevant to our purpose. They manipulated target prevalence and unexpectedly found no target prevalence effect in terms of miss rate and RTs. There were more false alarms in their low prevalence condition, which was quite uncommon in previous studies. Intuitively, a low target prevalence should be associated with fewer false alarms. They commented that perhaps these discrepant findings were due to differences between static and dynamic searches. However, as opposed to this speculation, our current findings clearly suggest that a target-rate effect can occur in a continuous and dynamic visual search. Thus, an alternative explanation of their results is warranted. To explain their findings, we believe the critical uniqueness of their task was their use of target-similar distractors. For instance, the relatively high false alarm rate in their experiments (~ 30%) was likely caused by these target-similar distractors. In their high-prevalence condition, the false alarm rate reduced to ~ 20%, while in their low prevalence condition, it increased drastically to ~ 60%. We believe this may reflect that when there were more targets in the experiment, there might be a higher chance for the participants to learn to discern them from target-similar distractors. Therefore, while a high false alarm rate in most other studies may reflect a liberal decision criterion, a high false alarm rate in Muhl-Richardson et al.’s study may mainly reflect the perceptual confusion between the target and the target-similar distractors. In our experiment, however, we have a relatively wide feature zone reserved for the target. Thus, distractors and targets are unlikely to get confused. As such, we observed similar target-rate effects to regular visual search.
Continuous visual search and event rates
The most important finding in vigilance literature was that vigilance performance drops over time, and a high event rate would increase such decrement (e.g., Mackworth, 1965). In vigilance tasks, the target rate and event rate are related but have distinct manipulations (Colquhoun, 1961; Mackworth, 1965). Previous studies have generally shown that event-rate and target-rate effects in vigilance tasks are mostly results of decision bias, except for very high event rates in which perceptual limitations may come into play (Parasuraman, 1979). In visual search literature, event rates were rarely systematically manipulated; in our experiments, they were not manipulated either. However, event rates could influence visual search in a similar manner, which warrants further research. For example, we may want to avoid a too-high event rate in visual search if it causes perceptual constraints. This way, the issue of event rate would arguably be more critical for continuous search compared with regular, discrete visual search. In real-life applications of discrete search, the event rate can often be controlled by the searcher (e.g., a security officer can stop at a certain piece of baggage for more careful inspection, or a radiologist can inspect a medical image one after another on his/her own pace). However, in real-life continuous search, the event rates may not be controlled by the searcher. For example, the security officer cannot control how fast people move in and out of the surveillance region, and the lifeguard cannot control the speed with which patrons swim at the beach. Therefore, the only way to avoid a high event rate from going too high is to plan ahead on the resource management of the business. For example, one may reduce the event rate by recruiting more searchers and dividing the monitoring regions. Future research on the nature and behavior of event-rate effects is important, especially for continuous visual search.
Target-rate effects on false alarms
In general, the current results indicated a target-rate effect on continuous visual search. Rare targets led to slower hit RTs and more misses. According to previous research on vigilance and visual search, decision bias may be a major cause of target-rate effects. However, according to this explanation, not only would a low target rate lead to a higher miss rate, but a high target rate should also lead to a more liberal criterion in accepting targets, with more false alarms ensuing. However, there were no target-rate effects on false alarms in our data. Instead, in all our experiments, the Bayesian analysis favored a lack of target-rate effect on false alarms, especially in Experiments 1 and 2.
We postulate three possible reasons for the lack of a target-rate effect on false alarms. The first possible reason is the floor effect. In our experiments, the number of false alarms per session was not high (3.8, 6.4, 2.2, and 1.3, respectively, for each experiment; the session durations were 15, 15, 10, and 7 min, respectively). However, they were not too close to zero either. Therefore, we are unsure how much our false alarms reflected unsystematic motor errors and how much they reflected genuine decision errors. We suppose there may be a mild floor effect, but it should not obscure the target-rate effect on false alarms altogether.
The second possibility is related to the way we analyzed the false alarm results. For instance, since continuous search is not separated by trials, we reported false alarm counts in our experiments because we do not have well-defined background events for calculating a false alarm rate like in regular discrete search. In discrete search studies, false alarm rates were calculated relative to the number of target-absent trials. As such, with the same false alarm count, the false alarm rate would be higher in a high-prevalence condition, where there were fewer target-absent trials. In continuous visual search, although the background events for false alarms to occur were ill-defined, they nevertheless occurred (i.e., the changing of non-target colors and orientations). Obviously, this should be taken into account in the analysis of false alarms, and it may not be correct to directly compare false alarm counts across target-rate conditions. It is possible that our lack of effect on false alarms is an illusion due to this incorrect comparison. Imagine dividing a search session into many imaginary time slices, and consider each time slice to be analogous to a search trial in a regular search task. If there was a target in a time slice, it was a “target-present slice”; otherwise, it was a “target-absent slice.” In this case, there would be more target-absent slices in a low-prevalence session. As such, if we calculate the false alarm “rate” relative to the number of “target-absent slices,” theoretically, the same false alarm count should result in a lower false alarm rate in a low-prevalence session and vice versa, consistent with the usual expectation.
The third possibility is related to a possible difference in the decision process between regular and continuous visual search. In regular visual search, Schwark et al. (2013) demonstrated that the search was driven by at least two types of decisions: search-based and prevalence-based decisions. A search-based decision means the observer finds and sees the target and then makes a target-present response. This decision is based on perception. Prevalence-based decision, on the other hand, is a cognitive one. For example, when the target prevalence is very high, the observer may think that it is a good bet to make a target-present response even if the target is not clearly seen. To distinguish these two kinds of decisions, Schwark et al. asked the participants to click on the target if they detected a target, but they also allowed the participants to press a target-present button (TP button) if they want to make a target-present response without locating the target. The TP button here represents the case wherein a response was made strategically, but not perceptually. It was found that the TP button was used more in higher prevalence and more difficult searches, providing evidence for prevalence-based decisions. Therefore, target prevalence may influence search performance in two ways. First, it may influence the search process. Wolfe and Van Vert (2010) suggested that target prevalence may alter our decision criterion when we inspect each search item during a search, leading to false detection or overlooking of targets. Second, it may influence strategic decisions. In prevalence-based decisions, even without perceiving any item as a target (whether it is, in fact, a target or not), one may still make a target-present response as a bet.
This distinction may be related to our lack of a false alarm effect. In our continuous visual search experiments, the search process is generally similar to regular searches. The observer inspects different search items throughout a search session, and, occasionally, the observer may accept some search items as targets, leading to detection responses. Target-rate manipulations may influence search-based decisions by changing the decision criterion used in each item inspection. However, continuous visual search is different from regular visual search because it is not separated by trials. It is reasonable to assume that prevalence-based decisions take individual trials as decision units. When making a prevalence-based decision, the cognitive process in an observer may be as follows: “I am not sure if there is a target in this trial, but since a target existed in most of the trials, let me make a target-present response for this trial.” However, this decision process may not be applied to a continuous visual search, where the observer sits and waits for a target without being forced to make a decision for each trial. For this reason, prevalence-based decisions may be much less common in continuous visual search. As such, in continuous visual search, we may expect that while the part of target-rate effects on false alarms attributed to search-based decisions may be preserved, the part due to prevalence-based decisions may diminish. Critically, based on the results of Experiment 1 in Schwark et al.’s study (2013, Fig. 5), whereas search-based detections (target click responses) were highly accurate, most false alarms actually came from prevalence-based detections (TP button responses). If it is generally true that most false alarms in visual search are due to prevalence-based decisions and that prevalence-based decisions are extremely reduced in continuous visual search, then it is reasonable to see a lack of target-rate effect on false alarms in continuous visual search.
The decision process
As mentioned above, a critical difference between regular and continuous visual search lies in their response choices, and this difference requires us to reconsider the approach to model the associated decision processes. In regular trial-by-trial visual search, one chooses between three possible responses at any moment: target present, target absent, and continue searching. A popular approach to model this decision is to view it as a diffusion process of gathering and gauging statistical evidence for or against the presence of a target. Should it reach the response criterion of either decision, a corresponding response could be made; otherwise, the search continues. In diffusion models, statistical evidence is relative; technically, it represents the (log) odds of one response over the other. Thus, in the realm of discrete visual search, the presence of a perceptual signal is statistical evidence for a target-present trial, whereas the lack of a perceptual signal is statistical evidence for a target-absent trial. In other words, diffusion models can integrate the fact that you have waited long enough but did not succeed in spotting any targets as a good sign of a target-absent trial.
However, in continuous visual search, the task is not to gauge evidence between target-present and target-absent responses. The lack of a target signal here is an uninteresting baseline state, which does not constitute any responses (no matter how long you rested in it). This analysis is true, at least when the number of targets in a session is unknown. On the other hand, there is only a target-present response, and it should be triggered upon the detection of a signal. This situation is more akin to the case of the signal detection theory (SDT), in which the decision criterion is a level of perceptual signal strength rather than the odds between two possible situations. Thus, one discerns whether an input is more likely to be a signal or a noise by comparing it with critical signal strength.
Although it seems that the SDT is a more suitable approach than diffusion models in understanding the decision process involved in continuous search, it should be noted that the current data analysis practices for SDT cannot be directly applied to continuous search. In SDT, researchers typically analyze hit, miss, correct rejection, and false alarm data and then transform them into d’ (sensitivity) and c (response criterion) measures. In continuous search, we do not have well-defined numbers of correct rejections, and false alarms cannot be expressed as a ratio between the number of false alarms and negative trials. In our experiments, we focused our analysis on hit RT and miss rates. Further theoretical work would be necessary to determine the best way to measure the sensitivity and response criteria in the continuous domain.