N2pc: 200–300 ms after stimulus onset (planned analyses)
Presence of the N2pc The first analysis investigated the presence of the N2pc component when the targets and foils appeared in the patterned or random locations from 200 to 300 ms after stimulus onset (Figs. 4 and 5). One-sample t tests of mean N2pc amplitude (compared to 0 μV) revealed a significant N2pc during the Exemplar Match trials, when the targets appeared in the patterned location (M = − 1.85 μV, SD = 3.01), t(18) = − 2.68, p = 0.015, d = − 0.61, and in the random location (M = − 2.71 μV, SD = 3.69), t(18) = − 3.20, p = 0.005, d = − 0.74. There was no significant N2pc when foils appeared in the patterned location (M = 0.10 μV, SD = 3.18), t(18) = 0.14, p = 0.891, d = 0.03, or in the random location (M = − 0.40 μV, SD = 2.36), t(18) = − 0.73, p = 0.474, d = − 0.17.
Omnibus ANOVA The next analysis investigated the effects of symbol position (patterned versus random) and trial types (Exemplar Match and Foil trials) on the mean amplitude. A 2 (symbol position: pattern and random) × 2 (trial type: Exemplar Match and Foil trials) ANOVA revealed a main effect of trial type, F(1,18) = 21.31, p < 0.001, η2p = 0.54, where Exemplar Match trials (M = − 2.28 μV, SD = 1.74) had a larger mean N2pc amplitude than Foil trials (M = − 0.15 μV, SD = 1.02). There was no main effect of symbol position, F(1,18) = 0.32, p = 0.581, η2p = 0.02, nor an interaction, F(1,18) = 0.25, p = 0.621, η2p = 0.01.
Mean amplitudes at the standard time window showed a significant N2pc component when targets appeared in both the patterned and random locations. There were no significant N2pc components when foils appeared in either location, suggesting possibly that foil effects did not occur, irrespective of where foils appeared. Although N2pc components emerged sometime in the 200–300 ms time window for Exemplar Match trials, there were no differences when the target appeared in the patterned nor random location. However, it is possible that visual patterns shifted covert attention sooner than 200 ms. To best select an optimal time window that captures N2pc components for all participants while minimizing noise, signed negative area (Gaspar & McDonald, 2018; Gaspar et al., 2016; Sawaki et al., 2012; Tay et al., 2019) was measured at a broad time window of 150–300 ms.
Signed negative area Signed negative areas were used to compute the magnitude of the N2pc component across a broad time window of 150–300 ms. Due to variability in noise with areas and bias to nonzero values, a nonparametric permutation test (e.g., Sawaki et al., 2012; Tay et al., 2019) was used in lieu of traditional one-sample t tests compared to zero. The dataset was permuted to create a distribution of expected values assuming the null hypothesis is true. Specifically, for each participant, individual trials were randomly assigned one of two possible conditions (patterned vs. random location), separately for Exemplar Match and Foil trials, to estimate noise while subtracting signal. The random assignment process occurred 500 times, and each time, a grand average of the four conditions (2 symbol positions × 2 trial types) was calculated. For each grand average, a signed negative area was obtained, with a total of 500 areas for each condition. Figure 6 shows the null distribution of the 500 signed negative areas for each condition. The p value was based on the proportion of times the observed area (i.e., area from grand average with correct assignment of conditions) was larger (in this case, more negative) than the permuted value (see Eq. 1 in Tay et al., 2019). The p value was considered significant when the observed value exceeded the 95th percentile of the null distribution.
For Exemplar Match trials, the signed negative area was significant when targets appeared in the patterned location (M = 0.32 μV, SD = 0.34), p = 0.002, and in the random location (M = 0.41 μV, SD = 0.35), p = 0.002. Similar results were found with the Foil trials, in the patterned location (M = 0.20 μV, SD = 0.26), p = 0.002, and in the random location (M = 0.15 μV, SD = 0.15), p = 0.002. Given that the effects for all conditions were well beyond what was to be expected under the null (i.e., top 5% values), it is unlikely that the effects were due to chance. A 2 (symbol position) × 2 (trial type) ANOVA revealed no main effect of symbol position, F(1,18) = 0.04, p = 0.849, η2p = 0.20. There was a main effect of trial type, F(1,18) = 24.58, p < 0.001, η2p = 0.58, with a larger N2pc area for Exemplar Match trials compared to Foil trials. There was also an interaction between symbol position and trial type, F(1,18) = 4.52, p < 0.048, η2p = 0.20. Pairwise comparisons did not reveal a difference between patterned and random location for either Exemplar Match trials, t(18) = − 0.67, p < 0.51, or Foil trials, t(18) = 0.54, p < 0.60.
Multivariate pattern analysis (MVPA) To examine whether scalp distributions were sensitive to spatial information (i.e., locations that contained visual patterns), an exploratory MVPA with support vector machines (SVMs) was implemented in MATLAB 2021a, with the EEGLAB (Delorme & Makeig, 2004) and ERPLAB (Lopez-Calderon & Luck, 2014) toolbox. The decoding procedures were adapted from Bae and Luck (2018, 2019) and were run separately for Exemplar Match and Foil trials. Briefly, for each participant’s preprocessed data, SVMs were trained to distinguish scalp responses (excluding HEOG and reference electrodes) at each time point when items appeared in the patterned location vs. random location. A threefold cross-validation (10 iterations) was used, where the data were randomly divided into three separate blocks with an equal number of trials. For each block, the trials were averaged to increase signal-to-noise ratio. For each iteration, two of the three blocks were randomly selected for training, and the last block was used for testing. Decoding accuracy was based on comparing the true label (i.e., patterned or random location) with the predicted label, with a chance performance of 0.50 (= 1/2). For each participant, decoding accuracy was averaged as a proportion across 60 decoding attempts (2 symbol positions × 3 cross-validations × 10 iterations) for each time point (60 for each trial type).
Decoding analysis First, the decoding performance was averaged across the broad time window of 150–300 ms (i.e., the likely window in which shift in covert attention occurred), and then compared to chance performance (0.50). A one-sample t test revealed decoding accuracy was well-above chance for both Exemplar Match trials (M = 0.71, SD = 0.14), t(18) = 6.35, p < 0.001, d = 1.46, and Foil trials (M = 0.55, SD = 0.06), t(18) = 3.19, p = 0.005, d = 0.73. To examine the differences in decoding performance between the trial types, a pairwise comparison revealed better performance for Exemplar Match trials than for Foil trials, t(18) = 4.13, p < 0.001, d = 0.95.
Second, a one-sample t test compared to chance was computed for each time point in the 150–300 ms time window. Then, clusters of contiguous significant time points were used to compute cluster-level t mass (i.e., the sum of t scores within a cluster). To control for Type I error, a null distribution of cluster-level t mass values was created based on permutation tests (see Bae & Luck, 2018). A permutation test iterated 10,000 times (a total of 10,000 t mass values assuming the null), and the p value of the observed t mass was considered significant if the observed t mass exceeded the top 95% of the null distribution (i.e., p < 10–4). For Exemplar Match trials, there was a significant cluster (p < 0.0001, one-tailed) across the whole 150–300 ms time window (Fig. 7), compared to the null distribution (critical t mass = 31.25, α = 0.05). For Foil trials, there was a significant cluster (p < 0.0001, one-tailed) between 212 and 262 ms, compared to the null (critical t mass = 26.29, α = 0.05).
Onset latency of N2pc
Jackknife latency A jackknife latency analysis was conducted using a − 0.75 µV threshold (Kiesel et al., 2008) for N2pc studies (methods described in Miller et al., 1998). Eighteen out of 19 participants’ data met the threshold for all four conditions (= 2 symbol positions × 2 trial types). The last participant’s data met the threshold for Exemplar Match trials in both the patterned and random location but only Foil trials in the patterned location. Therefore, only the data from Exemplar Match trials were included in the pairwise comparisons. A 2 (symbol position) × 2 (trial type) ANOVA revealed a main effect of symbol position, F(1,17) = 759.87, p < 0.001, η2p = 0.98, with a faster latency when items appeared in the patterned location (M = 137.42 ms, SD = 3.88) than in the random location (M = 226.39 ms, SD = 10.96). There was a main effect of trial type, F(1,17) = 491.49, p < 0.001, η2p = 0.97, with a faster latency for Exemplar Match trials (M = 156.21 ms, SD = 2.51) than for Foil trials (M = 207.67 ms, SD = 9.20). There was an interaction between symbol position and trial type, F(1,17) = 77.09, p < 0.001, η2p = 0.82. A pairwise comparison revealed a shorter latency when targets appeared in the patterned location (M = 124.42 ms, SD = 4.93) than in the random location (M = 188.00 ms, SD = 2.31), t(18) = − 47.50, p < 0.001, d = − 10.90. Similarly, latency was shorter when foils appeared in the patterned location (M = 150.42 ms, SD = 4.60) than in the random location (M = 265.00 ms, SD = 21.67), t(17) = − 19.15, p < 0.001, d = − 4.51.
Fractional area latency A fractional area latency analysis was conducted to detect potential early onset of N2pc components prior to the standard time window of 200–300 ms. Based on onset latencies with the jackknife approach, we chose a broad time window of 150–300 ms with a 50% negative area. Fifteen out of the 19 participants’ data met the threshold for all four conditions (= 2 symbol positions × 2 trial types). Three other participants’ data met the threshold for Exemplar Match trials in both patterned and random location but only Foil trials in the patterned location. Therefore, only the data with Exemplar Match trials were included in the pairwise comparisons. The last participant’s data met the threshold for both Exemplar Match and Foil trials in the patterned location but not in the random location. Therefore, these data were not included in any analyses. A 2 (symbol position) × 2 (trial type) ANOVA revealed no main effect of symbol position, F(1,14) = 2.58, p = 0.131, η2p = 0.16, nor a main effect of trial type, F(1,14) = 2.31, p = 0.151, η2p = 0.14. There was an interaction between symbol position and trial type, F(1,14) = 7.88, p = 0.014, η2p = 0.36. A pairwise comparison revealed no difference in latencies between the patterned (M = 238.11 ms, SD = 26.55) and random (M = 240.22, SD = 28.75) locations for Exemplar Match trials, t(17) = − 0.23, p = 0.82. However, latencies were shorter when foils appeared in the patterned location (pattern: M = 210.80, SD = 42.51), than in the random location (random: M = 248.00, SD = 30.47), t(14) = − 2.23, p = 0.042, d = − 0.58.
Data quality of ERP measures using standardized measurement error To further test the data quality of ERP measures for the N2pc component, the bootstrapped standardized measurement error (bSME; see Luck et al., 2019) was calculated for the fractional area latency, mean amplitude, and signed negative area. Briefly, for each participant’s data, a simulation of the experiment was conducted 10,000 times, per condition, by sampling randomly with replacement from the correct trials after artifact rejection. For each iteration, the averaged waveform was made, and the three measures (i.e., fractional area latency, mean amplitude, and signed negative area) were computed, with a total of 10,000 values each. After completing the iterations, the bSME is calculated, which is the standard deviation of all 10,000 values. In total, each participant had 12 bSME values (2 symbol positions × 2 trial types × 3 measures). To get the average data quality across participants, we calculated the root mean square (RMS), or the aggregate of bSME’s of all participants, resulting in 12 RMS values (2 symbol positions × 2 trial types × 3 measures).
Currently, there is no conventional method to determine an ideal threshold for a “good” RMS value, let alone specific for N2pc components. However, one preliminary approach is to compare the RMS to the standard deviation of the group mean (from the observed grand average waveform), which is influenced by both true differences between participants and measurement error. A lower RMS value likely indicates that the observed variability was driven by true differences, instead of measurement error. For fractional area latency, the RMS values were the following: Exemplar Match with patterned location, 17.37 (compared to SD = 26.55); Exemplar Match with random location, 25.59 (compared to SD = 28.75); Foil with patterned location, 14.09 (compared to SD = 42.51); Foil with random location, 14.42 (compared to SD = 30.47). For mean amplitude, the RMS values were the following: Exemplar Match with patterned location, 0.82 (compared to SD = 3.01); Exemplar Match with random location, 0.83 (compared to SD = 3.69); Foil with patterned location, 0.78 (compared to SD = 3.18); Foil with random location, 0.76 (compared to SD = 2.36). For the signed negative area, the RMS values were the following: Exemplar Match with patterned location, 0.08 (compared to SD = 0.29); Exemplar Match with random location, 0.07 (compared to SD = 0.24); Foil with patterned location, 0.08 (compared to SD = 0.21); Foil with random location, 0.08 (compared to SD = 0.11). For all measures and conditions, the RMS values were moderately or much lower than the sample standard deviation, which suggests strong precision (i.e., low noise level) in the ERP measurements.
Behavioral results To measure the effects of the symbol position and trial types on mean reaction time (RT) for correct trials, a 2 (symbol position) × 2 (trial type) ANOVA revealed a main effect of trial type, F(1,18) = 27.03, p < 0.001, η2p = 0.60, with a faster RT for Exemplar Match trials (M = 616.21 ms, SD = 81.40) than for Foil trials (M = 660.14 ms, SD = 92.30). There was no main effect of symbol position, F(1,18) < 0.35, p = 0.563, η2p = 0.02, nor an interaction between symbol position and trial type, F(1,18) < 0.18, p = 0.678, η2p = 0.01 (Fig. 8). To measure the same effects as RT on accuracy, a 2 (symbol position) × 2 (trial type) ANOVA revealed no main effect of trial type, F(1,18) = 0.96, p = 0.340, η2p = 0.0.05, no main effect of symbol position, F(1,18) = 0.39, p = 0.540, η2p = 0.02, and no interaction between trial type and symbol position, F(1,18) = 0.44, p = 0.514, η2p = 0.024.
2AFC results The two-alternative forced choice recognition task did not reveal an explicit preference for triplets (i.e., runes in the patterned location) over non-triplets (i.e., runes in the random location), t(18) = 0.66, p = 0.517, d = 0.15 (M = 0.51%, SD = 0.05). The participants provided an average confidence rating (scale from 1 to 4, 4 being highest confidence) of M = 2.74, SD = 0.28. In addition, participants did not explicitly report any suspicion of the symbols, nor did they report implementing any strategies during the search task.