Performance with the in situ device was compared to a conventional stationary display in two tasks. Novice users explored a data set derived from contrast-enhanced thoracic CT images of pulmonary vessels. In an initial navigation task, they were asked to trace along 3D vessels using 2D cross-sections and to determine where the vessels terminated. In a second task, participants reported the angular relation between two locations within the lung vasculature in the 3D space defined by the data set.
Methods
Participants
Thirteen naïve observers and three coauthors (four females and twelve males), inexperienced in interpreting medical images, participated, with informed consent. All were young adults with normal or corrected-to-normal vision in both eyes.
Stimuli
A set of 18 contrast-enhanced CT scans of the thorax was acquired. An expert identified the pulmonary vasculature in each scan and used colored spheres (radius 5 mm) to label three structures: the pulmonary artery (PA) as it exits the right ventricle of the heart (green sphere), the left atrium (LA) where the pulmonary veins drain (blue sphere), and one distal branch of a pulmonary vessel (red sphere), which the expert recorded as objectively an artery or a vein. The spheres were visualized as cross-sectional disks overlaid on the particular slice from the data being displayed. Using each scanned CT sequence twice, once with an artery and once with a vein, a stimulus set of 36 uniquely labeled vessels was generated; each was connected to either the PA or the LA, but not both. In general, these vessels can be identified in CT images only by their anatomical connection to the heart, rather than pixel intensity or local vessel morphology, so to a novice they are indistinguishable except by tracing along the vessel, slice by slice, to either the PA or LA. Figure 3 shows representative CT images. Each scan slices across the underlying branching structure of vessels; successive slices within the volume shift which vessels are visible and the size and location of those seen continuously.
The distance (in the axial dimension anatomically of the scan) between the red sphere (marking the unknown vessel) and the correct endpoint (either the PA or LA) was used to classify each vessel into one of three categories according to its distance from the endpoint: short (5 to 30 mm), medium (50 to 80 mm), or long (>100 mm). Across stimuli, the distances were uniformly distributed among these categories and the scan was displayed at scale.
Design and procedure
A 3 (Distance) × 2 (Display condition: in situ or ex situ) within-subjects design was implemented. Here, in situ visualization refers to visualization of the image data on the movable FRISM display, while ex situ refers to visualization on a fixed conventional display. The ex situ display was identical to the in situ display, except that it was stationary on a table directly adjacent to the space in which the FRISM display would be manipulated. In either case, movement through the data was controlled by physically moving the FRISM display on its boom arm (see Fig. 2). Six trials were performed in each condition; a unique stimulus (as defined by source data and PA versus LA target) was used for each of the resulting 36 trials. Trials were blocked into two sets of 18 trials by the viewing condition, with the presentation order of trials counterbalanced across blocks and participants using a Latin square. The testing order of the two viewing conditions was also counterbalanced to avoid bias from learning.
Participants performed the experiment in a room with overhead lighting eliminated. With the screen blanked out, participants moved the FRISM display until they found the red sphere in 3D space, which appeared in cross-section as a disk on the screen when the display encountered it. The corresponding slice of CT data then appeared on the screen, with the red sphere depicted inside an unknown pulmonary vessel. Participants were instructed to remember the location of the red sphere relative to the surrounding 3D space. The first task required navigation through the data set. Participants followed the vessel by moving the display while maintaining a continuous path from the starting point to the endpoint. Eventually, the vessel terminated at a slice in which the PA (marked with a green sphere) and LA (marked with a blue sphere) both appeared, at which point the participant made a forced-choice selection about which endpoint was connected to the starting point by means of a color-coded keypad. Participants were timed during the tracing from the red sphere at the origin to the endpoint sphere (blue or green).
The second task, spatial relations, was assessed immediately after the endpoint sphere was selected. The screen was again blanked out, and participants were asked to indicate the vertical (that is, gravitationally aligned) plane containing the centers of the red starting-point sphere and the selected endpoint sphere. They responded by rotating the blank FRISM display about the y-axis (see Fig. 2) until it was perceived to be parallel to the vertical plane connecting the starting and endpoint locations. Valid angles ranged from −80° to +80° relative to the starting position. Prior to the experimental trials, participants performed two to four sample trials in each viewing condition to demonstrate the tasks.
Results
Navigation task
The difficulty of navigation through pulmonary vasculature is limited by its intrinsic branching, which here produced a ceiling effect in performance: There were few errors in navigation (accuracy 95.8% in situ versus 93.1% ex situ). Although this small difference reaches the standard p < 0.05 significance level by 1-tail test (t(15) = 2.07, p = .028), under the directional prediction that the in situ display would be superior, the effect is not strong. There was also no significant difference in time to navigate (34.9 seconds for in situ, 38.5 seconds for ex situ). To provide a stronger test of whether the in situ display facilitates navigation, it would be necessary to bring performance below ceiling, possibly by constructing an artificial CT data set with more complex branching.
Spatial relations task
Correct and response angles were recorded as values within ± 180°, signed relative to the z-axis. In previous research assessing visualization of 3D relationships from 2D images, we found a type of error in which angular judgments are correct in magnitude, but reversed in direction, which was particularly prevalent with an ex situ display (Wu et al., 2010). Such “reversal errors” were clearly evident in the present data. We identified a reversal as occurring when the response differed in sign from the correct value and the absolute difference was greater than 90°. Although reversal errors were few, they followed the previous pattern that the in situ display produces fewer errors (12 versus 23 for ex situ, constituting 4.2% and 8.0% of responses, respectively); the difference reached significance by 1-tailed test, p < .05. To eliminate the effects of these errors in the subsequent analyses, the sign of the response angle for such trials was reversed.
The principal measure of performance on the visualization test is the degree to which the mean reported angle for a given stimulus matches the correct value for that stimulus. When response angles are regressed against actual values, the ideal slope would be 1.0. When such regressions were done for each display, the slope for FRISM was closer to the ideal than the slope for the conventional, ex situ display (mean slope = .94 for in situ versus .85 for ex situ, r
2 = .96 and .98, respectively, t(15) = 3.26, p = .005). (If the same test is done with the naïve participants alone, the results are essentially the same: mean = .92 for in situ versus .83 for ex situ, t(13) = 3.78, p = .003.)
This result is augmented by an analysis of individual performance in the spatial relations task, which divided the participants into two groups that corresponded to levels above and below the median of the slope averaged over the two displays. The relation between the response and actual angles is shown for each group and display in Fig. 4. The higher-performing group showed little advantage from in situ imaging, as their slopes statistically reached the ideal value of 1.0 in both cases (mean slope = 1.05 and 1.03 for in situ and ex situ, r
2 = .96 and .97, respectively). In contrast, the lower performers were substantially aided by the in situ display (mean slope = 0.84 and 0.68 for in situ and ex situ, r
2 = .94 and .94, respectively).
Performance with the two displays was highly correlated across individual participants, as shown in Fig. 5. Notably, the extent to which a participant’s slope for in situ exceeded that for ex situ (with no improvement indicated by points on the diagonal) tended to be greater, as a participant’s overall performance was poorer. In other words, in situ imaging provided the greatest help to those who needed it most.
Absolute errors in the reported angle between start and endpoints, which combines systematic and variable error, were analyzed with an ANOVA on modality × distance to the endpoint. Mean absolute error was reduced by in situ viewing (11.7° versus 14.7° for ex situ), F(1,15) = 6.65, p = .021. There was no systematic effect of distance, F(2,30) = 1.06 (mean = 14.36°, 12.30°, and 12.85° for short, medium, and long, respectively), nor was there a significant interaction, F(2,30) <1.
Note that the foregoing analyses ignore the few errors that arose in the navigation task, which could indicate a misconception of the correct angle. An alternate analysis on the spatial relations data, assigning the “correct” response angle to whichever ending location had been indicated in the navigation task, produced essentially equivalent results.