- Original article
- Open Access
Inversion effects in the expert classification of mammograms and faces
© The Author(s) 2018
- Received: 27 July 2017
- Accepted: 23 June 2018
- Published: 15 August 2018
A hallmark of a perceptual expert is the ability to detect and categorize stimuli in their domain of expertise after brief exposure. For example, expert radiologists can differentiate between “abnormal” and “normal” mammograms after a 250 ms exposure. It has been speculated that rapid detection depends on a global analysis referred to as holistic perception. Holistic processing in radiology seems similar to holistic perception in which a stimulus like a face is perceived as an integrated whole, not in terms of its individual features. Holistic processing is typically subject to inversion effects in which the inverted image is harder to process/recognize. Is radiological perception similarly subject to inversion effects? Eleven experienced radiologists (> 5 years of radiological experience) and ten resident radiologists (< 5 years of radiological experience) judged upright and inverted bilateral mammograms as “normal” or “abnormal”. For comparison, the same participants judged whether upright and inverted faces were “happy” or “neutral”. We obtained the expected inversion effect for faces. Expression discrimination was superior for upright faces. For mammograms, experienced radiologists exhibited a similar inversion effect, showing higher accuracy for upright than for inverted mammograms. Less experienced radiology residents performed more poorly than experienced radiologists and demonstrated no inversion effect with mammograms. These results suggest that the ability to discriminate normal from abnormal mammograms is a form of learned, holistic processing.
- Perceptual expertise
Breast cancer is the most prevalent cancer in women, with approximately 250,000 new cases reported each year. The ability to accurately diagnose breast cancer in an X-ray image is an essential medical skill for early detection and treatment. However, identifying breast cancer in an X-ray is not an easy perceptual task and requires many years of experience and practice. One of the key differences between an expert and novice radiologist is their ability to detect abnormalities within a medical image in the blink of an eye. It has been speculated that holistic processing (widely reported in face recognition) helps enable this rapid detection. In the present study, we test this claim by employing an inversion paradigm, which has been demonstrated to disrupt holistic processing. Resident and experienced radiologists were asked to identify abnormal and normal mammograms presented in their upright and inverted orientations. We found that experienced radiologists were more accurate at identifying abnormalities in upright mammograms than in inverted mammograms. By comparison, the radiology residents performed more poorly than the experienced radiologists overall and their performance was not affected by inversion. The expert inversion effect indicates that experienced radiologists employ holistic processes to assist their rapid detection of breast cancer and these processes are disrupted when the mammogram is turned upside down. Our results suggest that teaching methods in medical imaging may benefit by including a holistic approach in which students are trained in the rapid detection of many mammogram exemplars.
A hallmark of the human visual system is our ability to make rapid visual categorizations in fractions of a second, whether we are interpreting the meaning of a picture (Potter, Wyble, Hagmann, & McCourt, 2013), classifying a scene (Schyns & Oliva, 1994) or recognizing a familiar face (Grill-Spector & Kanwisher, 2005).
In a medical evaluation, diagnosis of chest radiographs and mammograms requires the detection and localization of the radiological abnormality (Kundel, Nodine, Conant, & Weinstein, 2007). After an initial glimpse, expert radiologists report that they have an intuition that a mammogram is likely to be normal or abnormal before any pathology is localized. In searching for signs of lung cancer, Kundel and Nodine (1975) found that radiologists could achieve a d’ of about 1.0 after a just 200 ms glimpse of a chest X-ray. This level of performance was nowhere near the level of d’ of 2.5 obtained during free-viewing conditions of these stimuli but, nevertheless, comfortably above chance performance. In mammography, Evans, Georgian-Smith, Tambouret, Birdwell, and Wolfe (2013) found a similar level of performance after a 250 ms exposure to mammograms. The rapid global analysis of the radiological image has been referred to as holistic processing and is the precursor to the subsequent stage involved in the localization of abnormalities (Carrigan et al., 2018).
In holistic processing, recognition relies on the integration of individual stimulus parts into an emergent whole representation that is qualitatively more than the summed representation of its individual parts. Face recognition is the prime example of holistic processing where recognition is based on the synthesis of facial features that yields a unique face that is more than the summed recognition of each individual facial feature. Three tasks have been applied as the gold standards for testing holistic face processes: the face composite task, the parts/wholes task, and the inversion task. In the “face composite task”, it has been demonstrated that participants find it difficult to selectively attend to one half of a face (e.g., top half) while ignoring information from the other half (e.g., bottom half) (Young, Hellawell, & Hay, 1987). In the face composite task, the whole face representation makes it difficult for participants to selectively attend to one region of the face, isolated from the whole face. In the “parts/wholes task”, participants exhibit better recognition when a face part (mouth) is displayed in the whole face than when displayed in isolation (Tanaka & Farah, 1993; reviewed in Tanaka & Simonyi, 2016). The parts/wholes task demonstrates that facial features are not represented in memory as individual parts, but are integrated into a whole face representation.
Perhaps the most widely used test of holistic face processes is the face inversion task (Yin, 1969). Although all objects are more difficult to recognize when inverted compared to upright, inversion disproportionately impairs the recognition of faces relative to other object classes (McKone & Yovel, 2009; Rossion, 2008; Yin, 1969). Turning a face upside down disrupts the normal holistic face processing and forces the participant to use a less optimal strategy based on analysis of specific features (wide-set eyes, square jaw, etc). Inversion has been shown to abolish the holistic interference observed in both the face composite task (Rossion & Boremanse, 2008; Young et al., 1987) and the whole face recognition advantage in the parts/whole task (Tanaka & Farah, 1993; Tanaka & Sengco, 1997).
Real world perceptual experts, such as birdwatchers or dog judges, are similar to face “experts” in that they recognize objects in their domain of expertise quickly, accurately, and at a specific level of categorization (Tanaka & Taylor, 1991). To facilitate their speeded precognition, it has been hypothesized that expert recognition demands the same kind of holistic processing that is employed in face processing. Therefore, it follows that expert object recognition should be susceptible to similar manipulations used in face recognition, such as inversion. In a seminal study, Diamond and Carey (1968) tested this prediction by asking dog judges and control participants to recognize upright and inverted photographs of dogs. They found that while the novices exhibited an inversion effect only for faces, dog experts showed a significant inversion effect for both faces and dogs. In other expert object recognition studies, inversion impairs the speed and accuracy of expert recognition processes (Ashworth, Vuong, Rossion, & Tarr, 2008; Campbell & Tanaka, 2018; Rossion & Curran, 2010; Rossion, Gauthier, Goffaux, Tarr, & Crommelinck, 2016) and limits the visual short-term memory capacity of the expert (Curby, Glazek, & Gauthier, 2009).
Although it has been speculated that mammogram expertise involves holistic strategies (Kundel et al., 2007), direct tests of holistic processing strategies in radiology have yet to be conducted. To investigate a possible link between holistic perception and mammogram expertise, we tested the effects of inversion on a group of experienced radiologists (> 5 years of radiology experience) and radiology residents (< 5 years of radiology experience). On average, an experienced mammographer evaluates between 1000 and 15,000 images per year (Evans et al., 2013) compared to resident radiologists, who see fewer than 300 cases during the course of their clinical training. Expert mammographers and residents have likely received similar formal mammography training, but it is the experts, with their extended experience, who exhibit evidence of rapid detection (Evans et al., 2013; Kundel & Nodine, 1975).
In our study, participants made a “normal/abnormal” decision to briefly presented upright and inverted mammograms. In order to rule out any age-related inversion effect, participants were asked to judge the facial expressions (e.g., neutral/happy) of briefly presented upright and inverted faces. Recognition of facial expressions, like facial identity, recruits holistic perception that is disrupted by inversion (Calder & Jansen, 2005). We made three predictions: First, given that virtually everyone is an expert in holistic expression perception, we expected that both the experienced and resident radiologists would show an inversion effect in their perception of expression (i.e., better detection of happy expressions in upright faces than inverted faces). Second, we hypothesized that the experienced radiologists (< 5 years of radiology practice) would be more accurate in their discriminations of upright mammograms than novice radiology residents. Finally, as evidence of their holistic strategies, we predicted that the experienced radiologists should show a greater inversion effect to mammograms (i.e., difference between upright and inverted recognition) than the resident radiologists.
Of 21 study participants, 11 were highly experienced radiologists who performed daily breast radiology screening and had at least 5 years of experience (eight female, three male; average age 56 years), average 18 years in practice (range 6 to 37 years) reading, on average, 6045 cases (range 1000 to 10,000) in the last year. The other ten participants were radiology residents who had fewer than 5 years of experience (five female, five male; average age 34 years), average 3 years in practice (range 2 to 5 years) reading, on average, 297 cases (range 20 to 500) in the last year. The expertise cut-off was based on previous studies (Evans et al., 2013; Nodine, Kundel, Mello-Thoms, et al., 1999) which suggested that radiologists with more than 5 years of experience had significantly better discrimination on rapidly presented mammograms. All study participants were recruited during the 2016 Radiology Society of North America (RSNA) conference in Chicago, Illinois, US. This study was approved by the Human Research Ethics Board at the University of Victoria, ethics protocol number 16–362. All participants had normal or corrected-to-normal vision and gave informed consent. The sample size was dictated by the availability of participants.
Stimuli and apparatus
Twenty morphs were developed from the NimStim Emotional Face Stimuli database (Tottenham et al., 2009), by overlaying a neutral and happy expression of the same individual and shifting the opacity. Faces were piloted to determine the level of difficulty that was sufficient to demonstrate an inversion effect. Higher percentage morphs (100% happy expression) were extremely salient; 40% happy, 60% neutral faces were found to be an optimal difficulty for demonstrating the face inversion effect. Control points were placed on salient features of each matching face and opacity was modified using the FantaMorphTM software package (v4.1, Abrosoft, http://www.fantamorph.com). At least 50 control points were placed on each face, and control points were added to remove obvious artifacts in the resulting morph. Hair and clothing information was removed with Adobe PhotoshopTM graphics program (v7.0, Adobe, http://www.adobe.com/photoshop). Faces were scaled to fit within a frame of 250 × 375 pixels and pasted on a black background (Fig. 1b). Images subtended a visual angle of approximately 5.6° vertically and 3.7° horizontally with participants sitting 50 cm from the screen.
For the ANOVA on the mammogram data, Experience (experienced radiologists, resident radiologists) was a between-group factor and Orientation (upright, inverted) was a within-group factor. For mammogram recognition sensitivity (d’), there was a main effect of Experience (F(1, 19) = 6.18, p < 0.02, ηp2 = 0.22; experts are better). A direct comparison using paired t-tests revealed that experts had better performance for upright mammograms (t(10) = 3.6, p = 0.0019, Cohen’s d = 0.45) compared to residents. There was no reliable difference for the inverted mammograms (p > 0.2). A Group by Orientation interaction was obtained (F(1, 19) = 11.91, p = 0.003, ηp2 = 0.09), reflecting the presence of an inversion effect for the experts but not residents. The inversion effect was assessed with paired t-tests. These reveal a significant inversion effect for experts (t(10) = − 2.8, p = 0.018, Cohen’s d = − 0.51). The effect for residents was not significant and, in any case, goes in the opposite direction from what would be expected (p > 0.33) (Fig. 3b). Reaction time data for correct mammogram trials showed no Experience (F(1,19) = 1.03, p > 0.32, ηp2 = 0.05) or Orientation (F(1,19) = 2.89, p > 0.11, ηp2 = 0.04) effects.
Recognition performance of radiology experts and residents for upright and inverted mammogram and face trials in terms of hits (hit), false alarms (fa), sensitivity (d’) and bias (c)
In this study, we employed the inversion task to test the holistic hypothesis of radiological expertise. Experts (> 5 years of experience) and residents (< 5 years of experience) were asked to classify upright and inverted mammograms as either normal or abnormal. As comparison stimulus, the same participants judged whether upright and inverted faces displayed a happy or neutral expression. For faces, both resident and experienced radiologists exhibited the classic face inversion effect where face expression discrimination was better in the upright orientation than the inverted orientation. For mammograms, experienced radiologists showed superior discrimination relative to resident radiologists for mammograms presented in their upright orientation. Although the overall performance of resident radiologists was worse than the experienced radiologists, their detection rates were unaffected by orientation and were essentially the same for upright and inverted mammograms. When the mammograms were inverted, the discrimination scores of the experienced radiologists dropped to the same level as demonstrated by resident radiologists. These results are consistent with previous studies of perceptual expertise showing that, with extensive domain-specific experience, experts access holistic information in an upright stimulus, but holistic information is impaired when the stimulus is inverted (Campbell & Tanaka, 2018; Diamond & Carey, 1986; Rossion, Gauthier, Goffaux, Tarr, & Crommelinck, 2016).
Although inversion impaired mammogram detection for the experienced radiologists, it did not completely abolish the expertise advantage. The performance of experienced radiologists in this study (d’ = 0.99) was comparable to detection levels of expert radiologists in previous studies for upright mammograms (d’ = 1.14) (experiment 1; Evans, Haygood, Cooper, Culpan, & Wolfe, 2016), albeit at a shorter exposure presentation of 500 ms. The residual expert effect for inverted mammograms suggest that other types of non-holistic information survived the inversion manipulation. Harley et al. (2009), for example, presented chest radiographs to experts for 500 ms and compared normal images to chest radiographs that were scrambled (e.g., segmenting image into 25 squares and shuffling their positions). Despite disrupting the global structure of the stimulus, the experts’ performance dropped slightly from d’ = 1.23 for the intact stimulus compared to d’ = 1.09 for the scrambled stimulus.
The holistic account of mammogram expertise is consistent with eye-tracking studies which show that, in comparison to non-experts, experts typically perform domain-related tasks with fewer fixations, longer saccades, and less coverage of the image (Krupinski, 1996; Manning, Ethell, Donovan, & Crawford, 2006). One study directly examined the eye position of expert breast radiologists and of novice radiology residents when reading digital mammograms (Kundel et al., 2007). They found that the median time for the eyes of the experts to reach the location of a cancerous nodule was 0.96 s from image onset whereas for the novices that time was 2.15 s. The authors speculated that response time for the experts was too fast to support a search-to-find strategy, implying that their search was being guided by a global representation of the mammogram. In the current study, the exposure duration was increased to 1000 ms in order for the experienced and resident participants to perform above chance levels. The extended exposure duration contrasts with previous “gist” studies where expert radiologists achieved reasonable detection scores after a much shorter presentation duration (i.e., 250 ms or less; Carrigan, Wardle, & Rich, 2018; Evans et al., 2016; Kundel & Nodine, 1975). Given that inverted and upright mammograms were randomly presented in the current study, it is possible that additional encoding time was required to, first, determine the orientation of the stimulus and, next, to apply the appropriate holistic and non-holistic strategy. In future studies, it would be useful to block the mammogram stimuli by orientation so participants will have the opportunity to prepare their detection strategy prior to the onset of the stimulus.
The emergence of a holistic strategy with radiological experience has implications for training medical students. In our study, the expert radiologists acquired holistic strategies implicitly over many years of clinical experience, evaluating thousands, even tens of thousands of mammograms images. It is interesting to speculate whether this perceptual knowledge can be taught explicitly during medical training by presenting medical students with many images of abnormal and normal mammograms, asking them to judge the normality of each image and providing the appropriate feedback. A perceptual expertise protocol might accelerate the learning process and allow radiological trainees to achieve expert performance more quickly. The perceptual expertise training approach has been successfully applied in other medical domains that require visual diagnosis. For example, participants showed reliable gains in their ability to detect melanoma skin lesions after four 30-min sessions of perceptual expertise training (Xu, Rourke, Robinson, & Tanaka, 2016).
The goal of this study was to measure the holistic processing of expert mammographers by employing the inversion test—a standard measure of holistic processing used in the face recognition research. The main finding was that experienced radiologists exhibited a robust inversion effect as evidenced by their better discrimination of upright mammograms than inverted mammograms. In contrast, the less experienced, resident radiologists performed more poorly than experienced radiologists and their discriminations were the same for upright and inverted mammograms. Critically, detection performance improved and the inversion effect increased for radiologists who had more experience diagnosing mammogram images, suggesting that holistic abilities developed as a function of perceptual experience. Our results have implications for training medical students by emphasizing the role of experience and the learning principles of perceptual expertise.
Alison Campbell assisted in data analysis.
Natural Sciences and Engineering Research Council of Canada (to JT) for travel expenses to the Radiology Conference of North America. National Eye Institute (US; award number EY017001; to JWW); supported data collection at the Radiology Conference of North America. National Cancer Institute (award number CA207490; to JWW); supported data collection at the Radiology Conference of North America.
Availability of data and materials
The dataset supporting the conclusions of this article is available in the figshare repository, Mammogram and Face Data, https://doi.org/10.6084/m9.figshare.5248513.v1 in CSV format.
MDC: design and experiment development, collection of data, data analysis, primary author. KKE: design, stimulus development, author. JMW: design, author. JWT: design, author. JB: experiment development, collection of data, author. All authors read and approved the final manuscript.
Ethics approval and consent to participate
Signed consent was obtained from human subjects prior to their participation in the study indicating their knowledge of the goals, publication, and possible health risks related to the study. This study was approved by the Human Research Ethics Board at the University of Victoria, ethics protocol number 16-362.
Consent for publication
Not applicable: manuscript does not contain individual details, data, or photos.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Ashworth, A. R. S., Vuong, Q. C., Rossion, B., & Tarr, M. J. (2008). Recognizing rotated faces and greebles: What properties drive the face inversion effect? Visual Cognition, 16, 754–784.View ArticleGoogle Scholar
- Calder, A. J., & Jansen, J. (2005). Configural coding of facial expressions: The impact of inversion and photographic negative. Visual Cognition, 12, 495–518.View ArticleGoogle Scholar
- Campbell, A., & Tanaka, J. W. (2018). Inversion impairs expert budgerigar identity recognition: a face-like effect for a nonface object of expertise. Perception, 1–13. https://doi.org/10.1177/0301006618771806.
- Carrigan, A. J., Wardle, S. G., & Rich, A. N. (2018). Finding cancer in mammograms: if you know it’s there, do you know where? Cognitive Research: Principles and Implications, 3, 10. https://doi.org/10.1186/s41235-018-0096-5.Google Scholar
- Curby, K. M., Glazek, K., & Gauthier, I. (2009). A visual short-term memory advantage for objects of expertise. Journal of Experimental Psychology: Human Perception and Performance, 35, 94–107.PubMedGoogle Scholar
- Diamond, R., & Carey, S. (1986). Why faces are and are not special: an effect of expertise. J. Exp. Psychol. Gen., 115(2), 107–117.Google Scholar
- Evans, K. K., Georgian-Smith, D., Tambouret, R., Birdwell, R. L., & Wolfe, J. M. (2013). The gist of the abnormal: Above-chance medical decision making in the blink of an eye. Psychonomic Bulletin & Review, 20, 1170–1175. Google Scholar
- Evans, K. K., Haygood, T. M., Cooper, J., Culpan, A.-M., & Wolfe, J. M. (2016). A half-second glimpse often lets radiologists identify breast cancer cases even when viewing the mammogram of the opposite breast. Proceedings of the National Academy of Sciences USA, 113(37), 201606187. https://doi.org/10.1073/pnas.1606187113.View ArticleGoogle Scholar
- Grill-Spector, K., & Kanwisher, N. (2005). Visual recognition: as soon as you know it is there, you know what it is. Psychological Science, 16, 152–160.View ArticlePubMedGoogle Scholar
- Harley, E. M., Pope, W. B., Villablanca, J. P., Mumford, J., Suh, R., Mazziotta, J. C., … Engel, S. A. (2009). Engagement of fusiform cortex and disengagement of lateral occipital cortex in the acquisition of radiological expertise. Cerebral Cortex, 19(11), 2746–2754.View ArticlePubMedGoogle Scholar
- Krupinski, E. A. (1996). Visual scanning patterns of radiologists searching mammograms. Academic Radiology, 3(2), 137–144.View ArticlePubMedGoogle Scholar
- Kundel, H. L., & Nodine, C. F. (1975). Interpreting chest radiographs without visual search 1. Radiology, 116(3), 527–532.View ArticlePubMedGoogle Scholar
- Kundel, H. L., Nodine, C. F., Conant, E. F., & Weinstein, S. P. (2007). Holistic component of image perception in mammogram interpretation: Gaze-tracking study. Radiology, 242(2), 396–402. https://doi.org/10.1148/radiol.2422051997.View ArticlePubMedGoogle Scholar
- Manning, D., Ethell, S., Donovan, T., & Crawford, T. (2006). How do radiologists do it? The influence of experience and training on searching for chest nodules. Radiography, 12(2), 134–142.View ArticleGoogle Scholar
- McKone, E., & Yovel, G. (2009). Why does picture-plane inversion sometimes dissociate perception of features and spacing in faces, and sometimes not? Toward a new theory of holistic processing. Psychonomic Bulletin & Review, 16(5), 778–797 Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=19815781.View ArticleGoogle Scholar
- Nodine, C. F., Kundel, H. L., Mello-Thoms, C., et al. (1999). How experience and training influence mammography expertise. Academic Radiology, 6, 575–585.View ArticlePubMedGoogle Scholar
- Potter, M.C., Wyble, B., Hagmann, C.E., McCourt, E.S. (2013). Detecting meaning in RSVP at 13 ms per picture. Attention, Perception, & Psychophysics 76: 270–279.Google Scholar
- Rossion, B. (2008). Picture-plane inversion leads to qualitative changes of face perception. Acta Psychologica, 128(2), 274–289. https://doi.org/10.1016/j.actpsy.2008.02.003.View ArticlePubMedGoogle Scholar
- Rossion, B. Boremanse, A. (2008). Nonlinear relationship between holistic processing of individual faces and picture-plane rotation: Evidence from the face composite illusion. Journal of Vision, 8(4):3, 1–13Google Scholar
- Rossion, B., & Curran, T. (2010). Visual expertise with pictures of cars correlates with RT magnitude of the car inversion effect. Perception, 39, 173–183.View ArticlePubMedGoogle Scholar
- Rossion, B., Gauthier, I., Goffaux, V., Tarr, M.J., Crommelinck, M. (2016). Expertise training with novel objects leads to left-lateralized facelike electrophysiological responses. Psychological Science, 13(3):250-257.Google Scholar
- Schyns, P. G., & Oliva, A. (1994). From blobs to boundary edges: Evidence for time- and spatial-scale dependent scene recognition. Psychological Science, 5, 195–200.View ArticleGoogle Scholar
- Tanaka, J. W., & Farah, M. J. (1993). Parts and wholes in face recognition. Quarterly Journal of Experimental Psychology, 46A(2), 225–245.View ArticleGoogle Scholar
- Tanaka, J. W., & Sengco, J. A. (1997). Features and their configuration in face recognition. Memory & Cognition, 25, 583–592.View ArticleGoogle Scholar
- Tanaka, J. W., & Simonyi, D. (2016). The “parts and wholes” of face recognition: a review of the literature. Quarterly Journal of Experimental Psychology, 218(August), 1–37. https://doi.org/10.1080/17470218.2016.1146780.Google Scholar
- Tanaka, J. W. & Taylor, M. (1991). Object categories and expertise: is the basic level in the eye of the beholder? Cog. Psychol. 23, 457–482.Google Scholar
- Tottenham, N., Tanaka, J. W., Leon, A. C., McCarry, T., Nurse, M., Hare, T. A., … Nelson, C. (2009). The NimStim set of facial expressions: Judgments from untrained research participants. Psychiatry Research, 168(3), 242–249.View ArticlePubMedPubMed CentralGoogle Scholar
- Xu, B., Rourke, L., Robinson, J. K., & Tanaka, J. W. (2016). Melanoma detection in photographs with the perceptual expertise training approach. Applied Cognitive Psychology, 30, 750–756.View ArticleGoogle Scholar
- Yin, R. K. (1969). Looking at upside-down faces. Journal of Experimental Psychology, 81(1), 141–145.View ArticleGoogle Scholar
- Young, A. W., Hellawell, D., & Hay, D. C. (1987). Configural information in face perception. Perception, 10, 747–759.View ArticleGoogle Scholar