Hands-on experience can lead to systematic mistakes: A study on adults’ understanding of sinking objects

Science learning and hands-on experiences

Many studies lament the challenge of science learning, the claim being that students’ naïve responses to phenomena conflict with conceptions established by the scientific community (for reviews, see Murphy & Alexander, 2008; Pfundt & Duit, 1993). While the exact nature of students’ knowledge is still debated (cf., Smith, diSessa, & Roschelle, 1993), the challenges that come with science learning are indisputable. They are pervasively documented in all aspects of science, including physics (e.g., Edens & Potter, 2003; Lee & Law, 2001; Mazens & Lautrey, 2003; Park & Han, 2002; Pozo & Gomez Crespo, 2005), chemistry (e.g., Chiu, Chou, & Liu, 2002; Harrison & Treagust, 2001; Boo & Watson, 2001), biology (e.g., Mikkilä-Erdmann, 2001; Windschitl, 2001), and astronomy (e.g., Diakidoy & Kendeou, 2001; Vosniadou & Brewer, 1992). It is therefore urgent to develop more effective science pedagogy, compared to typical instruction (e.g., Ohlsson, 1999, 2000).

In recent years, the adoption of diverse and integrated approaches to STEM education have called for inclusion of a more “hands-on” approach to teaching. The idea is to go beyond conveying material in pictorial two-dimensional format and endorse a “head, heart, and hands” pedagogy. It emphasizes not only the need for students’ minds, but also for their emotions (hearts) and their haptic experiences (hands) to be part of learning (e.g., Carlson & Sullivan, 1999; Ferguson & Hegarty, 1995; Sipos, Battisti, & Grimm, 2008). In line with these suggestions, there is indeed evidence that hands-on activities help with learning (e.g., Kontra, Goldin-Meadow, & Beilock, 2012; Kontra, Lyons, Fischer, & Beilock, 2015). For example, hands-on experiences in middle-school students’ science classes predict better science performance on a standardized test of achievement (Stohr-Hunt, 1996).

The call for hands-on activities has been further fueled by theoretical and empirical advances in the area of embodied cognition (e.g., Chemero, 2011; Gibbs, 2005; Wilson, 2002; Wilson & Clark, 2009). Proponents of the embodied-cognition theory claim that higher-level cognition is influenced by our bodily experience (e.g., Barsalou, 2008; Louwerse, 2007; 2008; Smith, 2005). And there is extensive empirical evidence to support this claim (for reviews, see Gibbs, 2005; Iverson & Goldin-Meadow, 2005; Spivey, 2008). It has lent credence to the pedagogical procedure of allowing the learner to actively experience real-life objects (e.g., Bilgin, 2006; Case & Fraser, 1999; Kahle & Damnjanovic, 1994).

At the same time, despite the optimism about hands-on learning, an unconditional endorsement of hands-on experiences is not confirmed unequivocally. For example, a study comparing learning in a fluid-mechanics course through video versus hands-on implementation found that students who watched videos performed just as well on assessments, or even better than the students who had hands-on experience (Abdel-Salam, Kauffman, & Crossman, 2006). Indeed, there has long been a debate over the efficacy of active hands-on activities versus static schematics in teaching science (e.g., Ma & Nickerson, 2006; McNeil & Jarvin, 2007; McNeil, Uttal, Jarvin, & Sternberg, 2009). In a commentary on the usefulness of concrete materials for learning, Brown, McNeil, and Glenberg (2009) caution against making the general assumption that concrete experience always leads to better learning. Rather, hands-on experience might sometimes relate to better learning, while at other times it may be unrelated to learning (Kirschner, Sweller, & Clark, 2006).

Understanding the physics of sinking objects

To better understand the role of hands-on experiences, we focused specifically on the physics domain of sinking objects. Inhelder and Piaget (1958) were among the first to look systematically at the development of people’s understanding of sinking and floating. They presented children with a series of everyday objects (e.g., utensils, tools, toys, materials) and asked them to decide whether they would sink or float in water. Since then, the range of tasks employed in this domain has expanded considerably. It includes making predictions about a single object (e.g., Kohn, 1993; Rappolt-Schlichtmann, Tenenbaum, Koepke, & Fischer, 2007; Skoumios, 2009; Unal, 2008), comparing pairs of objects (e.g., Castillo, Kloos, Richardson, & Waltzer, 2015; Kloos & Somerville, 2001; Penner & Klahr, 1996), and providing explicit explanations about predictions (e.g., Hsin & Wu, 2011; Meindertsma, 2014; Smith, Carey, & Wiser, 1985).

Overall, findings are typically taken to imply the presence of mistaken beliefs about sinking objects (e.g., Butts, Hofman, & Anderson, 1993; Chinn & Malhotra, 2002; Hardy, Jonen, Möller, & Stern, 2006; Kang, Scharmann, & Noh, 2004; Kloos & Somerville, 2001; Skoumios, 2009; Unal, 2008). For example, participants in Penner and Klahr’s (1996) study often picked the heavier object as the faster one - even after learning the inaccuracies of this strategy. This pattern of mistaken behavior was further corroborated by verbal responses about what determines the sinking behavior of objects: Predictions about sinking and floating appear to be focused on weight or size exclusively, rather than on mass distribution (e.g., Castillo & Kloos, 2013; Castillo et al., 2015; Smith et al., 1985; but see Kloos, Fisher, & Van Orden, 2010; Kohn, 1993; Rappolt-Schlichtmann et al., 2007). Thus, this domain is ideal to investigate science learning.

Many learning studies on the physics of sinking objects have incorporated hands-on experiences as part of their didactic choices (e.g., Kloos & Somerville, 2001). However, the efficacy of this choice is far from established. In fact, the separate effect of hands-on experiences is often confounded with effects of instruction or curriculum changes (e.g., Unal, 2008; Hardy et al., 2006; see also Klahr, Triona, & Williams, 2007). To our knowledge, the only sinking-objects study that looked at the relative effect of hands-on manipulations was with 5-year-olds and 6-year-olds (Butts et al., 1993). Those findings showed that hands-on manipulations did not lead to learning by themselves. Instead, only the combination of both instruction and hands-on manipulation showed improved learning. The goal of the current study was to expand on these findings and investigate the effects of hands-on experience versus viewing static images in adults.

Overview of the current study

In the current study,¹ adults had to predict which of two objects would sink faster in water. Objects were transparent containers that differed in their size and in the number of weights inside. They were combined into pairs in such a way that neither the number of weights nor the size of the container was predictive of relative sinking rate. Thus, in order to perform correctly, participants had to compare objects on the basis of a variable other than mass or volume. Our question was whether adults’ predictions are affected by the type of stimuli: Do real-life objects yield better or worse predictions than static images of the objects?

Experiment 1 investigated the role of real-life objects on naïve performance - prior to any training. Half of the participants were handed real-life objects that they could explore haptically (real-objects condition). The other participants were shown static images of the objects (static-images condition). In Experiment 2, we applied the same prediction task, but now looking at the performance of participants who had been given training beforehand about sinking objects. Finally, in Experiment 3, we looked at whether effects of hands-on experiences would persist when real objects are removed and replaced with static images.

Methods

Materials and apparatus

The objects were transparent glass containers that differed in size. Round aluminum discs could be placed inside the containers to obtain a desired density (see Appendix A for detailed dimensions). Depending on a container’s size and the number of weights inside, there were 12 unique objects. They were combined into pairs such that neither mass nor volume fully predicted the relative rate of sinking across all pairs. For example, in some pairs, the object that sank faster was the bigger and heavier container; and in other pairs, the object that sank faster was the smaller and lighter one. Figure 1 depicts several pairs to illustrate this point.

Real-life objects served as stimuli in the real-objects condition. For the static-images condition, we generated photographs of each unique pair of containers. A picture was 960 pixels wide and 720 pixels high. One picture showed two empty containers, each with a specific number of aluminum discs next to it. And the second picture showed the same two containers filled with the aluminum discs and closed with lids.

Procedure

Participants were tested individually in the laboratory, using DirectRT Precision Timing Software (2012 Version) to randomize the trials and record participants’ responses. Prior to the experiment proper, participants were introduced to the stimuli. They were first shown three empty containers of different sizes and several aluminum discs. They were then shown an image of two containers with discs inside them. They were told that the image represented a picture taken of the real objects in front of them. Next, participants were introduced to the task of predicting which of the two objects would sink faster when dropped in a tank of water. Participants’ prior knowledge about buoyancy was not assessed. No explanation was given about the underlying physics or how the participants should go about solving the task. The experiment started immediately.

There were 45 unique pairs of objects. Each possible pair was presented twice (with counter-balanced left-right position of objects). This yielded a total of 90 trials. The trials were presented in random order, with the caveat that a full set of 45 unique pairs was presented first, before any pair was repeated with its counterbalanced version.

In the real-objects condition, participants sat across from the researcher and in front of an opaque box that separated them (60 × 25 × 40 cm). The box served as a barrier behind which the researcher kept all 12 containers. Figure 2 provides a schematic overhead view of this arrangement. For each trial, objects were placed in the participant’s hands, and the participant had to choose the object they thought would sink faster. There was no time restriction for making a decision. After the participant made a choice, the experimenter recorded the choice on the computer and removed the containers from the participant’s hands. This ended the trial.

For the static-images condition, participants sat in front of the computer screen to view the images. Participants made their predictions using the keypad that had two marked choices (“left” and “right”). A trial started with the program presenting an image of two empty containers, each next to its respective stack of discs. After 1.5 seconds, the image was replaced with an image of the same two containers filled with the discs. Participants then had to decide which of the two containers would sink faster. There was no time restriction for making a decision. The trial ended when the participant marked a choice on the keypad.

Results and discussion

We first looked at the data in terms of proportion of correct predictions. Across all possible trials, adults performed above chance (mean proportion M _real-objects = .77, M _{static-images} = .82). However, they made characteristic mistakes on trials in which the faster object was small and light (see Panel 5 in Fig. 1 for an example). There were 20 trials of this kind. Figure 3A provides the mean performance on these trials, separated by condition (see Appendix B for the data on all other trial types). Interestingly, performance in the real-objects condition (M = .22) was significantly lower than performance in the static-images condition (M = .39), independent-sample t(51) = 2.01, p < .05; d _Cohen = .55. Thus, hands-on experiences appear to have negatively impacted performance, leading participants to make more systematic mistakes when predicting the sinking rate of real objects. In fact, only performance in the real-objects condition, but not performance in the static-objects condition, was significantly below chance (assuming a chance probability of 0.5), t(25) = 5.27; p < .001, d _Cohen = 1.01.

Overall, we found that individuals made more mistakes when they predicted the relative sinking rate of real objects, compared to when making predictions with static images. There are two possible explanations for this finding: On the one hand, it is possible that hands-on experiences highlight misleading features. Perhaps the holding and hefting of real objects highlighted the feature of heaviness, over and above the more subtle feature of mass distribution. On the other hand, it is possible that real-life objects led to more stable learning - incorrect learning, but nevertheless learning. Maybe adults’ higher average in the static-images condition was not the result of some insight about sinking objects, but the result of simply guessing. Indeed, the average performance of participants in this condition is indistinguishable from chance. And while only 18% of participants in the real-objects condition performed randomly at any point during the experiment, a contrasting 40% of participants did so in the static-images condition (one-tailed χ ² (1) = 3.19, p < .04). Experiment 2 was carried out to disambiguate between the two possible explanations and clarify the effect of the real-objects manipulation.

Methods

Results and discussion

The feedback training was successful. To illustrate, we report average accuracy on participants’ predictions during the second half of the training. Across all trials, accuracy was near ceiling for both conditions (M _real-objects = .91; M _{static-images} = .92). Even when considering only trials for which the lighter and smaller object sank fastest, performance was above chance (M _real-objects = .82; M _{static-images} = .78, ps < .01). There was no difference between conditions during training, whether we considered the full set of trials, F(1,51) = 1.91; p > .17, or only the subset of trials for which the faster object was small and light, t(51) = 1.19, p > .24. An analysis of the 95% confidence interval confirmed the overlap (CI _real-objects = .82 ± .04; CI _{static-images} = 0.78 ± .05). The crucial question, then, was how participants performed after the training, when they were asked to make predictions either with real-life objects or with static images.

Past the training, adults performed close to ceiling across all trials (M _real-objects = .89, M _{static-images} = .91). The exception was their performance on trials in which the faster object in a pair was small and light (see Appendix B for the performance on all other trial types). Figure 3B depicts the mean accuracy on these trials, separated by condition. As we found in Experiment 1, performance was again lower in the real-objects condition (M = .66) than the static-images condition (M = .82), t(51) = 2.95, p < .001. Thus, even though participants demonstrated similarly high performance during training, the effect of real-life objects nevertheless emerged. Following the training, performance dropped significantly for participants in the real-objects condition, repeated-measure t(27) = 2.68, p < .02, while it increased slightly for participants in the static-images condition, repeated-measure t(24) = 2.31, p < .03.

Results from Experiment 2 reaffirm that hands-on experiences might highlight the heaviness of objects and thus lead to mistaken performance. In order to understand the seriousness of these effects, we next examined whether the negative influence of hands-on experiences would persist over time.

Methods

Results and discussion

We focused again only on trials in which the faster object is small and light (see Appendix B for the performance on all other trial types). Recall from Experiment 1 that naïve performance was below chance for both groups, and that participants in the real-objects condition performed worse than participants in the static-images condition. Following the training session, participants’ accuracy improved in both conditions, from .22 to .81 in the real-objects condition, t(27) = 10.34, p < .01, and from .39 to .82 in the static-images condition, t(24) = 6.71, p < .01. This suggests that the training was indeed helpful for overcoming the initial mistakes on trials in which the faster object is small and light. Importantly, the difference between the two conditions disappeared after the training session: During the test phase, performance in the real-objects condition was indistinguishable (M = .81) from performance in the static-images condition (M = .82), t(51) = 0.23, p > .82. Analyses of the 95% confidence intervals confirmed these results (CI _real-objects = .81 ± .05; CI _{static-images} = .82 ± .06). This shows that after an intermediate training session, participants were all able to reach an equally high level of accuracy, regardless of whether they were initially exposed to hands-on experiences.

Overall, we found that while hands-on experiences may initially lead to mistaken patterns of performance when making predictions about sinking objects, these mistakes could be overcome with training. We next turn to a general discussion of the findings from this research.