Participants
In a diverse Midwestern elementary school (67.7% white) in the USA, 44 fifth grade students (10-year olds) participated as intact classes across 2 consecutive years with the same teacher. There were 26 students (56% boys) in the Embedded Phenomena condition and 18 students (50% boys) in the non-embedded condition. No differences were seen between boys (M = 5.09, SD = 2.22) and girls in spatial skills (M = 4.65, SD = 2.08, t <1), which replicates previous findings for this age group (Voyer, Voyer, & Bryden, 1995). Further, the embedded and non-embedded classes were matched on spatial skills as well as on earth science concepts and earthquake measurement at pre-test, all ts <1.10.
Manipulation and procedure
The two classes were followed over a period of 6 weeks as they participated in an earth science unit on the understanding of concepts about the earth’s layers and composition, the existence of tectonic plates, convection currents, plate boundaries, and the geological features that relate to the interactions of tectonic plates. Both classes received the same lessons on these topics during their regularly scheduled science classes. A breakdown of the lessons and differences across the two classes can be found in the Appendix. In addition, both classes completed a series of 15 simulated earthquake event activities in which they were required to compute and locate earthquake epicenters from seismograph data, and record this data in multiple representational formats. These activities were designed to support the acquisition of knowledge about earthquake measurement and skills in authentic seismological practice, including the determination of event distance and magnitude, and the use of trilateration to determine event epicenters. The representation of the data in multiple formats in persistent public classroom displays was intended to support the development of an understanding of the distributional characteristics of earthquakes across the dimensions of space, intensity, and time (see Fig. 2).
The main manipulation between the two classes was in how the students engaged in the earthquake activities. Students in the Embedded Phenomena condition experienced the simulated earthquakes (“RoomQuakes”) as being located within the physical space of the classroom. These RoomQuakes occurred at random times throughout the school day over a period of 4 weeks (no RoomQuakes occurred during the first and last weeks of the unit). Specifically, five of the RoomQuakes occurred during the regularly scheduled science class, seven occurred during other subjects throughout the school day, and three occurred out of school hours. The RoomQuakes were created by placing four 24-inch iMac computers and speakers around the classroom (see Fig. 3). Each computer served as a seismograph reading station, which depicted a continuously running strip chart recorder of ground vibration. When a RoomQuake occurred, the seismographs traced out characteristic waveforms (seismograms) that corresponded to the expected vibration at their specific locations due to an event at a particular location in the classroom, and a rumbling sound was generated by the speakers in the iMacs. In the time between events, the seismographs displayed random visual noise and no sound. When the rumbling began, students ran to collect data from each seismograph.
In the embedded condition, when a RoomQuake occurred during a non-science class the students would get up from their seats and go to their stations to begin the process of measuring the waves and using trilateration to locate the epicenter just as they would have if it occurred during their science class. After completing the task, they would go back to their regularly scheduled subject. To track the occurrence of RoomQuakes outside school hours, students would check the activity of the seismographs first thing each morning. If a RoomQuake had occurred there would be a recorded seismogram available at each station that included the waveform that was created, the date, and time of the event so students could measure and record the event. Having RoomQuakes occur during non-science classes and on evenings and weekends was intended to reinforce the idea that earthquakes are not predictable and can happen at any time.
Students in the non-embedded condition also worked with the same earthquake data, but in their case it was presented as a series of 15 historical earthquakes that had occurred in 1999 in Southern California. For this condition, the iMac computers were placed in a row with each representing a different reading station that was suggested to be located in Southern California and identified on a map. Students were able to go forward and backward through an archive of “snapshots” of the seismograms taken at each location during each earthquake event. The 15 earthquake activities were completed as part of a single lesson spanning three science class sessions. Although the earthquakes in this class were presented as historical data from California, the data matched that of the embedded class. Specifically, both classes had the same patterns of magnitude (more small earthquakes than large), time (large earthquakes tended to be followed closely in time by several smaller ones), and location (occurred along a fault line).
The actual process for determining epicenters (trilateration) also differed between conditions. The embedded class used calibrated dry-lines that were anchored at each of the seismograph reading stations. To determine the distance each reading station was from the event epicenter, students had to measure the length of the primary wave. The length of this wave was then used to calculate the length of the dry-line. One quarter of an inch (6.35 mm) was equal to 1 pre-determined unit of dry-line (approximately 1 foot; 304.8 mm) so, if the wave measured 2 inches (50.8 mm) the students needed to measure out 8 units of dry-line. Once each group determined the appropriate length of dry-line for the event, the group would sweep out circles with their bodies, which reflected the possible loci of solutions from the individual stations until they found the place where all four dry-lines (and bodies) coincided. The place where students converged represented the epicenter of the event. The non-embedded class used calibrated dry-lines that were pinned to a large map of Southern California (see Fig. 1). Again, one quarter of an inch (6.35 mm) of length for the primary wave was equal to 1 pre-measured unit of dry-line; however, in this condition 1 unit of dry-line was approximately 1 inch (25.4 mm) in length. Despite the relative difference in scale across the two conditions (whole classroom trilateration versus wall map trilateration), the entire process of measuring, locating, and recording an earthquake event took approximately 12 to 15 minutes in both conditions. This suggests that the difference in scale did not have an impact on the difficultly of completing the trilateration activities.
The process for determining event magnitude was the same across both conditions. In order to determine event magnitude, students had to measure the amplitude (or height) of the secondary wave. Both the height of the secondary wave (amplitude) and the length of the primary wave (distance) were required to determine the magnitude with a nomogram (see Fig. 4). Students marked the distance on the left of the nomogram and the amplitude on the right of the nomogram, then using a straightedge would connect those two marks. The point at which the line crossed the center on the nomogram would give students the magnitude of the event.
Students in both conditions were responsible for recording event data in multiple formats. They entered it in their individual workbooks and on public displays in the classroom, which included the magnitude of each event, the location, and a timeline for when each event occurred. In the Embedded Phenomena class, the location of each event was marked by hanging a polystyrene ball from the ceiling where the class determined the epicenter of that event to be. Each ball was color-coded to reflect the magnitude of that event. Students in the embedded class also recorded this data on a classroom map in their workbooks. In the non-embedded class, students recorded epicenter locations on a large map of Southern California and used color-coded stickers to reflect the magnitude. They also recorded this data on a small map of Southern California in their workbooks. Both classes completed the same “big ideas” worksheets and engaged in whole-class discussions about the earthquake simulation activities; the discussions were intended to help them reflect on the lessons, map across representations, and connect data collection activities to conceptual understanding. For an overview showing the differences between the two conditions see the Appendix.
Because the teacher, and not the research team, carried out the activities it was important that the teacher be consistent across both conditions. Therefore, two intact classes across 2 consecutive years served as the sample for this study, and the same teacher implemented both conditions. The teacher had extensive experience with the unit and had been involved in running pilot versions of both conditions in their classroom for several years prior to this study. The teacher also worked collaboratively with the research team, including a seismologist, to develop the learning goals, activities, lessons, and assessments for the unit. This collaboration was essential because it ensured that the lessons were scientifically accurate according to the seismologist, but also age and curriculum appropriate according to the teacher.
Measures
There were three primary learning goals for this unit. The first learning goal was related to developing an understanding of basic earth science concepts. This included understanding concepts about the earth’s layers and composition, the existence of tectonic plates, convection currents, plate boundaries, and the geological features that relate to the interactions of tectonic plates. The second learning goal was related to understanding the methods used in observing and measuring earthquakes as well as characteristics of earthquake data. For example, students were expected to develop an understanding of the distributional characteristics of earthquakes across the dimensions of space, intensity, and time and understand how information from wave information is used to determine these characteristics. The third learning goal was related to developing authentic skills in seismological practice. This included demonstrating skill in being able to compute and locate earthquake epicenters from seismograph data, determine event distance and magnitude, and use trilateration to determine event epicenters.
In order to examine the effectiveness of different versions of this unit for achieving the aforementioned learning goals, several sources of data were collected. A 20-item multiple-choice test was developed and given to both classes prior to beginning the unit and after the unit was completed. Ten of the items were related to understanding of the target earth science concepts and ten were related to understanding the methods used in observing and measuring earthquakes. The assessments were developed in a collaborative effort by the classroom teacher, based on his standard curriculum, an expert seismologist, and the rest of the research team. The pre-test and post-test items were designed using standard items from the content of the existing curriculum, including items found on standardized assessments from the National Assessment of Educational Progress (NAEP) and Illinois Standards Achievement Test (ISAT).
In addition, students’ seismological skills and their ability to work with multiple representations of data were assessed through hands-on skills assessments conducted in individual sessions with an experimenter. This post-unit assessment tested a student’s ability to locate the arrival of the primary and secondary waves, determine the distance between the arrivals of those waves, the amplitude of the earthquake, and the magnitude of the earthquake. In addition, students were asked to show the loci of potential epicenters using two different methods. First they had to find the epicenter by means of the method used in their class (either with strings in the classroom or strings on a map) and then they were asked to find an epicenter on a transfer task using three compasses on a piece of graph paper. There was a total of seven possible points. Both multiple-choice and skills assessments were piloted and revised to ensure their alignment with and coverage of the learning goals of the unit, and their appropriateness for the ability level of the students.
Prior to the beginning of the unit, students completed a 10-item paper-folding test (Ekstrom, French, Harman, & Derman, 1976). In this test, participants select which one of five possible patterns of holes will result after a piece of paper is folded and a hole is punched through it. Participants’ scores were computed as the number of correct responses. This test was chosen because it has commonly been used as a measure of spatial visualization skill, representing one’s ability to mentally transform or manipulate objects (Carroll, 1993). More specifically, it requires the manipulation of an internal representation as well as the transformation of three-dimensional elements. These task demands align with the demands of creating a mental representation of a fault line from continuously updating earthquake data; both incorporate three-dimensional information and both incorporate updating a mental representation through multiple transformation stages. Further, the paper-folding test has been demonstrated to be a strong predictor of performance or aptitude in STEM areas (Höffler & Leutner, 2007; Hsi, Linn, & Bell, 1997; Lord, 1987; Mayer & Sims; 1994; Siemonhowski & MacKnight, 1971) and, more specifically, in geosciences learning (Black, 2005; Jaeger et al., 2016; Sanchez, 2012; Sanchez & Wiley, 2010).