The Results: Augmented Reality Simulation verses Traditional

Classroom InstructionDr. Jonathan C. Renner, Ed.D.Department of Teacher EducationNeuta Hidatsa Sahnish College

New Town, ND

Full PDF of study

http://pqdtopen.proquest.com/pubnum/3628589.htmlhttp://pqdtopen.proquest.com/pubnum/3628589.html

Nueta Hidatsa Sahnish College

Department of Teacher Education

Purpose

• To determine if learning occurred with augmented reality instruction (using a mobile handheld digital device: iPod)

• To examine if there was a difference in approximately 50 students’ learning outcomes between an augmented reality simulation and traditional classroom instruction in the chemistry topic of Alpha, Beta, and Gamma radiation, as measured by a pre- and posttest methodology without group collaboration.

Theoretical Foundation

• Students use technology in a variety of ways

• Some students use the available technology to acquire knowledge through discovery, collect information from multiple sources, integrate the visual with the spatial (Oblinger & Oblinger, 2005), and seek self-directed learning experiences (Dieterle, Dede, & Schrier, n.d.).

• Thus, the learning style of these students may be in direct conflict with traditional classroom instruction. • These students may prefer a learning format that follows a

constructivist, game-based learning approach as found in an augmented-reality simulation that follows the learning methodology and constructivist principles of Dewey (1910), Piaget (1947), and Vygotsky (Dalgarno, 2002).

• An investigation into the existing literature covering the topic of game-based learning (GBL), computer games have been investigated for students’ learning outcomes (Carr & Bossomaier, 2011; Cheung et al., 2008; Foster, 2011; Kaufman et al., 2011; Kebritchi et al., 2010; Tuzun et al., 2009; Yip & Kwan, 2006).

• Previous studies that investigated augmented reality game-based learning used groups of two to eight participants working collaboratively, but learning was not assessed (Dunleavy, Dede, & Mitchell, 2009; Klopfer & Squire, 2008; Rosenbaum, Klopfer, & Perry, 2007; Schrier, 2006; Squire & Jan, 2007).

• Four studies were found that investigated augmented reality and learning outcomes: Tamsui (Chang, Wang, Lin, & Yang, 2009), Frequency 1550 (Huizenga, Admiraal, Akkerman, & ten Dam, 2009), Beetle Breeders, Beasties, Island Hoppers, and Chomp (Rosenheck & Perry, 2012), and Human Digestive System (Vilkoniene, 2009). • Of these studies, Chang, Wang, Lin, & Yang (2009), Huizenga,

Admiraal, Akkerman, & ten Dam (2009), and Vilkoniene (2009) demonstrated positive learning outcomes. Yet, none of these four studies controlled for collaboration or ensured that the individual outcomes were not the result of group collaboration.

• From this review, a gap was indentified in the existing literature that centered on the lack of evidence of improved individual learning outcomes, without group collaboration, from augmented reality games that used handheld digital devices.

Methodology

• Quasi-experimental, pre -and posttest, quantitative research design that compared a group of traditional classroom students without group collaboration with a group who used an augmented-reality simulation without group collaboration.

• A quantitative methodology was chosen for this study for its ability to compare two groups statistically and quantify results for significance.

Why Quasi-experimental?

• Because participants were not randomly assigned to groups, but rather purposely assigned to groups (Shadish, Cook, & Campbell, 2002) by the high school’s chemistry teacher based on the participant’s cumulative science GPA (Panoutsopoulos & Sampson, 2012). Assigning groups based on the science GPA, instead of randomly, had the goal of forming two groups with participants of similar ability, so that a similar baseline of ability was established between the groups.

• Pre- and posttest design in which measurements are taken immediately before and after an activity, and data analyses determines if one group’s posttest scores differed more significantly from the pretest scores than the other group’s scores (Weinfurt, 2000).

• Quantitative methodology was developed to numerically compare the results of a comparison group (traditional classroom instruction) with an experimental group (augmented-reality simulation). Both groups worked independently and without group collaboration as directly observed by the researcher.

Research Questions & Hypothesis

• RQ1: Does augmented reality affect high school students’ learning outcomes in chemistry, as measured by a pre- and posttest methodology when ensuring that the individual outcomes are not the result of group collaboration? • H1

0: Augmented reality has no impact on students’ learning outcomes.

• H11: Augmented reality has a positive impact on

students’ learning outcomes. • H1

2: Augmented reality has a negative impact on students’ learning outcomes.

• RQ2: Does augmented reality instruction or traditional classroom instruction have a greater positive impact on high school students’ learning outcomes in chemistry, as measured by a pre- and posttest methodology when ensuring that the individual outcomes are not the result of group collaboration?• H2

0: Augmented reality instruction and traditional classroom instruction have the same impact on students’ learning outcomes.

• H21: Augmented reality instruction has a greater positive impact

on students’ learning outcomes than traditional classroom instruction.

• H22: Traditional classroom instruction has a greater positive

impact on students’ learning outcomes than augmented reality instruction.

Sample

• Sample size of approximately 50 participants was determined by consulting formulae and a published table (Cohen, 1992). Eighty-nine prospective participants returned signed and completed consent forms. 45 were placed into the traditional classroom instruction group (Average science GPA of group was 3.18), and 44 were placed into the augmented reality simulation group (average science GPA of group was 3.13). On the day of the study due to student absence, participants not completing tests or withdrawing from the study, 42 of the traditional instruction group and 36 of the augmented reality group completed both pretest and posttest.

Validity

• Pre- and posttests have been used previously as chemistry textbook questions, they have had success at measuring learning, and have results that are generalizable.

• Threat to external validity may arise as an effect of testing. Since participants took a pretest, the pretest may increase or decrease the participant’s attention or interaction during the treatment (traditional classroom instruction or augmented reality simulation) (Campbell & Stanley, 1963). To minimize this, the researcher only used a multiple-choice pre- and posttest and then drew generalizable conclusions from the results.

• Face Validity• The pre- and posttest were claimed by the

textbook authors as subject matter experts (Glencoe Science White Paper, 2013; Hunt, 2013), were based upon National Science Education Standards (National Research Council, 1996), and research-based instructional strategies that were used to explain and justify the standards supporting curricula, and the assessment of student understanding through a variety (section and chapter assessments, standardized test practice, and supplemental materials such as ExamView Pro Testmaker CD-ROM).

• Criterion Validity• Study compared two levels of the independent variable

(traditional instruction and augmented reality simulation).

• Two possible treats to internal validity were identified. • First, a possible threat to internal validity was that after

taking the pretest, participants may have become familiar with the outcome measure and remembered responses for the posttest (Shadish, Cook, & Campbell, 2002). To prevent this from happening, the researcher scrambled the research questions and possible answers on the posttest.

• A second threat to internal validity may arise from the selection of subjects (Campbell & Stanley, 1963). This study did not use randomly assigned subjects, but rather subjects were divided based on GPA, then randomly assigned to a group.

Reliability

• Reliability of the data collected was based largely on the test instruments used in this study. The pre- and posttest were comprised of questions that have been used previously as chemistry textbook questions to gain understanding in radioactivity. The chemistry textbook questions have also been correlated to national science standards (Glencoe Science White Paper, 2013; Hunt, 2013). Because these textbook questions have been previously used to yield the designed results, they have been tested for stability reliability and internal consistency (Vogt & Burke-Johnson, 2011).

• Cronbach’s alpha (Cronbach, 1951; Pallant, 2010) and found .826 in the pretest and .835 for the posttest.

Procedure/Data Collection

• On the day of the study, two neighboring classrooms were used that had an adjoining door, allowing the researcher to observe all participants in both groups. One classroom was set-up for traditional classroom instruction and the other for the augmented reality simulation. The researcher called participants to one or the other room. Upon arrival at either site, participants were reminded that the study was voluntary, and were given a written child assent form to sign and verify their voluntary participation. After this form was signed and collected by the researcher, the 20 minute pretest was administered, 30 minute activity, and 20 minute posttest. The pretest and posttest were identical except the questions were in a different order. The traditional classroom group received teacher-led instruction of lecture and demonstrations. The augmented reality group participants were given an iPod. After a brief demonstration on the use of the iPod’s camera feature with the markers (QR codes), they conducted their self-led, 30 minute activity. In doing so, the participants would move around the classroom, and take a picture of the marker; the captured image of the marker would be sent wirelessly to the ARIS server; the ARIS server would receive the image, and compare it to other images in its database. When a match was found, it would send the event (picture, text, video, or other digital information) back to the participant’s iPod. The researcher collected all pre- posttests, and conducted all coding.

Data Analysis

• First a one-tailed paired t test looked at the difference between the pretest and posttest scores of each group. The null hypothesis of this test, stating that the scores were the same, was rejected, indicating that the posttest scores were higher than pretest scores for both groups. This indicated that further investigation was warranted.

• The paired t test performed for RQ1 showed that augmented reality instruction did lead to an improvement in test scores from the pretest to the posttest.

• A subsequent comparison of the mean scores of the pretest and posttest for each group indicated that a larger increase in mean scores occurred in the group that received traditional classroom instruction, as compared to the group who received the augmented reality simulation (See next slide data tables).

Variable

N Minimum Maximum Mean SD

Pretest 42 30.00 86.67 59.05 16.07Posttes

t42 43.33 96.67 76.03 14.24

Descriptive Statistics – Test Scores – Traditional Classroom Instruction

Variable

N Minimum Maximum Mean SD

Pretest 36 33.33 96.67 61.76 18.16Posttes

t36 40.00 96.67 69.17 16.55

Descriptive Statistics – Test Scores – Augmented Reality Instruction

• In order to test RQ2, an independent sample t test was used to determine whether the difference in scores between the groups was significant.

• For this test, after confirming the assumption of equal variances, the null hypothesis was rejected, indicating that the differences between pre- and posttest scores for the two groups were significantly higher for the traditional instruction group, thus upholding H2

2 of RQ2.

Results & Summary

• The combined results of these statistical analyses indicated that augmented reality simulation did increase posttest scores in the group of high school chemistry students. However, these results also indicated the increase in scores was less for the augmented reality group than it was for the group that received traditional classroom instruction.

• This study attempted to control variables by creating equal ability groups based on students’ science GPA; but in the end, the unpredictability of participant absence, withdrawing from the study, or not completing a test created a slight unevenness between the groups (groups may not have been equal by ability after all).

Educational/Scientific Importance

• This study contributed to the existing knowledge about game-based learning, and specifically augmented reality game-based learning, because previous studies were not conclusive as to whether augmented reality could improve individual students’ learning outcomes without group collaboration.

• Therefore, the present study would become the first quantitative experiment controlling for individual learning to validate augmented reality using mobile handheld digital devices that affect individual students’ learning outcomes without group collaboration.

• This topic is important to the field of education as it may help educators understand how students of the digital age learn and it may change the way students are taught. • Currently, teaching is mainly an educator-led, guided, two-dimensional or

pen-and-paper process. • This study demonstrated that learning can also be a self-directed,

constructivist approach through an augmented reality simulation.

References

• Academic ADL Co-Lab. (2007). Mad city mystery [augmented reality game]. Retrieved October 2, 2012, from http://www.academiccolab.org/randd

• Berkowitz, J. (n.d.). Sampling and sample size. Retrieved from http://www.columbia.edu/~mvp19/RMC/M6/M6.doc

• Carr, D., & Bossomaier, T. (2011). Relativity in a rock field: A study of physics learning with a computer game. Australasian Journal of Educational Technology, 27, 1042–1067. Retrieved from http://www.ascilite.org.au/ajet/ajet27/carr.pdf

• Chang, W.-C., Wang, T.-H., Lin, F. H., & Yang, H.-C. (2009). Game-based learning with ubiquitous technologies. IEEE Internet Computing, 13(4), 26–33.

• Chemistry: Matter and Change. (2005). ExamView Pro: Testmaker CD-ROM [software]. Chicago, IL: McGraw-Hill.

• Dieterle, E., Dede, C., & Schrier, K. (n.d.). Neomillennial learning styles propagated by wireless handheld devices. In M. Lytras & A. Naeve (Eds.), Ubiquitous and pervasive knowledge and learning management: Semantics, social networking and new media to their full potential. Hershey, PA: Idea Group.

• Dunleavy, M., Dede, C., & Mitchell, R. (2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of Science Education and Technology, 18, 7–22. doi:10.1007/s10956-008-9119-1

• Foster, A. N. (2011). The process of learning in a simulation strategy game: Disciplinary knowledge construction. Journal of Educational Computing Research, 45, 1–27. doi:10.2190/EC.45.1.a

• Grand Canyon University. (n.d.). IRB forms. Retrieved September 25, 2012, from http://dc.gcu.edu/documents/ethicsinstitutionalreview/institutionalreviewboardirb/irbforms~2/informedconsentformsocialbehaviorminriskdocx

References

• Harvard University. (2007). The River City project [multiuser virtual environment]. Retrieved October 1, 2012, from http://muve.gse.harvard.edu/rivercityproject/contributors/contributors.html

• Hoffmann, L. (2009). Learning through games. Communications of the ACM, 52(8), 21–22. doi:10.1145/1536616.1536624

• Hussain, Z., & Griffiths, M. D. (2009). Excessive use of massively multi-player online role-playing games: A pilot study. International Journal of Mental Health and Addiction, 7, 563–571. doi:10.1007/s11469-009-9202-8

• Huizenga, J. J., Admiraal, W. W., Akkerman, S. S., & ten Dam, G. (2009). Mobile game-based learning in secondary education: Engagement, motivation and learning in a mobile city game. Journal of Computer Assisted Learning, 25, 332–344. doi:10.1111/j.1365-2729.2009.00316.x

• Jenkins, H., & Henrichs, R. (2003). Supercharged [computer game]. Retrieved October 1, 2012, from http://www.educationarcade.org/gtt/proto.html

• Jha, N. K. 92008). Research Methodology. Chandigarh, India: Global Media.• Jonassen, D. H., Carr, C., & Yueh, H.-P. (1998). Computers as mindtools for engaging learners

in critical thinking. TechTrends, 43(2), 24–32. doi:10.1007/BF02818172• Kaufman, D., Sauve, L., & Renaud, L. (2011). Enhancing learning through an online secondary

school educational game. Journal of Educational Computing Research, 44, 409–428. doi:10.2190/EC.44.4.c• Kebritchi, M., Hirumi, A., & Bai, H. (2010). The effects of modern mathematics computer games

on mathematics achievement and class motivation. Computers & Education, 55, 427–443. doi: 10.1016/j.compedu.2010.02.007.

• Kevin, K. F. C., Morris, S. Y. J., F, L. L., Jimmy, H. M. L., Eric, T. H. L., Shang, J., & Marti, K. H. W. (2008). Farmtasia: An online game-based learning environment based on the VISOLE pedagogy. Virtual Reality, 12(1), 17-25. doi:http://dx.doi.org/10.1007/s10055-008-0084-z

References

•Klopfer, E., & Squire, K. (2008). Environmental detectives—The development of an augmented reality platform for environmental simulations. Educational Technology Research & Development, 56, 203–228. doi:10.1007/s11423-007-9037-6

•Lipsey, M. W. (1990). Design sensitivity: Statistical power for experimental research. Newbury Park, CA: Sage.•Lombardi, M. M., & Oblinger, D. G. (2007). Authentic learning for the 21st century: An overview (Educause Learning Initiative Paper 1). Retrieved April 28, 2012, from http://net.educause.edu/ir/library/pdf/ELI3009.pdf

•Oblinger, D. G., & Oblinger, J. L. (2005). Is it age or IT: First steps toward understanding the net generation. In D. G. Oblinger & J. L. Oblinger (Eds.), Educating the net generation (pp. 2.1–2.20). Washington, DC: EDUCAUSE.

•Panoutsopoulos, H., & Sampson, D. G. (2012). A study on exploiting commercial digital games into school context. Journal of Educational Technology & Society, 15(1), 15–27.

•Rice University. (2012). Nothing to rave about [computer game]. Retrieved October 1, 2012, from http://webadventures.rice.edu/ed/Teacher-Resources/_games/Reconstructors/_701/Game-Overview.html

•Rosenbaum, E., Klopfer, E., & Perry, J. (2007). On location learning: Authentic applied science with networked augmented realities. Journal of Science Education and Technology, 16, 31–45. doi:10.1007/s10956-006-9036-0

•Rosenheck, L., & Perry, J. (2012). Ubiqbio: A playful approach to learning biology with mobile games. Retrieved from http://www.isteconference.org/2012/uploads/ KEY_70132706/PerryRosenheckUbiqBio_RP.pdf

•Schrier, K. L. (2005). Revolutionizing history education: Using augmented reality games to teach history (Master’s thesis). Massachusetts Institute of Technology. Retrieved January 25, 2013, from http://erax1201.eliterax.com/research/theses/KarenSchrier2005.pdf

•Schrier, K. L. (2006). Using augmented reality games to teach 21st century skills. Retrieved from http://nguyendangbinh.org/Proceedings/Siggraph/2006/cd1/content/educators/schrier.pdf

•Soper, D. (2012). Statistics calculators: A-priori sample size calculator for students t-tests. Retrieved September 25, 2012, from http://www.danielsoper.com/statcalc3/calc.aspx?id=47

References

• Squire, K. (2007b). Video-game literacy: A literacy of expertise. In D. Leu, J. Coiro, M. Knobel, & C. Lankshear (Eds.), A handbook of new literacies research (pp. 639–667). Mahwah, NJ: Erlbaum. Retrieved January 13, 2013 from http://website.education.wisc.edu/kdsquire/tenure-files/04-video-game%20literacy.pdf

• Squire, K., & Jan, M. (2007). Mad city mystery: developing scientific argumentation skills with a place-based augmented reality game on handheld computers. Journal of Science Education and Technology, 16, 5–29. doi:10.1007/s10956-006-9037-z

• Tinzmann, M. B., Jones, B. F., Fennimore, T. F., Bakker, J., Fine, C., & Pierce, J. (1990). What is the collaborative classroom? Retrieved March 18, 2012, from http://www.ncrel.org/sdrs/areas/rpl_esys/collab.htm

• Tuzun, H., Yilmaz-Soylu, M., Karakus, T., Inal, Y., & Kizilkaya, G. (2009). The effects of computer games on primary school students’ achievement and motivation in geography learning. Computers & Education, 32, 68–77.

• Vilkoniene, M. (2009). Influence of augmented reality technology upon pupils’ knowledge about human digestive system: The results of the experiment. US–China Education Review. 6(1), 36–43.

• Wetcher-Hendricks, D. (2011). Analyzing Quantitative Data: An Introduction for Social Research. New York: Wiley & Sons.

• Yip, F. M., & Kwan, A. M. (2006). Online vocabulary games as a tool for teaching and learning English vocabulary. Educational Media International, 43, 233–249. doi:10.1080/09523980600641445