Virtual and augmented reality: Implications of game-changing ...
Investigating Student Flow Experience During a Mobile Augmented Reality Science Game
Transcript of Investigating Student Flow Experience During a Mobile Augmented Reality Science Game
Paper Presented at NARST 2013 1
Investigating Student Flow Experience
During a Mobile Augmented Reality Science Game
Denise M. Bressler, Lehigh University, Bethlehem, PA 18015 USA
Alec M. Bodzin, Lehigh University, Bethlehem, PA 18015 USA
2013 NARST Annual Conference
April 6 – 9, 2013
Rio Grande, Puerto Rico
Email: [email protected]
Abstract
Current studies have reported that secondary students are highly engaged while playing mobile
augmented reality learning games. This study investigated factors related to students’
engagement—as characterized by flow theory—during a collaborative augmented reality (AR),
forensic science mystery game using mobile devices. School Scene Investigators: The Case of
the Stolen Score Sheets is a vision-based AR game played inside the school environment with
quick response codes. A mixed methods approach was employed with 68 urban middle school
students. Data sources included pre- and post-surveys, field observations, and group interviews.
Results showed that neither gender nor interest in science were important predictors of
variability in flow experience. Gaming attitude uniquely predicted 23% of the variance in flow
experience. Student flow experience features included a flash of intensity, a sense of discovery,
and the desire for higher performance. The findings demonstrated a potential for mobile AR
science games to increase science interest and help students collaborate on learning tasks.
Implications for future research concerning mobile AR games for science learning are discussed.
Keywords: flow, engagement, game-based learning, quick response codes, augmented
reality, mobile technology
Paper Presented at NARST 2013 2
Introduction
In 2011, the National Research Council published the report Learning Science through
Computer Games and Simulations and highlighted ―games as worthy of future investment and
investigation as a means to improve science learning‖ (p. 2). The report explained that the
current research on student learning with science games is inconclusive. The report called for
more research to increase the field’s understanding of the role that games can play in science
learning and to develop scalable designs (National Research Council 2011).
Research studies have shown that players are highly engaged while playing mobile
augmented reality (AR) learning games (Facer et al. 2004; Schwabe & Göth 2005; Perry et al.
2008; Dunleavy et al. 2009; Dunleavy & Simmons 2011). Some researchers have found that
engagement is related to flow (Park et al. 2010; Admiraal et al. 2011; Ryu & Parsons 2012).
However, previous studies have reported technology implementation and network issues that
inhibit the scalability of these games in school settings. In response, we designed and developed
a mobile AR learning science game with Quick Response (QR) codes that utilized free web-
based software to address these issues.
The gameplay relies solely on QR codes, a vision-based AR technology that has
enormous potential but remains insufficiently researched in school settings (Dunleavy & Dede in
press). The technology is vision-based since players use the embedded camera on their mobile
device as their portal to virtual gameplay and it is considered augmented reality because
scanning the codes trigger game-related content to appear on the player’s device not otherwise
viewable to the naked eye. The game was created with the ARIS platform (ARIS 2012). ARIS,
short for Augmented Reality and Interactive Storytelling, offers not only a free iOS application
but also allows anyone to freely create games using their own open-source, web-based
programming environment. The user-friendly, drag-and-drop editor enables first-time
programmers to develop mobile games. In this study, we designed and implemented a narrative-
driven, inquiry-based mobile AR game and examined factors that contributed to the flow
experience of urban middle school learners.
Theoretical Framework
Flow as a Measure of Engagement
Complete immersion in an activity can be characterized by the theory of flow as first
posited by Csikszentmihalyi (1975a). Flow is a positive psychological state that is challenging,
intrinsically rewarding, and enjoyable (Csikszentmihalyi 1975a). Research studies across
different activities with many different populations—such as rock climbers, artists, and
scientists—continue to find consistency in how people describe their flow experience
(Csikszentmihalyi 1975b, 1997). While in flow, people feel involved in meaningful actions,
maintain a sense of control, and stay focused on a goal. Csikszentmihalyi (1975a) stated that the
flow experience ―seems to occur only when a person is actively engaged in some form of clearly
specified interaction with the environment‖ (p. 43).
Video game researchers (Sweetser & Wyeth 2005; Cowley et al. 2008) and educational
researchers (Shernoff et al. 2003; Rathunde & Csikszentmihalyi 2005a) have used flow
assessments to measure engagement. Hoffman and Nadelson (2010) summarized the existing
research on achieving flow while gaming and concluded that ―gaming promotes intrinsic
motivation, positive affect, and many aspects of the flow experience‖ (p. 248). Longitudinal data
Paper Presented at NARST 2013 3
from Shernoff et al. (2003) revealed that students feel significantly more engaged when they are
challenged and in control of their learning.
More recently, researchers have studied flow in technology-mediated learning
environments. Cooper (2010) reviewed published studies of such environments and concluded
that, ―studies indicate a positive relationship between flow and learning in technology
environments‖ (p. 4). Researchers have also used flow as a measure of student engagement with
educational computer games (Inal & Cagiltay 2007; Kiili & Lainema 2008; Grimley et al. 2011)
and more recently, mobile learning games (Park et al. 2010; Admiraal et al. 2011; Ryu &
Parsons 2012).
Mobile Learning Games Mobile learning games, while still in their infancy, show potential for promoting science
learning (Squire & Klopfer 2007) and developing scientific literacy (Squire and Jan 2007).
Using authentic contexts and appropriate game design, these games can provide a situated
learning and collaborative problem solving environment (Rosenbaum et al. 2007; Dede 2009;
Fotouhi-Ghazvini et al. 2009). These learning environments are supported by mobile technology
features that include location-awareness, connectivity, portability, and 1-to-1 personalization
(Klopfer et al. 2002) and also support media-based learning (Dieterle et al. 2007).
Mobile leaning games can be designed for learners to experience science content within
an authentic context. According to Klopfer and Sheldon (2010), ―the power of AR…lies in truly
augmenting the physical landscape, using digital technologies to enable students to see the world
around them in new ways‖ (p. 86). For example, inquiry-based narrative-driven games have
been designed to put students in professional science roles where they engage in scientific
inquiry (Rosenbaum et al. 2007; Squire & Klopfer 2007; Dunleavy et al. 2009) and
argumentation skills (Squire & Jan 2007; Mathews et al. 2008).
Collaborative teaching strategies, such as jigsaw pedagogy (Aronson 1978), are an
effective paradigm for mobile learning activities (Lai & Wu 2006). Studies have reported that
interdependent, differentiated roles can be designed to effectively scaffold collaborative problem
solving within mobile learning games (Squire & Jan 2007; Dunleavy et al. 2009). Specifically,
when students play unique roles, they can receive role-specific, individualized bits of
information. They can then educate other group members about their findings and collaborate to
solve the problem presented within the game.
Purpose of the Study
Students find mobile learning games challenging and enjoyable—critical elements of flow.
However, more research is needed to understand factors pertaining to student engagement during
mobile AR games from the perspective of flow. Using a mixed methods approach, this study
investigated the following questions:
1. What are the features of the flow experience for middle school students while playing a
mobile AR science game?
2. What relationship do gender, interest in science, and gaming attitude have to middle school
students’ flow experience while playing a mobile AR science game?
Paper Presented at NARST 2013 4
Game Description
School Scene Investigators: The Case of the Stolen Score Sheets (henceforth SSI) is a
forensic science mystery game played with mobile devices in a middle school building. This
narrative-driven, inquiry-based game was played on iPhones supplied by the researchers. As
students moved throughout their school, they encountered QR codes that they scanned to access
game-related information (see Figure 1). Sometimes, they conversed with a virtual character
(see Figure 2). At other times, scanning a code meant discovering evidence (see Figure 3).
Throughout gameplay, players were exposed to elements of investigative forensic science, such
as analyzing fingerprints, hair and other trace evidence. They also collaboratively solved
problems such as decoding a locker combination and determining motives of suspects.
Figure 1. One student scans a
QR code while the other reads
information just received.
Figure 2. Screenshot of the
beginning of a conversation
with the principal.
Figure 3. Screenshot of the
hair sample discovered at the
crime scene.
The game design takes advantage of the affordances of mobile technology while
minimizing some limitations documented by previous research (Dunleavy et al. 2009; Fotouhi-
Ghazvini et al. 2009). First, gameplay took place entirely inside the school to minimize the
influence of environmental factors, such as rain, snow, or extreme temperatures. Second, since
previous research demonstrated that GPS error can cause technical difficulties and player
frustration, the game relied solely on the vision-based AR technology of QR codes. Third,
research has shown that mobile players enjoy determining the order of their missions (Ardito et
al. 2009), so non-linearity was built into the game. Fourth, Folta (2010) found that players did
not like the open-ended nature of the game designed for her study; therefore, SSI has a win
state—a point when players know they have successfully completed the game. Finally, the game
is location-based yet scalable because the game space was designed to utilize generic school
locations such as a library, gym, and principal’s office. Almost any school teacher can
implement the game by printing the QR codes and posting them in relevant locations. To play
the game, students need access to WiFi-enabled iOS devices loaded with the ARIS application
developed by the University of Wisconsin-Madison (ARIS 2012).
Paper Presented at NARST 2013 5
Gameplay Overview
Students played the game in groups of three and four. Each player had an iPhone. To
promote an authentic context, students had differentiated roles. The roles of social networker,
science whiz, techie, and photographer were assigned based on student interest. The game’s start
screen presented a two-sentence description of the character’s strengths and interests.
The main gameplay screen contained a button each
for quests, a decoder, and the inventory. Quests helped
players navigate the game; they told players where to go and
what to do (see Figure 4). Complete with audio clues and
onscreen alerts, quests were automatically triggered at
different points during gameplay, often by scanning QR
codes. When quests required moving to a new location,
players would find a new QR code and scan it with the
decoder. Scanning would often add a new inventory item
and a quest would pop-up requiring the player to view the
new item in the inventory. The game is a mystery to find
out who stole test answers from the principal’s office. The
mystery involves analyzing four pieces of evidence and
investigating four suspects to solve the crime. Quests were
differentiated based on a player’s role. Groups were
required to stay together and therefore had to collaborate to
solve the mystery and complete the game. Four different
versions of the game were developed and implemented,
each with a different thief.
Methods
Setting
SSI was implemented as an enrichment activity and played inside the participants’ school.
The school is located in a diverse, urban area of Pennsylvania with mostly low-income
households. A large majority of students (81%) participate in the free and reduced lunch
program. Middle school was chosen as the target population because it is a critical time to foster
intrinsic interest in science and build students’ self-confidence in their potential for success in
science careers. Research has shown that girls’ interest in science decreases from age 12 to 16
(Lindahl 2007) and experience with science before high school plays an important role in
whether students go into science-related careers (Tai et al. 2006).
Participants The participants included 68 students who volunteered for the study from two science
teachers’ classes and the after-school program. The sample population included 51.5% male
(n=35) and 48.5% female (n=33) students. Participants ranged in age from 11- to 15-years-old.
Several grade levels were represented including eighth grade (47.1%), seventh grade (29.4%),
and sixth grade (23.5%).
Students played in groups of three and four and experienced the game one group at a
time. Group composition was created by the classroom teacher dictated mostly by which
students had returned consent forms. A total of 19 groups participated; 17 groups played during
Figure 4. Screenshot of a quest
that students receive during the
game.
Paper Presented at NARST 2013 6
school hours and two groups played after-school. Most groups were mixed gender (n=14);
however, there were three girl-only groups and two boy-only groups. Data collection took place
during a two-week period during March 2012.
Study Design
This study employed a mixed methods approach and used a convergent design with the
data-validation variant (Creswell & Plano Clark 2011). Data sources included: pre- and post-
surveys, field observations, and group interviews. All survey data was collected online using
Survey Monkey (2011), a web-based survey application.
Pre-survey. Immediately before starting the game, players completed the pre-survey
(see Appendix A). There were three parts to the pre-survey including, demographics, interest in
science, and gaming attitude. Interest in science was measured using 10 Likert-style items
derived from the Test of Science Related Attitudes (Fraser 1981), a valid and reliable instrument.
The complete instrument has seven scales; however, this study utilized only two scales:
enjoyment of science lessons and leisure interest in science. From the 20 original items, 10
questions were eliminated to reduce repetition about science lessons and to remove references to
out-of-date communication technologies. Negative items required reverse coding. Possible
scores ranged from 10 to 50. Cronbach’s alpha was .90 for this adapted instrument.
The third part of the pre-survey measured gaming attitude using 12 Likert-style items.
Items were selected from Bonanno and Kommers’s (2008) instrument that was developed to test
for different attitudinal components towards gaming. Their original instrument contained 21
items but their results showed significant Pearson chi-square tests for only 12 items. Those
significant items were selected although wording was adjusted slightly to eliminate references to
specific game titles and grade levels in school. Negative items required reverse coding. When
items were tested for reliability, results indicated one item should be dropped to increase
Cronbach’s alpha. The item (―I only use games when told to.‖) was deemed ambiguous and not
indicative of gaming attitude. The resulting instrument contained 11 items with a Cronbach’s
alpha of .73. Post-survey. Immediately after the game concluded, players completed the post-survey
(see Appendix B). The post-survey collected basic gameplay feedback and assessed flow. First,
students indicated which role they played and whether they felt more curious about forensics.
Second, flow experience was measured using 10 Likert-style items similar to the Short Flow
State scale (S FSS-2) developed by Jackson et al. (2010). The scale measures nine elements of
flow as outlined by Csikszentmihalyi (1996). All nine original items were used; however,
wording was adjusted to enhance understanding for urban middle school students. One question
was also selected from Jackson et al.’s (2010) Core Flow State scale (C FSS-2). This item was,
―I was in the zone.‖ It was included to represent the player’s overall experience of being in flow.
Possible scores ranged from 10 to 50. Cronbach’s alpha for this instrument was .80.
Field observations. To collect field observations during gameplay, a researcher
followed players throughout the school. While students were playing, the researcher recorded
observations about students’ focus on the task, physical movement between locations,
technology use, and collaboration. The researcher also chronicled dialogue when possible and
documented individual behaviors, group behaviors, and atypical interactions.
Group interview protocol. Immediately after finishing the post-survey, students
participated in a group interview. Since the game was designed to scaffold collaborative
problem solving, students were interviewed together to draw out inferences about their
Paper Presented at NARST 2013 7
collaboration. The interview protocol (see Appendix C) included several broad questions about
the gameplay experience and specific questions intended to illuminate further details concerning
post-survey answers. Interviews were audio recorded and later transcribed by the researcher.
Data Analysis
Data sets were analyzed independently. A multiple hierarchical regression analysis was
conducted to illuminate how much variance in flow experience was predicted by gender, interest
in science, and gaming attitude. A qualitative analysis uncovered features of the students’ flow
experience. Transcripts were coded and analyzed using ATLAS.ti, a qualitative analysis
software package. The researcher started with a provisional list of 25 descriptive codes based on
the conceptual framework, research questions, field observations, and group interview protocol
(Miles & Huberman 1994). Employing the constant-comparative method (Glaser & Strauss
1967), the researcher reviewed transcripts by starting with open-coding, then revisiting
transcripts to incorporate new codes and check coding of the original data. Eventually, the final
eight transcripts served as a confirmation sample in order to test for code saturation (Glaser &
Strauss 1967).
Throughout the coding process, detailed memos were kept that included possible
relationships between variables. With each level of coding, memos became more detailed,
developing into testable hypotheses. After coding the confirmation sample, hypotheses were
grouped under the broader themes and combined with hypotheses built while in the field.
Guided by the hypotheses, the researcher examined the data searching for co-occurrences of
codes. A member check confirmed dependability of findings.
Findings
Since the qualitative and quantitative data sets were triangulated to confirm the reliability
of the outcome variable (flow experience), the findings are presented as a merged analysis. Prior
to statistical analysis, aggregate scores were calculated for interest in science, gaming attitude,
and flow experience. Table 1 displays descriptive statistics for these aggregated scores.
Table 1 Descriptive statistics for predictors and flow experience (N = 68)
Variable M SD Observed
Range
Possible
Range
Interest in science
31.88 8.50 10.00-50.00 10.00-50.00
Gaming attitude 44.59 5.41 34.00-55.00 10.00-55.00
Flow experience 41.21 5.30 30.00-50.00 10.00-50.00
Note. Gender was excluded from this table because it is a dichotomous variable.
A preliminary one-way ANOVA test was conducted to determine if the player’s role
influenced flow experience. All roles were integral to gameplay except the photographer; groups
could play with or without someone as the photographer. No significant differences were
observed in flow experience scores based on role. The ANOVA test verified that the non-
integral role of photographer was just as effective as the integral roles of social networker,
science whiz, and techie.
Paper Presented at NARST 2013 8
Flow Experience
Flow experience was measured using the post-survey (see Appendix B); results are
displayed in Table 2. The mean flow experience score was 41.2 out 50 indicating that the
average player had a substantive flow-like experience. Additionally, no scores fell below the
mid-point score of 30 signifying all players experienced some level of flow. Field observations
further supported students’ experiencing elements of flow; in particular, the researcher observed
students’ enjoyment and concentration on the task. Students also referenced flow elements
during focus group interviews. Table 2 includes student statements aligning to flow elements.
Table 2 Quantitative and qualitative support for participants’ flow experience
Statement I strongly
agree I agree
I don’t
know
I
disagree
I
strongly
disagree
Quotations from Group Interviews
I was
challenged and I
felt I could meet
the challenge.
41.2%
(n=28)
45.6%*
(n=31)
8.8%
(n=6)
1.5%
(n=1)
2.9%
(n=2)
You’re thinking real hard to figure
it out.
The most I actually had to think of
something.
It’s hard but if you set your mind
to it, you can do it.
I think it’s like chess. It improves
your brain or something.
I did things
naturally
without thinking
too much.
19.1%
(n=13)
47.1%*
(n=32)
16.2%
(n=11)
14.7%
(n=10)
2.9%
(n=2) Yeah, we winged it.
It felt like I was a cop.
You might not realize you’re doing
it, but you’re doing it.
I had a strong
sense of what I
wanted to do.
36.8%
(n=25)
39.7%*
(n=27)
19.1%
(n=13)
2.9%
(n=2)
1.5%
(n=1) Trying to like find something. That
makes you determined to like find
it.
It was - wow - now you have to do
this - and we were like running
around, finding those barcodes and
everything.
I felt in control
of what I was
doing.
45.6%*
(n=31)
33.8%
(n=23)
13.2%
(n=9)
7.4%
(n=5)
0.0%
(n=0) We could get out there and do it
ourselves.
It was kind of like independent
learning…in a way.
Like you can solve it yourself. You
don’t like need someone.
It teaches you how to do things on
your own. Like individual.
I felt I was on
track towards
my goals.
39.7%
(n=27)
44.1%*
(n=30)
7.4%
(n=5)
7.4%
(n=5)
1.5%
(n=1) It inspired me to keeping going.
We kept getting closer to our goal.
It made me want to keep playing
because I wanted to find out who
did it.
Paper Presented at NARST 2013 9
I was totally focused on
what I was doing.
52.9%*
(n=36)
33.8%
(n=23)
10.3%
(n=7)
2.9%
(n=2)
0.0%
(n=0) You had to think about it more.
Center on what you are actually
focusing on.
It felt like nothing else
mattered.
23.5%
(n=16)
29.4%
(n=20)
36.8%*
(n=25)
10.3%
(n=7)
0.0%
(n=0) I think we were so into it…we didn’t
even notice it was a game.
I think that it was challenging. And it
made me focus more. And I didn’t
really focus on anything else because
I wanted to beat the challenge of it.
I lost my normal sense
of time.
36.8%*
(n=25)
29.4%
(n=20)
22.1%
(n=15)
10.3%
(n=7)
1.5%
(n=1) Time went quickly.
It was like time consuming.
It felt like 5 minutes. [Researcher
Note: It took 25 minutes to
complete.]
I felt like it was so short. You know
what they say - time flies when
you’re having fun!
I really enjoyed what I
was doing.
66.2%*
(n=45)
29.4%
(n=20)
4.4%
(n=3)
0.0%
(n=0)
0.0%
(n=0) It was really fun - a nice experience
to have.
The most fun I had in school.
It felt like it wasn’t science. I felt like
it was a different subject. I felt like it
was the subject of fun.
I was in the zone. 55.9%*
(n=38)
32.4%
(n=22)
10.3%
(n=7)
1.5%
(n=1)
0.0%
(n=0) It was a rush!
Out of body experience.
I kind of forgot that it was a game. I
kept thinking we were going to win a
prize or something.
Note. (*) denotes category with highest percentage of responses
Features of Flow
The first research question investigated flow experience features for middle school
students while playing a mobile AR science game. Qualitative data results were used to examine
flow experience features. Participants’ ability to experience flow seemed to stem from three
experiential features: Participants felt a flash of intensity, a sense of discovery, and the desire for
higher performance. Table 3 displays student quotations from interviews that support these
features.
Table 3 Qualitative support for features of flow
Features of flow Quotations from students
Flash of intensity I liked it because it was different than the normal science class. Like, it
was better than regular science class.
I felt like it was more eye-opening. Instead of just like the usual.
It’s better than sitting in a class reading.
It’s different. It’s like unique. We weren’t sitting here. Just like listening
to you say something repeatedly.
Paper Presented at NARST 2013 10
Sense of discovery
It was like a scavenger hunt.
It was actually like a Sherlock Holmes type thing.
I liked going around school and looking for the clues.
We got to go on an adventure around the school.
Desire for higher performance Different codes. Like really hard ones that take an hour to figure out.
Could have been longer. Could have had a more in-depth storyline…And
more detailed evidence.
There should have been more suspects too…Like when there’s more
people it’s more, it’s harder, more complicated.
According to Csikszentmihalyi (1990), games can be considered flow activities because
they "offer opportunities to go beyond the boundaries of ordinary experience" (p. 72). More
specifically, Csikszentmihalyi (1997) explained that ―flow experiences provide the flashes of
intense living‖ (p.31) in comparison to the dull background of a typical day. This intense verses
dull comparison was seen repeatedly in the focus group transcripts. The majority of students
noted that the game offered something ―different‖ or ―unique‖ compared to their typical
classroom learning.
Studies on flow activities have shown they have a common element—they all provide a
sense of discovery (Csikszentmihalyi 1990). Students often referenced the idea of discovery
when describing their overall experience and favorite part of the game using terms such as
―adventure‖ or ―scavenger hunt.‖ They also regularly mentioned elements of discovery as
enjoyable parts of the experience with statements about ―looking for clues‖ and ―figuring it out.‖
The sense of discovery in flow activities also leads to higher levels of performance
(Csikszentmihalyi 1990). According to Csikszentmihalyi (1990), one cannot experience flow
while doing the same activity at the same level forever; to stay in flow, participants need to
continue discovering new challenges. During group interviews, several groups freely discussed
making the game more challenging. Unlike students in previous studies who experienced
cognitive overload (Dunleavy et al. 2009), these participants seemed to experience the right
challenge-skill balance and were eager to challenge themselves more the next time with ―more
clues‖ and a ―longer‖ experience.
Gender, Interest in Science, and Gaming Attitude The second research question investigated the relationship that gender, interest in science,
and gaming attitude have on middle school students’ flow experience while playing a mobile AR
science game. Data was checked for multicollinearity. Pearson correlations are reported in
Table 4. Since all the values were low enough, all the predictors were kept in the model. Next,
Cook’s D values were checked and no outliers were biasing the dataset. Finally, the data was
checked for homoelasticity; the standardized plot revealed an even distribution.
Table 4 Correlations between predictors and flow experience (N = 68)
Measure Gender Interest in
science Gaming attitude Flow experience
Gendera
- -.314 -.035 .035
Interest in science - .096 .167
Gaming attitude - .489
Flow experience - a Gender was dummy coded using male = 0 and female = 1.
Paper Presented at NARST 2013 11
Based on the literature, gender was a possible confounding variable, so it was added to
the hierarchical regression analysis as Block 1. Results are displayed in Table 5. Gender only
explained 0.1% of the variance in flow experience, p = .78. Field observations revealed that boy-
only groups (n=2) behaved somewhat differently than girl-only groups (n=3) during gameplay.
Overall, girls seemed more methodical; they interacted quietly, seemed to read every detail, and
walked between locations. Generally, boy groups seemed comfortable with the technology and
game format; they also moved steadily between locations, sometimes running. Although their
gameplay experiences seemed different to the observer, gender did not influence flow
experience. Focus group findings illuminated details to explain this discrepancy; gender may
dictate what aspects of the game one finds most enjoyable. During group interviews, boys
seemed more apt to discuss enjoyment in using the phones and scanning the codes (―I liked using
the phone [and] scanning the barcodes‖); girls referenced the storyline (―I liked the storyline‖).
Table 5 Results for multiple hierarchical regression analysis predicting flow experience (N = 68)
Note. B = unstandardized regression coefficient; SE = standard error; Part r = semi-partial correlation coefficient. a Gender was dummy coded using male = 0 and female = 1.
*** p < .001.
Interest in science and gaming attitude were added as Block 2 in the regression analysis.
Gender was still not significant, p = .38. Interest in science also did not significantly predict
variance in flow experience, p = .18. However, gaming attitude predicted significant variability
in flow experience, p < .001. The squared semi-partial (part) correlation of gaming attitude was
.226; therefore, gaming attitude uniquely predicted 23% of the variance in flow experience.
Focus group data supported the finding that interest in science did not predict flow
experience. During group interviews, students expressed varied levels of interest in science.
Some students expressed moderate or high interest in science, with statements such as ―I love
science‖ or ―I like science.‖ Other students admitted to disliking science (―Yeah, I don’t like
science‖) and were often surprised at their enjoyment in the game, since it was a science game.
Emerging Themes
Two additional themes, building better relationships and increased interest in
investigative science emerged during analysis offering insight into the benefits of mobile AR
science games.
Model 1 Model 2
Variable B SE B Part r B SE B Part r
Gender a 0.365 1.296 0.035 1.049 1.191 0.095
Interest in science 0.095 0.071 0.145
Gaming attitude 0.468*** 0.106 0.475***
R2 0.001 0.262***
ΔR2 0.261***
Paper Presented at NARST 2013 12
Building better relationships. The game was designed to scaffold collaborative
problem solving and students stated that working with their fellow classmates went well.
Several groups were truly surprised by how well they worked together. Some students said they
normally argued with each other in class, yet they actually worked well as a team. As one girl
reflected on her gameplay experience, she stated, ―I thought we were going to argue or
something. But we actually didn’t.‖ Others indicated they did not necessarily associate with
each other, but they cooperated fine. One participant stated, ―I’m actually surprised we worked
well, because we don’t kinda hang out together.‖ In one extreme case, two boys admitted to not
getting along yet they both agreed they did a good job together during the game. Field
observations of these supposed adversaries showed that their group interactions were some of the
best of any group in the sample (see Figure 5).
Figure 5. Students working together to share virtual evidence collected during gameplay.
Increased interest in investigative science. As mentioned earlier, interest in science
was not a predictor of flow experience. However, after gameplay, 93% (n = 63) of players said
they were curious to learn more about forensics. Focus group data supported this dramatic
increase in interest: the way the participants experienced science within the game seemed to
change their perception of science which increased their interest.
Some students were going to forego playing the game because they thought science was
boring. Students in traditional learning environments spend ample time in didactic settings
(Rathunde & Csikszentmihalyi 2005b) and they are not engaged by the autocratic way science is
presented (Lyons 2006). One participant stated, "I honestly thought [the game] was going to be
boring. Like…because, you know, it’s science." Others were willing to give it a try, but only
Paper Presented at NARST 2013 13
because it was a game. One student stated, ―I wouldn’t mind if we played a game about science
but if it’s just science, I don’t want to do it.‖
After playing the game, students repeatedly discussed how they were able to have fun
and learn at the same time. The experiential nature of the game seemed to really excite them.
The game experience, which was authentic and investigative, seemed to shift their perception of
science. Some students discussed how science is usually boring to them but the game made it
fun. A few students explained in their own words how the pedagogy of the game more
accurately represented science. One student explained that, ―It’s a way closer look at science
than we get taught.‖ Finally, a seventh grade girl made this comment after the group interview
already concluded: "Thanks for spending your time here…I mean you like showed us that
science isn’t only writing and reading. It’s actually experimenting."
Discussion
Using a mixed methods approach, this study examined middle school students’ flow
experience while playing School Scene Investigators, a mobile AR science game. Results
showed that neither gender nor interest in science was important in predicting flow experience.
However, gaming attitude uniquely predicted 23% of the variance. Focus group transcripts
revealed important features of the flow experience: Participants felt a flash of intensity, a sense
of discovery, and the desire for higher performance. The game’s ability to help players build
better relationships and increase their interest in investigative science surfaced as emerging
themes.
Novelty Effect
The novelty effect is a reality when introducing a project of this nature to an underserved
audience. However, in this study, novelty manifested itself in an unusual way. Students did not
talk about the new technology as a primary driver of their enjoyment. Rather, running around
the hallways while classes were in session was the novel experience. The excitement of doing
something normally taboo was encapsulated in this comment: ―Running around school…it was
fun…it was like a playground!"
There was also something intrinsic within the pedagogy that seemed novel as well.
Participants’ comments revealed the qualities that made this a desirable learning experience: it
was an adventure, it required thinking, it facilitated teamwork, and one could have fun and learn
simultaneously. Players also highlighted the independent nature of the learning experience and
that it felt ―real.‖ The technology supported the novelty of an investigative, experiential way to
learn and promoted neomillennial learning styles (Dieterle et al. 2007).
Game Design Promotes Flow Experience
Students playing SSI were highly engaged; they were experiencing flow. Game design
played a large role in this outcome. First, the use of QR codes and wireless Internet decreased
technical frustrations experienced with hardware and software issues reported in previous studies
that used location-aware technologies such as GPS-enabled smartphones (Facer et al. 2004;
Dunleavy et al. 2009; Admiraal et al. 2011). In this study, field observations showed that
players occasionally got bumped back to the iPhone’s home screen or stuck on the decoder
screen, but it was never enough to induce frustration. This aligns with Csikszentmihalyi’s (1990)
idea that people in flow will temporarily tolerate a lack of stimulation if they believe stimulation
will resume shortly. Second, the vision-based AR made navigation easier. Park et al. (2010)
Paper Presented at NARST 2013 14
found that "mobility may have negative influences on learning performance, in that it asks the
learner to attend to more than one information source when using the handheld device" (p. 59).
However, the purely vision-based AR game format allowed players to focus on navigation or
information, rarely both at the same time. In this study, locational clues were embedded in
player quests, so before leaving each location, players knew where to go next. When players
were moving, they were not using the phones to navigate, thus reducing cognitive load. In other
studies with location-aware technologies (Dunleavy et al. 2009; Park et al. 2010) players needed
to navigate with the handheld while in motion, perhaps increasing cognitive load and decreasing
enjoyment. Third, the game design did not allow for social loafing or overpowering leadership,
thus collaboration was a source of enjoyment. Not all students had a quest at the same time; this
forced leadership to shift between different group members where no single member could
dominate group dynamics or slack off. This design meant all players equally contributed to the
overall success of the mission. Shifting quests between roles effectively leveraged the benefits
of cooperative learning with the jigsaw technique (Aronson & Patnoe 2011) along with social
constructivist learning (Palincsar 1998).
Predictors of Flow Experience Gaming attitude predicted 23% of the variance in flow experience. Research has shown
that action video gamers have improved cognitive abilities over non-gamers, especially in multi-
tasking (Green et al. 2012). Since SSI requires multi-tasking, gamers may achieve a better
balance between challenge and skill, thus reaching a more substantive flow experience.
Previous research has reported that boys are more interested in science (Gardner 1975, 1998) and
have more positive attitudes about gaming (Bonanno & Kommers 2008). Hoffman and
Nadelson (2010) found that gender was a significant predictor of engagement in gaming with
males twice as likely to be engaged as females. Since SSI was a science game, there was a
strong case that gender would predict flow experience—but it did not. Our findings were similar
to those reported by Ketelhut and Nelson (2010) who found that both boys and girls were equally
engaged in a narrative-driven, inquiry-based multi-user virtual environment. The answer to
equal engagement may again lie in game design, specifically, the narrative-driven, inquiry-based
design. Research has reported that girls prefer narrative-based computer games (Gorriz &
Medina 2000). Furthermore, game narratology plays an important role in whether girls achieve
flow while playing (Inal & Cagiltay, 2007). Whereas, for boys, the key elements for achieving
flow are the rules and gameplay characteristics. Therefore, this design-type of educational
gaming may be engaging both genders equally because it entices girls through narrative, but
offers interesting game characteristics that appeal to boys.
Gameplay Increased Science Interest By exposing students to an investigative science experience, gameplay increased interest
in forensic science, perhaps even science in general. This finding is also supported by Beylefeld
and Struwig (2007) who reported that their microbiology game positively influenced players’
attitudes and perceptions of the science content within the game. One student stated during the
focus group that playing SSI could get kids into "science to begin with, because personally I
don’t like science." While another mentioned, "it could make kids want to go into forensics."
This is a promising finding because experiences that peak students’ early interest in science lead
to long-standing interest in science (Lindahl 2007; Maltese & Tai 2010). Consequently, if
Paper Presented at NARST 2013 15
science games played with mobile AR technology increase interest in science, middle school
students playing AR science games could help promote a more scientifically-literate citizenship.
Critical Features of Flow Experience: Interdependence and Challenge
When asked about the overall game experience, students often stated: ―It was fun.‖
Delving deeper, students particularly enjoyed working together. This finding is supported by
previous research about flow occurring in groups, known as social flow. According to Walker
(2010), experiencing flow as a group is more enjoyable than experiencing it alone. In a related
study, Admirral et al. (2011) found that team flow in gameplay explained 24% of the variance in
group game performance. Playing the game as a group is not enough; the successful nature of
team collaboration hinges tremendously on positive interdependence. Walker (2010) found that
people who collaborated interdependently experienced more enjoyment in flow than those who
worked with less interdependence. Overall, given the interdependence of the players’ roles in
SSI, students were able to experience a lot of enjoyment from their social flow.
As mentioned earlier, game characteristics seemed to promote the right challenge-skill
balance for players to experience flow. Field observations of the decoding activity in the
mystery game yielded evidence of a suitable challenge-skill balance; many groups were very
challenged by decoding the locker combination, yet no one complained about the difficulty.
Additionally, players read a lot during gameplay—something they admitted to being bored by in
the classroom—yet very few mentioned the amount of reading as a complaint about the game.
This observation differed from findings in a previous study by Admirral et al. (2011) who found
students were reluctant to read text messages in their game.
Going beyond reading comprehension and tackling in-game challenges, participants
enjoyed having to think. While discussing their favorite part, participants referenced using your
―brain‖ and having to ―think about it more;‖ they also enjoyed ―figuring it out.‖ Additionally,
students were eager to experience future challenges demonstrating an increase in desire for
higher levels of performance—a direct result of their flow experience. Furthermore, learners
may wish to take on more complex game challenges when an instructional game provides
students with their own unique learning paths (Ryu & Parsons 2012). In summary, students were
undeterred by the challenge of solving the mystery and being challenged seemed critical to their
enjoyment of the gameplay experience.
Implications and Conclusion
This study’s findings contribute to the literature by providing support for a middle school
player’s flow experience while playing a mobile AR science game with QR codes. Gaming
attitude was an important significant predictor of flow experience; gender and interest in science
were not important predictors. The interdependent nature of gameplay seemed to increase social
flow. This study suggests that playing narrative-driven, inquiry-based science games can
increase interest in science for both genders. Since players desired higher levels of performance,
the study also indicates a role for these games in learning; while playing, students persevere at
challenges within their zone of proximal development (Vygotsky 1978), and upon success, desire
greater challenges.
This research also investigated vision-based AR, an AR technology lacking extensive
research, and determined its potential as a scalable design for school settings. The technology
was very stable; this minimized player frustration and may have increased enjoyment. Game
design reduced cognitive overload by enabling players to focus on either navigation or
Paper Presented at NARST 2013 16
information, rarely both at the same time. Gameplay took approximately 30 minutes and setup
required only a few minutes to post QR codes; therefore, similar games could be played within a
class period. Implementation also does not require additional chaperones since students never
leave the school building. Additionally, mobile devices only need wireless Internet access, not
cellular plans. Thus, implementing AR learning games with QR codes reduces overall financial
costs for schools seeking to invest in mobile technology use.
This study had several limitations. First, the small sample size was obtained from one
urban middle school; therefore, care must be taken when generalizing to other contexts. Within
this sample, all the kids had positive gaming attitudes; in a different sample, gaming attitude may
not be as predictive of flow experience. Second, groups played one at a time; therefore,
competition was practically non-existent. Increased competition could diminish players’ flow
experience. Several groups playing simultaneously might also change the logistics, such as
increasing wait times to scan codes, which could lessen player enjoyment. Third, this was an
enrichment activity and not a required classroom activity, so player enthusiasm could be
attributable to the novelty. Finally, forensic science has been popularized by the media and the
narrative was gender-neutral; a different storyline with another science focus could reduce
enjoyment for one or both genders.
Future research should further explore findings reported by this study, chronicle different
game designs, and determine other elements that contribute to players’ flow experience. Future
studies may wish to focus on how gender as a predictor of flow experience may change when
students work in single gender verses mixed gender groups. To investigate scalability, vision-
based AR game designs need to be tested with multiple groups playing simultaneously. Since
this game design takes place in school, future game designs could integrate actual lab
experiments within the context of gameplay. Additionally, since sparking early science interest
can lead to science careers (Tai et al. 2006), future research should study the short-term and
long-term impact on students’ interest in science after playing games of this genre. Overall, this
study makes a strong case for vision-based AR learning games as scalable design for schools to
utilize since players experience flow while playing and desire higher levels of performance.
Paper Presented at NARST 2013 17
References
Admiraal, W., Huizenga, J., Akkerman, S. & Ten Dam, G. (2011) The concept of flow in
collaborative game-based learning. Computers in Human Behavior 27, 1185-1194.
Ardito, C., Buono, P., Costabile, M. F., Lanzilotti, R. & Piccinno, A. (2009) Enabling interactive
exploration of cultural heritage: an experience of designing systems for mobile devices.
Knowledge, Technology & Policy 22, 79-86.
ARIS. (2012) Retrieved May 12, 2012, from http://arisgames.org/.
Aronson, E. (1978) The jigsaw classroom. Sage Publications, Beverly Hills, CA.
Aronson, E. & Patnoe, S. (2011) Cooperation in the classroom: The jigsaw method, 3rd edition.
Pinter & Martin Ltd, London.
Beylefeld, A. A. & Struwig, M. C. (2007) A gaming approach to learning medical microbiology:
students' experiences of flow. Medical Teacher 29, 933-940.
Bonanno, P. & Kommers, P. A. M. (2008) Exploring the influence of gender and gaming
competence on attitudes towards using instructional games. British Journal of
Educational Technology 39, 97-109.
Cooper, K. (2010) Go with the flow: engagement factors for learning in Second Life. Paper
presented at the 2010 Spring Simulation Multi-Conference, Orlando, Florida.
Cowley, B., Charles, D., Black, M. & Hickey, R. (2008) Toward an understanding of flow in
video games. Computers in Entertainment 6, 1-27.
Creswell, J. & Plano Clark, V. (2011) Designing and conducting mixed methods research, 2nd
edition. Sage Publications, Thousand Oaks, CA.
Csikszentmihalyi, M. (1975a) Play and intrinsic rewards. Journal of Humanistic Psychology 15,
41-63.
Csikszentmihalyi, M. (1975b) Beyond boredom and anxiety. Jossey-Bass, San Francisco.
Csikszentmihalyi, M. (1990) Flow: the psychology of optimal experience. Harper and Row, New
York.
Csikszentmihalyi, M. (1996) Creativity: flow and the psychology of discovery and invention.
Harper Collins, New York.
Csikszentmihalyi, M. (1997) Finding flow: the psychology of engagement with everyday life.
Harper Collins, New York.
Dede, C. (2009) Immersive interfaces for engagement and learning. Science 323, 66-69.
Dieterle, E., Dede, C. & Schrier, K. (2007) ―Neomillennial‖ learning styles propagated by
wireless handheld devices. In Ubiquitous and pervasive knowledge and learning
management: Semantics, social networking and new media to their full potential (eds M.
Lytras & A. Naeve), pp. 35–66. Idea Group, Hershey, PA.
Dunleavy, M. & Dede, C. (in press) Augmented reality teaching and learning. In Handbook of
Research on Educational Communications and Technology, Volume 2, 4th edition (eds
J.M. Spector, M.D. Merrill, J. Elen & M.J. Bishop). Macmillan, New York.
Dunleavy, M., Dede, C. & Mitchell, R. (2009) Affordances and limitations of immersive
participatory augmented reality simulations for teaching and learning. Journal of Science
Education and Technology 18, 7–22.
Dunleavy, M. & Simmons, B. (2011) Assessing learning and identity in augmented reality
science games. In Serious educational games assessment (eds L. Annetta & S. Bronack),
pp. 221–240. Sense Publishers, Rotterdam, The Netherlands.
Facer, K., Joiner, R., Stanton, D., Reid, J., Hull, R. & Kirk, D. (2004) Savannah: mobile gaming
and learning? Journal of Computer Assisted Learning 20, 399-409.
Paper Presented at NARST 2013 18
Folta, E. E. (2010) Investigating the impact on student learning and outdoor science interest
through modular Serious Educational Games: a design-based research study. (Doctoral
dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No.
3442557).
Fotouhi-Ghazvini, F., Earnshaw, R. A., Robison, D. & Excell, P. S. (2009) Designing augmented
reality games for mobile learning using an instructional-motivational paradigm. Paper
presented at the International Conference on CyberWorlds, Bradford, UK.
Fraser, B. (1981) TOSRA: test of science-related attitudes handbook. Australian Council for
Educational Research, Hawthorn, Victoria.
Gardner, P.L. (1975) Attitudes to science: a review. Studies in Science Education 2, 1–41.
Gardner, P.L. (1998) The development of males’ and females’ interests in science and
technology. In Interest and learning: Proceedings of the Seeon Conference on Interest
and Gender (eds L. Hoffmann, A. Krapp, K.A. Renninger & J. Baumert), pp. 41-57.
Institute for Science Education (IPN), Kiel, Germany.
Glaser, B. G. & Strauss, A. L. (1967) The discovery of grounded theory: strategies for
qualitative research. Aldine Publishing Company, New York.
Gorriz, C. M. & Medina, C. (2000) Engaging girls with computers through software games.
Communications of the ACM, 43(1), 42-49.
Green, C. S., Sugarman, M. A., Medford, K., Klobusicky, E. & Bavelier, D. (2012) The effect of
action video game experience on task-switching. Computers in Human Behavior 28,
984-994.
Grimley, M., Green, R., Nilsen, T., Thompson, D. & Tomes, R. (2011) Using computer games
for instruction: the student experience. Active Learning in Higher Education 12, 45-56.
Hoffman, B. & Nadelson, L. (2010) Motivational engagement and video gaming: a mixed
methods study. Educational Technology Research & Development 58, 245-270.
Inal, Y. & Cagiltay, K. (2007) Flow experiences of children in an interactive social game
environment. British Journal of Educational Technology 38, 455-464.
Jackson, S., Eklund, R. & Martin, A. (2010) The FLOW manual. Mind Garden, Inc., Queensland,
Australia.
Ketelhut, D. J. & Nelson, B. C. (2010) Designing for real-world scientific inquiry in virtual
environments. Educational Research, 52(2), 151-167.
Kiili, K. & Lainema, T. (2008) Foundation for measuring engagement in educational games.
Journal of Interactive Learning Research 19, 469-488.
Klopfer, E. & Sheldon, J. (2010) Augmenting your own reality: student authoring of science-
based augmented reality games. New Directions for Youth Development, Winter
2010(128), 85-94.
Klopfer, E., Squire, K. & Jenkins, H. (2002) Environmental detectives: PDAs as a window into
a virtual simulated world. Proceedings of the IEEE international workshop on wireless
mobile technologies in education, pp. 95-98, IEEE.
Lai, C. Y. & Wu, C. C. (2006) Using handhelds in a jigsaw cooperative learning environment.
Journal of Computer Assisted Learning 22, 284-297.
Lindahl, B. (2007) A longitudinal study of students' attitudes towards science and choice of
career. Paper presented at the National Association for Research in Science Teaching,
New Orleans.
Lyons, T. (2006) Different countries, same science classes: students’ experiences of school
science in their own words. International Journal of Science Education 28, 591-613.
Paper Presented at NARST 2013 19
Maltese, A. V. & Tai, R. H. (2010) Eyeballs in the fridge: sources of early interest in science.
International Journal of Science Education 32, 669-685.
Mathews, J., Holden, C., Jan, M.-F. & Martin, J. (2008) Sick at South Shore Beach: a place-
based augmented reality game as a framework for building evidence-based arguments.
Proceedings of 8th
International Conference on Learning Sciences 3, 89-90.
Miles, M. B. & Huberman, A. M. (1994) Qualitative data analysis: an expanded sourcebook,
2nd edition. Sage Publications, Thousand Oaks, CA.
National Research Council. (2011) Learning science through computer games and simulations.
The National Academies Press, Washington, DC.
Palincsar, A. S. (1998) Social constructivist perspectives on teaching and learning. Annual
Review of Psychology 49, 345-375.
Park, J., Parsons, D. & Ryu, H. (2010) To flow and not to freeze: applying flow experience to
mobile learning. IEEE Transactions on Learning Technologies, 3(1), 56-67.
Perry, J., Klopfer, E., Norton, M., Sutch, D., Sandford, R. & Facer, K. (2008, June). AR gone
wild: two approaches to using augmented reality learning games in zoos. Paper
presented at the 8th international conference for the learning sciences, Utrecht,
Netherlands.
Rathunde, K. & Csikszentmihalyi, M. (2005a) Middle school students' motivation and quality of
experience: a comparison of Montessori and traditional school environments. American
Journal of Education 111, 341-371.
Rathunde, K. & Csikszentmihalyi, M. (2005b) The social context of middle school: teachers,
friends, and activities in Montessori and traditional school environments. Elementary
School Journal, 106(1), 59-79.
Rosenbaum, E., Klopfer, E. & Perry, J. (2007) On location learning: authentic applied science
with networked augmented realities. Journal of Science Education & Technology 16, 31-45.
Ryu, H. & Parsons, D. (2012) Risky business or sharing the load? – social flow in collaborative
mobile learning. Computers & Education 58, 707-720.
Schwabe, G. & Göth, C. (2005) Mobile learning with a mobile game: design and motivational
effects. Journal of Computer Assisted Learning 21, 204-216.
Shernoff, D. J., Csikszentmihalyi, M., Schneider, B. & Shernoff, E. S. (2003) Student
engagement in high school classrooms from the perspective of flow theory. School
Psychology Quarterly, 18(2), 158-176.
Squire, K. & Jan, M. (2007) Mad city mystery: developing scientific argumentation skills with a
place-based augmented reality game on handheld computers. Journal of Science
Education & Technology 16, 5-29.
Squire, K. & Klopfer, E. (2007) Augmented reality simulations on handheld computers. Journal
of the Learning Sciences 16, 371-413.
SurveyMonkey. (1999-2011) SurveyMonkey. Retrieved from http://www.surveymonkey.com/
Sweetser, P. & Wyeth, P. (2005) GameFlow: a model for evaluating player enjoyment in games.
Computers in Entertainment, 3(3), 1-24.
Tai, R. H., Liu, C. Q., Maltese, A. V. & Fan, X. (2006) Planning early for careers in science.
Science 312, 1143-1144.
Vygotsky, L.S. (1978) Mind in society: development of higher psychological processes, 14th
edition. Harvard University Press, Cambridge, MA.
Walker, C. J. (2010) Experiencing flow: Is doing it together better than doing it alone? The
Journal of Positive Psychology, 5(1), 3-11.
Paper Presented at NARST 2013 20
Appendix A.
Pre-survey
Part 1: Demographics
1. Enter your game ID
2. How old are you?
11 [ ] 12 [ ] 13 [ ] 14 [ ] 15 [ ] 16 [ ]
3. What grade are you in?
6th grade [ ] 7
th grade [ ] 8
th grade [ ]
4. What is your gender?
Male [ ] Female [ ]
Part 2: Interest in science
5. This part of the survey contains a number of statements about science. There is no ―right‖
or ―wrong‖ answer. For each statement, select the column that represents how you feel
about that statement.
Statement I strongly
agree I agree
I don’t
know I disagree
I strongly
disagree
1. Science lessons are fun. 2. I would like to belong to a science
club.
3. I would like to be given a science
book or a piece of science equipment
as a present.
4. Science is one of the most
interesting school subjects.
5. I would like to do science
experiments at home.
6. Talking to my friends about science
after school would be boring.*
7. I would enjoy having a job in a
science laboratory during my summer
vacation.
8. The material covered in science
lessons is NOT interesting.*
9. I would enjoy visiting a science
museum on the weekend.
10. I would enjoy school more if there
were NO science lessons.*
Note. (*) denotes item requiring reverse coding
Paper Presented at NARST 2013 21
Part 3: Gaming attitude
6. This part of the survey contains a number of statements about gaming. There is no ―right‖
or ―wrong‖ answer. For each statement, select the column that represents how you feel
about that statement.
Statement I strongly
agree I agree
I don’t
know I disagree
I strongly
disagree
1. Given the opportunity to play a
real-time strategy game, I am afraid
that I might have trouble in navigating
through it.*
2. Games help me relax and do my
work better.
3. I could probably teach myself most
of the things I need to know about
games.
4. I can make the computer do what I
want it to do while playing a game.
^ 5. I only use games when told to.* 6. I need an experienced person
nearby when I’m using a game.*
7. Playing games does not scare me at
all.
8. I avoid playing games.* 9. If I get problems using a game, I
can usually solve them one way or the
other.
10. I hesitate to use a computer or
mobile device for playing games as
I’m afraid of making mistakes I can’t
correct.*
11. I will use games regularly during
the next 4 to 5 years.
12. Games make me feel
uncomfortable.*
Note. (*) denotes item requiring reverse coding
Note. (^) denotes item dropped after initial regression analysis
Paper Presented at NARST 2013 22
Appendix B.
Post-survey
Part 1: Gameplay feedback
1. Enter your game ID
2. What role did you play?
Techie [ ] Science whiz [ ] Photographer [ ] Social networker [ ]
3. Did you identify the correct thief on the first try?
Yes [ ] No [ ]
4. Did playing this game make you curious to learn more about forensics,
using science to solve a crime?
Yes [ ] No [ ]
Part 2: Flow experience
5. This part of the survey contains a number of statements that relate to your feelings during
the game. There is no ―right‖ or ―wrong‖ answer. For each statement, think about the
game you just finished. Then, select the column that represents how you feel about each
statement.
Statement I strongly
agree I agree
I don’t
know I disagree
I strongly
disagree
1. I was challenged and I felt I could
meet the challenge.
2. I did things naturally without
thinking too much.
3. I had a strong sense of what I
wanted to do.
4. I felt I was on track towards my
goals.
5. I was totally focused on what I was
doing.
6. I felt I was in control of what I was
doing.
7. It felt like nothing else mattered. 8. I lost my normal sense of time. 9. I really enjoyed what I was doing. 10. I was in the zone.
Paper Presented at NARST 2013 23
Appendix C.
Interview questions
1. What did you think of the experience you just had?
2. What did you enjoy most about the experience?
3. What did you like least about the experience?
4. Now that you’ve played this game, what do you think about forensic science-which is using
science to solve a crime?
5. How did it go working with your fellow students?
6. What did you learn while playing, if anything?
7. Was this an interesting way to have a learning experience? Why?
8. Could you see yourselves making a game like this?
9. Do you have anything else you want to say about your experience today?