Investigating Student Flow Experience During a Mobile Augmented Reality Science Game

23
Paper Presented at NARST 2013 1 Investigating Student Flow Experience During a Mobile Augmented Reality Science Game Denise M. Bressler, Lehigh University, Bethlehem, PA 18015 USA Alec M. Bodzin, Lehigh University, Bethlehem, PA 18015 USA 2013 NARST Annual Conference April 6 9, 2013 Rio Grande, Puerto Rico Email: [email protected] Abstract Current studies have reported that secondary students are highly engaged while playing mobile augmented reality learning games. This study investigated factors related to students’ engagementas characterized by flow theoryduring a collaborative augmented reality (AR), forensic science mystery game using mobile devices. School Scene Investigators: The Case of the Stolen Score Sheets is a vision-based AR game played inside the school environment with quick response codes. A mixed methods approach was employed with 68 urban middle school students. Data sources included pre- and post-surveys, field observations, and group interviews. Results showed that neither gender nor interest in science were important predictors of variability in flow experience. Gaming attitude uniquely predicted 23% of the variance in flow experience. Student flow experience features included a flash of intensity, a sense of discovery, and the desire for higher performance. The findings demonstrated a potential for mobile AR science games to increase science interest and help students collaborate on learning tasks. Implications for future research concerning mobile AR games for science learning are discussed. Keywords: flow, engagement, game-based learning, quick response codes, augmented reality, mobile technology

Transcript of Investigating Student Flow Experience During a Mobile Augmented Reality Science Game

Paper Presented at NARST 2013 1

Investigating Student Flow Experience

During a Mobile Augmented Reality Science Game

Denise M. Bressler, Lehigh University, Bethlehem, PA 18015 USA

Alec M. Bodzin, Lehigh University, Bethlehem, PA 18015 USA

2013 NARST Annual Conference

April 6 – 9, 2013

Rio Grande, Puerto Rico

Email: [email protected]

Abstract

Current studies have reported that secondary students are highly engaged while playing mobile

augmented reality learning games. This study investigated factors related to students’

engagement—as characterized by flow theory—during a collaborative augmented reality (AR),

forensic science mystery game using mobile devices. School Scene Investigators: The Case of

the Stolen Score Sheets is a vision-based AR game played inside the school environment with

quick response codes. A mixed methods approach was employed with 68 urban middle school

students. Data sources included pre- and post-surveys, field observations, and group interviews.

Results showed that neither gender nor interest in science were important predictors of

variability in flow experience. Gaming attitude uniquely predicted 23% of the variance in flow

experience. Student flow experience features included a flash of intensity, a sense of discovery,

and the desire for higher performance. The findings demonstrated a potential for mobile AR

science games to increase science interest and help students collaborate on learning tasks.

Implications for future research concerning mobile AR games for science learning are discussed.

Keywords: flow, engagement, game-based learning, quick response codes, augmented

reality, mobile technology

Paper Presented at NARST 2013 2

Introduction

In 2011, the National Research Council published the report Learning Science through

Computer Games and Simulations and highlighted ―games as worthy of future investment and

investigation as a means to improve science learning‖ (p. 2). The report explained that the

current research on student learning with science games is inconclusive. The report called for

more research to increase the field’s understanding of the role that games can play in science

learning and to develop scalable designs (National Research Council 2011).

Research studies have shown that players are highly engaged while playing mobile

augmented reality (AR) learning games (Facer et al. 2004; Schwabe & Göth 2005; Perry et al.

2008; Dunleavy et al. 2009; Dunleavy & Simmons 2011). Some researchers have found that

engagement is related to flow (Park et al. 2010; Admiraal et al. 2011; Ryu & Parsons 2012).

However, previous studies have reported technology implementation and network issues that

inhibit the scalability of these games in school settings. In response, we designed and developed

a mobile AR learning science game with Quick Response (QR) codes that utilized free web-

based software to address these issues.

The gameplay relies solely on QR codes, a vision-based AR technology that has

enormous potential but remains insufficiently researched in school settings (Dunleavy & Dede in

press). The technology is vision-based since players use the embedded camera on their mobile

device as their portal to virtual gameplay and it is considered augmented reality because

scanning the codes trigger game-related content to appear on the player’s device not otherwise

viewable to the naked eye. The game was created with the ARIS platform (ARIS 2012). ARIS,

short for Augmented Reality and Interactive Storytelling, offers not only a free iOS application

but also allows anyone to freely create games using their own open-source, web-based

programming environment. The user-friendly, drag-and-drop editor enables first-time

programmers to develop mobile games. In this study, we designed and implemented a narrative-

driven, inquiry-based mobile AR game and examined factors that contributed to the flow

experience of urban middle school learners.

Theoretical Framework

Flow as a Measure of Engagement

Complete immersion in an activity can be characterized by the theory of flow as first

posited by Csikszentmihalyi (1975a). Flow is a positive psychological state that is challenging,

intrinsically rewarding, and enjoyable (Csikszentmihalyi 1975a). Research studies across

different activities with many different populations—such as rock climbers, artists, and

scientists—continue to find consistency in how people describe their flow experience

(Csikszentmihalyi 1975b, 1997). While in flow, people feel involved in meaningful actions,

maintain a sense of control, and stay focused on a goal. Csikszentmihalyi (1975a) stated that the

flow experience ―seems to occur only when a person is actively engaged in some form of clearly

specified interaction with the environment‖ (p. 43).

Video game researchers (Sweetser & Wyeth 2005; Cowley et al. 2008) and educational

researchers (Shernoff et al. 2003; Rathunde & Csikszentmihalyi 2005a) have used flow

assessments to measure engagement. Hoffman and Nadelson (2010) summarized the existing

research on achieving flow while gaming and concluded that ―gaming promotes intrinsic

motivation, positive affect, and many aspects of the flow experience‖ (p. 248). Longitudinal data

Paper Presented at NARST 2013 3

from Shernoff et al. (2003) revealed that students feel significantly more engaged when they are

challenged and in control of their learning.

More recently, researchers have studied flow in technology-mediated learning

environments. Cooper (2010) reviewed published studies of such environments and concluded

that, ―studies indicate a positive relationship between flow and learning in technology

environments‖ (p. 4). Researchers have also used flow as a measure of student engagement with

educational computer games (Inal & Cagiltay 2007; Kiili & Lainema 2008; Grimley et al. 2011)

and more recently, mobile learning games (Park et al. 2010; Admiraal et al. 2011; Ryu &

Parsons 2012).

Mobile Learning Games Mobile learning games, while still in their infancy, show potential for promoting science

learning (Squire & Klopfer 2007) and developing scientific literacy (Squire and Jan 2007).

Using authentic contexts and appropriate game design, these games can provide a situated

learning and collaborative problem solving environment (Rosenbaum et al. 2007; Dede 2009;

Fotouhi-Ghazvini et al. 2009). These learning environments are supported by mobile technology

features that include location-awareness, connectivity, portability, and 1-to-1 personalization

(Klopfer et al. 2002) and also support media-based learning (Dieterle et al. 2007).

Mobile leaning games can be designed for learners to experience science content within

an authentic context. According to Klopfer and Sheldon (2010), ―the power of AR…lies in truly

augmenting the physical landscape, using digital technologies to enable students to see the world

around them in new ways‖ (p. 86). For example, inquiry-based narrative-driven games have

been designed to put students in professional science roles where they engage in scientific

inquiry (Rosenbaum et al. 2007; Squire & Klopfer 2007; Dunleavy et al. 2009) and

argumentation skills (Squire & Jan 2007; Mathews et al. 2008).

Collaborative teaching strategies, such as jigsaw pedagogy (Aronson 1978), are an

effective paradigm for mobile learning activities (Lai & Wu 2006). Studies have reported that

interdependent, differentiated roles can be designed to effectively scaffold collaborative problem

solving within mobile learning games (Squire & Jan 2007; Dunleavy et al. 2009). Specifically,

when students play unique roles, they can receive role-specific, individualized bits of

information. They can then educate other group members about their findings and collaborate to

solve the problem presented within the game.

Purpose of the Study

Students find mobile learning games challenging and enjoyable—critical elements of flow.

However, more research is needed to understand factors pertaining to student engagement during

mobile AR games from the perspective of flow. Using a mixed methods approach, this study

investigated the following questions:

1. What are the features of the flow experience for middle school students while playing a

mobile AR science game?

2. What relationship do gender, interest in science, and gaming attitude have to middle school

students’ flow experience while playing a mobile AR science game?

Paper Presented at NARST 2013 4

Game Description

School Scene Investigators: The Case of the Stolen Score Sheets (henceforth SSI) is a

forensic science mystery game played with mobile devices in a middle school building. This

narrative-driven, inquiry-based game was played on iPhones supplied by the researchers. As

students moved throughout their school, they encountered QR codes that they scanned to access

game-related information (see Figure 1). Sometimes, they conversed with a virtual character

(see Figure 2). At other times, scanning a code meant discovering evidence (see Figure 3).

Throughout gameplay, players were exposed to elements of investigative forensic science, such

as analyzing fingerprints, hair and other trace evidence. They also collaboratively solved

problems such as decoding a locker combination and determining motives of suspects.

Figure 1. One student scans a

QR code while the other reads

information just received.

Figure 2. Screenshot of the

beginning of a conversation

with the principal.

Figure 3. Screenshot of the

hair sample discovered at the

crime scene.

The game design takes advantage of the affordances of mobile technology while

minimizing some limitations documented by previous research (Dunleavy et al. 2009; Fotouhi-

Ghazvini et al. 2009). First, gameplay took place entirely inside the school to minimize the

influence of environmental factors, such as rain, snow, or extreme temperatures. Second, since

previous research demonstrated that GPS error can cause technical difficulties and player

frustration, the game relied solely on the vision-based AR technology of QR codes. Third,

research has shown that mobile players enjoy determining the order of their missions (Ardito et

al. 2009), so non-linearity was built into the game. Fourth, Folta (2010) found that players did

not like the open-ended nature of the game designed for her study; therefore, SSI has a win

state—a point when players know they have successfully completed the game. Finally, the game

is location-based yet scalable because the game space was designed to utilize generic school

locations such as a library, gym, and principal’s office. Almost any school teacher can

implement the game by printing the QR codes and posting them in relevant locations. To play

the game, students need access to WiFi-enabled iOS devices loaded with the ARIS application

developed by the University of Wisconsin-Madison (ARIS 2012).

Paper Presented at NARST 2013 5

Gameplay Overview

Students played the game in groups of three and four. Each player had an iPhone. To

promote an authentic context, students had differentiated roles. The roles of social networker,

science whiz, techie, and photographer were assigned based on student interest. The game’s start

screen presented a two-sentence description of the character’s strengths and interests.

The main gameplay screen contained a button each

for quests, a decoder, and the inventory. Quests helped

players navigate the game; they told players where to go and

what to do (see Figure 4). Complete with audio clues and

onscreen alerts, quests were automatically triggered at

different points during gameplay, often by scanning QR

codes. When quests required moving to a new location,

players would find a new QR code and scan it with the

decoder. Scanning would often add a new inventory item

and a quest would pop-up requiring the player to view the

new item in the inventory. The game is a mystery to find

out who stole test answers from the principal’s office. The

mystery involves analyzing four pieces of evidence and

investigating four suspects to solve the crime. Quests were

differentiated based on a player’s role. Groups were

required to stay together and therefore had to collaborate to

solve the mystery and complete the game. Four different

versions of the game were developed and implemented,

each with a different thief.

Methods

Setting

SSI was implemented as an enrichment activity and played inside the participants’ school.

The school is located in a diverse, urban area of Pennsylvania with mostly low-income

households. A large majority of students (81%) participate in the free and reduced lunch

program. Middle school was chosen as the target population because it is a critical time to foster

intrinsic interest in science and build students’ self-confidence in their potential for success in

science careers. Research has shown that girls’ interest in science decreases from age 12 to 16

(Lindahl 2007) and experience with science before high school plays an important role in

whether students go into science-related careers (Tai et al. 2006).

Participants The participants included 68 students who volunteered for the study from two science

teachers’ classes and the after-school program. The sample population included 51.5% male

(n=35) and 48.5% female (n=33) students. Participants ranged in age from 11- to 15-years-old.

Several grade levels were represented including eighth grade (47.1%), seventh grade (29.4%),

and sixth grade (23.5%).

Students played in groups of three and four and experienced the game one group at a

time. Group composition was created by the classroom teacher dictated mostly by which

students had returned consent forms. A total of 19 groups participated; 17 groups played during

Figure 4. Screenshot of a quest

that students receive during the

game.

Paper Presented at NARST 2013 6

school hours and two groups played after-school. Most groups were mixed gender (n=14);

however, there were three girl-only groups and two boy-only groups. Data collection took place

during a two-week period during March 2012.

Study Design

This study employed a mixed methods approach and used a convergent design with the

data-validation variant (Creswell & Plano Clark 2011). Data sources included: pre- and post-

surveys, field observations, and group interviews. All survey data was collected online using

Survey Monkey (2011), a web-based survey application.

Pre-survey. Immediately before starting the game, players completed the pre-survey

(see Appendix A). There were three parts to the pre-survey including, demographics, interest in

science, and gaming attitude. Interest in science was measured using 10 Likert-style items

derived from the Test of Science Related Attitudes (Fraser 1981), a valid and reliable instrument.

The complete instrument has seven scales; however, this study utilized only two scales:

enjoyment of science lessons and leisure interest in science. From the 20 original items, 10

questions were eliminated to reduce repetition about science lessons and to remove references to

out-of-date communication technologies. Negative items required reverse coding. Possible

scores ranged from 10 to 50. Cronbach’s alpha was .90 for this adapted instrument.

The third part of the pre-survey measured gaming attitude using 12 Likert-style items.

Items were selected from Bonanno and Kommers’s (2008) instrument that was developed to test

for different attitudinal components towards gaming. Their original instrument contained 21

items but their results showed significant Pearson chi-square tests for only 12 items. Those

significant items were selected although wording was adjusted slightly to eliminate references to

specific game titles and grade levels in school. Negative items required reverse coding. When

items were tested for reliability, results indicated one item should be dropped to increase

Cronbach’s alpha. The item (―I only use games when told to.‖) was deemed ambiguous and not

indicative of gaming attitude. The resulting instrument contained 11 items with a Cronbach’s

alpha of .73. Post-survey. Immediately after the game concluded, players completed the post-survey

(see Appendix B). The post-survey collected basic gameplay feedback and assessed flow. First,

students indicated which role they played and whether they felt more curious about forensics.

Second, flow experience was measured using 10 Likert-style items similar to the Short Flow

State scale (S FSS-2) developed by Jackson et al. (2010). The scale measures nine elements of

flow as outlined by Csikszentmihalyi (1996). All nine original items were used; however,

wording was adjusted to enhance understanding for urban middle school students. One question

was also selected from Jackson et al.’s (2010) Core Flow State scale (C FSS-2). This item was,

―I was in the zone.‖ It was included to represent the player’s overall experience of being in flow.

Possible scores ranged from 10 to 50. Cronbach’s alpha for this instrument was .80.

Field observations. To collect field observations during gameplay, a researcher

followed players throughout the school. While students were playing, the researcher recorded

observations about students’ focus on the task, physical movement between locations,

technology use, and collaboration. The researcher also chronicled dialogue when possible and

documented individual behaviors, group behaviors, and atypical interactions.

Group interview protocol. Immediately after finishing the post-survey, students

participated in a group interview. Since the game was designed to scaffold collaborative

problem solving, students were interviewed together to draw out inferences about their

Paper Presented at NARST 2013 7

collaboration. The interview protocol (see Appendix C) included several broad questions about

the gameplay experience and specific questions intended to illuminate further details concerning

post-survey answers. Interviews were audio recorded and later transcribed by the researcher.

Data Analysis

Data sets were analyzed independently. A multiple hierarchical regression analysis was

conducted to illuminate how much variance in flow experience was predicted by gender, interest

in science, and gaming attitude. A qualitative analysis uncovered features of the students’ flow

experience. Transcripts were coded and analyzed using ATLAS.ti, a qualitative analysis

software package. The researcher started with a provisional list of 25 descriptive codes based on

the conceptual framework, research questions, field observations, and group interview protocol

(Miles & Huberman 1994). Employing the constant-comparative method (Glaser & Strauss

1967), the researcher reviewed transcripts by starting with open-coding, then revisiting

transcripts to incorporate new codes and check coding of the original data. Eventually, the final

eight transcripts served as a confirmation sample in order to test for code saturation (Glaser &

Strauss 1967).

Throughout the coding process, detailed memos were kept that included possible

relationships between variables. With each level of coding, memos became more detailed,

developing into testable hypotheses. After coding the confirmation sample, hypotheses were

grouped under the broader themes and combined with hypotheses built while in the field.

Guided by the hypotheses, the researcher examined the data searching for co-occurrences of

codes. A member check confirmed dependability of findings.

Findings

Since the qualitative and quantitative data sets were triangulated to confirm the reliability

of the outcome variable (flow experience), the findings are presented as a merged analysis. Prior

to statistical analysis, aggregate scores were calculated for interest in science, gaming attitude,

and flow experience. Table 1 displays descriptive statistics for these aggregated scores.

Table 1 Descriptive statistics for predictors and flow experience (N = 68)

Variable M SD Observed

Range

Possible

Range

Interest in science

31.88 8.50 10.00-50.00 10.00-50.00

Gaming attitude 44.59 5.41 34.00-55.00 10.00-55.00

Flow experience 41.21 5.30 30.00-50.00 10.00-50.00

Note. Gender was excluded from this table because it is a dichotomous variable.

A preliminary one-way ANOVA test was conducted to determine if the player’s role

influenced flow experience. All roles were integral to gameplay except the photographer; groups

could play with or without someone as the photographer. No significant differences were

observed in flow experience scores based on role. The ANOVA test verified that the non-

integral role of photographer was just as effective as the integral roles of social networker,

science whiz, and techie.

Paper Presented at NARST 2013 8

Flow Experience

Flow experience was measured using the post-survey (see Appendix B); results are

displayed in Table 2. The mean flow experience score was 41.2 out 50 indicating that the

average player had a substantive flow-like experience. Additionally, no scores fell below the

mid-point score of 30 signifying all players experienced some level of flow. Field observations

further supported students’ experiencing elements of flow; in particular, the researcher observed

students’ enjoyment and concentration on the task. Students also referenced flow elements

during focus group interviews. Table 2 includes student statements aligning to flow elements.

Table 2 Quantitative and qualitative support for participants’ flow experience

Statement I strongly

agree I agree

I don’t

know

I

disagree

I

strongly

disagree

Quotations from Group Interviews

I was

challenged and I

felt I could meet

the challenge.

41.2%

(n=28)

45.6%*

(n=31)

8.8%

(n=6)

1.5%

(n=1)

2.9%

(n=2)

You’re thinking real hard to figure

it out.

The most I actually had to think of

something.

It’s hard but if you set your mind

to it, you can do it.

I think it’s like chess. It improves

your brain or something.

I did things

naturally

without thinking

too much.

19.1%

(n=13)

47.1%*

(n=32)

16.2%

(n=11)

14.7%

(n=10)

2.9%

(n=2) Yeah, we winged it.

It felt like I was a cop.

You might not realize you’re doing

it, but you’re doing it.

I had a strong

sense of what I

wanted to do.

36.8%

(n=25)

39.7%*

(n=27)

19.1%

(n=13)

2.9%

(n=2)

1.5%

(n=1) Trying to like find something. That

makes you determined to like find

it.

It was - wow - now you have to do

this - and we were like running

around, finding those barcodes and

everything.

I felt in control

of what I was

doing.

45.6%*

(n=31)

33.8%

(n=23)

13.2%

(n=9)

7.4%

(n=5)

0.0%

(n=0) We could get out there and do it

ourselves.

It was kind of like independent

learning…in a way.

Like you can solve it yourself. You

don’t like need someone.

It teaches you how to do things on

your own. Like individual.

I felt I was on

track towards

my goals.

39.7%

(n=27)

44.1%*

(n=30)

7.4%

(n=5)

7.4%

(n=5)

1.5%

(n=1) It inspired me to keeping going.

We kept getting closer to our goal.

It made me want to keep playing

because I wanted to find out who

did it.

Paper Presented at NARST 2013 9

I was totally focused on

what I was doing.

52.9%*

(n=36)

33.8%

(n=23)

10.3%

(n=7)

2.9%

(n=2)

0.0%

(n=0) You had to think about it more.

Center on what you are actually

focusing on.

It felt like nothing else

mattered.

23.5%

(n=16)

29.4%

(n=20)

36.8%*

(n=25)

10.3%

(n=7)

0.0%

(n=0) I think we were so into it…we didn’t

even notice it was a game.

I think that it was challenging. And it

made me focus more. And I didn’t

really focus on anything else because

I wanted to beat the challenge of it.

I lost my normal sense

of time.

36.8%*

(n=25)

29.4%

(n=20)

22.1%

(n=15)

10.3%

(n=7)

1.5%

(n=1) Time went quickly.

It was like time consuming.

It felt like 5 minutes. [Researcher

Note: It took 25 minutes to

complete.]

I felt like it was so short. You know

what they say - time flies when

you’re having fun!

I really enjoyed what I

was doing.

66.2%*

(n=45)

29.4%

(n=20)

4.4%

(n=3)

0.0%

(n=0)

0.0%

(n=0) It was really fun - a nice experience

to have.

The most fun I had in school.

It felt like it wasn’t science. I felt like

it was a different subject. I felt like it

was the subject of fun.

I was in the zone. 55.9%*

(n=38)

32.4%

(n=22)

10.3%

(n=7)

1.5%

(n=1)

0.0%

(n=0) It was a rush!

Out of body experience.

I kind of forgot that it was a game. I

kept thinking we were going to win a

prize or something.

Note. (*) denotes category with highest percentage of responses

Features of Flow

The first research question investigated flow experience features for middle school

students while playing a mobile AR science game. Qualitative data results were used to examine

flow experience features. Participants’ ability to experience flow seemed to stem from three

experiential features: Participants felt a flash of intensity, a sense of discovery, and the desire for

higher performance. Table 3 displays student quotations from interviews that support these

features.

Table 3 Qualitative support for features of flow

Features of flow Quotations from students

Flash of intensity I liked it because it was different than the normal science class. Like, it

was better than regular science class.

I felt like it was more eye-opening. Instead of just like the usual.

It’s better than sitting in a class reading.

It’s different. It’s like unique. We weren’t sitting here. Just like listening

to you say something repeatedly.

Paper Presented at NARST 2013 10

Sense of discovery

It was like a scavenger hunt.

It was actually like a Sherlock Holmes type thing.

I liked going around school and looking for the clues.

We got to go on an adventure around the school.

Desire for higher performance Different codes. Like really hard ones that take an hour to figure out.

Could have been longer. Could have had a more in-depth storyline…And

more detailed evidence.

There should have been more suspects too…Like when there’s more

people it’s more, it’s harder, more complicated.

According to Csikszentmihalyi (1990), games can be considered flow activities because

they "offer opportunities to go beyond the boundaries of ordinary experience" (p. 72). More

specifically, Csikszentmihalyi (1997) explained that ―flow experiences provide the flashes of

intense living‖ (p.31) in comparison to the dull background of a typical day. This intense verses

dull comparison was seen repeatedly in the focus group transcripts. The majority of students

noted that the game offered something ―different‖ or ―unique‖ compared to their typical

classroom learning.

Studies on flow activities have shown they have a common element—they all provide a

sense of discovery (Csikszentmihalyi 1990). Students often referenced the idea of discovery

when describing their overall experience and favorite part of the game using terms such as

―adventure‖ or ―scavenger hunt.‖ They also regularly mentioned elements of discovery as

enjoyable parts of the experience with statements about ―looking for clues‖ and ―figuring it out.‖

The sense of discovery in flow activities also leads to higher levels of performance

(Csikszentmihalyi 1990). According to Csikszentmihalyi (1990), one cannot experience flow

while doing the same activity at the same level forever; to stay in flow, participants need to

continue discovering new challenges. During group interviews, several groups freely discussed

making the game more challenging. Unlike students in previous studies who experienced

cognitive overload (Dunleavy et al. 2009), these participants seemed to experience the right

challenge-skill balance and were eager to challenge themselves more the next time with ―more

clues‖ and a ―longer‖ experience.

Gender, Interest in Science, and Gaming Attitude The second research question investigated the relationship that gender, interest in science,

and gaming attitude have on middle school students’ flow experience while playing a mobile AR

science game. Data was checked for multicollinearity. Pearson correlations are reported in

Table 4. Since all the values were low enough, all the predictors were kept in the model. Next,

Cook’s D values were checked and no outliers were biasing the dataset. Finally, the data was

checked for homoelasticity; the standardized plot revealed an even distribution.

Table 4 Correlations between predictors and flow experience (N = 68)

Measure Gender Interest in

science Gaming attitude Flow experience

Gendera

- -.314 -.035 .035

Interest in science - .096 .167

Gaming attitude - .489

Flow experience - a Gender was dummy coded using male = 0 and female = 1.

Paper Presented at NARST 2013 11

Based on the literature, gender was a possible confounding variable, so it was added to

the hierarchical regression analysis as Block 1. Results are displayed in Table 5. Gender only

explained 0.1% of the variance in flow experience, p = .78. Field observations revealed that boy-

only groups (n=2) behaved somewhat differently than girl-only groups (n=3) during gameplay.

Overall, girls seemed more methodical; they interacted quietly, seemed to read every detail, and

walked between locations. Generally, boy groups seemed comfortable with the technology and

game format; they also moved steadily between locations, sometimes running. Although their

gameplay experiences seemed different to the observer, gender did not influence flow

experience. Focus group findings illuminated details to explain this discrepancy; gender may

dictate what aspects of the game one finds most enjoyable. During group interviews, boys

seemed more apt to discuss enjoyment in using the phones and scanning the codes (―I liked using

the phone [and] scanning the barcodes‖); girls referenced the storyline (―I liked the storyline‖).

Table 5 Results for multiple hierarchical regression analysis predicting flow experience (N = 68)

Note. B = unstandardized regression coefficient; SE = standard error; Part r = semi-partial correlation coefficient. a Gender was dummy coded using male = 0 and female = 1.

*** p < .001.

Interest in science and gaming attitude were added as Block 2 in the regression analysis.

Gender was still not significant, p = .38. Interest in science also did not significantly predict

variance in flow experience, p = .18. However, gaming attitude predicted significant variability

in flow experience, p < .001. The squared semi-partial (part) correlation of gaming attitude was

.226; therefore, gaming attitude uniquely predicted 23% of the variance in flow experience.

Focus group data supported the finding that interest in science did not predict flow

experience. During group interviews, students expressed varied levels of interest in science.

Some students expressed moderate or high interest in science, with statements such as ―I love

science‖ or ―I like science.‖ Other students admitted to disliking science (―Yeah, I don’t like

science‖) and were often surprised at their enjoyment in the game, since it was a science game.

Emerging Themes

Two additional themes, building better relationships and increased interest in

investigative science emerged during analysis offering insight into the benefits of mobile AR

science games.

Model 1 Model 2

Variable B SE B Part r B SE B Part r

Gender a 0.365 1.296 0.035 1.049 1.191 0.095

Interest in science 0.095 0.071 0.145

Gaming attitude 0.468*** 0.106 0.475***

R2 0.001 0.262***

ΔR2 0.261***

Paper Presented at NARST 2013 12

Building better relationships. The game was designed to scaffold collaborative

problem solving and students stated that working with their fellow classmates went well.

Several groups were truly surprised by how well they worked together. Some students said they

normally argued with each other in class, yet they actually worked well as a team. As one girl

reflected on her gameplay experience, she stated, ―I thought we were going to argue or

something. But we actually didn’t.‖ Others indicated they did not necessarily associate with

each other, but they cooperated fine. One participant stated, ―I’m actually surprised we worked

well, because we don’t kinda hang out together.‖ In one extreme case, two boys admitted to not

getting along yet they both agreed they did a good job together during the game. Field

observations of these supposed adversaries showed that their group interactions were some of the

best of any group in the sample (see Figure 5).

Figure 5. Students working together to share virtual evidence collected during gameplay.

Increased interest in investigative science. As mentioned earlier, interest in science

was not a predictor of flow experience. However, after gameplay, 93% (n = 63) of players said

they were curious to learn more about forensics. Focus group data supported this dramatic

increase in interest: the way the participants experienced science within the game seemed to

change their perception of science which increased their interest.

Some students were going to forego playing the game because they thought science was

boring. Students in traditional learning environments spend ample time in didactic settings

(Rathunde & Csikszentmihalyi 2005b) and they are not engaged by the autocratic way science is

presented (Lyons 2006). One participant stated, "I honestly thought [the game] was going to be

boring. Like…because, you know, it’s science." Others were willing to give it a try, but only

Paper Presented at NARST 2013 13

because it was a game. One student stated, ―I wouldn’t mind if we played a game about science

but if it’s just science, I don’t want to do it.‖

After playing the game, students repeatedly discussed how they were able to have fun

and learn at the same time. The experiential nature of the game seemed to really excite them.

The game experience, which was authentic and investigative, seemed to shift their perception of

science. Some students discussed how science is usually boring to them but the game made it

fun. A few students explained in their own words how the pedagogy of the game more

accurately represented science. One student explained that, ―It’s a way closer look at science

than we get taught.‖ Finally, a seventh grade girl made this comment after the group interview

already concluded: "Thanks for spending your time here…I mean you like showed us that

science isn’t only writing and reading. It’s actually experimenting."

Discussion

Using a mixed methods approach, this study examined middle school students’ flow

experience while playing School Scene Investigators, a mobile AR science game. Results

showed that neither gender nor interest in science was important in predicting flow experience.

However, gaming attitude uniquely predicted 23% of the variance. Focus group transcripts

revealed important features of the flow experience: Participants felt a flash of intensity, a sense

of discovery, and the desire for higher performance. The game’s ability to help players build

better relationships and increase their interest in investigative science surfaced as emerging

themes.

Novelty Effect

The novelty effect is a reality when introducing a project of this nature to an underserved

audience. However, in this study, novelty manifested itself in an unusual way. Students did not

talk about the new technology as a primary driver of their enjoyment. Rather, running around

the hallways while classes were in session was the novel experience. The excitement of doing

something normally taboo was encapsulated in this comment: ―Running around school…it was

fun…it was like a playground!"

There was also something intrinsic within the pedagogy that seemed novel as well.

Participants’ comments revealed the qualities that made this a desirable learning experience: it

was an adventure, it required thinking, it facilitated teamwork, and one could have fun and learn

simultaneously. Players also highlighted the independent nature of the learning experience and

that it felt ―real.‖ The technology supported the novelty of an investigative, experiential way to

learn and promoted neomillennial learning styles (Dieterle et al. 2007).

Game Design Promotes Flow Experience

Students playing SSI were highly engaged; they were experiencing flow. Game design

played a large role in this outcome. First, the use of QR codes and wireless Internet decreased

technical frustrations experienced with hardware and software issues reported in previous studies

that used location-aware technologies such as GPS-enabled smartphones (Facer et al. 2004;

Dunleavy et al. 2009; Admiraal et al. 2011). In this study, field observations showed that

players occasionally got bumped back to the iPhone’s home screen or stuck on the decoder

screen, but it was never enough to induce frustration. This aligns with Csikszentmihalyi’s (1990)

idea that people in flow will temporarily tolerate a lack of stimulation if they believe stimulation

will resume shortly. Second, the vision-based AR made navigation easier. Park et al. (2010)

Paper Presented at NARST 2013 14

found that "mobility may have negative influences on learning performance, in that it asks the

learner to attend to more than one information source when using the handheld device" (p. 59).

However, the purely vision-based AR game format allowed players to focus on navigation or

information, rarely both at the same time. In this study, locational clues were embedded in

player quests, so before leaving each location, players knew where to go next. When players

were moving, they were not using the phones to navigate, thus reducing cognitive load. In other

studies with location-aware technologies (Dunleavy et al. 2009; Park et al. 2010) players needed

to navigate with the handheld while in motion, perhaps increasing cognitive load and decreasing

enjoyment. Third, the game design did not allow for social loafing or overpowering leadership,

thus collaboration was a source of enjoyment. Not all students had a quest at the same time; this

forced leadership to shift between different group members where no single member could

dominate group dynamics or slack off. This design meant all players equally contributed to the

overall success of the mission. Shifting quests between roles effectively leveraged the benefits

of cooperative learning with the jigsaw technique (Aronson & Patnoe 2011) along with social

constructivist learning (Palincsar 1998).

Predictors of Flow Experience Gaming attitude predicted 23% of the variance in flow experience. Research has shown

that action video gamers have improved cognitive abilities over non-gamers, especially in multi-

tasking (Green et al. 2012). Since SSI requires multi-tasking, gamers may achieve a better

balance between challenge and skill, thus reaching a more substantive flow experience.

Previous research has reported that boys are more interested in science (Gardner 1975, 1998) and

have more positive attitudes about gaming (Bonanno & Kommers 2008). Hoffman and

Nadelson (2010) found that gender was a significant predictor of engagement in gaming with

males twice as likely to be engaged as females. Since SSI was a science game, there was a

strong case that gender would predict flow experience—but it did not. Our findings were similar

to those reported by Ketelhut and Nelson (2010) who found that both boys and girls were equally

engaged in a narrative-driven, inquiry-based multi-user virtual environment. The answer to

equal engagement may again lie in game design, specifically, the narrative-driven, inquiry-based

design. Research has reported that girls prefer narrative-based computer games (Gorriz &

Medina 2000). Furthermore, game narratology plays an important role in whether girls achieve

flow while playing (Inal & Cagiltay, 2007). Whereas, for boys, the key elements for achieving

flow are the rules and gameplay characteristics. Therefore, this design-type of educational

gaming may be engaging both genders equally because it entices girls through narrative, but

offers interesting game characteristics that appeal to boys.

Gameplay Increased Science Interest By exposing students to an investigative science experience, gameplay increased interest

in forensic science, perhaps even science in general. This finding is also supported by Beylefeld

and Struwig (2007) who reported that their microbiology game positively influenced players’

attitudes and perceptions of the science content within the game. One student stated during the

focus group that playing SSI could get kids into "science to begin with, because personally I

don’t like science." While another mentioned, "it could make kids want to go into forensics."

This is a promising finding because experiences that peak students’ early interest in science lead

to long-standing interest in science (Lindahl 2007; Maltese & Tai 2010). Consequently, if

Paper Presented at NARST 2013 15

science games played with mobile AR technology increase interest in science, middle school

students playing AR science games could help promote a more scientifically-literate citizenship.

Critical Features of Flow Experience: Interdependence and Challenge

When asked about the overall game experience, students often stated: ―It was fun.‖

Delving deeper, students particularly enjoyed working together. This finding is supported by

previous research about flow occurring in groups, known as social flow. According to Walker

(2010), experiencing flow as a group is more enjoyable than experiencing it alone. In a related

study, Admirral et al. (2011) found that team flow in gameplay explained 24% of the variance in

group game performance. Playing the game as a group is not enough; the successful nature of

team collaboration hinges tremendously on positive interdependence. Walker (2010) found that

people who collaborated interdependently experienced more enjoyment in flow than those who

worked with less interdependence. Overall, given the interdependence of the players’ roles in

SSI, students were able to experience a lot of enjoyment from their social flow.

As mentioned earlier, game characteristics seemed to promote the right challenge-skill

balance for players to experience flow. Field observations of the decoding activity in the

mystery game yielded evidence of a suitable challenge-skill balance; many groups were very

challenged by decoding the locker combination, yet no one complained about the difficulty.

Additionally, players read a lot during gameplay—something they admitted to being bored by in

the classroom—yet very few mentioned the amount of reading as a complaint about the game.

This observation differed from findings in a previous study by Admirral et al. (2011) who found

students were reluctant to read text messages in their game.

Going beyond reading comprehension and tackling in-game challenges, participants

enjoyed having to think. While discussing their favorite part, participants referenced using your

―brain‖ and having to ―think about it more;‖ they also enjoyed ―figuring it out.‖ Additionally,

students were eager to experience future challenges demonstrating an increase in desire for

higher levels of performance—a direct result of their flow experience. Furthermore, learners

may wish to take on more complex game challenges when an instructional game provides

students with their own unique learning paths (Ryu & Parsons 2012). In summary, students were

undeterred by the challenge of solving the mystery and being challenged seemed critical to their

enjoyment of the gameplay experience.

Implications and Conclusion

This study’s findings contribute to the literature by providing support for a middle school

player’s flow experience while playing a mobile AR science game with QR codes. Gaming

attitude was an important significant predictor of flow experience; gender and interest in science

were not important predictors. The interdependent nature of gameplay seemed to increase social

flow. This study suggests that playing narrative-driven, inquiry-based science games can

increase interest in science for both genders. Since players desired higher levels of performance,

the study also indicates a role for these games in learning; while playing, students persevere at

challenges within their zone of proximal development (Vygotsky 1978), and upon success, desire

greater challenges.

This research also investigated vision-based AR, an AR technology lacking extensive

research, and determined its potential as a scalable design for school settings. The technology

was very stable; this minimized player frustration and may have increased enjoyment. Game

design reduced cognitive overload by enabling players to focus on either navigation or

Paper Presented at NARST 2013 16

information, rarely both at the same time. Gameplay took approximately 30 minutes and setup

required only a few minutes to post QR codes; therefore, similar games could be played within a

class period. Implementation also does not require additional chaperones since students never

leave the school building. Additionally, mobile devices only need wireless Internet access, not

cellular plans. Thus, implementing AR learning games with QR codes reduces overall financial

costs for schools seeking to invest in mobile technology use.

This study had several limitations. First, the small sample size was obtained from one

urban middle school; therefore, care must be taken when generalizing to other contexts. Within

this sample, all the kids had positive gaming attitudes; in a different sample, gaming attitude may

not be as predictive of flow experience. Second, groups played one at a time; therefore,

competition was practically non-existent. Increased competition could diminish players’ flow

experience. Several groups playing simultaneously might also change the logistics, such as

increasing wait times to scan codes, which could lessen player enjoyment. Third, this was an

enrichment activity and not a required classroom activity, so player enthusiasm could be

attributable to the novelty. Finally, forensic science has been popularized by the media and the

narrative was gender-neutral; a different storyline with another science focus could reduce

enjoyment for one or both genders.

Future research should further explore findings reported by this study, chronicle different

game designs, and determine other elements that contribute to players’ flow experience. Future

studies may wish to focus on how gender as a predictor of flow experience may change when

students work in single gender verses mixed gender groups. To investigate scalability, vision-

based AR game designs need to be tested with multiple groups playing simultaneously. Since

this game design takes place in school, future game designs could integrate actual lab

experiments within the context of gameplay. Additionally, since sparking early science interest

can lead to science careers (Tai et al. 2006), future research should study the short-term and

long-term impact on students’ interest in science after playing games of this genre. Overall, this

study makes a strong case for vision-based AR learning games as scalable design for schools to

utilize since players experience flow while playing and desire higher levels of performance.

Paper Presented at NARST 2013 17

References

Admiraal, W., Huizenga, J., Akkerman, S. & Ten Dam, G. (2011) The concept of flow in

collaborative game-based learning. Computers in Human Behavior 27, 1185-1194.

Ardito, C., Buono, P., Costabile, M. F., Lanzilotti, R. & Piccinno, A. (2009) Enabling interactive

exploration of cultural heritage: an experience of designing systems for mobile devices.

Knowledge, Technology & Policy 22, 79-86.

ARIS. (2012) Retrieved May 12, 2012, from http://arisgames.org/.

Aronson, E. (1978) The jigsaw classroom. Sage Publications, Beverly Hills, CA.

Aronson, E. & Patnoe, S. (2011) Cooperation in the classroom: The jigsaw method, 3rd edition.

Pinter & Martin Ltd, London.

Beylefeld, A. A. & Struwig, M. C. (2007) A gaming approach to learning medical microbiology:

students' experiences of flow. Medical Teacher 29, 933-940.

Bonanno, P. & Kommers, P. A. M. (2008) Exploring the influence of gender and gaming

competence on attitudes towards using instructional games. British Journal of

Educational Technology 39, 97-109.

Cooper, K. (2010) Go with the flow: engagement factors for learning in Second Life. Paper

presented at the 2010 Spring Simulation Multi-Conference, Orlando, Florida.

Cowley, B., Charles, D., Black, M. & Hickey, R. (2008) Toward an understanding of flow in

video games. Computers in Entertainment 6, 1-27.

Creswell, J. & Plano Clark, V. (2011) Designing and conducting mixed methods research, 2nd

edition. Sage Publications, Thousand Oaks, CA.

Csikszentmihalyi, M. (1975a) Play and intrinsic rewards. Journal of Humanistic Psychology 15,

41-63.

Csikszentmihalyi, M. (1975b) Beyond boredom and anxiety. Jossey-Bass, San Francisco.

Csikszentmihalyi, M. (1990) Flow: the psychology of optimal experience. Harper and Row, New

York.

Csikszentmihalyi, M. (1996) Creativity: flow and the psychology of discovery and invention.

Harper Collins, New York.

Csikszentmihalyi, M. (1997) Finding flow: the psychology of engagement with everyday life.

Harper Collins, New York.

Dede, C. (2009) Immersive interfaces for engagement and learning. Science 323, 66-69.

Dieterle, E., Dede, C. & Schrier, K. (2007) ―Neomillennial‖ learning styles propagated by

wireless handheld devices. In Ubiquitous and pervasive knowledge and learning

management: Semantics, social networking and new media to their full potential (eds M.

Lytras & A. Naeve), pp. 35–66. Idea Group, Hershey, PA.

Dunleavy, M. & Dede, C. (in press) Augmented reality teaching and learning. In Handbook of

Research on Educational Communications and Technology, Volume 2, 4th edition (eds

J.M. Spector, M.D. Merrill, J. Elen & M.J. Bishop). Macmillan, New York.

Dunleavy, M., Dede, C. & Mitchell, R. (2009) Affordances and limitations of immersive

participatory augmented reality simulations for teaching and learning. Journal of Science

Education and Technology 18, 7–22.

Dunleavy, M. & Simmons, B. (2011) Assessing learning and identity in augmented reality

science games. In Serious educational games assessment (eds L. Annetta & S. Bronack),

pp. 221–240. Sense Publishers, Rotterdam, The Netherlands.

Facer, K., Joiner, R., Stanton, D., Reid, J., Hull, R. & Kirk, D. (2004) Savannah: mobile gaming

and learning? Journal of Computer Assisted Learning 20, 399-409.

Paper Presented at NARST 2013 18

Folta, E. E. (2010) Investigating the impact on student learning and outdoor science interest

through modular Serious Educational Games: a design-based research study. (Doctoral

dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No.

3442557).

Fotouhi-Ghazvini, F., Earnshaw, R. A., Robison, D. & Excell, P. S. (2009) Designing augmented

reality games for mobile learning using an instructional-motivational paradigm. Paper

presented at the International Conference on CyberWorlds, Bradford, UK.

Fraser, B. (1981) TOSRA: test of science-related attitudes handbook. Australian Council for

Educational Research, Hawthorn, Victoria.

Gardner, P.L. (1975) Attitudes to science: a review. Studies in Science Education 2, 1–41.

Gardner, P.L. (1998) The development of males’ and females’ interests in science and

technology. In Interest and learning: Proceedings of the Seeon Conference on Interest

and Gender (eds L. Hoffmann, A. Krapp, K.A. Renninger & J. Baumert), pp. 41-57.

Institute for Science Education (IPN), Kiel, Germany.

Glaser, B. G. & Strauss, A. L. (1967) The discovery of grounded theory: strategies for

qualitative research. Aldine Publishing Company, New York.

Gorriz, C. M. & Medina, C. (2000) Engaging girls with computers through software games.

Communications of the ACM, 43(1), 42-49.

Green, C. S., Sugarman, M. A., Medford, K., Klobusicky, E. & Bavelier, D. (2012) The effect of

action video game experience on task-switching. Computers in Human Behavior 28,

984-994.

Grimley, M., Green, R., Nilsen, T., Thompson, D. & Tomes, R. (2011) Using computer games

for instruction: the student experience. Active Learning in Higher Education 12, 45-56.

Hoffman, B. & Nadelson, L. (2010) Motivational engagement and video gaming: a mixed

methods study. Educational Technology Research & Development 58, 245-270.

Inal, Y. & Cagiltay, K. (2007) Flow experiences of children in an interactive social game

environment. British Journal of Educational Technology 38, 455-464.

Jackson, S., Eklund, R. & Martin, A. (2010) The FLOW manual. Mind Garden, Inc., Queensland,

Australia.

Ketelhut, D. J. & Nelson, B. C. (2010) Designing for real-world scientific inquiry in virtual

environments. Educational Research, 52(2), 151-167.

Kiili, K. & Lainema, T. (2008) Foundation for measuring engagement in educational games.

Journal of Interactive Learning Research 19, 469-488.

Klopfer, E. & Sheldon, J. (2010) Augmenting your own reality: student authoring of science-

based augmented reality games. New Directions for Youth Development, Winter

2010(128), 85-94.

Klopfer, E., Squire, K. & Jenkins, H. (2002) Environmental detectives: PDAs as a window into

a virtual simulated world. Proceedings of the IEEE international workshop on wireless

mobile technologies in education, pp. 95-98, IEEE.

Lai, C. Y. & Wu, C. C. (2006) Using handhelds in a jigsaw cooperative learning environment.

Journal of Computer Assisted Learning 22, 284-297.

Lindahl, B. (2007) A longitudinal study of students' attitudes towards science and choice of

career. Paper presented at the National Association for Research in Science Teaching,

New Orleans.

Lyons, T. (2006) Different countries, same science classes: students’ experiences of school

science in their own words. International Journal of Science Education 28, 591-613.

Paper Presented at NARST 2013 19

Maltese, A. V. & Tai, R. H. (2010) Eyeballs in the fridge: sources of early interest in science.

International Journal of Science Education 32, 669-685.

Mathews, J., Holden, C., Jan, M.-F. & Martin, J. (2008) Sick at South Shore Beach: a place-

based augmented reality game as a framework for building evidence-based arguments.

Proceedings of 8th

International Conference on Learning Sciences 3, 89-90.

Miles, M. B. & Huberman, A. M. (1994) Qualitative data analysis: an expanded sourcebook,

2nd edition. Sage Publications, Thousand Oaks, CA.

National Research Council. (2011) Learning science through computer games and simulations.

The National Academies Press, Washington, DC.

Palincsar, A. S. (1998) Social constructivist perspectives on teaching and learning. Annual

Review of Psychology 49, 345-375.

Park, J., Parsons, D. & Ryu, H. (2010) To flow and not to freeze: applying flow experience to

mobile learning. IEEE Transactions on Learning Technologies, 3(1), 56-67.

Perry, J., Klopfer, E., Norton, M., Sutch, D., Sandford, R. & Facer, K. (2008, June). AR gone

wild: two approaches to using augmented reality learning games in zoos. Paper

presented at the 8th international conference for the learning sciences, Utrecht,

Netherlands.

Rathunde, K. & Csikszentmihalyi, M. (2005a) Middle school students' motivation and quality of

experience: a comparison of Montessori and traditional school environments. American

Journal of Education 111, 341-371.

Rathunde, K. & Csikszentmihalyi, M. (2005b) The social context of middle school: teachers,

friends, and activities in Montessori and traditional school environments. Elementary

School Journal, 106(1), 59-79.

Rosenbaum, E., Klopfer, E. & Perry, J. (2007) On location learning: authentic applied science

with networked augmented realities. Journal of Science Education & Technology 16, 31-45.

Ryu, H. & Parsons, D. (2012) Risky business or sharing the load? – social flow in collaborative

mobile learning. Computers & Education 58, 707-720.

Schwabe, G. & Göth, C. (2005) Mobile learning with a mobile game: design and motivational

effects. Journal of Computer Assisted Learning 21, 204-216.

Shernoff, D. J., Csikszentmihalyi, M., Schneider, B. & Shernoff, E. S. (2003) Student

engagement in high school classrooms from the perspective of flow theory. School

Psychology Quarterly, 18(2), 158-176.

Squire, K. & Jan, M. (2007) Mad city mystery: developing scientific argumentation skills with a

place-based augmented reality game on handheld computers. Journal of Science

Education & Technology 16, 5-29.

Squire, K. & Klopfer, E. (2007) Augmented reality simulations on handheld computers. Journal

of the Learning Sciences 16, 371-413.

SurveyMonkey. (1999-2011) SurveyMonkey. Retrieved from http://www.surveymonkey.com/

Sweetser, P. & Wyeth, P. (2005) GameFlow: a model for evaluating player enjoyment in games.

Computers in Entertainment, 3(3), 1-24.

Tai, R. H., Liu, C. Q., Maltese, A. V. & Fan, X. (2006) Planning early for careers in science.

Science 312, 1143-1144.

Vygotsky, L.S. (1978) Mind in society: development of higher psychological processes, 14th

edition. Harvard University Press, Cambridge, MA.

Walker, C. J. (2010) Experiencing flow: Is doing it together better than doing it alone? The

Journal of Positive Psychology, 5(1), 3-11.

Paper Presented at NARST 2013 20

Appendix A.

Pre-survey

Part 1: Demographics

1. Enter your game ID

2. How old are you?

11 [ ] 12 [ ] 13 [ ] 14 [ ] 15 [ ] 16 [ ]

3. What grade are you in?

6th grade [ ] 7

th grade [ ] 8

th grade [ ]

4. What is your gender?

Male [ ] Female [ ]

Part 2: Interest in science

5. This part of the survey contains a number of statements about science. There is no ―right‖

or ―wrong‖ answer. For each statement, select the column that represents how you feel

about that statement.

Statement I strongly

agree I agree

I don’t

know I disagree

I strongly

disagree

1. Science lessons are fun. 2. I would like to belong to a science

club.

3. I would like to be given a science

book or a piece of science equipment

as a present.

4. Science is one of the most

interesting school subjects.

5. I would like to do science

experiments at home.

6. Talking to my friends about science

after school would be boring.*

7. I would enjoy having a job in a

science laboratory during my summer

vacation.

8. The material covered in science

lessons is NOT interesting.*

9. I would enjoy visiting a science

museum on the weekend.

10. I would enjoy school more if there

were NO science lessons.*

Note. (*) denotes item requiring reverse coding

Paper Presented at NARST 2013 21

Part 3: Gaming attitude

6. This part of the survey contains a number of statements about gaming. There is no ―right‖

or ―wrong‖ answer. For each statement, select the column that represents how you feel

about that statement.

Statement I strongly

agree I agree

I don’t

know I disagree

I strongly

disagree

1. Given the opportunity to play a

real-time strategy game, I am afraid

that I might have trouble in navigating

through it.*

2. Games help me relax and do my

work better.

3. I could probably teach myself most

of the things I need to know about

games.

4. I can make the computer do what I

want it to do while playing a game.

^ 5. I only use games when told to.* 6. I need an experienced person

nearby when I’m using a game.*

7. Playing games does not scare me at

all.

8. I avoid playing games.* 9. If I get problems using a game, I

can usually solve them one way or the

other.

10. I hesitate to use a computer or

mobile device for playing games as

I’m afraid of making mistakes I can’t

correct.*

11. I will use games regularly during

the next 4 to 5 years.

12. Games make me feel

uncomfortable.*

Note. (*) denotes item requiring reverse coding

Note. (^) denotes item dropped after initial regression analysis

Paper Presented at NARST 2013 22

Appendix B.

Post-survey

Part 1: Gameplay feedback

1. Enter your game ID

2. What role did you play?

Techie [ ] Science whiz [ ] Photographer [ ] Social networker [ ]

3. Did you identify the correct thief on the first try?

Yes [ ] No [ ]

4. Did playing this game make you curious to learn more about forensics,

using science to solve a crime?

Yes [ ] No [ ]

Part 2: Flow experience

5. This part of the survey contains a number of statements that relate to your feelings during

the game. There is no ―right‖ or ―wrong‖ answer. For each statement, think about the

game you just finished. Then, select the column that represents how you feel about each

statement.

Statement I strongly

agree I agree

I don’t

know I disagree

I strongly

disagree

1. I was challenged and I felt I could

meet the challenge.

2. I did things naturally without

thinking too much.

3. I had a strong sense of what I

wanted to do.

4. I felt I was on track towards my

goals.

5. I was totally focused on what I was

doing.

6. I felt I was in control of what I was

doing.

7. It felt like nothing else mattered. 8. I lost my normal sense of time. 9. I really enjoyed what I was doing. 10. I was in the zone.

Paper Presented at NARST 2013 23

Appendix C.

Interview questions

1. What did you think of the experience you just had?

2. What did you enjoy most about the experience?

3. What did you like least about the experience?

4. Now that you’ve played this game, what do you think about forensic science-which is using

science to solve a crime?

5. How did it go working with your fellow students?

6. What did you learn while playing, if anything?

7. Was this an interesting way to have a learning experience? Why?

8. Could you see yourselves making a game like this?

9. Do you have anything else you want to say about your experience today?