Computers & Education 71 (2014) 206–221

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier .com/locate/compedu

Investigating the impact of an integrated approach to thedevelopment of preservice teachers’ technological pedagogicalcontent knowledge (TPACK)

Chrystalla Mouza*, Rachel Karchmer-Klein, Ratna Nandakumar, Sule Yilmaz Ozden,Likun HuSchool of Education, University of Delaware, 219 D Willard Hall, Newark, DE 19702, USA

a r t i c l e i n f o

Article history:Received 1 April 2013Received in revised form20 September 2013Accepted 23 September 2013

Keywords:Elementary educationImproving classroom teachingPedagogical issuesTeaching/learning strategies

* Corresponding author. Tel.: þ1 302 831 3108.E-mail addresses: cmouza@udel.edu (C. Mouza),

lhun@udel.edu (L. Hu).

0360-1315/$ – see front matter � 2013 Elsevier Ltd. Ahttp://dx.doi.org/10.1016/j.compedu.2013.09.020

a b s t r a c t

The purpose of this study is to describe an integrated pedagogical approach, aimed at advancing pre-service teachers’ learning on the use of technology and investigate its impact on participants’ knowledge(i.e., TPACK) and practice. The integrated approach juxtaposes an educational technology course withmethods courses and field experience through careful instructional design. Both quantitative andqualitative data were collected. Quantitative data were collected through a pre-post administration of theSurvey of Preservice Teachers’ Knowledge of Teaching and Technology. Qualitative data were collectedthrough open-ended survey responses and preservice teacher case narratives reporting on the designand implementation of technology-integrated lessons in a field placement. Finding revealed that par-ticipants experienced significant gains in all TPACK constructs. Further, findings indicated that partici-pants applied their knowledge in practice though there was variability in the ways in which knowledgedomains were represented in participants’ narratives. Findings have implications for teacher educationprograms and for researchers interested in the development and assessment of preservice teacherknowledge of teaching with technology.

� 2013 Elsevier Ltd. All rights reserved.

1. Introduction

Common Core Standards, now adopted by 45 states across the U.S., place increased attention to the use of new technologies as a way toacquire knowledge and skills in core subject matter areas (Common Core State Standards Initiative, 2010). The standards are not onlydesigned to promote benchmarks of student achievement in literacy and mathematics, but they are also intended to reflect the knowledgeand skills students need to succeed in college, the workplace, and life in a technological society (Common Core State Standards Initiative,2010). This shift makes it necessary for new teachers to enter the classroomwith the knowledge and skills required to design and implementrigorous standards-based lessons that emphasize strategic use of technology in support of curricular goals (Lawless & Pellegrino, 2007).

Existing research indicates that a critical factor influencing new teachers’ adoption of technology is the quantity and quality of tech-nology experiences included in their teacher education program (Agyei & Voogt, 2011; Tondeur et al., 2012). Although this generation ofpreservice teachers is more technologically savvy and actively engaged with digital media, knowledge and skills alone are not sufficientconditions for curricular use of technology in support of rigorous standards (Lei, 2009; Margaryan, Littlejohn, & Vojt, 2011). Niess (2008), inparticular, argues that since this generation of future teachers has not traditionally experienced their own content learning with digitaltechnologies, they need specialized instruction on how to teach their core contentwith technologywhile simultaneously guiding students inlearning about new forms of technology.

Recent calls have indicated that to prepare prospective teachers for effective use of technology, teacher education programs must helpthem build knowledge of content, good pedagogical practices, and technical skills as well as an understanding of how these constructsinteractively relate to one another (Koehler &Mishra, 2008). In fact, the interactions among content, pedagogy and technology form the core

karchmer@udel.edu (R. Karchmer-Klein), nandakum@udel.edu (R. Nandakumar), shule@udel.edu (S. Yilmaz Ozden),

ll rights reserved.

C. Mouza et al. / Computers & Education 71 (2014) 206–221 207

of what has been called technological pedagogical content knowledge (TPACK), a distinct type of flexible knowledge required for effectiveuse of technology in classroom teaching (Angeli & Valanides, 2009; Mishra & Koehler, 2006; Niess, 2005).

While researchers and teacher educators have embraced the TPACK framework with excitement (Graham, 2011), recommendations onhow to develop and assess TPACK vary widely in the literature (Koehler, Shin, & Mishra, 2012; Polly, Mims, Shepherd, & Inan, 2010; Tondeuret al., 2012). Further, limited research exists to date examining the actual technology topics addressed across teacher education institutions,the impact of these topics on teaching practice, and the empirical basis for the inclusion of these topics within the teacher educationcurriculum (Hew & Brush, 2007; Lawless & Pellegrino, 2007; Ottenbreit-Leftwich et al., 2012).

2. Purpose and significance of the study

The purpose of this study is to describe a pedagogical approach, aimed at advancing preservice teachers’ learning on the use of tech-nology and investigate its impact on participants’ knowledge (i.e., TPACK) and practice. The approach discussed in the study consisted of aneducational technology course with a specific set of technology topics offered in conjunction with methods courses and field experience. Thisintegrated approach aimed at fostering both technological knowledge as well as knowledge related to the intersections of content, pedagogyand technology. Existing literature indicates that the stand alone educational technology course can benefit when connected to methodscourses and field experiences that allow prospective teachers to teach particular content with technology (Niess, 2005, 2012). Yet, fewstudies articulate how such integration can be accomplished to effectively leverage preservice teachers’ knowledge bases (e.g., Harrington,2008).

Using a combination of self-reported survey and performance assessment measures we answer the following research question: Howand to what extent does participation in an educational technology course, offered in conjunction with methods courses and field experience,influence preservice teachers' TPACK development and practice? In our work, we acknowledge that simply juxtaposing the educationaltechnology course with methods courses and field experience without careful instructional design may not necessarily ensure TPACKdevelopment. Thus, in our analysis we articulate how that integration was accomplished and the ways in which it leveraged preserviceteachers’ learning. Findings from this work help improve our understanding of effective pedagogical models, instructional topics, andstrategies that help advance preservice teachers’ learning on the use of technology as well as ways in which we can measure such learning.As Tondeur et al. (2012) note, “it is of great importance to examine how teacher education programs influence preservice teachers to usetechnology in their future classrooms” (p. 135).

3. Conceptual framework

This work is grounded in the theoretical framework of TPACK, which is used to describe the knowledge base required for effective use oftechnology in teaching and learning (see Fig. 1; Mishra & Koehler, 2006). Beginning with Pierson’s (2001) initial articulation of the idea,TPACK has emerged as a key construct in teacher preparation to emphasize a content-specific orientation to technology integration (Angeli& Valanides, 2005; Koehler & Mishra, 2005; Margerum-Leys & Marx, 2004; Niess, 2005). Building upon Shulman’s (1986) conception ofpedagogical content knowledge (PCK), the TPACK framework includes three core knowledge domains namely, content knowledge (CK),pedagogical knowledge (PK), and technological knowledge (TK). It also includes the interactions among these three core domains, whichform four additional knowledge domains namely, pedagogical content knowledge (PCK), technological pedagogical knowledge (TPK),technological content knowledge (TCK), and technological pedagogical content knowledge (TPACK). Contextual knowledge is also includedas a part of the model (Kelly, 2008). Each type of teacher knowledge represented in the framework is shown in Fig. 1.

Fig. 1. TPACK framework: http://tpack.org.

C. Mouza et al. / Computers & Education 71 (2014) 206–221208

Despite the wide acceptance of the framework, there are different conceptions of TPACK in the literature. In a recent review of the TPACKliterature, Voogt, Fisser, Roblin, Tondeur, and van Braak (2012) uncovered three common views of TPACK that have developed over time: (a)TPACK as extended PCK (see Niess, 2005); (b) TPACK as a new and distinct body of knowledge (see Angeli & Valanides, 2009); and (c) TPACKas the interplay among three domains of knowledge and their intersections (see Mishra & Koehler, 2006). In this work we employ acombination of the latter two views. Althoughwe acknowledge TPACK as a distinct body of knowledge that is more than the sum of its parts,we also recognize that it is rooted in the TPACK sub-domains described in the framework (i.e., CK, TK, PK, PK, PCK, TCK, and TPK). As a result,when examining participants’ knowledge development over time we measure both TPACK as a distinct entity and its individual sub-domains.

It is important to clarify, however, that TPACK is not static or fixed, but a dynamic and flexible body of knowledge influenced by bothrapid changes in technology and the bidirectional relationship between knowledge and practice (Cox & Graham, 2009; Doering, Veletsianos,Scharber, & Miller, 2009; Mouza, 2009). As such, TPACK takes many forms and variations in practice and can be further developed throughiterative cycles of application and reflection (Kinuthia, Brantley-Dias, & Clarke, 2010). In fact, through extensive observations and datacollection with in-service teachers Niess, Lee, and Sadri (2007) identified five levels of TPACK development that included recognizing(knowledge), accepting (persuasion), adapting (decision), exploring (implementation), and advancing (confirmation). Using these levels as aguide, Niess et al. (2009) identified descriptors that characterize teachers’work in relation to four key themes: curriculum and assessment,learning, teaching, and access.We believe that this developmental viewof TPACK is particularly appropriate for preservice teachers who lackteaching experience and have not yet had an opportunity to go through iterative cycles of application and reflection.

4. Background literature

4.1. Approaches to the development of preservice teachers’ TPACK

A common strategy used to advance preservice teachers’ TPACK is the delivery of a technology course, which has been consistentlyoffered since the early 1990s (Niess, 2012; Polly et al., 2010). Specifically, a survey of 1439 institutions with teacher education programs inthe U.S. revealed that 85% of them offer an educational technology course (Kleiner, Thomas, & Lewis, 2007). This course has frequentlyfocused on learning about different technologies (e.g., word processors, presentation software, the Internet) along with their affordancesand constraints. Although there are many benefits associated with this approach including the ability to improve preservice teachers’ self-efficacy, provide a good overview of the use of technology in teaching, and develop a strong foundation of technology skills, it has notresulted in preservice teachers’ implementation of digital technologies in their teaching (Kay, 2006; Niess, 2012). As a result, Mishra,Koehler, Shin, Wolf, and DeSchryver (2010), suggest retaining the technology course but reconsidering its curriculum. They proposeengaging teachers with rich pedagogical, technological and content problems that maintain the inter-relationships through approachessuch as learning-by-design. Learning-by-design allows teachers to work in collaborative groups developing technological solutions toauthentic pedagogical problems (Koehler & Mishra, 2005).

In one of the earlier efforts to advance TPACK development through learning-by-design, Koehler and Mishra (2005) studied faculty andmaster students working collaboratively to transform a face-to-face course into an online course. Similarly, using a project-based approachin the context of an educational technology course, Wetzel, Foulger, and Williams (2009) engaged small groups of preservice teachersworking together to master specific technology tools. Subsequently, participants were responsible for teaching other classmates about theirselected tool, provide a picture of its use in a future classroom, and begin to use the new tool to accomplish learning goals. Other researchershave identified content and technology-based approaches, such as instructional modeling and application (Niess, 2005), TPACK-based casedevelopment (Mouza &Wong, 2009), instructional design through technology mapping (Angeli & Valanides, 2009), action research, inquiryand self-reflection (Cavin, 2008; Pierson, 2008).

While the above approaches have provided needed insights into the development of TPACK, Niess (2012) acknowledges that we mustcontinue investigating methods and learning trajectories for integrating the development of TPACK within entire teacher preparationprograms. In this study we examine the impact of an integrative approach to TPACK development that carefully juxtaposes the educationaltechnology course with methods courses and field experience. We also identify trajectories of preservice teacher learning using a devel-opmental approach to TPACK, as described by Niess et al. (2009).

4.2. Approaches to measuring TPACK

As researchers began to empirically investigate the impact of their instructional approaches, the issue of how to capture levels of un-derstanding in TPACK became prominent (Koehler et al., 2012). To date, however, there is no widely accepted and generally applicableinstrument for measuring teachers’ TPACK (Albion, Jamieson-Proctor, & Finger, 2010). In a recent review of the literature that involved 66studies, Koehler et al. (2012) identified five commonly used techniques: self-report measures, performance assessments, open-endedquestionnaires, interviews, and observations.

Self-report measures such as surveys are one of the most frequently used methods to measure TPACK (Koehler et al., 2012). The mostwidely used survey instrument has been developed by Schmidt et al. (2009), and is directly linked to the TPACK framework. The survey hasbeen constructed specifically for use with preservice teachers. Construct validity suggests that the instrument is reliable and could be usedwith confidence among preservice teachers majoring in elementary education (Albion et al., 2010). In a similar effort, Archambault andCrippen (2009) created a survey instrument linked to the TPACK framework specific to teaching online while Lee and Tsai (2010) pro-posed a survey instrument focusing specifically on web technology.

Performance assessments evaluate TPACK by directly examining participants’ performance on given tasks that often are designed torepresent complex, authentic, real-life tasks (Koehler et al., 2012). The earliest attempt to measure TPACK through performance assessmenthas been described by Koehler and Mishra (2005) who analyzed conversations between student-teachers and faculty who had to designonline courses through the period of one semester. Their analysis looked at how TPACK elements were represented in team conversations at

C. Mouza et al. / Computers & Education 71 (2014) 206–221 209

two different points in time. The study indicated that over time the initially separate topics of content, pedagogy, and technology becamemore strongly interconnected.

Similarly, Mouza andWong (2009) looked for evidence of TPACK constructs (i.e., PK, CK, PCK, TK, TPK, & TCK) in teacher artifacts such ascase narratives, to judge the existence of TPACK while Angeli and Valanides (2009) and Graham, Borup, and Smith (2012) looked for evi-dence of TPACK through a design task assigned to preservice teachers. Specifically, Angeli and Valanides (2009) scored participants’ designtasks on five criteria that included identification of suitable topics, appropriate representations to transform content, identification ofteaching strategies difficult to implement by traditional means, selection of appropriate tools and their pedagogical uses, and identificationof appropriate integration strategies. Along the same lines, Graham et al. (2012) used TPACK as a lens for understanding how preserviceteachers make decisions about the use of technology in their teaching in three content teaching design tasks during a pre-and post-treatment assessment. Researchers identified themes from preservice teachers’ rationales that mapped to the TPACK constructs.

In their analysis, Koehler et al. (2012), found that open-ended questionnaires, interviews and observations were used less frequently inefforts to measure TPACK. For example, So and Kim (2009) asked participants to write brief responses to prompts which were later codedusing the TPACK frame as a guide while Ozgun-Koca (2009) interviewed teachers on the advantages and disadvantages of calculator usageand the effects on teaching and learning. Similarly, studies involving observations typically used field notes or video-tapes which were latertranscribed and coded following a TPACK-based coding scheme (e.g., Suharwoto, 2006).

Despite the vibrant body of research, Koehler et al. (2012) pointed to a significant concern namely the short shrift given to issues ofreliability and validity. In particular, 90% of the studies examined provided no evidence of validity while 69% of the studies examinedprovided no evidence of reliability. These findings echo concerns identified by Kay (2006) in an early effort to evaluate strategies used toincorporate technology into preservice teacher education. Specifically, Kay also found that the research methods utilized in many studies togauge effectiveness left a lot to be desired, often employing a small or non-existent sample size or surveys without reliability and validityestimates.

In this study, we use a combination of self-reported survey data and performance assessment measures to report on the effectiveness ofan educational technology course offered in conjunction with methods courses and field experience. We include important contextualinformation on the teacher education program, as well as the course itself, and use a survey instrument (Schmidt et al., 2009) that has beenfound reliable for our sample (see Methods).

5. Rationale and description of the integrated approach to TPACK development

5.1. Rationale for an integrated approach

In recent years, a number of teacher preparation programs strive to reinforce their stand alone educational technology course by infusingtechnology in methods coursework and field experience, providing a more integrated approach to the development of preservice teachersTPACK (e.g., Gronseth et al., 2010). Niess and colleagues (e.g., Niess, 2005; Niess & Gillow-Wilson, 2013; Suharwoto & Niess, 2006), inparticular, conducted a number of studies examining preservice teachers’ development of TPACK in integrated teacher preparation pro-grams for mathematics and science teachers. Results indicated that those programs were effective in developing participants’ TPACK as wellas transfer of learning to classroom educational experiences. Although these efforts are critical, theyweremostly conducted in the context ofsubject-specific teacher preparation programs for mathematics and science teachers. As a result, little is known about how such integratedapproaches can support TPACK development among preservice elementary teachers who will be teaching a variety of subject areas ormiddle school teachers prepared outside subject-specific programs. In this section we provide information on the teacher education pro-gram in this study and describe the methods courses, field experience and educational technology course. Subsequently we explain howwebrought those elements together into an integrated approach to support preservice teachers’ learning.

5.2. Teacher education program

The Elementary Teacher Education (ETE) program in this study is a four-year undergraduate program accredited by the National Councilfor the Accreditation of Teacher Education (NCATE). Graduates of the program are eligible for elementary (K–5) and middle school (6–8)teacher certification. The program curriculum is divided in three areas in terms of coursework; the general studies courses which developpreservice teachers’ content knowledge, the professional studies courses (e.g., methods) which prepare preservice teachers for their futureclassroom, and the concentration courses which help preservice teachers’ develop expertise in one of six concentration areas includingmiddle school English, mathematics, science, social studies, and special education. Consistent with contemporary research that advocatesthe necessity of field experiences in teacher education, the program curriculum is designed to provide preservice teachers with a range offield experiences in a variety of classroom settings beginning with freshman year.

5.3. Methods courses

Helping preservice teachers develop knowledge and skill in how to teach is a critical component of teacher preparation (NCATE, 2010).Effective teachers not only know their subject matter but they are also able to create a stimulating learning environment and applypedagogical strategies, including technology-rich strategies, that engage students while helping them improve their achievement (NCATE).The ETE program requires a number of methods courses in literacy, social studies, mathematics, and science. Those courses are divided inthree blocks that include the elementary block, middle school block, and special education block. All methods courses focus on curriculumand appropriate methods for teaching subject matter concepts to elementary or middle school students. Activities include review, devel-opment and evaluation of curriculum materials, teaching strategies including strategies that utilize technology, as well as researchassessment on student learning. Methods faculty model a variety of ways in which technology can be used in conjunction with specificcontent (e.g., literacy, mathematics, science, social studies) and pedagogical strategies.

C. Mouza et al. / Computers & Education 71 (2014) 206–221210

5.4. Field experience

Clinical preparation or field experience is a key component of teacher preparation. In fact, the Blue Ribbon Panel Report (2010)commissioned by NCATE placed clinical practice at the center of transforming teacher education. Acknowledging the importance of clin-ical practice, the ETE program employs a variety of field experiences including early field experiences, methods field experiences, andstudent teaching. Further, efforts are made to place preservice teachers in classrooms that demonstrate effective use of technology.

Early field experiences begin in the freshman year and provide preservice teachers the opportunity to observe experienced teachers andlearn about the classroom environment and how to interact with children.Methods field experiences are taken in conjunctionwith methodscourses designed to develop preservice teachers’ teaching skills and provide participants with the opportunity to teach lessons to an entireclass, including lessons that integrate technology. During that time, preservice teachers are in the field for three full weeks each semesterallowing them to observe and experience the daily routines of a classroom teacher including their use of technology, see growth in studentlearning, and develop strong relationships with cooperating teachers. Early and methods field experiences are designed to prepare pre-service teachers for student teaching where they gradually take over classroom responsibilities for a period of one semester.

5.5. Educational technology course and description of the integrated approach

The educational technology course, titled Integrating Technology in Education, runs for 15 weeks every semester and is required for allpreservice teachers within the ETE program. The purpose of the educational technology course is to (a) introduce prospective teachers totechnologies available for use in classroom content areas (e.g., concept mapping software, interactive manipulatives, Internet resources, andWeb 2.0 tools); (b) pedagogical considerations with these technologies; and (c) teaching and learning practices that effectively combinethese technologies with content and pedagogy. Course instructors model a variety of ways in which these technologies can be used tosupport student learning. Table 1 provides an overview of key course content and activities.

Our pedagogical approach to helping preservice teachers acquire a deeper understanding of the interactions among technology, contentand pedagogy involved simultaneous participation in the educational technology course, methods courses and field experience. Whendesigned carefully, this model is consistent with key markers of effective teacher preparation in the use of technology recently described inthe research literature (Tondeur et al., 2012). These markers include: (a) theory to practice connections, (b) opportunities for instructionaldesign, (c)modeling, (d) authentic experiences, and (e) opportunities for enactment and reflection. Below we articulate how the integration ofeducational technology coursework, methods courses and field experience was designed to meet these markers of effective teacherpreparation (see also Table 1). We also discuss the ways in which such integration leveraged changes in preservice teachers’ learning.

5.5.1. Theory to practice connectionsOne of our key goals in juxtaposing the educational technology course with methods courses and field experience was to facilitate a

theory to practice connection by allowing preservice teachers to connect what they learn from the university classroom to the schoolclassroom and vice versa. We have developed recursive communication with the methods faculty where we inform each other of the tools,strategies and ideas emphasized in the courses. This approach allows us to provide unified content that supports learning in important ways.

Table 1Description of educational technology course in relation to markers of effective teacher preparation.

Marker Activities/assignments Description

Instructional design Lesson critique Participants identify a technology-integrated lesson in a content area of their choice from a lesson-plan portal called Thinkfinity and prepare a critique discussing the instructional objectives of thelesson, the strengths and weaknesses of the learning activities, the content and technologystandards addressed, and the role of technology in relation to the lesson’s objective.

Lesson development Concept mapping. Participants practice using concept mapping software by writing a compare andcontrast essay. In small groups, they reflect on their experiences and generate ideas that integrateconcept mapping in a content area of their interest.Inquiry-based. Participants learn how to design inquiry-based activities aroundweb-based resourcesin order to reinforce or teach a literacy, social studies, mathematics or science concept. First,participants decide the content area and specific standards/skills they want students to practice(e.g., what should students know, understand or be able to do). Second, they search for web-basedresources that support students as they learn/practice these skills. Third, they design inquiry-oriented activities that engage students with the web resources as they learn content-specificconcepts. Finally, they describe the mechanisms in which student work will be assessed.Web 2.0 tools. In small groups, participants learn about web a 2.0 tool such as blogs, wikis, orpodcasts. Subsequently, they generate a lesson idea that focuses on the integration of the selectedtool in a classroom setting and prepare sample work using the selected tool (e.g., blog or wiki entry,podcast, etc.). Finally, in jigsaw groups participants share their work with their peers.

Authentic experiences Technology inventory Participants construct an inventory of technological resources available in their field placement. Theinventory needs to identify both hardware and software resources to help participants understandthe kinds of technologies typically available in K–12 settings and where they are located (e.g.,computer lab, classroom, library, etc.).

Classroom implementation In consultation with their cooperating teacher, participants implement a technology-integratedlesson developed in the course (e.g., concept mapping, inquiry) into their field placement classroom.

Reflection Case development Upon completing the implementation of a technology-integrated lesson in their field placement,participants develop a reflective case that discusses their experience following specific writing andreflection prompts.

Theory to practice Educational technology course integrated with methods courses and field experience to facilitate learning and implementation oftechnology with content and pedagogy.

Role models Educational technology faculty, methods faculty and cooperating teachers in the field model uses of technology.

C. Mouza et al. / Computers & Education 71 (2014) 206–221 211

For example, during their methods courses preservice teachers learn lesson-planning strategies for teaching particular content (PK & PCK)while during their educational technology course they learn how to use concept mapping software (TK). Afterward, as part of theireducational technology course, they are required to develop a lesson using concept mapping software in a content area of their choiceconsidering the characteristics of the students in their field placement (TCK & TPK). Finally, after consulting with their cooperating teacherthey are required to implement a technology-integrated lesson in their field placement and prepare a reflective narrative documenting theirexperience and lessons learned (TPACK).

5.5.2. Instructional designTPACK is a framework that emphasizes the importance of preparing teachers to make effective choices in their uses of technology when

teaching specific content to a particular student population (Tondeur et al., 2012). Toward this end, preservice teachers need repeatedopportunities to examine instructional design, practice planning, and prepare materials that integrate technology tools. The educationaltechnology course takes a two-step approach to developing knowledge of instructional design: lesson critique and lesson development.

Initially, participants are asked to identify a technology-integrated lesson in a content area of their choice from a lesson-plan portal calledThinkfinity. Subsequently, they prepare a critiquewhich discusses the instructional objectives of the lesson, the strengths andweaknesses ofthe learning activities presented in the lesson, the content and technology standards addressed, and the role of technology in relation to thelesson’s objective. This is a challenging assignment because it requires preservice teachers to think deeply about content, pedagogy andtechnology as well as how these three constructs combine to develop effective instruction. They must draw upon knowledge gleaned in theeducational technology course in conjunction with the content and pedagogy learned in their content and methods courses.

After spending considerable time critiquing published lessons, preservice teachers are given repeated opportunities to design their ownfor their methods field experience classrooms. Utilizing a variety of technology tools and applications, including interactive whiteboards,concept mapping, Internet inquiry, and collaborative tools, the plans must reflect an understanding of the TPACK framework. For instance,when learning about the pedagogical uses of concept maps, preservice teachers are asked to generate a lesson idea that utilizes electronicconcept mapping software to support a learning goal within a content area of their choice. Additionally, they are required to create a sampleconcept map that would help students in their field placement understand the task.

5.5.3. Role modelsResearch indicates that preservice teachers who had opportunities to observe the implementation of technology-rich units intomethods

courses reported greater technological skills and more ideas on how to use technology with students (Polly et al., 2010). As the instructorsfor the educational technology course, we provide the most relative models of technology integration since we choose the tools and ap-plications preservice teachers will explore in our class. Therefore, for each tool and application introduced, wemodel its use and implementan activity requiring preservice teachers to learn about it and then apply it to an educational context. For example, in preparing lecturematerials delivered and posted through our course management systemwemodel the use of the interactive tool, Voicethread, which allowsusers to embed video, audio and text to deliver presentations. Later in the semester, preservice teachers are asked to open a free VoiceThreadaccount, watch several video tutorials, and explore public VoiceThreads. These basic steps expose preservice teachers to the tool and allowthem to practice the technical skills necessary to effectively engagewith it (TK). Subsequently, while completing a unit on the characteristicsof digital texts, preservice teachers read a related article and then present their ideas in Voicethread following a set of guiding questionswhere they use text, audio or video. This process provides them with an explicit use of the technology tool within the context of learningabout reading and writing digital texts.

By juxtaposing the educational technology course with methods courses and field experience, we seek to provide preservice teacherswithmodels beyond our own. Research indicates that preservice teachers greatly benefit from experiencing effective technology integrationin the field and tend to disregard practices taught at the university if they are not used in real classroom settings (e.g., Korthagen & Kessels,1999). Specific examples of theways inwhichmethods faculty and cooperating teachers modeled the use of technology are presented in theFindings section.

5.5.4. Authentic experiencesAccording to Tondeur et al. (2012) many teacher education programs emphasize the importance of providing preservice teachers with

authentic technology experiences, including increased opportunities for hands-on work. In our course, preservice teachers have repeatedopportunities to engage with technology including hands-on activities utilizing graphic organizers, Internet inquiry, and web 2.0 tools.Additionally, preservice teachers apply their learning into practice by implementing technology-integrated lessons designed in our courseinto their field placement classroom. This exercise enables them to witness use of technology first hand with their own students. This isanother example of how the integration of the educational technology course with field experience proves beneficial.

5.5.5. ReflectionTo support the development of TPACK several researchers emphasize the importance of designing, implementing and reflecting on

teaching experiences that incorporate appropriate technologies. Mishra et al. (2010) propose spiraling stages of more complex instructionaldesign activities where TPACK reflection is completed at the end of the process. This type of metacognitive reflection allows preserviceteachers to consider technology, content and pedagogy and their inter-relationship when considering specific instructional problems. Duringtheir participation in the educational technology course, we make it possible for preservice teachers to reflect on their ideas, beliefs andexperiences with respect to technology through a case development process (Mouza & Karchmer-Klein, 2013). The process of case devel-opment progresses incrementally throughout the semester and is divided in various stages where preservice teachers design or identify acurricular-based technology-integrated lesson, enact the lesson in their field practicum, andwrite a reflective case on the implementation andoutcomes of the lesson. To facilitate the process of reflection, a series of writing and reflection prompts are provided that help preserviceteachers engage in systematic and detailed analysis of their practice. The process of case development heavily depends on the juxtaposition ofeducational technology coursework, with methods courses, and field experience as it allows preservice teachers to bring together theknowledge bases of technology, content and pedagogy and provide particular contexts for enactment and reflection in real classrooms.

C. Mouza et al. / Computers & Education 71 (2014) 206–221212

6. Methods

6.1. Participants

Participants included 88 preservice teachers enrolled in 4 sections of the required educational technology course offered in conjunctionwith methods courses and field experience during one semester. All sections utilized the same syllabus, which instructors developed andrefined collaboratively. Further, instructors worked together throughout the duration of the study to ensure fidelity of implementation. Onesection was taught by the first author, two sections were taught by the second author and a fourth section was taught by a third instructor.All participants majored in elementary teacher education (i.e., ETE majors) and were typical undergraduate students in their early 20s.Further, all participants concentrated in a middle school content area such as English (14.8%), mathematics (9.9%), science (16%), socialstudies (9.9%), or special education (58%). Of the 88 students 95% were female and 5% were male. More than half were juniors (58%) whilethe rest were seniors (42%).

6.2. Data collection

6.2.1. TPACK surveyData were collected through the Survey of Preservice Teachers’ Knowledge of Teaching and Technology, a survey instrument developed

around the TPACK framework (Schmidt et al., 2009). This instrument is the most universally used survey designed specifically for preserviceteachers majoring in elementary education and focuses on the content areas that they would be teaching in their future classrooms. Thesurvey emphasizes teachers’ self-assessed levels of knowledge in each of the TPACK domains. The survey was administered to all preserviceteachers twice; at the beginning (first class) and end (last class) of the course. It was administered online and response rates were 100% forthe initial survey and 91% for the follow-up survey.

The survey consists of 8 items designed to collect demographic data and 47 items focusing on preservice teacher knowledge (TK, CK, TCK,PK, PCK, TPK, and TPACK).1 These items use a 5-point Likert-scale response format that ranges from Strongly Disagree to Strongly Agreewithadditional space for open-ended comments. Additionally, 3 open-ended questions ask preservice teachers to describe specific episodeswhere university faculty, cooperating teachers in their field placement, or themselves effectively combined content, technologies and teachingapproaches in a classroom lesson. Participants are asked to include in their descriptionwhat content was being taught, what technology wasused, and what teaching approaches were implemented.

Although the survey instrument was found to be reliable by earlier studies (Schmidt et al., 2009), we also assessed the reliability for oursample. The Cronbach alpha of a scale should be greater than 0.70 for items to be used together as a scale (Nunnally, 1978). Results of ouranalyses of Likert-scale responses revealed that all subscales and the instrument as a whole achieved alpha levels exceeding 0.70 indicatingthat the survey was reliable (see Table 2).

6.2.2. Performance assessment: case reportsTo triangulate survey data and gain a better understanding of preservice teachers’ TPACK development we have also examined quali-

tative data collected through performance assessments completed in the educational technology course. A key assignment of the courseengages preservice teachers in a case development project, which progresses through stages that allow participants to design, enact andreflect on their own technology-integrated experiences in their field placement. To culminate the project, participants write a reflective casereport following a series of writing and reflection prompts that scaffold inquiry and reflection. Each case report is divided in two sections; acase narrative and a case reflection and is approximately 1000 words.

For the purpose of this study, we have collected all case reports and associated artifacts in order to examine the way in which preserviceteachers’ recognized and discussed interactions among technology, content and pedagogy in their ownpractice. Although cases are also self-reported, they provide important insights into preservice teachers’ strategic thinking in the process of planning, organizing and imple-menting their lessons (Leatham, 2008). Such data are essential for considering participants’ knowledge development as well as applicationof knowledge into practice. Given the volume of the data, a stratified random sample of 22 case reports (25% of the data) was selected thatrepresented approximately 5 cases from each section of the educational technology taught. This selection ensured that our sample wasrepresentative of the larger population of preservice teacher participants.

6.3. Data analysis

6.3.1. TPACK surveysSurvey datawere analyzed using both quantitative and qualitativemethodologies. Quantitative analyses included descriptive statistics, t-

tests and reliability analyses. Likert-scale itemswere initially scored based on guidelines provided by Schmidt et al. (2009) and subsequentlywere exported into Excel and the Statistical Package for the Social Sciences (SPSS) where means and standard deviations were calculated foreach knowledge component and for TPACK as a whole. To test for the significance of the gain score (post measure-pre measure), a repeatedmeasures t-test was conducted on each of the scales.

Open-ended responses describing specific episodes where university faculty, cooperating teachers in their field placement, or preserviceteachers themselves effectively combined content, technologies and teaching approaches in a classroom lesson were analyzed usinginductive approaches (Glaser & Strauss, 1967). Specifically, initially all preservice teachers’ responses to the three open-ended questions inboth the pre and the post administration of the survey were read repeatedly by two researchers to generate a starting list of episodes thatdemonstrated specific uses of technology with content and pedagogy. Subsequently, using the constant comparative method (Miles &

1 The original survey includes 55 items. However, 8 items were not relevant to our work and although we have administered the survey as a whole, analysis for those itemsis not reported in this manuscript. Those items were considered when calculating the reliability of the instrument as whole.

Table 2Reliability of the survey scales.

Scale Number of items Cronbach’s alpha

TK 7 0.885CK 12 0.820PK 7 0.838PCK 4 0.849TCK 4 0.856TPK 5 0.726TPACK 8 0.884Total items 55 0.932

C. Mouza et al. / Computers & Education 71 (2014) 206–221 213

Huberman, 1994), redundant themes or themes that fit together conceptually were combined and less frequently represented themes wereeliminated. This approach, helped identify common themes that cut across participants’ responses. Specifically, the analysis revealed thatparticipants discussed content and pedagogy primarily in relation to three types of technologies: (a) digital content (e.g., videos, websites,and online content); (b) presentation technologies (e.g., PowerPoint, LCD projectors, document cameras, interactive whiteboards); and (c)content-specific software.

6.3.2. Case reportsTo identify evidence of TPACK domains in student planning and decision making, we turned to the 22 selected case reports. Specifically,

following an a priori coding scheme guided by existing literature and refined through earlier studies (see Table 3; Mouza, 2009, 2011), welooked to identify evidence of TK, TPK and TPACK in participants’ case reports. Those constructs are of particular importance to researchersin educational technology because they mark a distinctive move from general technology integration (TPK) to content-specific technologyintegration (TPACK; Graham, 2011). Using this coding scheme, two researchers independently coded all case narratives. Inter-rater reliabilitywas calculated at 90%. All differences were discussed and resolved through consensus. Once all data were coded, percentages werecalculated in order to quantify emergent patterns.

Additionally, we havemade a qualitative assessment on the level of TPACK in relation to teaching represented in preservice teachers’ casereports utilizing descriptors provided by Niess et al. (2009) that included: (a) recognizing: technology is used for rote activities; (b) accepting:technology is peripheral to classroom instruction; (c) adapting: technology is used to reinforce student learning; (d) exploring: technologysupports exploration of concepts and high-level thinking; and (e) advancing: a variety of technology tools are used consistently to facilitatedeep understanding of content.

7. Findings

7.1. Preservice teachers’ knowledge development

Scores on each of the scales associated with the Survey of Preservice Teachers Knowledge, TK, CK, PK, PCK, TCK, TPK, TPACK, and the TOTAL(total score on all items) were computed for each participant at the beginning of the course (pre measure) and at the end of the course (postmeasure). To test for the significance of the gain score (post measure-premeasure), a repeatedmeasures t-test was conducted on each of the

Table 3Coding scheme representing TPACK.

Technological knowledge (TK)Evidence:Operating computer hardwareUsing standard software tools (e.g., MS Word, PowerPoint, Internet browsers, email)Installing and removing peripheral devices (e.g., USB drives, microphones) & softwareTroubleshooting equipmentUsing appropriate vocabulary (e.g., technology terms)

Technological pedagogical knowledge (TPK)Evidence:Motivating students through technologyDifferentiating instruction when technology is usedAbility to organize collaborative work with technologyHolding students accountable for equipment usedDeveloping strategies for assessing student work with technologyKnowing about the existence of a variety of tools for particular tasksKnowing about the time required to teach with particular technologiesAbility to envision potential student problems with particular technologies and plan relevant activities to support those studentsGenerating alternatives in the event of technological failuresAbility to explain a computer procedure to students (e.g., through modeling)

Technological pedagogical content knowledge (TPACK)Evidence:Use of technology to facilitate subject-specific pedagogical methods (e.g., science inquiry, primary sources in social studies, etc.)Use of technology to facilitate content representationUse of technology to address learner content understanding (e.g., prior content knowledge, address misconceptions, improve content understanding)

Table 4Preservice teachers’ TPACK.

Subtests # of Items Mean for pre survey Mean for post survey Mean difference Std. deviation t df P sig. (2 tailed) Effect size

TK 7 3.367 3.660 0.293 0.448 5.809 78 0.000 0.65CK 12 3.879 4.018 0.139 0.354 3.498 79 0.001 0.39PK 7 3.794 4.251 0.458 0.502 8.094 78 0.000 0.91PCK 4 3.619 4.119 0.500 0.620 7.212 79 0.000 0.81TCK 4 3.253 3.929 0.676 0.698 8.72 80 0.000 0.97TPK 5 3.810 4.241 0.431 0.481 7.903 77 0.000 0.89TPACK 8 3.555 4.008 0.453 0.596 6.672 76 0.000 0.76TOTAL 55 3.637 3.982 0.345 0.301 8.513 54 0.000 1.15

C. Mouza et al. / Computers & Education 71 (2014) 206–221214

scales. These results are tabulated in Table 4. As seen in Table 4 there is a significant gain (P < 0.05) on all scales. The last column of Table 4reports the effect size for all scales. The effect size denotes the increase in the mean score in standard deviation units. For example, for scaleTK, the effect size is 0.65, which indicates that the average score on TK has improved by 0.65 standard deviation units. This is considered alarge increase.2

For all scales that have shown significant improvement, the effect size is large except for CK where the effect size is medium. This findingis not surprising since neither the educational technology course nor the methods courses or field experience focused directly on CK. Incontrast, improvements in PK and TK and their interactions with other knowledge domains (PCK, TCK, TPK and TPACK) can be largelyattributed to the integrated approach we employed that allowed participants simultaneous enrollment in methods courses and accom-panied field experience through carefully designed activities. As one participant noted: “Mymethods classes this semester have boostedmyconfidence in specific subject areas such as social studies and literacy.” Similarly, commenting on advances in her PK one participant noted:“From my experiences in the field, I have especially learned how to manage a classroom.” These knowledge bases are essential whenthinking and planning technology-integrated lessons.

7.2. Modeling of TPACK by university faculty, cooperating teachers, and preservice teachers

7.2.1. Modeling of TPACK by university facultyWhen participants were asked to describe a specific episode where a university faculty effectively combined content, technologies and

teaching approaches all participants were able to provide such descriptions. The majority of the examples presented described instanceswheremethods faculty utilized digital content such as video clips and online resources (e.g., videos, website, etc.) related to the topic at hand.Describing an observed use of technology in social studies one participant explained:

In my social studies the instructor demonstrated a way of using teaching artifacts. Each group was given an old object and was asked tomake observations and predictions as to what it represented. After filling out a worksheet and discussing our ideas in groups, we weregiven the opportunity to investigate the artifact online. For example, some of the artifacts had the names of manufacturing companiesprinted on them so we could search those names online and see if we can get any hints as to the function of the item. This model waseffective because we were able to participate in an investigation/inquiry, while learning about history at the same time.

A sizable number of participants described instances wheremethods faculty used presentation technology, such as PowerPoint, to create amore interactive learning experience while a smaller number of participants described the use of content-specific software such as a spinnergame used to teach probability, an interactive geometric board, the use of graphic organizers and electronic books in literacy, the use ofGlobal Positioning System (GPS) and video conferencing software in social studies, and the use of scientific probeware in science. Aparticipant explained:

In my science class the professor modeled how to use probeware in middle school classrooms. We learned that during investigationabout the human body and the circulatory system, the students could use a heart rate probe to learn how the heart works during variousactivities.

7.2.2. Modeling of TPACK by cooperating teachersDespite efforts to place preservice teachers in classrooms that model effective use of technology, our findings indicated challenges in this

area. Findings from this work demonstrated a somehow narrow application of technology in teaching and learning. When asked to describea specific episode where a cooperating teacher combined content, technologies and teaching approaches the majority of participantsdiscussed uses of digital content. In a representative example on the use of digital content a participant explained:

In a third grade classroom the students used the Internet to learn about the Titanic. First they were assigned a personwho was on boardthe Titanic and researched that person on websites given to them by the teacher. Later, they used technology to watch a documentaryabout the Titanic. The students really enjoyed this lesson and wanted to learn more about the Titanic after the lesson was through.

Further, a large number of participants reported on the use of presentation technologies including LCD projectors to display PowerPointpresentations, document cameras (e.g., Elmo), and interactive whiteboards. Content-specific uses of technology were more limited withexamples focusing on the use of graphic calculators in mathematics, integrated learning software in literacy and mathematics such asAccelerated Reader (http://www.renlearn.com/ar/) and AcceleratedMath (http://www.renlearn.com/am/), and the use of global positioning

2 Effect size < 0.3 is small, 0.3–0.5 is medium, and >0.5 is large (Cohen, 1988).

C. Mouza et al. / Computers & Education 71 (2014) 206–221 215

systems (GPS) in social studies. Content-specific applications of technology were more prominent in the post survey responses. A possibleexplanation for this finding is that preservice teachers were more likely to notice content-specific uses of technology after their exposure tothe TPACK framework. Overall, however, it is important to note that few episodes demonstrated seamless integration of technology, contentand pedagogy as envisioned by the TPACK framework. One of the most successful examples was described as follows:

My cooperative teacher used a lot of technology in the classroom. One lesson focused on the use of linear equations for a table of datavalues. Each student had a graphing calculator, tried to come up with an equation, and sent it to the Smartboard so all of the equationswere graphed on the board at once. Then the class discussed which equations were correct and which ones were not. They also discussedhow they could change them to correct them.

7.2.3. Modeling of TPACK by preservice teachersFindings indicated a fairly limited understanding of TPACK among preservice teachers at the beginning of the study. When asked to

describe a teaching episode in which they effectively demonstrated combining content, technologies and teaching approaches at thebeginning of the educational technology course, the majority of the open-ended survey responses (57%) revealed no opportunities to teachsuch a lesson. This percentage is consistent with the percentage of participants completing their first methods field experience (specialeducation track) indicating that preservice teachers do not have opportunities to demonstrate those practices early in their teacher edu-cation program. Participants in the content area track were completing their second methods field experience and were able to describehow they combined content, technologies and teaching approaches. Those examples, however, focused primarily on the use of digitalcontent such as online resources, presentation technologies such as the use of PowerPoint, and even more primitive tools such as overheadprojectors. One participant reported characteristically:

My teaching partner and I used a PowerPoint presentation for a social studies lesson on landforms. That is the closest I have come.Honestly, I had enough to worry about teaching my first-ever classroom lesson. I didn’t want to add another variable that mightmalfunction.

In contrast, post survey responses paralleled significant improvements in preservice teachers’ TPACK documented in the survey data (seeTable 4). When asked to describe a teaching episode in which they effectively demonstrated combining content, technologies and teachingapproaches at the end of the course all participants provided such examples. The majority of the participants implemented one of theactivities they designed in their educational technology course. The remaining participants implemented a technology-integrated lessonidentified and modified from a lesson-plan portal (i.e., Thinkfinity). As discussed earlier, implementation of a technology-integrated lessonwas required and so was reflection upon that implementation. The majority of the examples described focused on the use of digital content.Instead of simply describing the use of a particular website or video clip, however, many participants (35%) described a web-based inquiryactivity designed during their participation in the course, which utilized digital resources with content and pedagogy. Describing theimplementation of such activity a participant characteristically explained:

I was able to implement the web-based inquiry activity developed in the technology class with students in seventh grade. The activityrequired students to visit a number of websites identified for them and learn more about plant and animal cells. Then they had tocomplete a worksheet where they defined each of the cell’s organelles and a venn diagram where they compared and contrasted thecells. In the end, they wrote a compare and contrast essay.

Other participants described inquiry activities that placed students in the role of historians learning more about immigrant’s life andtransition through Ellis Island or the role of scientists learning more about flower dissection through interactive online activities.

A sizable number of participants described the use of presentation technologies including interactive whiteboards, and content-specificsoftware such as the use of graphic organizers to write persuasive essays. Overall, the practices described by participants at the end of thecourse oftenmirrored the types of practices observed in their methods courses and field experience. Yet, they also revealed that participantswere able to bring together different knowledge bases to design and implement lessons that integrated technologywith specific content andsound pedagogical practices.

7.3. Representation of TPACK in preservice teachers’ practice

To triangulate preservice teachers’ self-report survey data and gain a closer understanding of the ways in which participants recognizedand demonstrated relationships among technology, content and pedagogy (TPACK) we turned to participants’ reflective cases where theyreported on the design and implementation of a technology-integrated lesson in their field placement. In our analysis we looked to identifyevidence of TK, TPK and TPACK in participants’ case reports – constructs that were of particular importance to the objectives of this study(see Fig. 2 and Table 5). Additionally, we looked to identify whether there were differences in how TPACK was discussed and represented inparticipants’ narratives and the ways in which the integrated approach influenced TPACK development. As shown in Fig. 2 and Table 5, ouranalysis indicated that case reports provided ample evidence of TK, TPK and TPACK. This finding is consistent with our survey results, whichdemonstrated significant improvements in participants’ knowledge development over the course of the study. Below we present the majorthemes that cut across participants’ responses related to TK, TPK and TPACK and their application in practice.

7.3.1. Role of TKSurvey data indicated that preservice teachers experienced significant growth in their TK over the course of the study. Analysis of

preservice teachers’ cases further indicated that all reports provided evidence of TK. This is important because one of the primary objectivesof the educational technology coursewas to improve preservice teachers’ TK.When discussing TK, preservice teachers provided examples ofboth their own and their students’ knowledge with respect to technology (see Table 5). They also identified technical difficultiesencountered during the implementation of their lesson and discussed issues of access by describing the technology resources available at

Fig. 2. Representation of TPACK in preservice teachers’ case reports.

C. Mouza et al. / Computers & Education 71 (2014) 206–221216

their field placement. Discussing his own TK, for example, one participant noted how he had to work his way around the use of conceptmapping tool, Inspiration, introduced during the educational technology. He noted: “I had completed my concept mapping project a longtime ago and thus forgot how to maneuver around Inspiration. Specifically, I could not remember how to draw the arrows and it took me afew seconds to get acquainted with the program again”.

While discussing their students’ TK, preservice teachers often noted their students’ strengths and weaknesses in relation to technologiesintroduced in their lesson. One participant, for example, explained: “Because the students were both familiar with the Internet and thegeneral operating system, it did not take much time to introduce the technology. Once we worked on the Internet, however, I helped themdevelop familiarity with the different features of the website”. The majority of the participants noted that no major technical difficultieswere encountered during the implementation of their lesson, whichwas often surprising. For example, one preservice teacher noted: “Therewere no issues with the Internet working properly or the sound being turned on and loud enough on the computers. This allowed the lessonto run more smoothly”.

Although it is natural to expect that preservice teachers will exhibit TK after their participation in an educational technology course,findings indicated that TK was not the dominant form of knowledge represented in case reports. Rather, technology was more likely to berepresented in relation to pedagogy (TPK) as well as content and pedagogy (TPACK). This finding is important because it demonstrates thatissues related to TK did not dominate preservice teachers’ case reports. Rather, these issues were more likely to be situated in the context ofthe TPACK framework introduced and modeled throughout our integrated approach.

7.3.2. Role of TPKAnalysis of preservice teachers’ case reports indicated the prevalence of the TPK construct. This finding is consistent with survey results

that demonstrated a large increase in participants’ TPK over the course of the study. As shown on Table 5, across the 22 case reports wecoded 86 instances of TPK which represented 48% of all coded constructs (see also Fig. 2). These instances revealed that preservice teachersfrequentlymodeled TPK practices observed and experienced in their educational technology course. For example, prior to introducing a new

Table 5Coding examples and number of coding instances.

Construct Coding example Number ofcodinginstances

Mean SD

TK The only difficulties I had with technology were at the end of the lesson when I tried to print what thestudents had created from the website. It asked if I would like to print it, but everything came out with blankquestions when I tried.

37 1.7 1.4

TPK Although it may be tempting to step in and “show students how things work,” they need to be able toexperiment (safely and appropriately) with technology.

86 3.9 1.8

I wanted to make sure students completed the activity and had a chance to play the interactive game on theInternet because I knew they would enjoy it and I could help them answer questions if they were unsureabout things. If I were to teach this again, I would break the lesson up over a few days and make sure toreserve a computer lab in advance. This would provide student with an opportunity to explore the websiteat their own pace and in more detail.I bookmarked the website that we were going to be using on all of the computers so it would be easy for thestudents to access.

TPACK The major goals for this lesson were to help students identify different forms of energy and how theseenergy forms are transferred and transformed in energy phenomena. Students constructed a graphicorganizer representing the flow of energy from one point to another (adapting level).

55 2.5 1.3

Students used an inquiry-oriented activity on flowers and their functions. Using a collection of Internetresources, they learned the parts of the flower and the jobs of each part. Then, during the second part of thelesson, students applied the information gathered the first day to understand the process of pollination andidentify at least one pollinator and its role in the process. These activities were completed in preparation of aflower dissection. The background knowledge allowed students to focus on the dissection and theinterconnectedness of all the parts rather than worry about new terms (exploring level).

C. Mouza et al. / Computers & Education 71 (2014) 206–221 217

technology to students, preservice teachers often modeled its use by creating a classroom product much like a practice experienced in theireducational technology course. One participant explained:

I knew that students would not be able to make a concept map by themselves right away, so I made one on the teacher computer andprojected it on the screen in front of the classroom. After we read a book, I started with the main concept that included the word Animalsand then asked the students if they could remember the five animal groups discussed in the book. I also asked them to name animals thatfit with each category. As they provided the information, I filled out the bubbles on the graphic organizer.

Interestingly, this activity parallels one conducted in the educational technology course when introducing the use of graphic organizersthrough the tool Inspiration.

Similarly, other preservice teachers discussed how they often introduced their selected technology to students, provided instructions,and then let them explore while they simply walked around the room observing and helping as needed. This activity also parallels the waysin which educational technology faculty introduced new technologies in the course.

Further, preservice teachers’ reports provided evidence that practices associated with TPK, were influenced by what they observed andexperienced in their field placement context. For example, in trying to envision her students’ skills prior to the implementation of her lesson,one preservice teacher explained:

My expectations of the student’s technology skills were not very high. In the three weeks that I had been in my field placementclassroom, the only time I had ever seenmy students use a computerwas once aweek during their computer lab special. My students hadnever used the computers in the back of the room, which I used for this lesson, while I was there. I expected them to be somewhatfamiliar with computers from their weekly special, but I also expected that they would need a lot of guidance.

Other students, however, discussed the implementation of practicesmodeled by their cooperating teachers. One preservice teacher noted:“When I implemented my lesson, I started by going onto the main teacher computer and walking students through exactly what they weregoing to be doing. This is a technique that their computer teacher told me that she normally does with themwhen they have computer lab”.

7.3.3. Levels of TPACKConsistent with survey results that demonstrated large increase in preservice teachers’ TPACK over the course of the study (see Table 4),

all 22 case reports exhibited instances of TPACK. As shown on Table 5, we have coded 55 TPACK instances representing 31% of all TPACKconstructs. Despite significant growth in preservice teachers’ knowledge development, findings indicated that there was variability in theways in which TPACK was represented in their case reports. This finding is not surprising given that preservice teachers acquire TPACKprogressively and do not suddenly display this knowledge in their practice (Niess et al., 2007). Considering the five levels of TPACK discussedby Niess et al. (2007, 2009) in relation to participants’ teaching, our analysis also demonstrated growth in participants’ TPACK. Qualitativesurvey results, for example, indicated that at the beginning of the course many teachers (57%) had no opportunities to teach lessons thateffectively combined technology with content and pedagogy. Although participants acknowledged the important role of technology theyhad difficulty using it to support student learningwithout prior experiences. Evenwhen preservice teachers described TPACK instances fromtheir practice, the examples provided focused primarily on uses of technology tools independent of the content and pedagogical strategiesemployed (e.g., I used PowerPoint). These findings indicated that as a group, preservice teachers recognized the value of technology butexhibited limited understandings and application of technology in classroom contexts in connection with content and pedagogy.

Analysis of case reports indicated that all participants accepted the value of technology in teaching and learning as they incorporated it toaddress specific content in their field placement classrooms. Although in some cases activities were peripheral to instruction, preserviceteachers acknowledged the importance of technology in reinforcing concepts covered in class and discussed future plans in relation to theiruse of technology. A preservice teacher who used computer games to help reinforce mathematics skills previously addressed in class, noted:

I was surprised with howwell the students did with themath.When at their desks in their classroom, they struggle to get the math doneand are very distracted. In the computer lab, while playing fun and interactive games, students were able to solve problems much moreeasily and quickly. They made fewer mistakes than they normally do on their worksheets without computers.

At the same time, the preservice teacher continued to discuss the use of technology in relation to content, noting: “The math games didnot take away from classroom instruction because they allowed students to practice the same skills they normally do solely on paper”.

Further, our analysis indicated that the majority of the participants were able to progress to the adapting level (n ¼ 17) but only a smallnumber of participants progressed to the exploring level (n ¼ 5). Our data did not support analysis beyond the exploring level since casereports only presented one instance of technology-integrated practice. As such, we could not provide evidence consistent with the char-acteristics of advancing, such as active and consistent acceptance of technologies as tools for teaching and learning. Below we provide twoexamples that illustrate the TPACK knowledge at the adapting and exploring levels as illustrated in preservice teachers’ case reports.

Briana’s case: adapting level of TPACK. Briana’s case was situated in a middle school in a sub-urban area. Briana implemented a lesson onthe transformation of energy in science. This was a topic the students in her field placement were studying at the time. Her lesson, titledHow Energy Moves in Liquids had two major objectives: (a) students will understand that if the temperature of a liquid rises, the amount ofenergy in the liquid will also rise; and (b) students will understand that the temperature of a liquid affects how quickly/slowly food coloringwill disperse in that liquid. To accomplish her objectives Briana identified and used two pieces of technology, a document camera (Elmo) andan interactive whiteboard (i.e., SmartBoard) to demonstrate how energy moves in liquids. Briana noted that students were familiar with theuse of the interactive whiteboard because they had used it in the past to create graphs and diagrams.

To begin the activity, Briana used two beakers to place a drop of red food coloring in hot water and a drop of blue coloring in cold water.Using the document camera she allowed students to observe her actions on the interactive whiteboard while simultaneously a studentvolunteer used the interactive pen on the whiteboard to draw circles pointing to the areas in which the food color had spread. Additionalvolunteers performed the same activity 1 min and 2 min later to show change in the dispersion of the coloring and point out that the coldwater still had huge areas of clear water compared to the hot water indicating that the food color was dispersing slower in cold water.

C. Mouza et al. / Computers & Education 71 (2014) 206–221218

In this activity, Briana demonstrated knowledge of using the document camera and interactive whiteboard (TK) to demonstrate a scienceinvestigation and scaffold classroom discussion. In identifying appropriate tools, she also considered her students’ prior knowledge withtechnology and their prior familiarity with the use of the interactive whiteboard (TPK). Further, Briana demonstrated the ability to weavetechnology with content-specific activities required to facilitate student learning in her field placement (TPACK).

Briana’s TPACK is consistent with the adapting level because she used technology to enhance or reinforce ideas that students hadpreviously learned. Despite that, Briana’s instructional strategies were primarily teacher-directed. Students were asked to observe theteachers’ actions but did not have opportunities to engage with the technology.

Kate’s case: exploring level of TPACK. Kate’s case was situated in a sub-urban elementary school. The major objective of the lesson was forstudents to understand the lives of pilgrims living in Plymouth during the first Thanksgiving. The students were asked to use an interactiveInternet resource (http://www.scholastic.com/scholastic_thanksgiving/) that allowed them to assume the perspective of the Pilgrimstraveling to the new world on the Mayflower as well as take a virtual tour of the Mayflower. They also conducted research to identifyinformation related to the trip on theMayflower, theways inwhich Pilgrims interacted with the natives, and the events that transpired overthe course of the Pilgrim’s first year living in Plymouth. Finally, students created timelines with major events of the Pilgrims first year in theNew World and wrote a letter from the perspective of the Pilgrims living in Plymouth during the first Thanksgiving.

In this activity, Kate demonstrated knowledge of using the Internet (TK) to facilitate student research. In doing so, she considered herstudents prior knowledge of both technology and content (TPK), such as their ability to use the Internet as a resource, navigate through awebsite, and use word processing programs as well as their understanding of what happened between the Pilgrims and the NativeAmericans after the Pilgrims arrived in America. Additionally, Kate demonstrated the ability to integrate interactive Internet resources withcontent-specific activities and representations required to facilitate student understanding (TPACK).

Unlike Briana’s case, Kate engaged students in higher-level thinking activities that included inquiry and decision making. Kate, placedtechnology in the hands of her students allowing them to explore and engage with interactive web-based activities and resources thatimproved their understanding of content. As Kate noted, her students were surprised to find out about the length of the Pilgrim’s voyage, thedifferent parts of theMayflower, and thedifferent living arrangementswhile traveling. Kate’s case acknowledged the importance of interactiveonline resources in engaging and supporting student learning. Although at this point, we have no evidence that Kate will continue exploringand incorporating technologies for numerous topics, we have beginning evidence to indicate that Kate has approached the exploring level.

8. Limitations

Two limitations are evident in this work. First, data were collected from self-reported measures such as surveys and case reports.Although case reports provide rich accounts of preservice teachers’ thinking in the process of planning, organizing and implementingcurricular uses of technologies they are still generated from the perspective of the participants. Nevertheless, we believe that the combi-nation of those two sources provided a reliable measure of preservice teachers’ TPACK over time.

Second, recent empirical work with the Survey of Preservice Teachers’ Knowledge of Teaching and Technology, has questioned its constructvalidity. Two studies that used a modified version of the survey indicated that the knowledge domains of TPACK could not be reproducedthrough exploratory factor analysis, indicating that teachers may have difficulty distinguishing among TPACK constructs (Archambault &Barnett, 2010; Koh, Chai, & Tsai, 2010). Similar results are reported by a recent study that examined the construct validity of theauthentic version of the survey (Shinas, Yilmaz Ozden, Mouza, Karchmer-Klein, & Glutting, 2013). Using factor analysis and a large sample ofpreservice teachers in the U.S. this study also revealed that participants did not always make conceptual distinctions between the TPACKdomains. Despite that, the Survey of Preservice Teachers’ Knowledge of Teaching and Technology is the most mature instrument developed todate that can be used to collect quantitative data among large samples of preservice teachers, and as such this study makes an importantcontribution to the field.

9. Discussion, conclusion and implications

This study examined how and to what extent participation in an integrated approach that juxtaposed an educational technology coursewith methods courses and field experience through careful instructional design, influenced preservice teachers’ TPACK development andpractice. Quantitative and qualitative data collected through the Survey of Preservice Teachers’ Knowledge of Teaching and Technology and casereports revealed that participants experienced significant gains in all TPACK constructs. Effect sizes were large in all areas except CK, whichwas not particularly targeted in this approach. Further, all participants were able to apply their learning to classroom practice as theyimplemented technology-integrated lessons in their field placement classrooms and reflected on their experience. Specifically, qualitativedata indicated that preservice teachers progressed from recognizing to accepting levels of TPACK with the majority moving to the adaptinglevel and few progressing to the exploring level.

Our findings indicated that the educational technology course, methods courses and field experience collectively exposed preserviceteachers to a variety of TPACK models that contributed to their learning and subsequent practice in their field placement classrooms.Nevertheless, evidence from participants’ case reports revealed important insights as to how such learning occurred. Specifically, it wasapparent that the greatest source of TK was the educational technology course. Cooperating teachers were instrumental in exposing pre-service teachers’ to interactive whiteboards such as the SmartBoard but for the most part, preservice teachers attributed their TK to theactivities conducted in the educational technology course. This finding continues to reinforce the value of educational technology courses indeveloping preservice teachers comfort level with technology.

Similarly, cooperating teachers appeared to contribute primarily toward PK. In many cases, preservice teachers discussed generalpedagogical practices as well as pedagogical practices related to technology (TPK) observed in their field placement classroom (e.g.,classroommanagement, monitoring technology use) and the ways inwhich they influenced their own application of technology in practice.Chai, Koh, and Tsai (2010) demonstrated that PK has the largest impact on technology use in the classroom and thus the field experience canfoster growth in pedagogy that helps develop TPACK. Finally, methods courses contributed primarily toward the development of PCK as theyprovided strategies for teaching in one’s content area as well as TCK as they discussed and modeled interactions among content and

C. Mouza et al. / Computers & Education 71 (2014) 206–221 219

technology. In their open-ended survey responses participants reported a number of content-specific uses of technology and the ways inwhich they influenced their own thinking in teaching within the specific discipline.

The above findings support an existing body of literature indicating that educational technology courses can provide a strong foundation oftechnology skills and a good overview of the use of technology in teaching (Kay, 2006). Further, findings indicate that when taken inconjunction with methods courses and field experience stand alone educational technology courses can significantly influence preserviceteachers’ ability to combine content, pedagogy and technology in the design and implementation of technology-integrated lessons (TPACK).Analyzing evaluation reports from a number of projects funded by the federally funded project Preparing Teachers to Teach with Technology(PT3), Polly et al. (2010) also found that without methods courses and field experiences, preservice teachers are left with technology skills butlimited understanding about how to implement technology into their classroom. Yet, limited empirical evidencewas provided to support thisclaim. In this work, we articulated ways in which such integration can leverage preservice teachers’ learning and practice.

Two challenges were evident in our efforts to support preservice teacher learning through an integrative approach. The first challengewas related to the ways in which university faculty combined content, technologies and teaching approaches. Although findings indicatedthat participants had opportunities to observe practices where university faculty combined content, technologies and teaching approaches,the majority of practices observed focused on the use of digital content and presentation technologies. Further, in many instances par-ticipants reiterated the same observed models indicating that only a small number of faculty actually modeled how to combine content,technologies and teaching approaches effectively in their instruction.

Opportunities to observe the implementation of technology-rich units into methods courses is important because they help preserviceteachers develop positive attitude toward technology use in the classroom and acquire technology skills that are more strongly connected touse in K–12 settings (Pierson & Thompson, 2005; Tondeur et al., 2012). Niess (2012) urges teacher educators to re-think methods courses inways that emphasize and provide learning experiences at the intersection of content, pedagogy and technology. Given the important role ofmodeling, administrators and teacher education programs might have to allocate more resources to methods faculty interested in modelingeffective technology use, such as time for designing technology-rich instructional units,mentoring, and content-specific software or resources.

The second challenge was related to field placements where frequently access to technology was limited and cooperating teachersprovided few examples of effectively integrating technology, content and pedagogy. When participants were asked to describe practiceswhere their cooperating teachers combined content, technology and teaching approaches in their field placement most responsesconcentrated again on the use of digital content and presentation technologies. This finding parallels similar accounts reported by ProjectTomorrow (2011) on K–12 teachers’ use of technology. Specifically, in a national survey of K–12 educators Project Tomorrow found thatteacher use of technology remained sporadic or lesson dependent.

This finding is also consistent with existing literature pointing to difficulty in identifying technology-rich field experiences for preserviceteachers and cooperating teachers with an understanding and facility in teaching their subject matter with appropriate technologies (e.g.,Niess, 2012). Reasons for this difficulty often include lack of technological resources, school culture that does not value innovation, schools’reluctance to participate in professional development projects, and a lack of alignment between technology use in teacher educationprograms and K–12 schools (Hammond et al., 2009; Polly et al., 2010). Studies documenting preservice teachers’ learning in technology-richfield placements, however, reported promising outcomes including positive attitudes toward technology, frequent use of technology, andmore instances of preservice teaching using technology to support student learning (Strudler, Archambault, Bendixen, Anderson, & Weiss,2003). As a result, teacher education programs must engage in more concerted efforts of identifying field placements rich in technologyresources and models that integrate technology, content and pedagogy.

A major strength of the integrated approach was the opportunity it provided preservice teachers to design, enact and reflect upon theimplementation of a technology-integrated lesson in a real classroom. Although in many instances, preservice teachers described practicesthat paralleled those observed in their university and field placement classrooms, enactment and reflectionwas key to helping participantsrecognize and accept use of technology. Further, some preservice teachers were able to progress to higher levels of TPACK development,such as those associatedwith the adapting and exploring levels, and discuss future plans related to technology use. Voogt et al. (2012), foundthat preservice teachers are generally more likely to design rather than enact technology-rich lessons. Findings from this work indicate thatteacher education programs should include opportunities for the enactment of technology-integrated lessons. Setting such expectations hasproven effective in getting preservice teachers to use technology during clinical preparation and in developing the knowledge, skills anddispositions consistent with the TPACK framework (Dexter & Riedel, 2003; Niess, 2005).

In conclusion, this work provided a detailed account of an integrated instructional approach that can help preservice teachers advancetheir TPACK, and the ways inwhich such integration can be used to leverage preservice teacher learning. This account includes descriptionsof the actual technology topics and assignments addressed in the educational technology course and the types of technology uses observedin methods courses and field experiences. Our findings indicate that this approach holds promise for advancing preservice teachers’ TPACKas illustrated in both the quantitative and qualitative data. At the same time, it points to the need of providing preservice teachers withmorediverse TPACK models that extent beyond the use of digital content and presentation technologies. Arguably this is difficult to beaccomplished outside a subject-specific teacher preparation program like the one in this study (Niess, 2012). Findings from this work helpestablish an empirical basis related to best practices in TPACK development.

References

Agyei, D. D., & Voogt, J. M. (2011). Exploring the potential of the will, skill, tool model in Ghana: predicting prospective and practicing teachers’ use of technology. Computers &Education, 56, 91–100.

Albion, P., Jamieson-Proctor, R., & Finger, G. (2010). Auditing the TPACK confidence of Australian pre-service teachers: the TPACK confidence survey (TCS). In D. Gibson, &B. Dodge (Eds.), Proceedings of Society for Information Technology & Teacher Education international conference 2010 (pp. 3772–3779). Chesapeake, VA: AACE.

Angeli, C., & Valanides, N. (2005). Preservice teachers as ICT designers: an instructional design model based on an expanded view of pedagogical content knowledge. Journalof Computer-Assisted Learning, 21(4), 292–302.

Angeli, C., & Valanides, N. (2009). Epistemological and methodological issues for the conceptualization, development, and assessment of ICT-TPCK: advances in technologyand pedagogical content knowledge (TPCK). Computers & Education, 52, 154–168.

Archambault, L. M., & Barnett, J. H. (2010). Revisiting technological pedagogical content knowledge: exploring the TPACK framework. Computers & Education, 55, 1656–1662.

C. Mouza et al. / Computers & Education 71 (2014) 206–221220

Archambault, L., & Crippen, K. (2009). Examining TPACK among K–12 online distance educators in the United States. Contemporary Issues in Technology and Teacher Education,9(1), 71–88.

Blue Ribbon Panel Report on Clinical Preparation and Partnerships for Improved Student Learning. (2010). Transforming teacher education through clinical practice: A nationalstrategy to prepare effective teachers. NCATE. Retrieved on 25.04.13 from http://www.ncate.org/LinkClick.aspx?fileticket¼zzeiB1OoqPk%3D&tabid¼715.

Cavin, R. (2008). Developing technological pedagogical content knowledge in preservice teachers through microteaching lesson study. In K. McFerrin, et al. (Eds.), Proceedingsof Society for Information Technology and Teacher Education international conference 2008 (pp. 5214–5220). Chesapeake, VA: AACE. Retrieved from http://www.editlib.org/p/28106.

Chai, C. S., Koh, J. H. L., & Tsai, C. C. (2010). Facilitating preservice teachers’ development of technological, pedagogical, and content knowledge (TPACK). Educational Technology& Society, 13(4), 63–73.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Common Core State Standards Initiative. (2010). Common Core State Standards for English language arts & literacy in history/social studies, science, and technical subjects.

Washington, DC: National Governors Association Center for Best Practices and the Council of Chief State School Officers.Cox, S., & Graham, C. R. (2009). Diagramming TPACK in practice: using an elaborated model of the TPACK framework to analyze and depict teacher knowledge. TechTrends:

Linking Research & Practice to Improve Learning, 53(5), 60–69.Dexter, S., & Riedel, E. (2003). Why improving preservice teacher educational technology preparation must go beyond the college’s walls. Journal of Teacher Education, 54(4),

334–346.Doering, A., Veletsianos, G., Scharber, C., & Miller, C. (2009). Using the technological, pedagogical, and content knowledge framework to design online learning environments

and professional development. Journal of Educational Computing Research, 41(3), 319–346.Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine Publishing Company.Graham, C. R. (2011). Theoretical considerations for understanding technological pedagogical content knowledge (TPACK). Computers & Education, 57, 1953–1960.Graham, C. R., Borup, J., & Smith, N. B. (2012). Using TPACK as a framework to understand teacher candidates’ technology integration decisions. Journal of Computer Assisted

Learning, 28(6), 530–546. http://dx.doi.org/10.1111/j.1365-2729.2011.00472.x.Gronseth, S., Brush, T., Ottenbreit-Leftwich, A. T., Strycker, J., Serdar, A., Wylie, E., et al. (2010). Equipping the next generation of teachers: technology preparation and practice.

Journal of Digital Learning in Teacher Education, 27(1), 30–36.Hammond, M., Crosson, S., Fragkouli, E., Ingram, J., Johnston-Wilder, P., Johnston-Wilder, S., et al. (2009). Why do some student teachers make very good use of ICT? An

exploratory case study. Technology, Pedagogy and Education, 18(1), 59–73.Harrington, R. A. (2008). The development of pre-service teachers’ technology specific pedagogy (Unpublished doctoral dissertation). Corvallis, OR: Oregon State University.Hew, K. F., & Brush, T. (2007). Integrating technology into K–12 teaching and learning: current knowledge gaps and recommendations for future research. Educational

Technology Research and Development, 55, 223–252.Kay, R. (2006). Evaluating strategies used to incorporate technology into preservice education: a review of the literature. Journal of Research on Technology in Education, 38(4),

383–408.Kelly, M. A. (2008). Bridging digital and cultural divides: TPCK for equity of access to technology. In AACTE Committee on Innovation and Technology (Ed.), The handbook of

technological pedagogical content knowledge (TPCK) for educators (pp. 31–58). NY: American Association of Colleges of Teacher Education and Routledge.Kinuthia, W., Brantley-Dias, L., & Clarke, P. A. J. (2010). Development of pedagogical technology integration content knowledge in preparing mathematics preservice teachers:

the role of instructional case analyses and reflection. Journal of Technology and Teacher Education, 18(4), 645–669.Kleiner, B., Thomas, N., & Lewis, L. (2007). Educational technology in teacher education programs for initial licensure (NCES 2008-040). Washington, DC: National Center for

Education Statistics, Institute of Education Sciences, U.S. Department of Education.Koehler, M. J., & Mishra, P. (2005). What happens when teachers design educational technology? The development of technological pedagogical content knowledge. Journal of

Educational Computing Research, 32, 131–152.Koehler, M., & Mishra, P. (2008). Introducing TPCK. In AACTE Committee on Innovation and Technology (Ed.), The handbook of technological pedagogical content knowledge

(TPCK) for educators (pp. 3–29). New York: American Association of Colleges of Teacher Education and Routledge.Koehler, M., Shin, T. S., & Mishra, P. (2012). How do we measure TPACK: let me count the ways. In R. R. Ronau, C. R. rakes, & M. L. Niess (Eds.), Educational technology, teacher

knowledge, and classroom impact: A research handbook on frameworks and approaches (pp. 16–31). Hershey, PA: IGI Global.Koh, J. H. L., Chai, C. S., & Tsai, C. C. (2010). Examining the technological pedagogical content knowledge of Singapore pre-service teachers with a large-scale survey. Journal of

Computer Assisted Learning, 26, 563–573.Korthagen, F. A., & Kessels, J. P. A. M. (1999). Linking theory and practice: changing the pedagogy of teacher education. Educational Researcher, 28(4), 4–17.Lawless, K., & Pellegrino, J. W. (2007). Technology into teaching and learning: knowns, unknowns, and ways to pursue better questions and answers. Review of Educational

Research, 77(4), 575–614. http://dx.doi.org/10.3102/0034654307309921.Leatham, K. (2008). The development of technological pedagogical content knowledge in “technology, pedagogy and mathematics” courses in the U.S. In K. McFerrin, et al.

(Eds.), Proceedings of Society for Information Technology & Teacher Education international conference 2008 (pp. 5277–5283). Chesapeake, VA: AACE.Lee, M. H., & Tsai, C. C. (2010). Exploring teachers’ perceived self efficacy and technological pedagogical content knowledge with respect to educational use of the World Wide

Web. Instructional Science, 38, 1–21.Lei, J. (2009). Digital natives as preservice teachers: what technology preparation is needed? Journal of Computing in Teacher Education, 25(3), 87–97.Margaryan, A., Littlejohn, A., & Vojt, G. (2011). Are digital natives a myth or reality? University students’ use of digital technologies. Computers and Education, 56(2), 429–440.Margerum-Leys, J., & Marz, R. W. (2004). Teacher knowledge of educational technology: a case study of student/mentor teacher pairs. Journal of Teacher Education, 26(4),

427–462.Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage Publications.Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: a framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054.Mishra, P., Koehler, M. J., Shin, T. S., Wolf, L. G., & DeSchryver, M. (2010, March). Developing TPACK by design. In J. Voogt (Ed.), Developing TPACK. Symposium conducted at the

annual meeting of the Society for Information Technology and Teacher Education (SITE). San Diego, CA.Mouza, C. (2009). Does research-based professional development make a difference? A longitudinal investigation of teacher learning in technology integration. Teachers

College Record, 111(5), 1195–1241.Mouza, C. (2011). Promoting urban teachers’ understanding of technology, content, and pedagogy in the context of case development. Journal of Research on Technology in

Education, 44(1), 1–29.Mouza, C., & Karchmer-Klein, R. (2013). Promoting and assessing pre-service teachers’ technological pedagogical content knowledge (TPACK) in the context of case devel-

opment. Journal of Educational Computing Research, 48(2), 127–152.Mouza, C., & Wong, W. (2009). Studying classroom practice: case development for professional learning in technology integration. Journal of Technology and Teacher Education,

17(3), 175–292.NCATE. (2010). What makes a teacher effective. National Council for the Accreditation of Teacher Education. Retrieved 30.07.13 from http://www.ncate.org/LinkClick.aspx?

fileticket¼JFRrmWqa1jU%3d&tabid¼361.Niess, M. L. (2005). Preparing teachers to teach science and mathematics with technology: developing a technology pedagogical content knowledge. Teaching and Teacher

Education, 21, 509–523.Niess, M. L. (2008). Guiding preservice teachers in developing TPCK. In AACTE Committee on Innovation and Technology (Ed.), The handbook of technological pedagogical

content knowledge (TPCK) for educators. New York: American Association of Colleges of Teacher Education and Routledge.Niess, M. L. (2012). Rethinking pre-service mathematics teachers’ preparation: technological, pedagogical and content knowledge (TPACK). In D. Polly, C. Mims, &

K. Persichitte (Eds.), Developing technology-rich, teacher education programs: Key issues (pp. 316–336). Hershey, PA: IGI Global.Niess, M., & Gillow-Wilson, H. (2013). Advancing K–8 teachers’ STEM education for teaching interdisciplinary science and mathematics with technologies. Journal of Com-

puters in Mathematics and Science Teaching, 32(2), 219–245.Niess, M. L., Lee, K., & Sadri, P. (2007, April). Dynamic spreadsheets as learning technology tools: developing teachers’ technology pedagogical content knowledge (TPCK). In

Paper presentation for the American Education Research Association annual conference. Chicago, IL.Niess, M. L., Ronau, R. N., Shafer, K. G., Driskell, S. O., Harper, S. R., Johnston, C., et al. (2009). Mathematics teacher TPACK standards and development model. Contemporary

Issues in Technology and Teacher Education, 9(1), 4–24.Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw Hill.Ottenbreit-Leftwich, A., Brush, T., Strycker, J., Gronseth, S., Roman, T., Abaci, S., et al. (2012). Preparation versus practice: how do teacher education programs and practicing

teachers align in their use of technology to support teaching and learning? Computers & Education, 59, 399–411.

C. Mouza et al. / Computers & Education 71 (2014) 206–221 221

Ozgun-Koca, S. A. (2009). The views of preservice teachers about the strengths and limitations of the use of graphing calculators in mathematics instruction. Journal ofTechnology and Teacher Education, 17, 203–227.

Pierson, M. E. (2001). Technology integration practice as a function of pedagogical expertise. Journal of Research on Computing in Education, 33, 413–430.Pierson, M. (2008). Teacher candidates reflect together on their own development of TPCK: edited teaching videos as data for inquiry. In K. McFerrin, et al. (Eds.), Proceedings

of Society for Information Technology and Teacher Education international conference 2008 (pp. 5305–5309). Chesapeake, VA: AACE. Retrieved from http://www.editlib.org/p/28122.

Pierson, M., & Thompson, M. (2005). The re-envisioned educational technology course: if addition isn’t possible, try division. Journal of Computing in Teacher Education, 22(1),31–36.

Polly, D., Mims, C., Shepherd, C., & Inan, F. (2010). Evidence of impact: transforming teacher education with preparing tomorrow’s teachers to teach with technology (PT3)grants. Teaching and Teacher Education, 26, 863–870.

Project Tomorrow. (2011). The new 3 E’S of education: Enabled, engaged, empowered. Speak up 2010 national findings. Retrieved 28.05.12 from http://www.tomorrow.org/speakup/pdfs/SU10_3EofEducation_Educators.pdf.

Schmidt, D. A., Baran, E., Thompson, A. D., Mishra, P., Koehler, M. J., & Shin, T. S. (2009). Technology pedagogical content knowledge (TPACK): the development and validationof an assessment instrument for preservice teachers. Journal of Research on Technology in Education, 42(2), 123–149.

Shinas, V. H., Yilmaz Ozden, S., Mouza, C., Karchmer-Klein, R., & Glutting, J. (2013). Examining domains of technological pedagogical content knowledge using factor analysis.Journal of Research on Technology in Education, 45(4), 339–360.

Shulman, L. S. (1986). Those who understand: knowledge growth in teaching. Educational Researcher, 15(2), 4–14.So, H., & Kim, B. (2009). Learning about problem based learning: student teachers integrating technology, pedagogy and content knowledge. Australasian Journal of

Educational Technology, 25, 101–116.Strudler, N., Archambault, L., Bendixen, L., Anderson, D., & Weiss, R. (2003). Project THREAD: technology helping restructure educational access and delivery. Educational

Technology Research and Development, 51(3), 41–56.Suharwoto, G. (2006). Developing and implementing a technology pedagogical content knowledge (TPCK) for teaching mathematics with technology. In C. Crawford,

D. Willis, R. Carlsen, I. Gibson, K. McFerrin, J. Price, & R. Wever (Eds.), Proceedings of Society for Information Technology and Teacher Education international conference 2006(pp. 3824–3828). Chesapeake, VA: AACE.

Suharwoto, G., & Niess, M. L. (2006, March). How do subject specific teacher preparation programs that integrate technology throughout the courses support the devel-opment of mathematics preservice teachers’ TPCK (technology pedagogical content knowledge)?. In Paper presentation for the Society of Information Technology andTeacher Education (SITE) annual conference. Orlando, Florida.

Tondeur, J., van Braak, J., Sang, G., Voogt, J., Fisser, P., & Ottenbreit-Leftwich, A. (2012). Preparing pre-service teachers to integrate technology in education: a synthesis ofqualitative evidence. Computers & Education, 59, 134–144.

Voogt, J., Fisser, P., Roblin, N. P., Tondeur, J., & van Braak, J. (2012). Technological pedagogical content knowledge – a review of the literature. Journal of Computer AssistedLearning, 29(2), 109–121. http://dx.doi.org/10.1111/j.1365-2729.2012.00487.

Wetzel, K., Foulger, T. S., & Williams, M. K. (2009). The evolution of the required educational technology course. Journal of Computing in Teacher Education, 25(2), 67.