Assessing Cultural Awareness and Linguistic Competency of

EFL Learners in a CMC-based Active Learning Context

Wen-chi Vivian Wu

Associate Professor and First Author

Providence University

200 Chung-chi Rd.

Taichung , Taiwan 43301

wcwu@pu.edu.tw

Michael Marek

Associate Professor

and Corresponding Author

Humanities 409A

Wayne State College

Wayne, Nebraska, 68787, USA

mimarek1@wsc.edu

Abstract: Most studies examining Computer Mediated Communication to enhance EFL

learning have been limited to evaluating test scores, questionnaires, and interviewing students.

This study used a holistic assessment model with qualitative analysis of student reflective

essays and descriptive analysis of pre- and post-test oral performance by students in a

university English conversation course in Taiwan. The instructional design included Internet

videoconferences with a native speaker in the United States and an emphasis on critical

thinking about cultural differences and similarities. Although the class was too small for

statistical analysis, the students' oral scores improved an average of 44.5% from the beginning

to end of the semester. The analysis of the essays showed that students engaged in reflective

thinking about 70% of the items they observed in their essays, which is evidence of critical

thinking. Qualitative analysis revealed four major recurring themes in the students' essays, 1)

this was a different kind of class from any they have taken before and the instructional design

was helpful, 2) the importance of history and underlying meaning of iconic cultural elements,

such as holidays and festivals, 3) their development of a growing multicultural worldview,

and 4) their growing confidence as a result of the course.

Keywords: EFL, CMC, cultural awareness, critical text analysis, holistic

assessment, linguistic competence, videoconference, critical thinking, portfolios

1. Introduction

EFL classroom interactions are often interesting and enjoyable for learners, but do they

lead to actual improvements in English ability or cultural awareness? This study presents a

model for multi-dimensional assessment in such learning contexts to provide rich evaluation of

student learning outcomes as well as assessment of the effectiveness of a technology-rich

instructional design. There has been little research to date closely focused to the holistic

assessment methodologies and international collaboration used in this study, making it fertile

ground.

The site of this study was Taiwan, where the educational system treats English as a

classroom subject and learning happens only in certain locations or places, like schools or

English cram (special tutoring) schools. These settings rarely include meaningful interaction with

native English speakers (Huan, 1989; Kwon, Shih, Renandya, & Koike, 2000).

On the other hand, there is a growing body of literature about use of the Internet and

technology to enrich EFL instruction (Liu & Chen, 2007; Payne & Ross, 2005; Tudini, 2003; Wu

& Marek, 2009). The Internet, in particular, allows students to expand their horizons far beyond

the limits of the classroom, with previously non-existent access to other cultures and language

users (Felix, 2002; Hauck & Youngs, 2008; Rogerson-Revell, 2003; Shawback & Terhune, 2002),

allowing students to aspire toward a future globally-involved version of themselves, therefore

producing higher motivation (Dörnyei, 2005; Lamb, 2004). The general theme in the literature is

that such technology-rich learning provides authentic, realistic, and natural interactive

experiences that are much-desired by students (Bax, 2003; Sengupta, 2001).

EFL programs that make extensive use of technology, like many other academic

programs, frequently face a requirement to review what students have learned as a tool for

program improvement (Peim & Flint, 2009). Assessment, therefore, is often an expectation for

EFL programs, and that assessment needs to be grounded in the context of the class and the

program in order for the assessment to be truly meaningful (Fox-Turnbull, 2006). Technology

can be assessed, and technology can be used to conduct academic assessment, but the current

study addresses the challenges of assessing academic programs that make rich use of technology,

beyond simple tabulation of grades (Field, Freeman & Dyrenfurth, 2004).

In light of the drawbacks of Taiwan’s EFL learning environment, and the need for

assessment techniques that fit educational contexts which make strong use of technology, the

Taiwanese and American authors of this paper collaborated on a project that combined

student-centered instruction, active learning, and authentic materials in the form of Internet

videoconferences with a native speaker, plus assessment via multiple techniques going beyond

simple grades. To accomplish this, the American consistently, via Internet videoconference,

interacted over an entire semester with learners in an advanced English conversation class taught

by the Taiwanese researcher.

The Taiwanese collaborator is a native speaker of Mandarin, teaching Conversational

English at a large private university in central Taiwan. The American collaborator is a native

English speaker teaching at a public college in the central United States. Both have educational

philosophies that downplay traditional lecture/memorization and emphasize student-centered

active learning.

1.1. Purpose of the Study

The learning objectives of the conversation class were to enhance the students’ cultural

knowledge and to increase their linguistic competency, using a methodology grounded in

interactive media and Computer Mediated Communication (CMC). The companion research

goals of the project were:

1. To demonstrate assessment of cultural awareness of students about other cultures, and

2. To demonstrate assessment of changes in student English linguistic competency.

The study was delimited to making these demonstrations in the context of the type of

student-centered instruction, featuring active learning and authentic materials in the form of

Internet videoconferences with a native speaker, which is typical of past research by the authors

(see the literature review). Although a wide range of data was collected during the semester, this

study was also delimited to two specific sets of data, the final reflective essays written by

students as part of their final portfolios to measure the students’ critical thinking ability about the

target culture, and the oral pre- and post-tests to assess actual changes in the students’ linguistic

competency.

1.2. Significance of the study

There are many studies which have attempted to measure outcomes of cross-cultural

leaning programs and/or effectiveness of inter-cultural curricula (Menard-Warwick, 2009; Su,

2011; Zaltsman, 2009). However, Kauffman, Martin, and Weaver (1992) argued long ago that

traditional approaches, like surveys and interviews, tend to be too limiting and “reductionistic”

(p. 144) and called for new holistic, synergistic, and multifaceted approaches. In view of this, the

researchers decided to adopt the portfolio as an assessment tool (Jacobson & Florman, 2011; Su,

2011), including an analysis of the degree of cross-cultural awareness by locating evidence of

critical thinking in student-generated texts, a qualitative examination of themes in the essays, and

an examination of the improvement in student ability via pre- and post-test recorded oral answers

to specific questions.

Holistic approaches like this are at the cutting edge of measuring knowledge about the

target culture, but this study is significant for an additional reason. In this conversation class,

students were provided with the opportunity to improve their oral communication skills while

reflecting on both the target and local culture, as inspired by their CMC with the American

professor. Such emphasis on foreign culture is rare in conversation classes Taiwan.

2. Literature Review

2.1. Drawbacks of EFL Environment

This study was situated in Taiwan, where there are few English speaking people outside

of the classroom and English is considered to be a foreign language (Huan, 1989). South Korea

and Taiwan, as well as Japan, have made English a mandatory part of compulsory education, but

these countries lack a surrounding community of native speakers, which makes it premature to

consider their learning environments to be English as a Second Language (Kwon, Shih,

Renandya, & Koike, 2000). This lack of English speakers outside the classroom increases the

challenge for EFL instructors (Parker, Heitzman, Fjerstad, Babbs, & Cohen, 1995) to develop

authentic language-learning experiences to supplement conventional in-class lessons (You,

2003).

Computer Mediated Communication (CMC) was used in this study because, particularly

when using the Internet for interactivity, CMC has been shown to engage and motivate students

(Peterson, 2009). When the CMC interaction includes voice communication, it is particularly

effective for improving the listening and speaking skills of the student (Mahfouz & Ihmeideh,

2009), although multiple CMC modes, both synchronous and asynchronous, can foster

cooperative intercultural relationships and learning that strengthen both confidence and fluency,

particularly when used together (AbuSeileek, 2007; Sha, 2009; Stewart & File, 2007).

Only creating an environment as close to the target culture as possible, that is natural and

organic, will enable and motivate students to learn advanced English (Lin, 2003; Wu, Yen, &

Marek, 2011). Online learning, employing authentic interaction with native English speakers in a

way that helps the students understand the usage and connotations that flow from cultural

references, can provide a solution, but the prerequisite is that the students must see the potential

long-term value of learning English for membership in the broader community of the world

(Dörnyei, 2005). In this study, it was the goal that the CMC interactions with the native speaker

would increase the motivation of the students and give them an additional feel for Dörnyei’s

international community, in addition to the specific information in the native speaker’s lessons.

2.2. The importance of program evaluation

The researchers were inspired by the John M. Norris position paper, “The Why (and How)

of Assessing Student Learning Outcomes in College Foreign Language Programs” (2006). Norris

discussed the challenges and opportunities that the movement to assess student learning creates

for foreign language instructors. According to Norris, the potential of assessment, if done right, is

to facilitate student learning and also to improve programs, making a suitable assessment

approach vital. In order to assess their programs, instructors must evaluate students by gathering

information about their learning (Sullivan, 2006). Brislin and Yoshida (1994) said cultural

awareness is an important factor in assessing student learning outcomes because attitudes toward

the target culture and skills in communication play an equally important role. In Taiwan,

assessment of EFL learning has usually been limited to paper-based examinations with

right-wrong answers. The researchers intended to use this study to demonstrate models of

assessment that could contribute to a more holistic examination of student learning outcomes.

2.3. Portfolio Analysis

Portfolio assessment is a notable assessment tool that is relatively new to Taiwan (Su,

2011). A portfolio is defined as “a purposeful collection of student work” (Paulson, Paulson, &

Meyer, 1991, p. 60). It is a tool faculty members can use to prompt, guide, and assess student

learning and students have significant responsibility for using the portfolio to identify and

articulate what they have learned (Jacobsen & Florman, 2011). Although students self-select the

contents of the portfolio, guidance and direction are still provided by the instructor regarding

what content best reflects their learning and growth. Even though the technique has been widely

adopted in other countries for specific contexts, portfolio assessment is not widely regarded by

Taiwanese instructors as an alternative to traditional testing for core courses. However, because

the portfolio in an EFL class constitutes a cross-cultural encounter in itself, with the students

expressing their own cross-cultural experiences, portfolios can strengthen learning, as well as

provide a vehicle for student and program evaluation (Lo, 2010).

This selection of portfolio content is the initial step required for assessment and is

described by Hill and Ruptic (1994) as “The process of gathering evidence and documenting a

child’s learning and growth” (p. 8). Another definition of portfolio assessment echoes the point

of Hill and Ruptic stating that “the purpose of good assessment is to inform instruction and,

simultaneously, to provide…accurate and meaningful information regarding the student’s

progress (Gillespie, Fore, Gillespie, & Leavell, 1996, p.480). Therefore, the goal of assessment is

shifted from merely looking at the final product of student work but rather to identifying a wide

range of factors influencing individual learning, growth, and progress. For this study, one

element of the portfolio was assessed, a final reflective essay, even though the students’ grades in

the class included a complete review of their portfolios.

2.4. Critical Thinking

Critical thinking refers to meaningful, logical, and goal-oriented thinking (Bean, 2011). It

has not traditionally been stressed in the educational system in Taiwan in part because the

national college entrance examinations are primarily multiple choice questions. In high school,

therefore, students are not trained to perform the kind of critical thinking that would be required

for extensive essay questions and, as a result, are most likely to be taught by rote memorization.

Critical thinking plays an important role in the English writing of students, such as the

reflective essays used in the instructional design of this study, because it is related to the

cognition and use of strategies by the students (Gokhale, 1995; Halpern, 1997). To promote

critical thinking by students, researchers have developed multiple methods that can be employed,

including collaborative learning, on-line communication, and research paper writing (Johnson &

Johnson, 1986; Beckman & Hensel, 2009).

2.5. Instructional methodology

The literature review above led the researchers to the conclusion that the most advanced

instructional techniques should provide a way to overcome insufficient access and inadequate

exposure to native speakers and the cultures of the English language. Previous research has

shown that when the instructional methodology stresses authentic and interactive engagement

using interactive Internet technology, it builds confidence and produces increases in student

knowledge about the target culture; and also that student centered, active learning, using

technology for cross-cultural interactions, contributes to improvement in the English proficiency

level of learners (Wu & Marek, 2010). Specifically, “live” videoconferences with native speakers,

both scheduled and ad hoc via social media, have been shown in many studies to be effective in

improving the motivation, confidence and ability of EFL students (Juell, Brekke, Vetter &

Wasson, 1996; Ko, 2012; Liaw & Master, 2010; O’Dowd, 2005; Ramirez, 1998; Saw, Majid,

Abdul Ghani, Atan, Idrus, Rahman & Tan, 2008; Smyth, 2011; Whyte, 2011). Without careful

consideration of the instructional design, however, CMC for language instruction may end up

being online lecture and not provide significant interaction. As a result, students could tend to be

passive, as if they were watching television (Gillies, 2008).

This project, therefore, adapted and expanded an instructional design already used over

several semesters by the researchers, employing videoconferences to connect foreign language

students with native speakers (Marek, 2008; Wu & Marek, 2007; Wu & Marek, 2008; Wu &

Marek, 2009). Previously, the authors had only examined student perceptions of changes in

motivation, confidence, and ability stemming from the instructional design, but in this semester,

the researchers performed independent analysis of changes in ability and cultural knowledge.

2.6. Learning and teaching conceptual framework

There are three mainstream concepts that shape EFL discourse about teaching and

learning in the 21st Century -- Constructivism, Scaffolding, and Communicative Language

Teaching (CLT). Constructivism is an understanding of learning and teaching which is widely

accepted in the United States and takes advantage of insights into how people learn. It uses a

social philosophy in which student collaboration and student/teacher interaction leads learners to

evolve or “construct” their own knowledge foundations (Brandon & All, 2010; Bruning, Schraw

& Ronning, 1999). Constructivism depends on the building of positive relationships among

students and between students and instructors. In this way, a learning community is formed that

uses active learning and helps members construct new knowledge that is well-connected to other

knowledge already held by the students (Roehler & Cantlon, 1997).

Scaffolding is a primary tool used by teachers to foster this interaction (Lee, 2003), a

technique in which teachers initially give high support for the task to be accomplished, such as

being available to help with translation when an EFL student is interacting online with a native

speaker. The teacher, thereafter, slowly withdraws support to encourage the student to be

independent. It this way, the student develops confidence and the ability over time to function

without direct intervention from the teacher.

Communicative Language Teaching is a philosophy that emphasizes actual ability to

communicate, such as between individuals in actual interaction, as opposed to emphasizing

formal knowledge of grammar, syntax, and other technical elements of a language (Savignon &

Wang, 2003). In the CTL approach, conveying meaning is the primary consideration in language

use and technical accuracy is secondary, resulting in markedly different instructional strategies.

3. Research Methods

3.1. Participants and Instructional Design

This research was conducted during a required advanced English Conversation class

taught by the Taiwanese researcher, with 16 sophomores (14 female and 2 male) majoring in

English at a private four-year academic university in central Taiwan. Since all of them had

successfully passed a basic English conversation course and the intermediate level of a

standardized test (General English Proficiency Test), their English ability in oral communication

was sufficient to allow them to create theme-based presentations in English and interact with the

native speaker. Pseudonyms were used in data analysis to conceal the true identities of the

students.

Four times during the semester, the American researcher spoke “live” with the 16

Taiwanese students in the English class in Internet videoconferences (via SKYPE). The

American had previously provided video presentations about American cultural topics such as

American holidays, American vacations and national parks, entertainment in America, and

American clothing. Each lesson consisted of the recorded video presentation by the American

teacher, preparation of a presentation on a related topic by the students, a student "live"

presentation back to the American teacher in class via SKYPE, and follow-up conversation

between the native speaker and the student group about the topic during the videoconference.

The vocabulary was kept close to the ability of the students, but they were also challenged by

adding selected new vocabulary words specific to the cultural elements being discussed.

The instructional design required the students to be involved actively in their own

learning, using critical thinking to analyze the American’s presentations, deciding their own

presentation topics, researching selected topics, and then preparing and delivering the

presentation (see Appendix 1). During regular class time, the classroom teacher in Taiwan

provided suggested questions/topics that the students could use in writing their reflective papers,

and provided examples of comments addressing these questions that served as good examples of

critical thinking. In addition, scaffolding from team members and the instructors also played an

essential role in guaranteeing successful student presentations, including advice and suggestions

about initial brainstorming as a team for ideas, content development of the presentations, actual

composing written texts for presentations, and spiritual support and rapport with other team

members.

In order to supplement the learning activities related directly to the presentations,

students were required to keep reflective journals containing critical thinking about the

presentations of their peers, and about the feedback from the teacher and their peers about their

own presentations. The students also prepared final portfolios for submission at the end of the

semester, containing their compiled reflective journals, the materials they had collected for their

group presentations, and a final reflective paper.

The entire instructional design was crafted in light of Constructivist theory, in which

quality learning occurs when students are effective in fitting new information with things they

already know, thus constructing meaning as the result of classroom activities. In Constructivist

learning models, students have significant control over their learning methods, strengthening the

ability of the students to make connections in their own minds (Sengupta, 2001). In each step of

the project (see Appendix 1) students had significant control and the need to make connections.

Training by the Taiwanese local teacher helped the students to develop strategies to make these

decisions and connections effectively.

The classes met once a week for 100 minutes a week. Presentations were grouped so that

more than one group presented during a given class period, typically half the class one week and

the remainder of the class the following week. This provided approximately 30 minutes per

group for each presentation, up to half of which was interaction with the American following the

formal presentation.

The total classroom time spent conducting videoconferences was about 12 hours. The

instructional design, however, also required extensive interaction by the students in English as

part of planning their presentations back to the American, and included question and answer

periods at the end of the presentation in which both the American and classmates could ask

impromptu questions. The researchers judged that the time spent using English provided

extensive use and development of the students’ English skills.

3.2 Assessment Strategies

The instructional design used multiple assessment techniques to evaluate student learning

that resulted from the instructional strategies in this project, because no single assessment tool

can give a complete picture about student learning, knowledge about the target culture, and

cross-cultural experiences. This paper focuses on two major components of the classroom

evaluation, oral tests taken by each student at the beginning and end of the semester, recorded in

audio for analysis, and final reflective essays written by each student.

Both of these components were assessed via forms of Critical Text Analysis (CTA), a

process of analyzing a text (any print, aural, or visual content) via multidimensional analysis.

CTA scrutinizes the text for ideas and meaning, which are, themselves, socially constructed

(Bourne & Jewitt, 2003). Therefore, Critical Text Analysis is an appropriate approach for

evaluation of student portfolios (Ingulsrud, et al., 2002).

3.2.1 Critical Text Analysis Strategy

Each student in the class wrote a final reflective paper following a framework

recommended by the Taiwanese teacher. Resource material for preparing the paper included a

statement of expectations for the conversation class, an observational journal, PowerPoint

presentations, video recordings of the American’s presentations, and video recordings of in-class

videoconference interaction. The papers were about 1,000 words, written in English. See

Appendix 2 for prompt questions provided to the students.

The essays were first evaluated for cross-cultural recognition and reflection by the

American researcher and three of his American colleagues who taught communications-related

subjects and had the necessary experience to perform the Critical Text Analysis. Although this

study used a fundamentally qualitative methodology, findings from the CTA examination were

quantified in order to help assess the degree of cultural awareness of the students. Table 1 shows

the format used for critical text analysis of the essays. This descriptive analysis was not intended

to be generalized to other populations, but simply to describe the progress of the students in this

study.

The reflective essay analysis was based on a critical thinking approach. It is a primary

tenant of EFL instruction that learning about the native culture of the target language is integral

to learning a language (Liaw & Master, 2010). This is because advanced use of an L2 language,

and particularly English, requires understanding and use of cultural references and idioms.

Critical thinking is vital to understanding these references and idioms, interpreting them in light

of the student’s L1 culture, and applying the understandings to language use (Beckman & Helsel,

2009). This study employed critical thinking as a research variable because of its importance to

effective L2 language use.

Raters scrutinized the text in order to locate examples where students noticed or

recognized things that were significant to them. In other words, what students recognized and

what they choose to observe were important points in evaluation of their text. The concepts of

this recognition were grouped into the three categories: physical, intercultural, and personal

aspects based on the rubric of Ingulsrud, et al. (2002) as described below. For example:

a. Physical: Grand Canyon is large and deep.

b. Intercultural: American weddings usually take place in church and have receptions

after the wedding.

c. Personal: I would like to visit American someday to have fun.

Raters were also asked to determine whether there was actual reflection involved, based

on the recognition. The process of reflection was defined by John Dewey (1966) as critical

thinking in which students connect what they have observed or recognized to the knowledge base

in their native culture by critiquing both target culture and native culture and making

comparisons and contrasts between the two.

Combining the ideas of Dewey (1966) and Ingulsrud, et al. (2002), the researchers used a

model of critical thinking in which the first stage of critical thinking is recognition. Advanced

critical thinking, however, was not considered to have taken place unless the student performed

processes such as comparing and contrasting, analyzing consequences and outcomes, and other

advanced reflective thinking skills.

The rubric to quantify the Critical Text Analysis was based on the following two

postulates by the researchers:

a. Refection can happen only if recognition exists first. In other words, recognition is the

perquisite for reflection.

b. Therefore, recognition is more common in an individual’s thinking process, and

presumably occurs more frequently. In other words, there are times where recognition can be

located in a text without any accompanying reflection.

The formula used by the researchers for the critical text analysis portion of this study was

inspired by Ingulsrud, et al. (2002), although the researchers made significant changes to

Ingulsrud’s methodology. Ingulsrud used a construct in which cultural awareness (A) was

defined as recognitions (C) plus reflections (R), A=R+C. Based on their review of literature,

however, the authors of the current study concluded that awareness was less important than

critical thinking about the students’ own and other cultures thus addressing a flaw that Ingulsrud,

et al., identified in their own study, that a student with numerous but superficial recognitions

could still achieve a high score with no critical thinking.

The researchers, therefore, used the following formula, inspired by Ingulsrud, et al. (2002)

to measure the degree of critical thinking about other cultures:

C

RA

In this construct, A stands for cultural critical thinking in the EFL context. R and C stand

respectively for the number of reflection and recognition incidents in the text. Ideally, there

would be at least one reflection for every recognition, if a student reflects on every recognition

incident. In reality, the students sometimes performed more than one dimension of reflection for

a single recognition. The highest score was defined as indicating the highest degree of cultural

critical thinking in the EFL context. Table 1 shows the rubric worksheet used by the reviewers to

tabulate recognitions and reflections, also adapted from Ingulsrud.

Table 1

Critical Text Analysis of Essays

Student Pseudonym: Cross-Cultural Recognition (C) Reflection on Recognition (R)

Notes Physical Intercultural Personal Home, Culture

& Society

Student’s Past Student’s

Future

Total Recognitions__________ Total Reflections_________

3.2.2 Pre- and Post-Test Assessment Strategy

Each student also took an oral pre-test at the beginning of the semester and the same test

at the conclusion of the semester. The tests, which asked questions about the American cultural

information to be taught in during the semester. For example, to align with the lesson about

American holidays, the oral test included the question “what is your favorite American holiday

and why? Explain how Americans celebrate this holiday.” The students were provided the

questions that they would be answering on the pre-test in advance and the same questions were

used on the post-test in order to allow the students to prepare and express their own thoughts.

The researchers judged that particularly on the pre-test, if the EFL students had not received the

questions in advance, they would be highly intimidated by the unexpected questions. The oral

tests were recorded for qualitative audio analysis and evaluated by the same American college

teachers who performed the essay Critical Text Analysis. The evaluation rubric for the oral tests

included up to five points each for fluency, pronunciation, grammar, vocabulary, and content

knowledge. Each category had descriptions suggesting point values. See Appendix 3 for the

rubric, which was inspired by the 6+1 Trait model of assessment (Northwest Regional

Educational Laboratory, n.d.).

3.3 Data Analysis

For assessment, the four Americans read the final essays of the students to evaluate the

students’ linguistic competence and cultural awareness. The evaluators performed initial sample

evaluations together as a group, to ensure understanding of the rubrics and methodology. The

evaluators then worked independently to assess the essays and verbal tests. The same American

reviewers also listened to and separately evaluated the 16 pre-test audio recordings and 16

post-test audio recordings made by students, also using an evaluation rubric based on the 6+1

Traits model of evaluation (Northwest Regional Educational Laboratory, n.d.).

The points determined by the rubrics for each of the 16 students in each of three

categories (essay recognitions, essay reflections, and pre-test/post-test improvement) were

analyzed by Excel spreadsheet for basic descriptive percentages and averages. Because of the

small size of the class, the researchers did not attempt further statistical analysis of the data.

In order to understand the experiences of the students in more detail, qualitative analysis

was also performed on the final essays of the 16 students in the class to identify recurring themes.

The main points made by students were sorted and grouped into themes and the actual statements

of the students were retrieved so that their specific words could be quoted to most clearly express

their ideas.

4. Descriptive Analysis

The American reviewers read and independently evaluated the 16 student essays using

Critical Text Analysis. This qualitative analysis was quantified for descriptive analysis. The data

for recognitions ranged from one student listing only seven recognitions, determined by one

reviewer, to another student making 50 recognitions, determined by another reviewer. The data

for reflections ranged from seven reflections, identified by three of the four reviewers for one

student, to 29 reflections identified by the remaining reviewer for another student. The reviewers

found an average of 21.5 recognitions in each paper and an average of 14.8 reflections per paper.

Applying the A=R/C formula, the researchers determined that the students averaged .705

reflections per recognition, defined as a cultural critical thinking (A) value of 70.5%.

The same American reviewers listened to and separately evaluated the 16 pre-test audio

recordings and 16 post-test audio recordings made by students, also using Critical Text Analysis,

with the recording serving as aural texts. On the pre-test, students averaged 14.6 of a possible 25

points. The low score was seven for one student and a high of 20 for three different students,

rated by various reviewers. On the post-test, the students averaged 20.8 of a possible 25 points,

for an average improvement of 44.5%. Individual students ranged from a 12.28% improvement

to a 68% improvement. Table 2 shows the pre- and post-test scores, and the percent of

improvement in each category of the rubric.

Table 2

Pre- and Posts Test scores

Total Possible Pre-Test Post-Test Improvement

Fluency 5 2.8 4.2 51.50%

Pronunciation 5 3 3.3 11.10%

Grammar 5 3 3.8 25.00%

Vocabulary 5 3 4.6 52.80%

Content 5 2.7 5 87.50%

Overall 25 14.5 20.9 44.50%

In using this rubric, the researchers knew that the content knowledge of the students at the

beginning of the semester would be low, because the material had not yet been addressed in class.

Similarly, the researchers assumed that at the end of the semester the content knowledge would

be higher, because the material had, by then, been studied in depth during the semester. Two

thirds of the overall improvement of the students, however, came from the other four rubric

categories.

5. Qualitative Findings

Critical Text Analysis revealed four major themes recurring throughout the students’

essays. Each of the four themes was addressed by at least three-quarters of the students. They

were 1) this was a different kind of class from any they have taken before and the instructional

design was helpful, 2) the importance of history and underlying meaning of iconic cultural

elements, such as holidays and festivals, 3) their development of a growing multicultural

worldview, and 4) their growing confidence as a result of the course.

5.1. A Different Kind of Class

The students recognized that their class had been taught in a different way from past

classes they had taken and they believed that they benefitted from the alternative instructional

design and from the integration of technology. “This oral training class is quite different from

what I took last semester,” said Student Three. “In this class [the teacher] not only pays a lot of

attention to correcting our pronunciation but also encourages us to speak more bravely and think

more critically.”

“It is not easy to speak English in front of everybody,” commented Student Seven.

“And if you make one mistake, you will feel embarrassed about it. Then you become more

nervous. But I think I have overcome the drawback and now I can easily have conversation in

English. This is the first time I think that a class is really useful.”

Student Two said she had learned the need to “think in English” as opposed to literal

translation from Chinese into English. She mentioned the words “Taiwan Up” used in a

fireworks display at the Taipei 101 building. “We understand that it means Taiwan will be better

in many aspects,” she said, “but the native speakers did not understand…and people all over the

world saw this mistake with the fireworks.” She said she wants to be able to think in English so

as to not make such mistakes.

Student Nine began the semester feeling that spoken English was her greatest personal

need but was surprised that the class contained so much discussion of cultures delivered via

technology. “I had anticipated English Oral Training with some nervousness,” she said. “I didn’t

expect the English Oral Training to widen my knowledge, and I really thank the teacher. I have

gained a lot in this semester.”

5.2. History and Underlying Meaning

Many of the students commented that the critical thinking skills learned in the class had

led them to understand the importance of knowing the history and origin of their own cultural

icons, promoting cultural understanding. Many such comments were in the context of the online

presentations about American and Taiwanese holidays/festivals during the semester.

“It seems that we gradually forget the origin and root of every festival or holiday

belonging to our country,” commented Student One. “I think spending time celebrating on the

specific day is not the last purpose, but we should have deep thought about why we celebrate the

day.”

Student Eleven agreed, saying that the critical thinking required for her group’s Internet

presentations to the American caused to her to understand her own culture better. “We all know

we have to celebrate our significant holidays,” she said, “but most of us have no idea about its

background…if we didn’t have this presentation, I think I will never know the information about

so many stories.”

Student Nine wrote about many contrasts and similarities between American and

Taiwanese celebrations, concluding that, “I like the conviviality in Western culture, but I also

like the coded meaning of the holiday in traditional Chinese culture.”

Other students saw a similar need to understand history and traditions in other domains.

Student One said that after reading about the original of the American Yosemite National Park

for her online presentation, “something occurred in my mind…that what we should value was

not only the concrete national park but the influential origin.” Student Eight said that her group

researched the historical background of Fort San Domingo in Taiwan. “It was a special

experience to make me realize the historical antiques in Taiwan,” she said.

5.3. Multicultural Worldview

The students often used their essays to reflect on the need for a multicultural worldview,

and believed that the technology-rich experience of the class helped them cultivate such an

attitude. “The oral training class allowed me to broaden my world view to understand cultures

about food, festival, living style, and activities in the United States,” Student Sixteen said. “I

insist that college students should know what is happening outside of Taiwan…because now

there is a word called ‘Global Village.’ Distance between each country is not far anymore.

They are all neighbor to us and we are supposed to realize what is happening in their country.”

“It’s very complicated and challenging [to understand American culture] because I’m not

a Western and I’ve lived in Taiwan so far,” Student Six said. “I’ve got all information about

America by TV, newspapers and the Internet. Therefore, I have no idea whether the information

is true or false.” She expressed her appreciation of the authentic cultural components of the class

allowed by the online interaction.

“We’ve learned many things from time to time,” observed Student Fourteen. “Like at the

very first beginning, we always said ‘foreigners’ but afterwards we know that we should say

‘native speakers of English.’ This word is very disrespectful…so I’m glad that I learned this

important information.”

The student essays contained many anecdotes about improved understanding of American

culture resulting from the online interaction in the class. “Something that impressed me so much

was Christmas ‘stockings’ because I thought it was Christmas ‘socks’ before,” said Student

Fifteen, recalling a vocabulary correction from the American. “This information helped me a lot

due to it corrected my understanding about Christmas.”

5.4. Growing Confidence

Most of the student essays reflected on their growing confidence in the use of English.

They uniformly believed that the instructional methodology of the class, connecting them to a

native speaker via technology, made them more confident in their English speaking and listening

skills.

Student Eight observed that talking with a native English speaker, as she did online in this

class, is a significantly different experience from speaking with classmates, who themselves

often do not have strong English skills. “It is such a shame for an English department college

student walking on the street and seeing there is an American walking toward you, [but] you are

not going to say hi to him or her,” she said. “So, lucky as I am, I have a great opportunity to have

online learning with [the American]. We cherished the time we had on the stage [at the front of

the classroom talking online with the American]. I would like to learn again by this way and I am

so glad that I have had this opportunity to learn.”

“In the past, I was afraid to speak English with native speakers,” said Student Nine, “but

now I cannot only discuss and have a conversation with a native speaker bravely, but also

significantly increase my English speaking ability. American culture is not as strange as before I

learned it. I have more self-confidence to talk to foreigners bravely.”

Student Ten actively strategized ways of improving her English. “I have to listen to ICRT

[an English language radio station in Taiwan] every night to train my listening. And I have to try

my best to speak English in the class…because if I don’t do it every day, I only improve a little.”

6. Discussion

This project represented the first attempt of the researchers to move beyond past research

in online EFL interaction, in which findings were based purely on student perceptions, into

independent assessment of student learning. The findings show that students thought critically

about the subjects discussed in class, and improved their fluency, pronunciation, grammar,

vocabulary, and content knowledge. The content knowledge would be expected to increase as a

result of learning during the semester, but the five possible maximum points for content

knowledge did not exclusively account for the 44% average improvement across the entire pre-

to post-test range. The overall instructional methodology, grounded in technology, was

responsible for engaging the students in active learning.

It is not surprising that students varied in their display of critical thinking, measured in

their reflective essay calculations of recognitions and reflections. Particularly in Taiwan, where

critical thinking is not emphasized in many primary and secondary schools (Wu, 2006), some

students have little experience, whereas others have more advanced critical thinking ability by

the time they reach the college level. In addition, although the instructors used multiple

techniques to attempt to foster motivation, some students may have taken the class more

seriously than others.

In structuring the Critical Text Analysis for this study, the authors used the methodology

of Ingulsrud, et al. (2002), but addressed a limitation that the authors of that study identified

about their own work, i.e. the lack of emphasis on critical thinking in their evaluation formula.

The current study followed recommendations in the literature and stressed critical thinking about

the cultural information presented via technology in the classroom (Beckman & Hensel, 2009).

The American researcher’s online presentations addressed cultural similarities and differences,

and the student’s presentations back to the American required critical thinking to compare and

contrast related elements in the American and Taiwanese culture, fostering critical thinking

skills.

The themes identified in the qualitative analysis substantially reinforce past findings

about the merits of instructional designs and uses of technology that stress student-centered

instruction, active learning, and authentic materials in the form of Internet videoconferences with

a native speaker (Peterson, 2009). The students understood the benefits of the instructional

design clearly and appreciated that the critical thinking skills taught in the class would help them

continue to improve their understanding of other cultures after their formal educations end.

Therefore, the authors believe that they succeeded in both goals of the current research

project, demonstrating assessment of cultural awareness and critical thinking of students about

other cultures, and demonstrating assessment of changes in student English linguistic

competency. These successful demonstrations were conducted in the context of the learning

objectives of the class, enhancing the students’ cultural knowledge and increasing their linguistic

competency, using a methodology grounded in interactive media and CMC.

7. Final Thoughts

The new factor presented in this paper, external assessment via Critical Text Analysis, has

validated past findings of student perceptions that the authors’ instructional technology and

design leads to improved ability, i.e. linguistic competency. The findings also show that the

instructional design employed in the project can be used to stimulate critical thinking about the

multiple cultures encompassed by EFL learning in the 21st Century.

Assessment is a central issue in the evaluation of the effectiveness of use of technology in

the EFL classroom (Sullivan, 2006) and in the evaluation of academic programs based on the

outcomes of their students. Technology is not used for its own sake. Rather, it is used to enrich

the classroom materials, achieve course goals, and assist the students to understand the accents of

native speakers, as well as the nuanced references and metaphors used by native speakers, which

are vital to idiomatic use of the English language. In the EFL environment, therefore, effective

use of technology is not just important but vital to preparing the student to function in the

cosmopolitan international culture postulated by Dörnyei (2005). As EFL has evolved from

training to interact only with native speakers, to training to interact with others who also speak

English as a second or foreign language, the need for diverse international experiences in the

classroom has burgeoned. Technology, used effectively, can help prepare the students for the

multi-dimensional perspective they must have to function in the world marketplace.

In the final analysis, the overarching goals of any EFL classroom are to 1) help the

students achieve the necessary level of ability in the target language, and to 2) leave them

motivated to continue using and learning the new language after their formal instruction ends.

In the second decade of the 21st Century, the EFL roadmap is clear to achieving these goals. It

uses student-centered classrooms, authentic materials that intrigue and engage the students, and

active learning that promotes critical thinking. Technology, and particularly the Internet, provides

a rich source for these authentic materials, up to an including live interaction with English

speakers in other countries. This study has, again, validated these instructional techniques and

has, furthermore, demonstrated a methodology by which multimodal assessment can lead to rich

understanding of the experiences of the students in order to even further use technology to center

the learning experience directly on the students.

References

AbuSeileek, A. F. (2007). Cooperative vs. individual learning of oral skills in a CALL

environment. Computer Assisted Language Learning, 20(5), 493-514.

Bax, S. (2003). CALL-past, present and future. System, 31, 13-28.

Bean, J. C. (2011). Engaging ideas: The professor’s guide to integrating writing, critical thinking,

and active learning in the classroom (2nd

ed.). San Francisco: John Wiley & Sons, Inc.

Beckman, M. & Hensal, N. (2009). Making explicit the implicit: Defining undergraduate

research. Council on Undergraduate Research, 29(4), 40-44.

Bourne, J., & Jewitt, C. (2003). Orchestrating debate: A multimodal analysis of classroom

interaction. Reading, 37(2), 64-72. doi: 10.1111/1467-9345.3702004

Brandon, A. F., & All, A. C. (2010). Constructivism Theory Analysis and Application to

Curricula. Nursing Education Perspectives, 31(2), 89-92.

Brislin, R. & Yoshida, T. (1994). Intercultural communication training: An introduction.

Thousand Oakes, CA: Sage Publications.

Bruning, R. H., Schraw, G. J., & Ronning, R. R. (1999). Cognitive psychology and instruction

(3rd ed.). Upper Saddle River, NJ: Prentice-Hall.

Dewey, J. (1966). Democracy and education. NY: Free Press.

Dörnyei, Z. (2005). The psychology of the language learner: Individual differences in second

language acquisition. Mahwah, NJ: Lawrence Erlbaum.

Felix, U. (2002). The web as a vehicle for constructivist approaches in language teaching.

ReCALL, 14(1), 2–15.

Field, D. W., Freeman, S. A., & Dyrenfurth, M. J. (2004). Holistic Assessment of Students in

Undergraduate Industrial Technology Programs. Journal of Technology Studies, 30(4),

78-85.

Fox-Turnbull, W. (2006). The Influences of Teacher Knowledge and Authentic Formative

Assessment on Student Learning in Technology Education. International Journal of

Technology & Design Education, 16(1), 53-77. doi:10.1007/s10798-005-2109-1

Gillespie, C. S., Fore, K. L., Gillespie, T. D., & Leavell, A. G. (1996). Portfolio assessment:

Some questions, some answers, some recommendations. Journal of Adolescent and adult

Literacy, 39(4),480-491.

Gillies, D. G. (2008). Student perspectives on videoconferencing in teacher education at a

distance. Distance Education, 29(1), 107-118.

Gokhale, A.A. (1995). Collaborative learning enhances critical thinking. Journal of Technology

Education, 7(1), 22-30.

Halpern, D. F. (1997). Critical thinking across the curriculum: A brief edition of thought and

knowledge. Mahway, NJ: Lawrence Erlbaum Associates, Publishers.

Hauck, M. & Youngs, B. L. (2008). Telecollaboration in multimodal environments: The impact

on task design and learner interaction. Computer Assisted Language Learning, 21(2),

87-124.

Hill, B. C., & Ruptic, C. (1994). Practical aspects of authentic assessment: putting the pieces

together. Norwood, MA: Christopher Gordon.

Huan, T. L. (1989) Age, motivation, and second language learning. Paper presented at The

Proceedings of the Sixth Conference on English Teaching and Learning in the Republic of

China, Taipei, Taiwan.

Ingulsrud, J. E., Kai, K., Kadowaki, S., Kurobane, S. & Shiobara, M. (2002). The assessment of

cross-cultural experience: Measuring awareness through critical text analysis.

International Journal of Intercultural Relations, 26, 473-491.

doi:10.1016/S0147-1767(02)00030-5

Jacobson, W. & Florman, J. (2011). Teaching through portfolios. Thriving in Academe 28(5). 5-7.

Juell, P., Brekke , D., Vetter, R.J., & Wasson, J. (1996). Evaluation of computer conferencing

tools for conducting collaborative seminars on the Internet. International Journal of

Educational Telecommunications, 2(4), 233-248.

Johnson, R. T., & Johnson, D. W. (1986). Action research: Cooperative learning in the science

classroom. Science and Children, 24, 31-32.

Kauffmann, N., Martin, J. N., & Weaver, H. D, with Weaver, J. (1992). Students abroad:

Strangers at home. Yarmouth, ME: Intercultural Press, Inc.

Ko, C. (2012). A case study of language learners’ social presence in synchronous CMC. Recall,

24(2), 66-84. doi:10.1017/S0958344011000292

Kwon, O., Shih, Y. H., Renandya, W., Koike, I. (2000). Opening Address. Plenary Symposium:

English Education in East Asia for the 21st Century: Seeking for Common Ground,

Okinawa, Japan.

Lamb, M. (2004) Integrative motivation in a globalizing world. System, 32(1), 3-19.

Lee, E. (2003). Learning through scaffolded assistance during interaction in the EFL classroom:

The teacher's role to make an effective learning environment. Journal of Pan-Pacific

Association of Applied Linguistics, 7(2), 189-213.

Liaw, M. L. & Master, S. B. (2010). Understanding telecollaboration through an analysis of

intercultural discourse. Computer Assisted Language Learning, 23(1), 21-40.

Lin, M. H. (2003, Sep. 14). Tsai Ying-Wen knows extremely well the international affairs, create

a new milieu through the use of negotiation for Taiwan. Merit Times.

Liu, G., & Chen, A. S. (2007). A taxonomy of Internet-based technologies integrated in language

curricula. British Journal of Educational Technology, 38(5), 934–938. doi:

10.1111/j.1467-8535.2007.00728.x

Lo, Y. (2010). Implementing reflective portfolios for promoting autonomous learning among

EFL college students in Taiwan. Language Teaching Research, 14(1), 77-95.

Mahfouz, S. M. & Ihmeideh, F. M. (2009). Attitudes of Jordanian university students towards

using online chat discourse with native speakers of English for improving their language

proficiency. Computer Assisted Language Learning, 22(3), 207-227.

Menard–Warwick, J. (2009). Co-Constructing Representations of Culture in ESL and EFL

Classrooms: Discursive Faultlines in Chile and California. Modern Language Journal,

93(1), 30-45. doi:10.1111/j.1540-4781.2009.00826.x

Marek, M. W. (2008). Internet videoconferencing to improve EFL learning. Paper presented at

the 2008 Conference on English Teaching and Global Communication, Chienkuo

Technology University, Changhua City, Taiwan.

Norris, J. M. (2006). The why (and how) of student learning outcomes assessment in college FL

education. Modern Language Journal, 90(4), 590-597.

doi:10.1111/j.1540-4781.2006.00466_2.x

Northwest Regional Educational Laboratory (n.d.). 6+1 Trait scoring. Retrieved May 4, 2013

from http://educationnorthwest.org/webfm_send/773

O'Dowd, R. (2005). The use of videoconferencing and e-mail as mediators of intercultural

student ethnography. In J. A. Belz & S. L. Thorne (Eds.), Internet-mediated Intercultural

Foreign Language Education. Heinle: Boston.

Parker, J. E., Heitzman, S. M., Fjerstad, A. M., Babbs, L. M., Cohen, A. D. (1995). Exploring the

role of foreign language in immersion education. Highland, D., Lee, P. W., Milcham, J.,

Weber, R. R. (Eds.). Second language acquisition theory and pedagogy. (pp. 235-253).

Mahwah, NJ: Lawrence Erlbaum Associates.

Paulson, F. L., Paulson, P., & Meyer, C. (1991). What makes a portfolio a portfolio? Educational

Leadership, 48, 60-63.

Payne, J. S., & Ross, B. M. (2005). Synchronous CMC, working memory, and L2 oral

proficiency development. Language Learning and Technology, 19(3), 35-54.

Peim, N., & Flint, K. (2009). Testing Times: Questions concerning assessment for school

improvement. Educational Philosophy & Theory, 41(3), 342-361.

doi:10.1111/j.1469-5812.2008.00438.x

Peterson, M. (2009). Learner interaction in synchronous CMC: A sociocultural perspective.

Computer Assisted Language Learning, 22(4), 303–321.

Ramirez, M. (1998). Conversations from afar: Improving conversation skills and cultural

understanding through videoconferencing. Paper presented at the annual meeting of the

Sunshine State teachers of English to speakers of other languages. (ERIC Document

Reproduction Service No. ED424770)

Roehler, L. R., & Cantlon, D. J. (1997). Scaffolding student learning: Instructional approaches

and issues. Cambridge, MA: Brookline Books.

Rogerson-Revell, P. (2003). Developing a cultural syllabus for business language e-learning

materials. ReCALL, 15(2), 155–169.

Savignon, S. J. & Wang, C. (2003). Communicative language teaching in EFL contexts: Learner

attitudes and perceptions. International Review of Applied Linguistics in Language

Teaching, 41(3), 223-249.

Saw, K. G., Majid, O., Abdul Ghani, N., Atan, H., Idrus, R. M., Rahman, Z. A., & Tan, K. E.

(2008). The videoconferencing learning environment: Technology, interaction and learning

intersect. British Journal of Educational Technology, 39(3), 475-485. doi:

10.1111/j.1467-8535.2007.00736.x

Sengupta, S. (2001). Exchanging ideas with peers in network-based classrooms: An aid or a pain?

Language Learning & Technology, 5(1), 103-143.

Sha, G. (2009). AI-based chatterbots and spoken English teaching: A critical analysis. Computer

Assisted Language Learning, 22(3), 269-281.

Shawback, M.J., & Terhune, N.M. (2002). Online interactive courseware: Using movies to

promote cultural understanding in a CALL environment. ReCALL, 14(1), 85–95.

Smyth, R. (2011). Enhancing learner-learner interaction using video communications in higher

education: Implications from theorising about a new model. British Journal Of

Educational Technology, 42(1), 113-127. doi:10.1111/j.1467-8535.2009.00990.x

Stewart, I. A. D., & File, P. (2007). Let’s Chat: A conversational dialogue system for second

language practice. Computer Assisted Language Learning, 20(2), 97-116.

Su, Y. (2011). The effects of the cultural portfolio project on cultural and EFL learning in

Taiwan’s EFL college classes. Language Teaching Research, 15(2), 230-252.

doi:10.1177/1362168810388721

Sullivan, J. H. (2006). The importance of program evaluation in collegiate foreign language

programs. The Modern Language Journal, 90, 590-593.

Tudini, V. (2003). Using native speakers in chat. Language Learning and Technology, 7(3),

141-159.

Whyte, S. (2011). Learning to teach with videoconferencing in primary foreign language

classrooms. Recall, 23(3), 271-293. doi:10.1017/S0958344011000188

Wu, W. V. (2006). EFL optimal learning environment: Perspectives of faculty and students in a

Taiwanese technical university, USA, Vermillion, SD: The University of South Dakota.

Wu, V. & Marek, M. W. (2007). Taiwanese student reaction to English language Internet

teleconferencing for enriching EFL classroom instruction. Journal for Language

Teaching, (41)2, 124-137. doi:10.4314/jlt.v41i2.6094

Wu, V. & Marek, M. (2008). Enhancing learner motivation to study English via

videoconferencing with a native speaker. In J. Luca & E. Weippl (Eds.), Proceedings of

World Conference on Educational Multimedia, Hypermedia and Telecommunications

2008 (pp. 4481-4489). Chesapeake, VA: AACE. (Vienna)

Wu, W. V. & Marek, M. W. (2009). The Impact of Teleconferencing with Native English

Speakers on English Learning by Taiwanese Students. International Journal on

E-Learning. 8(1), pp. 107-126. Available from ERIC database (EJ820820).

Wu, W. V., & Marek, M. W. (2010). Making English a “habit”: Increasing confidence, motivation,

and ability of EFL students through cross-cultural, computer-assisted interaction. Turkish

Online Journal of Educational Technology, 9(4), 100-112. Available from ERIC database

(EJ908076).

Wu, W-c. V., Yen, L. L., & Marek, M. W., (2011). Using online EFL interaction to increase

confidence, motivation, and ability. Educational Technology & Society, 14(3), 118–129.

You, Y. H. (2003). A study on motivation to learn English among the freshmen in the four-year

program. Project Report on Improvements on Faculty Members of St. John’s Institute of

Technology.

Zaltsman, R. (2009). The Challenge of Intercultural Electronic Learning: English as Lingua

Franca. Ricardo, F. J. (Ed). Cyberculture and New Media. (pp. 99-117). Amsterdam/New

York, NY.: Editions Rodopi B. V.

Appendix #1

Sequence of this Project

Activity Teacher Student

1. Choose a topic • •

2. Write texts & video record presentations •

3. Upload both text & video recording •

4. Watch the video presentations prior to each

videoconferencing section

•

5. Go through texts if necessary • •

6. As a group, decide on topics for each presentation •

7. Conduct on-line research for individual part of their

presentation

•

8. Make PowerPoint’s presentations or pictures •

9. In-class live group presentation (interaction with the two

instructors and among students)

• •

10. Review the process and procedure •

11. Write journals for each presentation (their own and the

other groups)

•

12. Make Portfolios •

Appendix #2

The students received the following “prompt questions” in order to stimulate their critical

thinking for their reflective essay final papers in the class:

a. What were the differences and similarities that you observed or recognized?

b. What are your reflections based on what you have organized?

c. What new knowledge and experience did you obtain? Please give specific examples.

d. To what degree did your English ability improve in terms of the four skills?

e. Did you broaden your worldview by taking this class?

f. Have you understood America more deeply and thoroughly?

g. Have you changed in your way of thinking? How has the experience of interacting

with the native speaker affected your future plans or goals regarding your English

study?

Appendix #3 - Pre- and Post-test Evaluation Rubric

Fluency

5. The speaker speaks confidently and naturally with no distracting hesitations.

Ideas flow smoothly.

3. The speaker hesitates several times, but generally seems to know the

desired words, even if it is necessary to think about them a bit.

1. The speaker has many hesitations and great difficulty remembering or

selecting words.

5

Pronunciation

5. Pronunciation is accurate, with correct inflections, numbers of syllables and

other correct nuances of pronunciation.

3. Pronunciation is satisfactory; however words sometimes have incorrect

inflections or are otherwise sometimes hard to understand.

1. Pronunciation is very hard or impossible to understand by a native speaker.

5

Grammar

5. The speaker speaks with no more incorrect grammar than a native speaker

would.

3. The speaker occasionally uses inappropriate verb tenses and/or incorrectly

uses parts of speech, however the speaker has the ability to correct grammar

without prompts.

1. The speaker makes frequent use of inappropriate verb tenses and/or

incorrectly constructs sentences or uses parts of speech.

5

Vocabulary

5. Vocabulary is sufficient to be understood in most settings and words are used

with their correct meaning.

3. Vocabulary is moderate, although the speaker sometimes needs help

identifying the correct words. There are only occasional problems with correct

meanings of words.

1. Vocabulary is very limited and/or incorrect words are often used.

5

Content

5. The speaker is knowledgeable about the subject and provides a significant

level of detail, given the time available.

3. The speaker is aware of the subject and attempts to provide relevant ideas

about it. Provides some details.

1. Speaker seems to have little or no understanding of the subject. Statements

are superficial or not relevant.

5

Total 25