Our Policies, their Text: German Language Students' Strategies with and Beliefs about Web-Based...

21
Our Policies, Their Text: German Language Students’ Strategies with and Beliefs about Web-Based Machine Translation Kelsey D. White and Emily Heidrich University of Wisconsin-Madison Abstract Most educators are aware that some students utilize web-based machine translators for foreign lan- guage assignments, however, little research has been done to determine how and why students utilize these programs, or what the implications are for language learning and teaching. In this mixed-methods study we utilized surveys, a translation task, and interviews to discover some of the students’ beliefs and strategies in using such tools. Our results suggest that students hold a range of beliefs about the usefulness of machine translators for writing assignments, and that many of them can articulate and reflect upon the strategies that they use. We discuss implications for the use of machine translation in the foreign language curriculum based on the example of university German. § The idea of machine translation is certainly not a new one; Descartes and Leibnitz specu- lated on the possibility, and it was first used for full sentences in the 1950s (Hutchins & Sommers, 1992). Students have been using translation software for foreign language (FL) assignments since the popularization of personal computers and the ability to exploit the full potential of web-based machine translation (WBMT) tools is a skill that is increasingly valued by employers (Niño, 2008). Yet whether or not WBMT has a place in the FL curriculum is still debated. Much of the research has focused on the limitations of these programs, including ways to detect and prevent their use or misuse (Fountain & Fountain, 2009; Steding, 2009). As researchers point out, however, the software is continually being improved, and it appears that students employ WBMT regardless even of policies that ban it outright (Garcia & Pena, 2011; Larson-Guenette, 2013; Steding, 2009). Such policies turn this potential tool into a taboo, thereby leaving students to surreptitiously use WBMT with nowhere to turn for help or training in how to use it to greatest benefit for language learning and use. Indeed, depriving students of real-world, electronic resources appears to stand directly in opposition to the recommendations from the American Council on the Teaching of Foreign Languages (ACTFL) for helping a student become “a 21st Century skilled learner” (ACTFL, 2011, p. 3). In that document, ACTFL and the Partnership for 21st Century Skills (P21) define technologically literate students as “productive global citizens [who] use appropriate technolo- gies when interpreting messages, interacting with others, and producing written, oral, and visual messages” (p. 14). Some researchers have suggested regarding WBMT as a tool for pro- moting electronic literacy and linguistic awareness (e.g., Alley, 2005; Burton, 2003; Williams, 230

Transcript of Our Policies, their Text: German Language Students' Strategies with and Beliefs about Web-Based...

Our Policies, Their Text: German LanguageStudents’ Strategies with and Beliefs aboutWeb-Based Machine Translation

Kelsey D. White and Emily HeidrichUniversity of Wisconsin-Madison

Abstract

Most educators are aware that some students utilize web-based machine translators for foreign lan-guage assignments, however, little research has been done to determine how and why students utilizethese programs, or what the implications are for language learning and teaching. In this mixed-methodsstudy we utilized surveys, a translation task, and interviews to discover some of the students’ beliefs andstrategies in using such tools. Our results suggest that students hold a range of beliefs about the usefulnessof machine translators for writing assignments, and that many of them can articulate and reflect upon thestrategies that they use. We discuss implications for the use of machine translation in the foreign languagecurriculum based on the example of university German.

The idea of machine translation is certainly not a new one; Descartes and Leibnitz specu-lated on the possibility, and it was first used for full sentences in the 1950s (Hutchins &Sommers, 1992). Students have been using translation software for foreign language (FL)assignments since the popularization of personal computers and the ability to exploit the fullpotential of web-based machine translation (WBMT) tools is a skill that is increasingly valued byemployers (Niño, 2008). Yet whether or not WBMT has a place in the FL curriculum is stilldebated. Much of the research has focused on the limitations of these programs, including waysto detect and prevent their use or misuse (Fountain & Fountain, 2009; Steding, 2009). Asresearchers point out, however, the software is continually being improved, and it appears thatstudents employ WBMT regardless even of policies that ban it outright (Garcia & Pena, 2011;Larson-Guenette, 2013; Steding, 2009). Such policies turn this potential tool into a taboo,thereby leaving students to surreptitiously use WBMT with nowhere to turn for help or trainingin how to use it to greatest benefit for language learning and use.

Indeed, depriving students of real-world, electronic resources appears to stand directly inopposition to the recommendations from the American Council on the Teaching of ForeignLanguages (ACTFL) for helping a student become “a 21st Century skilled learner” (ACTFL,2011, p. 3). In that document, ACTFL and the Partnership for 21st Century Skills (P21) definetechnologically literate students as “productive global citizens [who] use appropriate technolo-gies when interpreting messages, interacting with others, and producing written, oral, andvisual messages” (p. 14). Some researchers have suggested regarding WBMT as a tool for pro-moting electronic literacy and linguistic awareness (e.g., Alley, 2005; Burton, 2003; Williams,

230

2006), but they do not seem to take into consideration that many programs’ policies preventstudents from receiving the training they would need to use it productively. Thus the time is ripefor educators and administrators to consider whether such policies help or hinder students inreaching these ACTFL goals, as well as our overall goal of facilitating language learning andstudents’ successful use of German in the world. As an important first step toward developinginformed and productive approaches and policies regarding WBMT, and to understandwhether current policies are meeting students’ needs, we should first explore why learners useWBMT programs and how they interact with them. The purpose of this article is to begin thisexploration.

Review of the Literature

Only a small amount of research focuses on students’ beliefs surrounding the use of WBMTtools, and as it is, this research is often clearly influenced by teachers’ opinions and centerslargely on the pros and cons of using such tools (e.g., Niño, 2009b; Steding, 2009). Of theavailable scholarship, only two studies include student perspectives. In Niño’s (2009b) study ofadvanced learners of Spanish, she found that in particular, weaker students favored the use ofWBMT. Additionally, Larson-Guenette (2013) collected data from first- through third-yearlearners of German and reported that they had mixed beliefs about using online resources (e.g.,dictionaries and translators). Participants cited speed, simplicity of results, and improvement ofaccuracy as perceived advantages; and academic dishonesty, increased dependency onelectronic tools, and unreliability of Internet resources as potential disadvantages. Despite theirresearch, a gap remains in our understanding of students’ beliefs specific to WBMT softwareand their abilities to interact with these tools.

Several authors point to some strategies for enhancing this type of electronic literacy (seeAppendix A for a list of training resources). For example, Niño (2009a) recommends findingways to use translation programs in class to raise awareness of what these systems cannot yetdo, thereby helping them recognize some of the intricacies of language. Similarly, Cook (2010)promotes the use of translation and Alley (2005) and Burton (2003) advocate using WBMTprograms to demonstrate syntactic discrepancies to students, as part of implementing ACTFL’sStandard 4.1 (“Students demonstrate understanding of the nature of language throughcomparisons of the language studied and their own”; National Standards, 1996). Beyondlearning strategies for the actual use of WBMT software, according to Hall (2001), it is equallyimportant for students themselves to reflect on the propriety of electronic resources, because“how well we prepare language learners to meet the social, political, and economic challengesof the next several decades will depend in part on our success in integrating technology into theforeign language curriculum” (p. 60). As Krawer (1995) demonstrates, it is not the software it-self, but the users of that software who determine the utility of machine translation systems.

Critical reflection on the usefulness of pedagogical tools is crucial, because learners need tomake theirown informeddecisions about the technologies theyuse (Levy, 2009). Userswho donot understand WBMT’s limitations are likely to trust in the product simply because it comesfrom the Internet (Niño, 2009b; Steding, 2009). More specifically, they may rely on such pro-grams due to accessibility, speed of the results, and the fact they are usually free. Phillips (1998)points out that technology is “seductive” (p. 25) to students, who often have positive reactionseven to electronic resources that do not advance their abilities or learning. Presumably thisoverall attitude has not changed since the late 90s. In contrast, Lewis (1997) observes thatstudents initially have low expectations of WBMTandshows that theyaredisappointed with theraw linguistic output; only the students who engage further with the software end up with

WHITE/HEIDRICH: STUDENTS AND WEB-BASED MACHINE TRANSLATION 231

positive or realistic attitudes about it.1 However, according to Williams (2006), it is not impor-tant whether students initially evaluate WBMT positively or negatively, as long as they do reflecton it; only then will they have the opportunity to develop critical perspectives of these tools andtheir role in learning and communicating in German.

It is also useful to note research findings on the use of WBMT as a resource for FL writing.Software applications are, of course, still imperfect and lack the versatility and cultural under-standing required for highly accurate and nuanced translation (Niño, 2009b; Sedrak, 2000).Therefore, in order to make use of this tool toward producing second language (L2) texts, theuser would first have to recognize the types of language errors that typically are produced byWBMT, if we can identify “typical” errors at all. For translation from German into English,WBMT is particularly flawed when working with texts containing large amounts of pragmaticinformation and polysemous lexical items (Calude, 2003).2 In addition to word choice,problems typically involve syntax and style (Lewis, 1997). According to Lewis, the user mustalso consider text type and target readership to judge the quality of output, determine whethercomprehensibility is impaired, and ultimately, improve the machine-generated output.Williams (2006) points out that this process can not only be beneficial in terms of functionalelectronic literacy (e.g., using technology), but also for developing critical literacy (e.g., ques-tioning technology) and rhetorical literacy (e.g., producing or influencing technology).

In sum, there are essentially two types of literature available on WBMT: (1) publications thatcensure the use of WBMT and provide educators with recommendations for its detection andprevention; and (2) articles that provide recommendations for how studentsand teachersmightuse WBMT as a resource for promoting electronic and linguistic awareness. Little research hasbeen done to determine whether students engage strategies for using WBMT without specifictraining, and to discover what they hope to gain by using the tools even when they areforbidden. This imperative for empirically driven scholarship on this important aspect of FLeducation and everyday life led to the following research questions:

1. What strategies do students employ before and during translation of a text from their first language(L1) into the L2 (German) using WBMT?

2. What strategies do students employ in editing WBMT textual output?3. What beliefs do students report about their interactions with WBMT?

Answers to these questions, even preliminary or tentative ones, would go a long way towardlaying the groundwork for further empirical study of this important but often ignored compo-nent of many L2 German learners’ experiences.

232 UP 46.2 (Fall 2013)

1 Though the findings from Lewis are valid, it should be noted that raw WBMT outputs have substantiallyimproved in the last decade.

2 For example, the English word “left” is polysemous: an adjective indicating direction, a noun indicatingpolitical leanings or a location, and also the past and past participle of the verb “to leave.” Depending on theinput of the English sentence containing the word “left”, the translator may provide a mistranslation. Many id-ioms are also dependent on pragmatic and contextual information; for instance, WBMT would never trans-late “It’s all Greek to me!” to the equivalent German idiom of “Ich verstehe nur Bahnhof.” Furthermore,WBMT encounters pragmatic difficulties because English does not have the distinctions between formal andinformal terms of address or noun genders that exist in German. For example, if one writes, “You have thepen. I need it.” into Google Translate, the output is “Sie haben den Stift. Ich brauche es.” At the very least, thesecond sentence should read “Ich brauche ihn,” where ihn refers to the masculine-gendered noun Stift. Addi-tionally, Google Translate automatically uses the formal terms of address Sie instead of the informal du,which may be inappropriate for the situation.

Methods

Participants

In order to answer these three research questions, we conducted a study with intermediatelearners of German in one intact class. The 18 participants were adult second-year students ofGerman at a large Midwestern research university where the use of WBMT programs has beencategorically prohibited for any work submitted for academic credit.3 In an initial questionnaireabout themselves and their uses of WBMT, all students reported English as their L1 and theyeither had completed the first year German courses at the same university or enrolled in thiscourse through a placement test. Study participants included eleven male and seven femalestudents ranging in age from 18 to 24 years-old. Fifteen out of the 18 participants had takenclasses in FLs in addition to German. Taken together, these students reported that they usedWBMT for an average of 27.7% of FL assignments, though one individual reported using suchprograms for all assignments, whereas six participants indicated that they never used transla-tion software for coursework.

In addition to the surveys, five of the 18 (two female and three male students) agreed to par-ticipate in follow-up interviews to further discuss their beliefs about WBMT and the strategiesthey used in the tasks with WBMT tools. The interviews were conducted face-to-face by one ofthe researchers who was not the instructor. The sessions lasted approximately ten minutes andtook place in a private setting outside of the regular classroom. Interviews were semi-scripted tofoster consistency between interviews but still allow for flexibility in asking relevant follow-upquestions.

Materials and Procedures

During their regular class hour, the students convened in a university computer lab, wherethey had the option of either participating in this study or engaging in a similar web-basedwriting assignment, but which would not be part of our data. For the students who participatedin the research, the order of the tasks in the study is shown in Figure 1. First, they completed theinitial part of an online survey that collected background information and their beliefs about theimportance of different features (e.g., grammatical accuracy, precise vocabulary, content, etc.)in German writing, and their usual uses of different electronic resources when writing in L2German.

After completing the first part of the questionnaire, we administered a task in which studentstranslated from their L1 to the L2. Although going from L2 to L1 is more common in profes-sional translation, we chose this method because it more closely reflects what students do whenfacing a writing assignment. Furthermore, Cook (2010) provides several arguments for movingfrom familiar to foreign. For example, by tapping into their existing knowledge of their own lan-guage, translation activities can give learners a sense of confidence that is lacking when the L1 is“artificially andautocratically” takenaway fromthem(p.133). He later continues, stating that

WHITE/HEIDRICH: STUDENTS AND WEB-BASED MACHINE TRANSLATION 233

3 Normally at this institution, students who are caught using WBMT on work submitted for academic creditface severe penalties. During the consent process, volunteers for this study were informed that their responseswould not be linked to their names or identities, and no matter what they shared about their uses of WBMT,there would be no effect on their course grades.

Humans teach and learn by moving from the familiar to the unfamiliar, by building new knowledge ontoexisting knowledge. Language learning and teaching are no exception to this general rule. Translation is justsuch a bridge between the familiar and the unfamiliar, the known and the unknown. To burn that bridge or topretend that it does not exist, hinders rather than helps the difficult transition which is the aim of languageteaching and learning. Learners moreover need that bridge to maintain the links between their languages andidentities. (p. 155)

Readers who are interested in further works advocating the place of translation in the curricu-lum in general should refer to Cook (1991, 1997, 2007, 2008, and 2010) and Witte, Harden,and Ramos de Oliveira Harden (2009).

For our task, the participants wrote a paragraph in English describing a picture prompt. Theillustration, taken from a children’s book containing only images, depicts a little boy sitting in hisroom at night with his dog; both are looking at a frog in a jar.4 The participants were instructedsimply to “write a paragraph,” but were given no definite word count. Forty-five minutes wereallotted for all tasks including the writing exercise.

While writing their paragraphs in English, the students were not told that their text wouldlater be used in a translation task. After all participants had completed the paragraph in their L1,the researchers informed them of the assignment and gave them the option to edit their work tomake it more suitable for accurate translation into German using WBMT. The participants thencopied and pasted what they had written into Google Translate (GT). This was selected for thestudy because it is among the most sophisticated WBMT tools available, as well as the one withwhich most students likelywould already be familiar. All 18 thenalso had the option to edit whatwas generated by the GT program. Finally, the participants completed the second online ques-tionnaire. This questionnaire asked participants to report their feelings about their use of theWBMT in the previous translation task. Items were marked on a 5-point Likert scale to indicatetheir level of agreement or disagreement with both positive and negative statements about thenature of their interactions with this WBMT tool. (Both questionnaires are displayed in Appen-dix B.) All participants completed the tasks in the range of seven to 25 minutes.

234 UP 46.2 (Fall 2013)

Figure 1. Order of Tasks in the WBMT Classroom Writing Assignment

2. L1 writing task

5. Option to editWBMT output

1. Survey(backgroundinformation &demographics

3. Informed ofWBMT task;option to editL1 text

4. Translationvia GoogleTranslate

6. Questionnaire(feelings aboutWBMT task)

4 The story in question is Mercer Mayer’s Frog, where are you? (1969), which has been used in numerousstudies of language acquisition and development, including the landmark study by Berman and Slobin(1994).

Results

This section first explains the types of errors that occur in GT output and students’ editedtexts. Then we respond to each of the research questions in turn, including an explanation of themethods used to answer each one.

Table 1. Examples, by Error Types

Type of Error Definition of Error Example

Verbal

TenseThe verb tense in the translated textdoes not match the English input.

One night, the boy captured a frog. �

Einer Nacht erfasst der Junge einen Frosch.

ConjugationThere is a lack of subject-verb

agreement.

They observed a frog that they caught. �

Sie beobachteten einen Frosch, den siegefangen hat.

Nominal

Declension

A noun, pronoun, adjective, orarticle does not accurately

demonstrate number, case, and/orgender.

With the moonlight and the companionshipof his pets, he has no fear. � Mit den

Mondschein und die Kameradschaft mitseiner Haustiere_ hat er keine Angst.

Contextual

Word Choice

The German lexical item and/or partof speech is not appropriate for the

context, or a necessary word ismissing.

His room is quite clean, save for the clotheson the floor. � Sein Zimmer ist relativ

sauber, speichern Sie die Kleider auf demBoden.

TypoThe English was entered

inappropriately, leading to amistranslation.

Animals were the boys passion.� Tierewaren die Jungs Leidenschaft.

Structural

Phrasing

The manner of expressing an idea isdifferent in English and German; a

one-to-one translation is notappropriate.

There is a striped T-shirt on the floor. � Esist ein gestreiftes T-shirt auf dem Boden.

ReducedComplexity

The student deleted information toavoid unfamiliar words/structures

(which occurs only in edited texts).

I would think that it is bedtime, because...� (GT) Ich würde denken, dass es

Schlafenszeit ist, weil …� (Student) Ichdenke, dass es Schlafenszeit ist, weil …

Word OrderThe elements of the sentence are in

the wrong order.

There is a little boy who is playing in hisroom with his dog. � Es gibt einen kleinen

Jungen, der spielt in seinem Zimmer mitseinem Hund.

Auxiliary

There is either an unnecessaryauxiliary, an inaccurate auxiliary(haben vs. sein), or a necessary

auxiliary verb is missing.

…, where a small boy is sitting in his room.� …, wo ein kleiner Junge ist in seinem

Schlafzimmer sitzten.

WHITE/HEIDRICH: STUDENTS AND WEB-BASED MACHINE TRANSLATION 235

Translations were produced in German by GT from students’ English input. The errors pro-duced in these machine translations and the students’ post-translation edited texts could bedivided into four main categories, which also contained several subcategories (in parentheses):verbal (tense and conjugation), nominal (declension), contextual (word choice and typos), andstructural (phrasing, reduced complexity, word order, and auxiliaries). Each student’s textswere examined line by line and errors were coded in these four categories as well as their subcat-egories. Examples of each type of error are presented in Table 1. Selected examples of students’English input, the unedited GT output, and the corresponding final products can be found in Ap-pendix C.

In addition, we used T-units from the different versions of the texts as a measure of complex-ity, which is defined as one main clause plus any attached dependent clauses (Hunt, 1965). Wecalculated the total number of T-units in both the German translated texts and the students’ ed-ited texts, as well as the number of total T-units that contained errors. In the eighteen uneditedGerman texts produced by GT, there are 91 T-units, of which 71 contain errors. Problems withsentence structure were the most frequent types of errors; the fewest related to verbal issues (i.e.,

tenseandconjugation). Figure2 shows thenumberof errors in theGT outputsbycategory.Figure 3, below, shows the number of errors in unedited GT output by subcategory. We see

that the highest number of errors was found in phrasing, whereas the fewest errors were due totypos and incorrect tenses.

Research Question 1: What strategies do students employ before and during translationof a text from their first language (L1) into the L2 (German) using WBMT?

This question was answered largely through follow-up interviews, in which students ex-plained their approaches to the translation task.

The majority of our interviewees, or four out of the five, reported that they tried to use bothsimple/familiar structures and vocabulary for the L1 input phase. Students also reported refin-ing their English grammar and removing slang for smoother translation. One learner who had

WHITE/HEIDRICH: STUDENTS AND WEB-BASED TRANSLATION 236

14

46 46

109

0

20

40

60

80

100

120

Verbal Nominal Contextual Structural

Figure 2. Types and Number of Errors in Unedited GT Outputs, by Category

previously used GT also mentioned that he would “get a little bit more specific with vocabulary”so that the translator tool would not mis-translate “some words that would have multiple mean-ings.” As we observed during the translation activities, some students also used online dictio-naries, though we noted that this strategy was more often employed during the post-translationediting phase.

Research Question 2: What strategies do the studentsemploy while editing WBMT textual output?

In the unedited GT texts, we discovered the most errors in phrasing (29.8% of all errors),declension (21.4%), word choice (20%), and word order (15.8%). Recognizing and correctingthese types of errors proved to be difficult for our participants. Out of the eighteen participants,it is noteworthy that three did not make any edits to the text produced by GT, and two additionalstudents only changed one word in their final texts. (One changed “wakeup” to “aufwachen”and another changed “würde denken” to “denken.”) When one of the interviewees was askedwhy he did not edit the output from GT, he stated, “I think there’s an assumption that peoplethink that translation programs are really wrong, you know, but I’ve used it before to find out theright sentence structure and it always made sense to me, so I guess I didn’t really think of it thatmuch.” This statement, coupled with the fact that 5 of the 18 students made minimal to no edits,further supports the thought that students may have implicit trust in the results that come fromWBMT. To discover trends in students’ final paragraphs, we again used the coding systemsystem (see Table 1) and found evidence of errors in all four categories. Figure 4 shows thedistribution of the numbers of errors in students’ essays after editing the translated text. TheGT-produced texts predominantly contained structural problems, which remained true for stu-dents’ edited texts, though the number was reduced from 109 to 86. The number of contextualerrors was also reduced (from 46 to 39); however, there was an increase in the number of verbalerrors (from 14 to 17) and in nominal errors (from 46 to 57) in students’ final versions.

This shows that instances of incorrect case and gender assignment produced by GT weretypically left in their erroneous forms after the students’ editing. Verbal constructions with prob-

237 UP 46.2 (Fall 2013)

11

34

64

3

43

46

9

5

0 10 20 30 40 50 60 70

Auxiliary

Word Order

Phrasing

Typo

Word Choice

Declension

Conjugation

Tense

Figure 3. Types and Number of Errors in Unedited GT Output, by Subcategory

lems produced by GT also were often left unedited, especially those containing auxiliary verbs.A frequently occurring error of this type was induced by students’ use of present progressiveforms in the English input phase. Although this structure does not exist in German, the programwas unable to identify that the English auxiliary verb is unnecessary.5 The students also did notrecognize these types of errors and left them unchanged. Although there were overall fewerstructural problems during this phase, we did see some examples of oversimplifying passages.For example, one student reduced the amount of information (and thereby decreased the num-ber of clauses) because she could not navigate the complex structure given in the GT output.This resulted in a grammatically correct sentence, yet one that did not contain all of the informa-tion she had originally written. The T-units in the students’ edited texts also were tallied; the totalnumber here is 92, of which 71 contained errors. When compared with the raw GT output, thetotal number of T-units increased by one, though the number of phrases with errors remainedthe same. When analyzing students’ texts, we discovered that several clauses had been deletedaltogether. The total number of T-units increased because some participants broke downlengthier, more complex structures into separate sentences.

Figure 5 shows students’ post-edited errors by subcategory. Phrasing, declension, and wordchoice errors dominate the types of issues that remained after students edited the texts pro-duced by GT.

In the post-translation editing phase, the numbers of errors in each subcategory was re-duced, except for tense (which increased from five to six), conjugation (from nine to 11), anddeclension (from 46 to 57). The participants examined and edited mainly vocabulary andstructures. In terms of vocabulary, we observed students using online dictionaries to check un-familiar lexical items, which resulted in a reduction of word choice errors from 43 to 37. Stu-dents were also able to recognize that GT frequently used the phrase da ist (to say “there is”),

238 UP 46.2 (Fall 2013)

17

57

39

86

0

10

20

30

40

50

60

70

80

90

Verbal Nominal Contextual Structural

Figure 4. Type and Number of Errors in Post-Translation/Student-Edited Texts, by Category

5 Although English distinguishes between the present (e.g., he sits) and the present-progressive tenses (e.g.,he is sitting), German uses the same present-tense form (e.g., er sitzt) in both cases. In one specific example, astudent wrote “he is sitting at the end of his bed,” and GT produced *"er ist am Ende seines Bettes sass“ in-stead of ”er sitzt am Ende seines Bettes."

which they adjusted to the more accurate es gibt, thereby decreasing the number of phrasing er-rors from 64 to 42. However, when faced with less familiar and more complex sentence struc-tures, students showed an inability to correct errors. At least one interviewee had completelyeliminated sentences containing the passive voice, which had not yet been introduced in class.This suggests that students could not use the WBMT software to produce writing that was abovethe level that they would be able to produce on their own.

Research Question 3: What beliefs do students report about their interactions with WBMT?

This question was answered by reviewing the pre-task questionnaire, which asked about theimportance of different features (e.g., grammatical accuracy, precise vocabulary, content, etc.)in German writing, as well as responses from the post-task questionnaire and follow-up inter-views. Additionally, during the interviews, some students’ thoughts that occurred as they com-pleted the task emerged.

Students indicated the personal level of importance and level of importance to the teacherof different features in German writing. Students rated the various features on a five-point scale,with 1 representing totally unimportant and 5 as very important. We determined the rank bymeasuring the mean of students’ responses in each category. The top two and last two catego-ries seem to match up well between the importance to the student and the students’ perceptionsof the importance to the teacher. For the students, accurate vocabulary and content were themost important, whereas complex grammar and sophisticated vocabulary were least impor-tant. The students perceived that content was the most important to the teachers (receiving thehighest meanof any category), followed by accurate vocabulary and overall style of the compo-sition.

WHITE/HEIDRICH: STUDENTS AND WEB-BASED MACHINE TRANSLATION 239

59

3042

237

57

11

6

0 10 20 30 40 50 60

Reduced Complexity

Word Order

Typo

Declension

Tense

Figure 5. Types and Number of Errors in Students’ Edits of GT Texts, by Subcategory

Table 2. Student Beliefs About Importance of Features in German Writing

Writing FeaturesImportance to Student Importance to Teacher

Meana SD Rank Mean SD Rank

Accuratevocabulary in

writing4.44 0.62 1 4.06 0.73 2

Content (e.g.,underlyingreasoning,

argumentation)

4.39 0.61 2 4.61 0.61 1

Grammaticalaccuracy in writing

4.06 0.42 3 3.67 0.77 5

Overall style (e.g.,text organization

3.78 1.00 4 4.06 0.54 2

Expression ofstudent voice in

German3.50 1.04 5 3.72 1.07 4

Complex grammar 3.33 0.69 6 3.39 0.70 6

Sophisticatedvocabulary

3.17 0.79 7 3.33 0.91 7

aScale: 1 = totally unimportant, 5 = very important

In Table 3, we show students’ responses from the online post-task questionnaire item,“When you used the online translator just now, how did it make you feel?” Students indicatedtheir level of agreement or disagreement on a five-point scale, with 1 indicating strongly dis-agree and 5 indicating strongly agree. We determined the rank by measuring the mean of stu-dents’ responses in each category. The top of the table indicates where students most stronglyagreed. These items include both positive and negative statements. Taken together, studentsstrongly agreed that they were not sure if they had used the tool in a sophisticated way. To illus-trate this point, one individual stated, “I understand that I can’t write in German as well as I canin English, and that’s the way it’s going to be.” On the questionnaire, many students also felt likeusing WBMT was cheating (despite being told to use it for the task), including one intervieweewho explained, “I felt like I was cheating a little bit - just because when you’re told to write some-thing in German, you’re supposed to write it the way you write it, not have someone else trans-late it for you.” On a more positive note, students generally felt like WBMT helped them betterorganize what they wanted to say and to find the right words. One interviewee admitted, “I’mnot saying if I would have translated it myself it would have been the same, but once I see ittranslated in a different way it makes more sense to me.” However, students tended to stronglydisagree with the idea that they were giving their best efforts by using the WBMT resource.

240 UP 46.2 (Fall 2013)

Table 3. Students’ Beliefs About Their Uses of WBMT

Question Meana SD

I am wondering whether I used this resource sophisticatedly,i.e., whether it made my writing better or worse.

3.59 0.71

I feel like I might have cheated. 3.59 1.18

I feel like it helps me organize what I want to say more clearly. 3.53 0.87

I feel like it helps me use words that fit what I want to say. 3.41 0.94

I feel like it helps me deliver my best work, for getting a good grade. 3.12 1.11

I feel like it helps me spell more sophisticatedly. 2.94 1.20

I feel like it helps me use words that are new and sophisticated. 2.94 1.03

I feel like it helps me use more complex grammatical structures. 2.88 1.05

I feel like it helps me develop better content. 2.82 0.88

I feel like it helps me deliver my best work, for my own satisfaction. 2.76 0.90

I feel like it helps my voice emerge more distinctly. 2.71 0.99

I feel like it helps improve my style. 2.65 1.06

I feel I am giving my best effort by using this resource. 2.06 0.97aScale: 1= strongly disagree, 5= strongly agree

Discussion

Although we examined three separate research questions pertaining to both students’attitudes toward and strategies in using WBMT, we found that in some ways, beliefs and behav-iors are closely interconnected, which will be further explained here. Additionally, we discussimplications and limitations of the study.

Beliefs and Strategies

The trends in the students’ strategies tended to correspond with their beliefs, demonstratinga strong connection between the two. In the interviews, one student remarked, “I don’t trustanything that comes out of those, because I know how inaccurate they can be, even with basicstuff.” Those who expressed skepticism about using WBMT in the survey also were likely tomention in the interviews that they had used strategies such as adjusting their English feedertexts. In fact, four out of five interviewees stated that they had adjusted the text to include only“simple” or “specific” vocabulary and structures to get a more accurate output. One participantexplained that he “tried to almost think like a third-grader essentially ... because as Germangoes, we’re at a pretty low level at this point. I don’t know complex German sentence structure,so even if it came out right, I wouldn’t know if it’s right.” Another participant noted that simplic-ity in the input is more important than editing the output. He did not, however, believe that thesimplicity was problematic; he said that his input resulted in a text so accurate that it could be“equivalent to a native speaker.” Students like him, who had previous successful experiencesand felt that using WBMT had helped them tended to make fewer edits. This confidence in thesoftware led them to leave the output in its erroneous form.

The majority of the participants (15 out of 18), however, did edit the GT output in some way.They tended to focus on structural elements, which were also mentioned by all five of the inter-viewees. In the end, they felt that the output was anywhere from “close enough” to “extremely

WHITE/HEIDRICH: STUDENTS AND WEB-BASED MACHINE TRANSLATION 241

accurate,” although two of them stated that the level of sophistication was very low, citing “lessflowery” language and a “third-grade writing style.” One student explained that his strategy forediting the output was “only doing things that I know I’m going to get right, even if that meanssacrificing exactly what I wanted to say... There’s a lot of things I wanted to say or describe, but Ididn’t necessarily know how to do it, so I thought of something else to say. … Anything that Iwasn’t familiar with I either cutoutorworded differently, so I knew it was at least close to right.”

Despite the time they spent manipulating the text and their feelings that the output was gen-erally accurate, the students ultimately felt that WBMT could not help them improve their writ-ing style or express their own voice. This was revealed in the post-task questionnaire, as well asin our interviews. Four of the five interviewees said that they do feel they have a personal voicein German, but that they simply did not have the skills with WBMT for it to reflect their own style.Therefore, we can conclude that students feel that using WBMT (and in some cases, editing itsoutput) can beused to producea linguistically accurate text, butnot necessarilywriting that theywould call their own. However, as indicated in our pre-task survey, students placed high impor-tance on linguistic accuracy and lower importance on sophistication, complexity, and express-ing their own voice. Perhaps this is an explanation for why many students continue to useWBMT despite its prohibition and known shortcomings.

In sum, students noted both pros and cons for using WBMT to produce FL writing. Asillustrated by one interviewee, “it takes less brainpower to punch it into a translator, but it takesmore time.” Students’ mixed beliefs and various behaviors led to several implications for thisstudy.

Implications

The intrapersonal variation in students’ stances on WBMT tools shows that some are unsurehow to use these resources properly. They are likely receiving conflicting messages (if anyguidance at all) about the usefulness of WBMT because, as discussed in the literature review,WBMT is criticized by many educators as a mere a tool for cheating, while others see it as aviable resource for learning. Although they did display mixed feelings about the usefulness ofthe WBMT tool, our participants recognized that the output was imperfect and many showedwillingness to engage with the text to improve linguistic accuracy; however, all of the finalGerman paragraphs contained errors. This shows that no learner’s intuited strategy resulted ina perfect interaction with the machine interface. The problems of the wide range of studentviews on WBMT and the inaccuracies in students’ edited texts could likely be mitigated ifinstructors would work with students to demonstrate the limitations of WBMT, and then utilizethe training strategies that demonstrate how to use these programs in order to produce moreaccurate writing (see Appendix A).

Our research also supports the claim that students will still use their L1 and WBMT programsin spite of known shortcomings and non-use policies (Cook, 2010; Fountain & Fountain, 2009;Larson-Guenette, 2013; Steding, 2009). Participants in our study reported using WBMT for anaverage of 27.7% of assignments, despite being informed by the instructor and in the syllabusthat it is categorically prohibited. Therefore, instructors should train students to recognize andcorrect the types of errors common to WBMT (including those that have surfaced in this studyand in Calude, 2003). Yet beyond this, teachers, department chairs, or the institution at largeshould consider revising policies to better reflect skills, needs, and ideas of 21st century learners,including those set forth by ACTFL and P21 (2011). Arguments against translation in the FLclassroom (e.g., that it is dull and demotivating, that it hinders acquisition and processing, that itis non-communicative, or that it is not a necessary skill) are unfounded (Cook, 2010). WBMT

242 UP 46.2 (Fall 2013)

activities not only develop translation skills, but also helpdevelopdeeperknowledgeof both theL1 and L2. Cook (2010) also argues that the ability to translate and use real-world translationtools should be important in contemporary curricula because translation activities are out-come-oriented and the focus-on-form is an offshoot of a communicative need rather than anend in itself. WBMT therefore has the potential to develop linguistic awareness while beingtask-based and communicative. As stated by Williams (2006), well-designed WBMT activitiesalso improve electronic, critical, and rhetorical literacy.

We could use any number of metaphors to describe the case of WBMT, such as “the genie isout of the bottle” or “you can’t unring a bell.”The tools are here and students are using them; theimperative for language educators is not only to guide students toward effective and productiveuse of WBMT, but also to explore and exploit WBMT’s potential as a learning tool. Of course,further research is needed to determine what sorts of training strategies would be most effectivefor students with differing proficiency levels, but what we offer is a good starting point.

Limitations of the Study

Although as German teachers we are confident with the results of our exploratory study, werecognize the limitations of the project. First, students were only allowed to use one translationprogram (GT), and our participant group was small and relatively homogenous, so the resultsare not generalizable to other programs or student populations. Students with different L1s orproficiency levels may have a different understanding of the errors produced by the translationprogram, or may interact differently altogether with WBMT.

Next, we should note that the instructor for the class was one of the researchers, which mayhave led to less honest self-reports about the use of WBMT among some participants. Weattempted to mitigate this by having multiple researcherspresent to answerquestionsduring theonline task and by identifying students’ responses by IP address only. We also recognize the in-herent difficulties associated with self-reports. When using these types of selected responsequestionnaires to report on beliefs, there is always a possibility that participants’ views may notbe wholly or accurately represented, or that the same people may report different beliefs at dif-ferent times. This drawback was somewhat mitigated by the results from the interviews, whichwereconductedbya memberof the research teamwho was not the interviewees’ instructor.

A final limitation is that students may have been aware that this assignment would not begraded, so some may have been more concerned with completing the task quickly, as opposedto achieving the best results in their writing. For future research, we recommend that student useof WBMT tools be observed in the context of actual classroom assignments with diverse studentpopulations. This would of course necessitate first removing the prohibition against WBMT.

Conclusion

Although umbrella policies that ban the use of WBMT are well intended to encouragelearner autonomy and creativity, it may be that it is depriving students of the opportunity to takeadvantage of a tool at their disposal both in the educational setting and outside the classroom(Cook, 2010). As we have shown, many students will use WBMT regardless of its limitationsand/or course policies. Therefore, instead of seeking to detect and punish students who areusing this resource, we hope we have shown that it would be more fruitful to train students howto use WBMT as a support for learning, in particular for German writing development (c.f.,Steding, 2009). In addition, in order to meet ACTFL and P21’s goal of providing learners with

WHITE/HEIDRICH: STUDENTS AND WEB-BASED MACHINE TRANSLATION 243

21st-century skills, it not only would be necessary to teach students how to use new tools, butalso how to avoid misusing them. As long as such machine translation programs continue to beimproved and refined, educators also will need to continually reassess policies and practices.Research such as the present study will thus remain important for understanding and employ-ing WBMT to the benefit of students and learning.

Acknowledgments

We would like to extend our sincere appreciation to Derek Drake and Yi-Chen Lee, whowere instrumental in helping design the original study and collect data. We would also like tothank the members Department of German at UW-Madison who have helped this paper alongin its many forms. Any errors that remain in this document are solely our own.

References

Alley, D. C. (2005). Using computer translation websites to further the objectives of the foreign language standards. In C. M.Cherry & L. Bradley (Eds.), Languages and language Learners. Dimension 2005, (pp. 63–74). Valdosta, GA: SouthernConference on Language Teaching. http://www.eric.ed.gov/PDFS/ED503091.pdf.

American Council on the Teaching of Foreign Languages (ACTFL) & The Partnership for 21st Century Skills (2011). 21st

Century Skills Map. http://www.actfl.org/sites/default/files/pdfs/21stCenturySkillsMap/p21_worldlanguagesmap.pdf.Berman, R. A., & Slobin, D. I. (1994). Relating events in narrative: A crosslinguistic developmental study. Hillsdale, NJ:

Lawrence Erlbaum Associates.Burton, C. (2003). The online translator: Implementing National Standard 4.1.Hispania: A Journal Devoted to the Teaching

of Spanish and Portuguese, 86 (2). 320–321.Calude, A. S. (2003). Machine translation of various text genres. Te Reo — The New Zealand Linguistic Society Journal, 46,

67–94. http://www.calude.net/andreea/MT.pdf.Cook, G. (1991). Indeterminancy, translation and the expert speaker. Triangle 10 (Proceedings of the 10th British Coun-

cil/Goethe Institut/Ens-Credif Triangle Colloquim): 127–141. Paris: Didier.Cook, G. (1997). Translation and language teaching. In M. Baker (Ed.): The Routledge Dictionary of Translation Studies, pp.

117-120. London: Routledge. (Revised Edition ‘Foreign Language Teaching’ in 2009, 2nd Edition, M. Baker & G.Saldanna, Eds., pp. 112–115.)

Cook, G. (2007). A thing of the future: Translation in language learning. International Journal of Applied Linguistics, 17 (3),396–401.

Cook, G. (2008). Plenary: An unmarked improvement: Using translation in ELT. In B. Beaven (Ed.), IATEFL 2007Aberdeen Conference Selections: 76–86. University of Kent: IATEFL.

Cook, G. (2010). Translation in language teaching: An argument for reassessment. Oxford: Oxford University Press.Fountain, C., & Fountain, A. (2009). A new look at translation: Teaching tools for language and literature. In C. Wilkerson

(Ed.), Empowerment through collaboration: Dimension 2009, (pp.1–15). Valdosta, GA: Southern Conference on Lan-guage Teaching.

Garcia, I., & Pena, M. I. (2011). Machine translation-assisted language learning: Writing for beginners. Computer AssistedLanguage Learning, 24 (5), 471–487.

Hall, J. K. (2001). Methods of teaching foreign languages: Creating a community of learners in the classroom. Upper SaddleRiver, NJ: Prentice Hall.

Hunt,K. (1965).Grammatical structures written at three grade levels.NCTE Research report No.3.Champaign, IL:NCTE.Hutchins, W. J., & Somers, H. L. (1992). An introduction to machine translation. http://www.hutchinsweb.me.uk/IntroM

T-TOC.htm.Larson-Guenette, J. (2013). “It’s just reflex now”: German language learners’ use of online resources. Die

Unterrichtspraxis/Teaching German, 46 (1), 62–74.Levy, M. (2009). Technologies in use for second language learning. The Modern Language Journal, 93, Focus Issue,

769–782.Lewis, D. (1997). Machine translation in a modern languages curriculum. Computer Assisted Language Learning, 10 (3).

255–271.Martin, R., & McHone-Chase, S. (2009). Translation resources on the web: A guide to accurate, free sites. College &

Research Libraries News, 70 (6), 356–359.

244 UP 46.2 (Fall 2013)

Mayer, M. (1969). Frog, where are you? New York: Dial Books for Young Readers.The National Standards in Foreign Language Education Project (1996). Standards for Foreign Language Learning: Pre-

paring for the 21st Century. New York: The National Standards in Foreign Language Education Project.Niño, A. (2008). Evaluating the use of machine translation post-editing in the foreign language class. Computer Assisted

Language Learning, 21 (1), 29–49.Niño, A. (2009a). Internet and language teaching/learning: Reflections on online emerging technologies and their impact on

foreign-language instruction. In R. Oxford & J. Oxford, (Eds.), Second language teaching and learning in the Net Gener-ation (pp. 23–30). Honolulu: University of Hawaii, National Foreign Language Resource Center.

Niño, A. (2009b). Machine translation in foreign language learning: Language learners’ and tutors’ perceptions of its advan-tages and disadvantages. ReCALL, 21 (2), 241–258.

Phillips, J. K. (1998). Media for the message: Technology’s role in the Standards. CALICO, 16 (1), 25–36.Sedrak, N. (2000). Will translation devices replace humans? Dialog on Language Instruction, 14 (1–2), 31–36.Somers, H. (2003). Machine translation in the classroom. In H. Somers, (Ed.), Computers and Translation: A translator’s

guide (pp. 319–340). Amsterdam: John Benjamins.Steding, S. (2009). Machine translation in the German classroom: Detection, reaction, prevention. Die Unterrichtspraxis/

Teaching German, 42 (2), 178–189.Williams, L. (2006). Web-based machine translation as a tool for promoting electronic literacy and language awareness. For-

eign Language Annals, 39 (4), 565–578.Witte, A., Harden, T., & Ramos de Oliveira Harden, A. (2009). From translating to translation in foreign language learning.

In A. Witte, T. Harden, and A. Ramos de Oliveira Harden (Eds.,): Translation in Second Language Teaching andLearning, pp. 92–111. Frankfurt: Peter Lang.

Zhang, Z. & Yu, K. (2002). Machine translation or machine-aided translation? Dalian Ligong Daxue Xuebao Shehui KexueBan/Journal of Dalian University of Technology (Social Sciences), 23 (3), 54–58.

Appendix A. List of Training Resources

Alley, D. C. (2005). Using computer translation websites to further the objectives of the foreign language standards. In C. M.Cherry & L. Bradley (Eds.), Languages and language Learners. Dimension 2005, (pp. 63-74). Valdosta, GA: SouthernConference on Language Teaching. http://www.eric.ed.gov/PDFS/ED503091.pdf.� Demonstrates how WBMT can be used to make comparisons between the L1 and L2 (see National Standard

4.1), thus helping students understand the nature of language and translation� Gives examples of the limitations of translation software� Provides classroom activities with examples in Spanish for solving language problems common to lower-level

studentsFountain, C., & Fountain, A. (2009). A new look at translation: Teaching tools for language and literature. In C. Wilkerson

(Ed.), Empowerment through collaboration: Dimension 2009, (pp.1–15). Valdosta, GA: Southern Conference on Lan-guage Teaching.� Discusses how instructors can minimize inappropriate use of WBMT� Provides examples in Spanish of how translation can be used productively for teaching language and literature

Lewis, D. (1997). Machine translation in a modern languages curriculum. Computer Assisted Language Learning, 10 (3),255–271.� Provides an overview of the main features of a program called Power Translator along with sample output for

German-English translations� Demonstrates that students can improve their knowledge of German grammar by using a machine translator

Niño, A. (2009). Machine translation in foreign language learning: Language learners’ and tutors’ perceptions of its advan-tages and disadvantages. ReCALL, 21 (2), 241–258.� Lists four main uses for machine translation and provides examples of each

WBMT as a bad model (i.e., inappropriate uses)WBMT as a good model (i.e., best practices)WBMT for vocational use (e.g., quality assessment, pre- and post-editing)WBMT as a tool for computer-assisted language learning

� Points out the strengths and weaknesses of WBMT programs applicable to all language pairsSomers, H. (2003). Machine translation in the classroom. In H. Somers, (Ed.), Computers and Translation: A translator’s

guide (pp. 319–340). Amsterdam: John Benjamins.� Gives information and examples in French and German for training translators, business people, scientists,

and technical writers to use WBMT� Discusses evaluation and editing of WBMT output

WHITE/HEIDRICH: STUDENTS AND WEB-BASED MACHINE TRANSLATION 245

� Shows how WBMT can be used as a bad model to demonstrate weaknesses of the softwareWilliams, L. (2006). Web-based machine translation as a tool for promoting electronic literacy and language awareness. For-

eign Language Annals, 39 (4), 565–578.� Explains some of the specific limitations of WBMT with examples from French� Provides a specific pedagogical plan for using WBMT to promote language awareness and electronic literacy

For general translation activities (that are not specific to WBMT), see the following resources:Butzkamm, W. & Caldwell, J. A. W. (2009). The Bilingual Reform: A Paradigm Shift in Foreign Language Teaching.

Tübingen: Narr Studienbücher.Deller, S. & Rinvolucri, M. (2002). Using the Mother Tongue. London: English Teaching Professional, Delta Publishing.Duff, A. (1989). Translation. Oxford: Oxford University Press.González Davies, M. (2004). Multiple Voices in the Translation Classroom: Activities, Tasks, and Projects. Amsterdam: John

Benjamin.

Appendix B. Questionnaires 1 & 2

(Please note, these questionnaires have been altered from their original electronic format, which were conductedusing Qualtrics survey software.)

Questionnaire 1. Demographics and Initial Questions

1. What is your age?a. 18b. 19–24c. 25–35d. 36–50e. 50–65f. 66+

2. What is your gender?a. Maleb. Femalec. Other

3. Is English your first language?a. Yesb. No

If English is NOT your first language, what is your first language?

4.Which foreign languages have you formally studied, even for a little while? Mark all that apply.a. Germanb. Spanishc. Russiand. Frenche. Chinesef. Japaneseg. Other ______

5. How would you assess your German writing proficiency relative to that of your classmates, on average?a. Much Worseb. Worsec. Somewhat Worsed. About the Samee. Somewhat Betterf. Betterg. Much Better

246 UP 46.2 (Fall 2013)

6. How would you assess your German writing proficiency relative to that of an educated native speaker of German,on average?

a. Much Worseb. Worsec. Somewhat Worsed. About the Samee. Somewhat Betterf. Betterg. Much Better

7. How would you assess your writing proficiency in your native language, relative to that of other educated nativespeakers of your language?

a. Much Worseb. Worsec. Somewhat Worsed. About the Samee. Somewhat Betterf. Betterg. Much Better

8. Please use the rating scales to answer the following questions: 1 (totally unimportant) to 5 (very important).

How important is grammatical accuracy in writing German to you personally?1 2 3 4 5 Unsure/Don’t Know

How important is accurate vocabulary in writing German to you personally?1 2 3 4 5 Unsure/Don’t Know

How important is complex grammar in writing German to you personally?1 2 3 4 5 Unsure/Don’t Know

How important is sophisticated vocabulary in writing German to you personally?1 2 3 4 5 Unsure/Don’t Know

How important is your overall style (e.g., text organization) in writing German to you personally?1 2 3 4 5 Unsure/Don’t Know

How important is the content (e.g., underlying reasoning, argumentation) of your German writing to youpersonally?1 2 3 4 5 Unsure/Don’t Know

How important is it for you personally that you give expression to your voice you write in German?1 2 3 4 5 Unsure/Don’t Know

How important is grammatical accuracy in writing German to your teacher?1 2 3 4 5 Unsure/Don’t Know

How important is accurate vocabulary in writing German to your teacher?1 2 3 4 5 Unsure/Don’t Know

How important is complex grammar in writing German to your teacher?1 2 3 4 5 Unsure/Don’t Know

How important is sophisticated vocabulary in writing German to your teacher?1 2 3 4 5 Unsure/Don’t Know

How important is your overall style (e.g., text organization) in writing German to your teacher?1 2 3 4 5 Unsure/Don’t Know

WHITE/HEIDRICH: STUDENTS AND WEB-BASED MACHINE TRANSLATION 247

How important is the content (e.g., underlying reasoning, argumentation) of your German writingto your teacher?1 2 3 4 5 Unsure/Don’t Know

How important is it for your teacher that you give expression to your voice when you write in German?1 2 3 4 5 Unsure/Don’t Know

9. Please indicate the appropriate percentage (0–100%)

How many of your foreign language assignments do you write on a computer?%

For how many foreign language writing assignments do you estimate you use an online dictionary?%

For how many foreign language assignments do you estimate you use an automatic spell & grammar check?%

For how many foreign language assignments do you estimate you use an online translator?%

Questionnaire 2. Post-Translation Task Questionnaire

When you used the online translator just now, how did it make you feel? Please indicate your dis/agreementwith the following statements:

Strongly DisagreeDisagreeNeither Agree nor DisagreeAgreeStrongly Agree

I feel I am giving my best effort by using this resource.I am wondering whether I used this resource sophisticatedly, i.e., whether it made my writing better or worse.I feel like I might have cheated.I feel like it helps me deliver my best work, for my own satisfaction.I feel like it helps me deliver my best work, for getting a good grade.I feel like it helps me use more complex grammatical structures.I feel like it helps me use words that fit what I want to say.I feel like it helps me use words that are new and sophisticated.I feel like it helps me organize what I want to say more clearly.I feel like it helps me spell more sophisticatedly.I feel like it helps me develop better content.I feel like it helps improve my style.I feel like it helps my voice emerge more distinctly.

248 UP 46.2 (Fall 2013)

Appendix C. Examples of Participant Responses

(English Input, Raw GT Output, Edited German Text)

Below are some selected participant responses for the translation task. We chose these examples to representsome of the common types of responses, errors, and editing choices (or lack thereof).

Text with Example of Reduced Complexity in Edited Text

English Input

A boy sits on a stool in his bedroom at the foot of his bed. It is nighttime. The boy’s clothes are on the floor and he isin his pajamas. In a jar before him he has a frog. A dog inspects the jar. A bird is nearby. The boy appears to be contentwith his animals.

Google Translate Raw Output

Ein Junge sitzt auf einem Stuhl in seinem Schlafzimmer am Fußende seines Bettes. Es ist Nacht. Der Junge dieKleider auf dem Boden und er ist in seinem Pyjama. In ein Glas vor ihm hat er einen Frosch. Ein Hund inspiziert dasGlas. Ein Vogel ist in der Nähe. Der Junge scheint Inhalt mit seinen Tieren sein.

Edited German Text

*Ein Junge sitzt auf einem Stuhl in seinem Schlafzimmer bei der Bett. *Seine Kleider liegen auf dem Boden und erträgt Pyjamas. *Ein Glas vor ihm hat einen Frosch.

No Changes Made from Raw GT Output to Edited German Text

English Input

The picture is of a bedroom. There is a boy and a dog looking at a frog in a jar. The light is on in the room. Clothesare on the floor. It is dark outside and you can see the moon.

Google Translate Raw Ouput

Das Bild besteht aus einem Schlafzimmer. Es ist ein Junge und ein Hund suchen ein Frosch in einem Glas. DasLicht wird in dem Raum. Die Kleidung ist auf dem Boden. Es ist dunkel draußen und man kann den Mond sehen.

Edited German Text

Das Bild besteht aus einem Schlafzimmer. Es ist ein Junge und ein Hund suchen ein Frosch in einem Glas. DasLicht wird in dem Raum. Die Kleidung ist auf dem Boden. Es ist dunkel draußen und man kann den Mond sehen.

Text with Example of Complex English Input

English Input

The young boy gazed contently at his puppy’s excitement upon receiving a gift. The bullfrog had been hoppingaround the pond nearby the boy’s home for some time now. Catching the frog had been much easier than expected.

WHITE/HEIDRICH: STUDENTS AND WEB-BASED MACHINE TRANSLATION 249

As he let his clothes dry out from the pond, he wondered what he ought to do with the frog. Should I keep him, or sethim free?

Google Translate Raw Ouput

Der Junge blickte zufrieden auf seine Welpen Aufregung beim Empfang ein Geschenk. Der Ochsenfrosch hatterund um den Teich in der Nähe des Jungen zu Hause schon seit einiger Zeit Hopping. Fangen der Frosch war vieleinfacher als erwartet. Als er seine Kleider trocknen aus dem Teich lassen, fragte er, was er soll, um mit dem Frosch zutun. Sollte ich ihn, oder ihn frei?

Edited German Text

Der Junge blickted zufrieden auf seine Welpen Aufregung beim Empfang ein Geschenk. Der Ochsenfrosh hatgegen um den Teich in der Nähe des Jungens Haus vor einiger Zeit Hopping sein. Fangen der Frosch war einfacher alserwartet. Als er hat Kleidung aus dem Teich trocknen, fragte er was er soll mit dem Frosch zu tun. Sollte ich ihn, oderihn frei?

Text with Examples of Egregious Google Translate Errors in Phrasing and Word Choice

English Input

Sitting in his room at night, a young boy and his dog look at the frog he caught in a jar. For a boy’s room his is quiteclean, save the clothes on the floor. Through the open window you can see the moon shining in the night sky.

Google Translate Raw Ouput

Sitzen in seinem Zimmer in der Nacht, ein kleiner Junge und sein Hund Blick auf die Frosch fing er in einem Glas.Für einen Jungen seines Zimmer ist sehr sauber, speichern Sie die Kleider auf dem Boden. Durch das geöffneteFenster können Sie sehen, der Mond schien in den Nachthimmel.

Edited German Text

Sitzen in seinem Zimmer in der Nacht, ein kleiner Junge und sein Hund blicken an die Frosch er fing in einemGlas. Für ein Jungenzimmer ist es sehr sauber, außer der Kleider auf dem Boden. Durch das geöffnete Fenster kannman ansehen den Mond scheint in den Nachthimmel.

250 UP 46.2 (Fall 2013)