Psychological Approaches to Translation: Eye Tracking and New Frontiers

Psychological Approaches to Translation: Eye Tracking and New Frontiers

UNSW Interpreting & Translation Seminar Series

March 25th, 2014

Dr. Stephen Doherty E-mail: [email protected] Room: 266 Morven Brown Building

Outline

1.  The Interdisciplinarity of Translation Studies 2.  Key Concepts of Eye Tracking 3.  Translation Evaluation and Usability 4.  Enhancing Viewer Experience with Captions 5.  Summary

The Old & The New

•  Established practice of translation vs. relatively new development of an independent discipline;

•  Historical links to literature and linguistics; •  Different approaches: descriptive, comparative, prescriptive; •  Shifts from equivalence to purpose; •  Interactions with other disciplines:

–  Cultural aspects; –  Sociology; –  Globalisation; –  Technology.

•  Relationship with the translation industry.

Theory & Practice

•  Growth of translation studies in university programmes;

•  Translator training, assessment, development; •  Research into translation:

–  Translation as a product; –  Translation as a process.

•  Translation process studies: –  Focus on translators (human!) as they read,

translate, interact with tools, evaluate; –  Need for evidence-based research and external

methods, e.g. psychology and cognitive science.

Outline


History of Reading Studies

•  Javal (1879) – reading process not smooth; •  Huey (1908) – reading, fixations on certain words; •  Buswell (1922) – first non-invasive tracker using

beams of light, studies of reading and picture viewing; •  Adler & Fliegelman (1934) – focus and physiology; •  Yarbus (1967) - task and physiological effects,

cognitive effort; •  Rayner (1970) – time to take stock; •  Just & Carpenter (1980) – strong eye-mind

hypothesis.

Main Theories

•  Eye Mind Hypothesis; •  Cognitive Load Theory; •  Working Memory Models

(Attention, Storage, Retrieval);

•  Motivation and Attrition; •  Multimodal Processing

Model.

Comprehension Failure (Sentence)

This is a sample sentence to show you how eye tracking works.

Linear Reading Pattern


Comprehension Difficulty (Lexical Item)


Shift of Attention


Wide Range of Applications •  Physiology (e.g. disease, rehabilitation); •  Cognitive psychology (e.g. memory, attention, motivation); •  Reading studies (e.g. reader type, education, subtitles); •  Writing studies (e.g. linguistic features, learning models); •  Marketing (e.g. brand recognition, attraction); •  Human-computer interaction (e.g. usability, system testing, video games); •  Clinical psychology (e.g. detection of pathology, therapies for autism).

Technologies

•  Search Coil: – Most accurate; – Most invasive.

Technologies

•  Head Mounted: – Less accurate; – Less invasive; – Portable.

Technologies

•  Remote: – Less accurate; – Not invasive; – Less Portable.

Translation Process Studies

•  O’Brien (2006) – translators & translation memories; •  Jakobsen & Jensen (2008) – reading & translating; •  Sharmin et al. (2008) – translation & direction; •  Pavlov & Jensen (2009) - translation & direction; •  Hvelplund (2011) – allocation of attention/cognitive

resources.

Eye Tracking

Video Capture

Human Measures Time & Task Measures

User Activity Data Psychometric Questionnaires

Outline


Background

•  Increased need for translation •  Diversity of content and users •  Rise in prevalence of machine translation [MT] both

off- and online •  Mixed reports of quality – attitudes and expectations •  Divergence in R&D – translation studies/computer

science •  Academic and commercial contexts •  Divergence in evaluation methods and metrics:

–  Human and/or automatic evaluation metrics

Interaction (Reading)

Comprehensibility Readability

Reader Text

Research Question 1

•  Can eye tracking be used as a means of evaluating translation quality? –  Is it consistent with other

metrics? –  What are the differences

between human and automatic metrics?

–  Can it be applied to practice?

Method

•  Establish a baseline for the translation quality; •  Improve the quality using established ‘controlled

language’ (linguistic pre-processing); •  Evaluation with real end-users of technical support

documentation (sentences, paragraphs, different interactions);

•  Comparison of human, automatic, and eye tracking metrics;

•  Using eye tracking, interviews, gaze replay, psychometric questionnaire.

Input Output

Controlled Input

MT Systems

Controlled Output

1. If you will be performing such an activity and want to avoid the warning, you can temporarily disable the program. 2. If you perform such an activity and want to avoid the warning, you can temporarily disable the program.

1. Si vous serez effectuer une telle activité et que vous voulez éviter l'avertissement, vous pouvez désactiver temporairement le programme. 2. Si vous effectuez une telle activité et que vous voulez éviter l'avertissement, vous pouvez désactiver temporairement le programme.

1. Wenn Sie die Durchführung einer solchen Tätigkeit werden und wollen, um die Warnung zu vermeiden, können Sie vorübergehend deaktivieren Sie das Programm. 2. Wenn Sie eine solche Tätigkeit und durchführen wollen, um die Warnung zu vermeiden, können Sie vorübergehend deaktivieren Sie das Programm.

1. このようなアクティビティを実行し、警告を回避するする場合は、プログラムを一時的に無効にすることができます。 2. このようなアクティビティを実行し、警告を回避したい場合は、プログラムを一時的に無効にすることができます。

Highlights

•  Application: –  Controlled language is worth it when humans are concerned; –  Focus on which rules work best; –  Improvements along the workflow; –  Predictors of readable/comprehensible text.

•  Lessons Learned: –  Humans can be unreliable, machines can be too rigid; –  Mixed methods adds great value, especially for human process research; –  Including many points of view, even if you don’t agree with them!

•  For details: –  Doherty, S. & O’Brien, S. 2009. Can Machine Translation Output be Evaluated through Eye Tracking?

Machine Translation Summit XII. –  Doherty, S., O’Brien, S. & Carl, M. 2010. Eye tracking as an MT Evaluation Technique. Machine

Translation, 24, 1. –  Doherty, S. 2012. Investigating the Effects of Controlled Language on the Reading and Language on

the Reading and Comprehension of Machine Translated Texts: A Mixed-Methods Approach using Eye Tracking. Dublin City University, PhD Thesis.

Usability of Machine Translation

Research Question 2

•  How are people using machine translation? –  Are needs being met? –  What solutions are users

finding? –  What needs to be focused on

next?

Method

•  Investigate the usability of MT from English into French, German, Spanish, and Japanese;

•  Use of real texts, users, and tasks; •  Application of ISO TR 16982 to study (user

satisfaction and efficiency with set product in set scenario under set conditions);

•  Using eye tracking, interviews, gaze replay, psychometric questionnaire.

Highlights

•  Source text scored best with best rated usability in all measures; •  Language distance was a significant factor (FR, SP just behind); •  Greatest difficult with German and Japanese materials; •  However, they all got the job done! •  Use of multiple regression models to predict best user experiences:

–  Comprehension decreases tasks time thus satisfaction is improved; –  Satisfied users recommend to others; –  Good instructions decrease cognitive effort and increase task completion.

•  Functional usage of translation, and MT, esp. with freely available tools online; •  A caveat is the representative quality and the damage to translation technology; •  For details:

–  Doherty, S. & O’Brien, S. 2012. A User-Based Usability Assessment of Raw Machine Translated Technical Instructions. The Tenth Biennial Conference of the Association for Machine Translations in the Americas.

–  Doherty, S. & O’Brien, S. 2014. Assessing the Usability of Raw Machine Translated Output: A User-Centred Study using Eye Tracking. International Journal of Human-Computer Interaction, 30, 1.

Teaching Translation Technology

•  Helped us to understand approaches to teaching translation technology;

•  Let us see how users actually use MT - what works well; •  Incorporation of MT into syllabus using these insights;

•  For details: –  Doherty, S., Kenny, D., & Way, A. 2012. Taking Statistical Machine Translation to the Student

Translator. Proceedings of the 10th Conference of the Association of Machine Translation in the Americas.

–  Doherty, S. & Moorkens, J. 2013. Investigating the Experience of Translation Technology Labs: Pedagogical Implications. The Journal of Specialised Translation, 19.

–  Moorkens, J., Doherty, S., O’Brien, S. & Kenny, D. 2013. A Virtuous Circle: Laundering Translation Memory Data using Statistical Machine Translation. Perspectives: Studies in Translatology.

–  Kenny, D. & Doherty, S. 2014. Statistical Machine Translation in the Translation Curriculum: Overcoming Obstacles and Empowering Translators. The Interpreter and Translator Trainer.

–  Doherty, S. & Kenny, D. 2014. The Design and Evaluation of a Statistical Machine Translation Syllabus for Translation Students. The Interpreter and Translator Trainer.

Outline


Western Adoption

Western Asian

Function 1.  Inter- and intra-lingual 2.  Convey information 3.  Other devices for

humour…

1.  Inter- and intra-lingual 2.  Convey information 3.  Highlighting device 4.  Summarisation 5.  Sign-posting 6.  Humour…

Structure 1.  Little real estate 2.  One or two colours

typically

1.  More real estate 2.  Colourful 3.  Animations 4.  Picture-in-picture 5.  Effects 6.  Multiple subtitles

Prevalence 1.  Low 1.  High (cultural and linguistic norms)

Effects 1.  Useful for purpose 2.  Tiring

1.  Love/hate

Research Question 3

•  What effect do captions have on the viewer experience? (if any) –  Do viewers even read them? –  Do they increase

comprehension? –  Do they increase

satisfaction? –  Do they increase comedic

effect?

Highlights

•  Captions are viewed regardless of their relatedness; •  Similar processing effort and viewer behaviours for different types of

captions; •  Significant effects of humour and comprehension found as a result of

captions; •  But, diminished satisfaction when obstructive and unrelated to the

programme content;

•  For details: –  Sasamoto, R. & Doherty , S. 2014. Towards the Optimal Use of Impact Captions: The Current State, and

Future Research. Proceedings of the 5th ASIA Conference. –  Sasamoto, R. & Doherty, S. (forthcoming). Manipulating the Minds of Others: The Use of Subtitles for

Viewer Manipulation. Media Psychology. –  Doherty, S. & Sasamoto, R. (forthcoming). The Use of Impact Caption Subtitles for Enhancing (and

Manipulating) Viewer Experiences. Poetics: Journal of Empirical Research on Culture, Media, and the Arts

Viewers familiar with subtitles of this

nature

Viewers spent time and effort on

all subtitles

Subtitles aided understanding

Subtitles added to the humour

Viewer satisfaction higher

Dynamic subtitles obstructive when unrelated Annoyance reported

Viewer satisfaction lower

Most Recently

•  ‘Hookability’; •  Facial expressions; •  Social interactions around viewing experiences; •  Further evidence for applications (focus on specific audiences); •  Ethical and policy implications;

•  For details: –  Sasamoto, R., O’Hagan, M. & Doherty, S. (under review). Deconstructing Impact Captions

on Japanese Variety Shows: A Multimodal Analysis of the Use of Creative Inserts on TV. –  Doherty, S., Sasamoto, R. & O’Hagan, M. (under review). Hooking and Retaining Viewer

Attention: A Multimodal Analysis of Impact Captions on TV.

Outline


Summary

•  Great development and growth in translation studies, especially for interdisciplinary research;

•  Fruitful application of psychological concepts, methods, and tools to the study of translation – both as a process and as a product;

•  Interesting findings for research, education, and industry;

•  Flexibility of this approach to new scenarios and applications;

•  An exciting collaborative new frontier!

Dr. Stephen Doherty E-mail: [email protected]

Room: 266 Morven Brown Building

Psychological Approaches to Translation: Eye Tracking and New Frontiers

Documents

Transcript of Psychological Approaches to Translation: Eye Tracking and New Frontiers