The Co-Definition of Self: Conversations in Virtual Reality

Matteo CANTAMESSEa,1

aCSRPC, Dipartimento di Psicologia Università Cattolica del Sacro Cuore, Milano, Italy

Abstract. Conversation analysis can take the form of a qualitative methodology for the exploration of discursive productions, whose main goal is the formulation of hypotheses for reading psychosocial interaction through descriptive models of interlocution. Therefore, in this study, conversations in a shared Virtual Environment have been analyzed in order to understand the specific structure, dynamics, and phenomenology of Virtual Reality effects on the “interactive micro-chains” that constitute the communicative thread of daily experience.

Keywords. Presence, Self, Social Interaction, CyberTherapy, Social Presence

Introduction

In recent years, the wide diffusion of multi-user Virtual Environment (VE), on-line games, and other artifacts shaped the research on collaboration in Virtual Reality (VR). There is a large interest on the study of social interaction per se, as noted by Schroeder [1]: a small or large group behavior, social conventions, and trust dynamics are just few example of themes investigated by the means of shared Virtual Environment.

Understanding how to use virtual reality to support collaborative interaction presents a substantial challenge for the designers and users of this emerging technology [2]. First, Virtual Environments (VEs) are created to serve a purpose, so they must be designed by explicitly considering intended users’ tasks and goals [3]. Second, the possibility of negotiation, both of actions and of their meaning, plays a key role in providing a satisfactory sense of cooperation. This is even truer for networked VEs, where cooperation and collaboration are the key features of the experience [4].

Particularly, there are two basic difficulties in studying VR-mediated interaction. The first difficulty lies in daily situations’ ambiguity and unpredictableness: considering the world of experience as an open system, it is not possible to plan and design every type of situation which could be generated while carrying out a series of tasks, as required in order to build a suitable cooperation environment [2]. Another issue is the fundamental diversity from the points of view of actors implied in the interaction: as noted by Gasser “two agents cannot have identical representations […] shared knowledge is impossible” [5]. Cooperation, according to Gasser, is simply a moment of practical understanding, because the actors are good at pragmatically tuning their own activities ‘as if’ they had common knowledge with other actors [2]. According to Galimberti and colleagues [4], for these reasons, the only way to

1 Corresponding Author: Via Nirone 15, Milano, Italy; E-mail: matteo.cantamesse@unicatt.it.

211

understand interaction in VE is by analyzing the subjects involved in the environment in which they operate, and the new processes and activities developed during interactions. A major difficulty in studying interaction can be identified in the respect of the ecological situation in which the interaction takes place.

A possible solution could be to use the method called analysis of conversations, a qualitative methodology for the exploration of discursive productions, whose main goal is the formulation of hypotheses for reading psychosocial interaction through descriptive models of interlocution [6-8]. Therefore, in this study we will examine conversations to understand their specific structure, dynamics, and phenomenology in order to understand the effect of Virtual Reality on the “interactive micro-chains” [9] that constitute the communicative thread of daily experience.

1. Method

In order to compare conversational dynamics in real life and in virtual reality, we created two experimental conditions during which couples of participants negotiated a narration using some Peanuts® cartoons. First condition was based on a CVE (Collaborative Virtual Environment) and the second condition took place inside the laboratory room. During their interaction participant were requested to choose 6 cartoons out of 46 and to create a story using them. Cartoons were randomly chosen from the Peanuts official website archive, and were reproduced on paper boxes or on virtual boxes inside the CVE.

A total of 48 participants (24 male and 24 female) were recruited for the experiment via e-mail, newsgroup, and poster announcements in Milan, Italy. Participants’ ages ranged from 18 to 29 (M= 23.30, SD= 3.13). Participants were divided by sex and randomly assigned to two groups: the Virtual Reality group (VR) and the Real Life group (RL), resulting in 12 males and 12 females for each group. They were then divided in couples, balanced for sex, resulting in 12 couples for group, 4 constituted only by females, 4 mixed, and 4 constituted by males. Since any level of acquaintanceship can have an effect on conversational dynamics, we paid particular attention in order to create couples with participants reciprocally unknown to each other.

The virtual environment was constituted by three rooms and two corridors, each linked by doors. Participants started in two separated rooms connected to the main space by two corridors divided by a window. The main room contained two sets of boxes (textured with Peanuts® comics) and a table. In order to obtain a high perceptual realism, several elements have been manipulated, according to Rademacher [10]: the used VE was characterized by high shadow softness, a wide variety of shape and number of objects, an average of 5 light sources in each room, a detailed sound environments, and coherent physics behaviors.

Participants were represented with a 3D human-like avatar of the same sex of the participant who controlled its movements using the mouse and the right-button for walking forward; the actions (opening the door, picking up a box, dropping it) were performed by using the mouse left button.

The VR software engine adopted offers two opportunities to show the action of object grasping: representing a gripping tool or representing objects as floating in front of the participant holding it. In order to increase the self perception [11], the gripping solution was preferred. To evaluate the impact of this solution on the perception of the

212

realism during the VR experience, a survey has been conducted: 43 participants (20 male, 23 female; age M= 29.23, SD= 4.21) were asked to describe a picture, randomly selected between two screenshots out of the different points of view. Only 8 of them described the “strange tool.” As noted by Hoorn [12], VR experience is not based only on the realism, but also on the relevance of its elements, dynamics, and actions allowed. In the VE used in this study, the grasping tool was relevant for the action, and its appearance, although not realistic, was not in contrast with its function. Considering the survey results, and the relevance of the experience, the grasping tool solution has been adopted in the VE.

2. Analysis

The research questions posed by presence in VR required an integrated approach, as suggested by Riva [13] and Galimberti [4]: the Complementary Explorative Multilevel Data Analysis (CEMDA) integrates qualitative and quantitative procedures, therefore it has been used as a framework for this analysis. CEMDA incorporates complementary use of both methods, depending on the particular research stage or the initial assumptions that need to be taken in consideration.

Conversation analysis usually does not require quantitative information of interactions analyzed, but the greater differences between conditions shall be reported: VR conversations’ average number of utterance is 178.2 per actor, while face-to-face average length is 97.4 utterance per actor. This mere information tells us something about the complexity characterizing VR-based conversations and the effort required of the actors in order to accomplish the same task as in face-to-face. The general structure of interaction underlines the co-definition of meanings and actions, but a deeper investigation highlights some recurrent processes characterizing the action of doing something with someone else in a collaboratively constructed context, specifically in VR.

In order to understand how Virtual Reality experience can shape the inter-subjectivity in interaction, the relative position that speakers took during interaction has been analyzed. While interacting, each actor can direct the other to do something, or could ask for help: each turn [14] has been coded as “+-” if the speaker directly assumed a “high” position , and as “-+” if the speaker accepted and adopted the “low position” casted by the previous turn [7, 15]. An iterative coding procedure has been adopted. After defining the rules for the two categories, 6 transcripts have been coded independently by two coders. The results of their coding have been compared for consistency of code application for both categories. According to Carey [16], a 2-by-2 contingency table has been constructed, reporting the absence or presence of each node by both coders across all the utterances coded. Such table was constituted by two agreement and two disagreement cells and Cohen’s Kappa [17, 18] had been calculated. The code “+-” scored .93, while the “-+” code scored .89. A Mann-Whitney test has been conducted on frequencies and a significant difference has been found for the +- code (Z=-5.78; p<.001) and for the -+ code (Z=-4.57; p<.001), with the VR condition scoring higher than the f2f condition for both. But a Mann-Whitney test on proportional frequencies showed a significant difference only for the -+ code (Z=-2.22; p<.05) with the VR condition scoring higher than the f2f condition, and no significant difference has been found for the +- code. This difference can indicate that VR

213

conversations require “interactants” to continuously define and affirm their role, but that there is no difference on the interactive position taking and casting pattern.

3. Discussion

The first quantitative analysis of conversations produced in this study shows that interaction in VR requires quite the double of turns than interaction in face-to-face conditions, in order to perform the same task. This information tells little about interactive characteristics of a dialogical sequence, but is a confirmation of the greater complexness and articulation of VR interaction. We found that conversations in virtual reality are usually composed by 7 different phases, while face-to-face ones only by 3. If we look at the common phases, we can see that they are all about the task: in details, there is a Task defining sequence, very common in RV conversation and far less in F2F ones; the Task sequence, composed by every exchange finalized at completing the task assigned, and the Closure, when subjects reported the story they created.

But differences are far more interesting: VR conversations are characterized by two opening sequences devoted to the negotiation of the environment, of the actors themselves, and of the action. The first sequence, that we called “tuning”, is similar to the summon/answer” sequence identified in phone conversation by Schegloff [19], but there is a difference, due to the specific media involved: while by phone the “summon” action is performed by calling, here it should be explicitly performed by the speaker. But the significance for the overall conversational sequence is the same: it starts the interaction. Sometimes the tuning sequence is followed by a salutation form, completely absent from the real life situation. Immediately after that, actors in VR performed what we called a Co-defining sequence, finalized at the negotiation of self, and three sub-sequences emerged: the self-affirmation, the reciprocal affirmation and the agnition of the other. This sub-sequence is in relation to the existence of the speaker in VR, the existence of the other, and their shared reciprocal recognition, and it supports the inter-comprehension process by creating a conversational device helping each actor to:

• ensure the intelligibility by the other of his own utterance; • be sure of his own comprehension of other’s utterance; • identify the environment and his relative position and make it identifiable by

the other; • identify the actions performed and objects involved and make them

identifiable by the other. These are the basis of any successful conversational interaction, as noted by

Galimberti [6]: what changes here is that they are explicitly faced and solved by each actor.

Allport defined social psychology as “an attempt to understand and explain how the thought, feeling, and behavior of individuals are influenced by the actual, imagined, or implied presence of others” [20]: by exploring the reciprocal affirmation sequence in its dialogical structure we can see that there are two symmetrical adiacency pairs, showing that for an actor is not enough to see the other, but he also need to be sure that the other sees him. A deep analysis of the conversational interplay highlights that actors not only co-define the virtual environment, but also their representations of it: when they don’t share their relative positions, or their mutual understanding of the situation, the interaction needs to be continuously tuned, they stop doing the task to fall

214

back on the description of the environment around there, and their position in it, their actions, and their intentions. From a psychosocial point of view, the sense of being in the VE can be defined as the result of the reciprocal confirmation, and can be defined as the reciprocal presence of self and the actual, imagined, and implied other.

References

[1] R. Schroeder, Social Interaction in Virtual Environments: Key Issues, Common Themes, and a Framework for Research, in The Social Life of Avatars, R. Schroeder, (Edr), 2002.

[2] C. Galimberti, et al., Communication and cooperation in networked environments: An experimental analysis, Cyberpsychology and Behavior,. 4 (1) (2001), 131-146.

[3] T. Rodden, J. Mariani, and G. Blair, Supporting cooperative applications, International Journal of Computer Supported Cooperative Work 1 (1992).

[4] C. Galimberti, et al., Characteristics of interaction and cooperation in immersive and nonimmersive virtual environments, in Towards CyberPsychology: Mind, Cognitions and Society in the Internet Age, G. Riva, and C. Galimberti, Editors. 2000, IOS Press: Amsterdam.

[5] L. Gasser, Social conceptions of knowledge in action: Distributed Artificial intelligence foundations and open systems semantics, Artificial Intelligence 47 (1991).

[6] C. Galimberti, La conversazione: prospettive sull’interazione psicosociale, Milano, Guerini e Associati, 1992

[7] C. Galimberti, and E. Ghinato, La conversazione in trappola: congegni sperimentali e problemi di metodo, Archivio di Psicologia, Psichiatria e Neurologia, LIII (1) (1992).

[8] C. Kerbrat-Orecchioni, Universali e variazioni culturali nei sistemi conversazionali, in La conversazione: prospettive sull’interazione psicosociale, C. Galimberti, (Eds), Guerini, Milano, 1992.

[9] D. Reiss, and D. Klein, Paradigm and pathogenesis: a family-centered approach to problems of etiology and treatement of psychiatric desorders, in Family interaction and psychopathology: theories, methods and findings, T.Jacob, (Ed), Plenum Press, New York, 1987.

[10] P. Rademacher, J. Lengyel, E. Cutrell, and T. Whitted, Measuring the Perception of Visual Realism in Images. in 12th Eurographics Workshop on Rendering Techniques, 2001.

[11] M. Garau, et al., The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environmen, in SIG-CHI Conference on Human Factors in Computing Systems, Fort Lauderdale, FL, 2003

[12] J.F. Hoorn, E.A. Konijn, and G.C. van der Veer, Virtual Reality: Do Not Augment Realism, Augment Relevance, Upgrade 4 (1) (2003).

[13] G. Riva, Virtual Reality as communication tool: a socio-cognitive analysis, in Communications Through Virtual Technology: Identity Community and Technology in the Internet Age, G. Riva, and F. Davide (Eds), IOS Press, Amsterdam, 2001.

[14] S.C. Levinson, Pragmatics, Cambridge, Cambridge University Press, 1983. [15] M.F. Agnoletti, Baru, and J. Defferrard, Expérimenter la conversation, Verbum 12 (1) (1989). [16] J.W. Carey, M. Morgan, and M.J. Oxtoby, Intercoder Agreement in Analysis of Responses to Open-

Ended Interview Questions: Examples from Tuberculosis Research. Field Methods 8 (3) (1996). [17] J. Cohen, A coefficient of agreement for nominal scales, Educational and Psychological Measurement

20 (1) (1960). [18] J. Cohen, Weighted kappa: Nominal scale agreement with provision for scaled disagreement of partial

credit, Psychological Bulletin 70 (4) (1968). [19] E.A. Schegloff, The routin as achievement, Human Studies 9 (2-3) (1986). [20] G. Allport, The historical background of social psychology, in Handbook of social psychology, G.

Lindzey, and E. Aronson (Eds), Random House, New York, 1985.

215