Evaluating the reflexive practices in a learning experience

Andreina Bruno & Laura Galuppo & Silvia Gilardi

Received: 21 July 2010 /Revised: 20 December 2010 /Accepted: 18 April 2011# Instituto Superior de Psicologia Aplicada, Lisboa, Portugal and Springer Science+Business Media BV 2011

Abstract Reflexivity is a primary requirement for professional work. Our aim was todescribe a methodology suitable for detecting the development of reflexive practice throughthe analysis of 59 Master’s degree students’ journals. We explore the use and changes ofreflexive practice in relation to the settings and activities of the course using analysis of themental language: We analyse how the reflexive practice is characterized, how it evolvesthrough time and how it can be promoted. Data analysis shows that reflexivity is notmonodimensional but multidimensional. Reflexive practice increases during the course:The different dimensions vary over time according to different organizational andeducational conditions. This methodology allows us to formulate new hypotheses aboutwhich elements of the setting and process are effective in supporting participants’ reflexivepractice.

Keywords Reflexivity . Mental language . Evaluation . Professional development .

Learning . Reflective journal

Introduction

The emphasis placed on reflexive practice as a primary competence for professional work(Schön 1983) has brought about a rethinking of educational processes for professionaldevelopment, particularly in areas connected with the health profession (Mann et al. 2007),the teaching profession (Barnett and O’Mahony 2006; Kreber 2005), the training ofpsychologists (Sax 2006; Mayo 2004) and management education (Cunliffe 2004).

Eur J Psychol EducDOI 10.1007/s10212-011-0061-x

A. Bruno (*)Department of Anthropological Sciences, Psychology Section, Università degli Studi di Genova, Genoa, Italye-mail: andreina.bruno@unige.it

L. GaluppoDepartment of Psychology, Università Cattolica del Sacro Cuore di Milano, Milan, Italy

S. GilardiDepartment of Labour and Welfare Studies, Università degli Studi di Milano, Milan, Italy

Although experimentation in higher education is extensive, it also appears to be ratherfragmented. This scenario could be owed to a number of reasons: There is still noagreement about what is meant by reflection (Kember et al. 2008; Kreber 2005), nor aboutthe methodology required to analyse and evaluate it (Mann et al. 2007).

The aim of this paper was to describe a methodology suitable for detecting thedevelopment of the reflexive practice within a specific educational setting.

After presenting several definitions of reflective thinking and reflexivity, we propose theconcept of reflexive practice, linked with the developmental and psychoanalytic studies onmentalizing processes. The paper then describes a methodology suitable for analysing,within a specific educational experience, how reflexive practices are characterized, howthey evolve through time and which educational conditions allow them to be promoted.

Defining reflexivity

Under the general heading of reflection, there are various descriptions of its nature thatsuggest different educational approaches (Dewey 1933; Schön 1983; Mezirow 1991; Boudand Walker 1998).

It is possible to identify at least three general approaches to describing this concept.In the first approach, reflection is a form of mental process delineated as an inquiring

attitude aiming at improving problem solving. The focus is on goal-directed behaviours andon cognitive activity, and less attention is paid to the role of the emotions (Moon 1999).

In the second approach, the emphasis is on awareness of self in action: Reflection refersto the beliefs, values, feelings and implicit assumptions used in setting and solving aproblem (Mezirow 1991). The attention paid to these aspects has led some scholars (Warinet al. 2006) to recommend a shift from reflective practice to reflexive practice. Reflexivitymeans considering the involvement of the practitioner’s professional and personal valuesand frames and their impact on his/her working activity (Schön 1983).

Recently, this second approach has undergone some reappraisal: Reflection is interpretedas a ‘socially situated, political, collective process’ (Reynolds and Vince 2004, p. 6) thatoccurs within social and organizational contexts. The emphasis is on reflexivity as adialogical and relational activity: It is not an internal, cognitive process, but a form ofknowledge that is built up through dialogue and negotiation (Cunliffe 2004).

In light of this debate, we propose that reflexivity is a conversation with the situation(Schön 1983), which includes a dialogue with one’s own and others’ minds.

Moving on from this definition, an important contribution which deepens the characteristicsof such a dialogue derives from studies on mentalizing in the psychoanalytical tradition (Fonagyand Target 1996, 1997) as well as developmental studies on the theory of mind (Antonietti etal. 2006). Historically, psychoanalytical and developmental studies on mentalizing and thoseabout practitioners’ reflexivity belong to different research traditions, each of themcharacterized by their own constructs and methodology. We suggest that studies onmentalizing, particularly those by Fonagy and Target (1996, 1997), can prompt a new andmeaningful way to define and evaluate the use of reflexivity in work and educational contexts.

Reflexive practice and mentalizing

Fonagy and Target (1997) define the reflective function as the whole of the psychologicalprocesses which sustain mentalization, i.e. the ability to use mental states to understand and

A. Bruno et al.

to explain one’s own and others’ behaviour (for example, ‘I took this action because Ithought…’; ‘He decided in that way because he believed… or because he felt angry…’).This ability entails the recognition that mental states are representations which can befallible and can change because they are only based on one of a wide range of possibleperspectives (Fonagy and Target 1996). According to these authors, the mentalization andthe theory of mind can be considered as synonymous: Displaying a mentalizing attitudemeans perceiving oneself and others as characterized by mental states. These mental statesinclude both those with a non-epistemic nature (wishes, intentions, emotions) and thosewith an epistemic nature (beliefs, reasoning, inferences).

Moreover, the ability to mentalize implies both a self-reflexive component, concerningthe attribution to the self of mental states, and an interpersonal component, which refers tothe process of conferring mental states on others.

Allen (2006) suggests that such an ability to mentalize is not a stable characteristic, but asituated process that can be activated and put to different uses depending on the relationalcontexts. It is an action: something we do or cannot do. Consequently, he prefers the term‘mentalizing’ rather than ‘mentalization’.

Following this theoretical framework, we define reflexive practice as the process ofmentalizing, i.e. of attributing mental states to oneself and to others and of explaining one’sown and others’ actions with reference to those mental states.

Reflexive practice—as a process of mentalizing—is useful for adults at work as it isclosely related to social and communication skills and to the processes of knowledgeconstruction. For example, an interaction based on mentalizing sustains cooperativenegotiation and effective communication: If we mentalize, we consider the other as anautonomous person with a different perspective, and we will make an effort to understandand to influence each other. Without mentalization, the other is transformed into an object,dehumanized (Allen 2006).

Regarding knowledge construction, recent studies also highlighted the relationship betweentheories of mind and epistemological beliefs (Fagnant and Crahay 2010). Lalonde andChandler (2002) state in effect that the capacity to appreciate the interpretative nature ofknowing needs to encompass an understanding of the interpretative character of the mind.

Since in every profession reflexive practice appears to be necessary to deal with complextasks and situations, it is therefore important to discuss the methodologies suitable forevaluating whether an educational context activates and sustains such a practice.

How to evaluate reflexive practice

In order to evaluate reflexive practice, adult educational literature suggests both quantitativetools, e.g. questionnaires (Mamede and Schmidt 2004; Sobral 2001; Biggs et al. 2001;Kember et al. 2000), and qualitative techniques. Journals are often proposed for use ineducational programmes both to sustain and to assess reflexive thinking (Gleaves et al.2008; Hubbs and Brand 2005). Reflective journals are often structured: Their analysisusually requires a coding scheme for assessing the level of students’ reflection (Kember etal. 1999; Wong et al. 1995).

An interesting example is that described by Kember et al. (2008), proposed as guidancein assessing the level of reflection in written works (journal entries, essays, contributions todiscussions on web forums). It allows assessing the level of a piece of writing by using thefollowing four categories: Habitual action/non-reflection; Understanding; Reflection;Critical reflection.

Evaluating the reflexive practices in a learning experience

The ‘Habitual action/non-reflection’ level occurs when the written work shows noevidence of the writer attempting either to reach an understanding of the concept or theorywhich underpins the topic, or to interpret it, or to form a view. At the ‘Understanding’ level,there is evidence of understanding of a concept or topic, but theory is not related to personalexperience, real-life applications or practical situations. At the ‘Reflection’ level, conceptsare related to other knowledge and interpreted in relation to personal experiences. Finally, atthe ‘Critical reflection’ level, there is ‘evidence of a change in perspective over afundamental belief of the understanding of a key concept or phenomenon’ (Kember et al.2008, p. 379).

We agree with Kember’s claim of a precise and non-equivocal definition of the differentlevels of reflection, but we believe that preliminary work is needed to specify whichindicators can be used to recognize the presence (and then the level) of reflexive processesin written texts.

In the last 15 years, research on theory of mind has highlighted the relations betweenlanguage and mentalizing (Shatz et al. 1983; Zanobini et al. 1998; Milligan et al. 2007).The cultural perspective of these studies considers the production of words denotingpsychological states, such as wishes, beliefs and emotions, as being an evidence of theability to understand and interpret mental states (Bretherton and Beeghly 1982; Antoniettiet al. 2006; Hughes et al. 2007). Therefore, the ability to refer to mental states throughlanguage is seen as an ecological index of the theory of mind (Symons 2004; Lecce 2009).Following these authors, in this paper, we consider the use of mental language as anindicator of reflexive practice.

Traditionally, studies on the theory of mind have been focused on children. In ourresearch, we focus on young adults in educational contexts aimed to promote a professionalapproach to work. We propose the analysis of mental language as a powerful tool to detectwhen reflexivity is utilized in written texts. By mental language, we mean a lexiconcomposed of terms (nouns, verbs, adjectives, adverbs) that refer to mental states: cognitive(thoughts, beliefs, reasoning); emotive (fear, sadness, happiness); and volitive (wishes). Ifreflexivity, as in our case, is defined as a conversation with one’s own and other minds,mental language represents the means by which each individual expresses his/her ownknowledge about his/her own mind and that of others. Following this theoreticalframework, we assume that the active use of such a lexicon indicates that an individualis using reflexive thinking.

The advantage of considering mental language is that it represents a precise and notsocially desirable indicator. As a matter of fact, the writer can or cannot use a mentallanguage without being aware of doing something desirable for the reader.

The analysis of mental language can therefore help us recognize, as a first step, thepresence and the characteristics of writers’ reflexive thinking; in a second step, it may helpus assess the level of their reflexive thinking.

The present article shows the first step of the work, and future contributions willdescribe our use of mental language to detect different levels of reflexive thinking.

The study

Research questions

This study aims at describing a methodology for analysing the use of participants’ reflexivethinking within a learning experience. More specifically, the aim of the study was to

A. Bruno et al.

explore how participants use their reflexive thinking, and how it changes in relation to thesettings and activities of the course, through the analysis of participants’ reflective journals.

Our research questions are:

1. What are the characteristics of the mental language used by participants in describingthe learning process?

2. How does the mental language change in relation to their learning experience?

The context

The context of the study was the course ‘Methods and techniques for business training’,scheduled during the first year of a Master’s degree in work and organizational psychologyin a university in the north of Italy. Two of the authors of this paper were the teacher andthe teacher assistant of the course.

During the course, a 60-h unit of 3-h meetings twice a week for 3 months, students weretaught to plan and conduct a training programme: After an introductory phase in which theyattended both theoretical lectures and active learning sessions (e.g. psychological games,role playing), they were required to become trainers themselves ‘for a day’, working insmall groups of seven or eight to plan and conduct a 3-h training session within the course.The aim of the course was to help students increase their competencies by experiencingprofessional practice and being involved with the teachers and their classmates in periodicalsessions of analysis and reflection on the activities that were being tried out.

During the programme, the students were also invited to write a personal journal athome. Each student could decide if, how and when to write her/his journal. No structuredgrid was provided. The only requirement was to describe the first and the final meeting ofthe course and the experience of conducting a didactic unit as ‘trainer for a day’. At the endof the course, every student was given the chance to discuss his/her journal personally withthe teachers, with the aim of going back together over the learning journey and using his/her writings as a starting point for revision and reconstruction of their learning experience.These meetings were optional and scheduled after the examination and had no impact onthe mark awarded. The decision to read and to discuss students’ journals after the examinationwas motivated by the awareness of a ‘social desirability’ risk.We supposed that students, withinan academic context, did not feel free to describe and reflect on their learning experience, beingworried about their teachers’ expectation and evaluation. As a matter of fact, at the beginning ofthe course, they often asked how to write their journals and what to include. These requestswere widely discussed in the classroom in order to allay students’ doubts and fears. At the endof the programme, almost all the students requested a meeting with the teachers in order todiscuss the journal and the course experience.

Method

Subjects

The sample consists of 59 students who attended one of three different sessions (2003/2004, 2004/2005 or 2006/2007) of the course in ‘Methods and techniques for businesstraining’. Every session was attended on average by 35 students aged 23–28, mostly (80%)females. They all have a bachelor’s degree in psychology and attended the Master’s degreein work and organizational psychology.

Evaluating the reflexive practices in a learning experience

Design and tools

Each student wrote an unstructured journal (without any temporal limitations or anystructured grid). Every journal is about 20 pages long and reports students’ descriptions ofalmost every meeting of the course. Particular attention is given to the first and the finalmeetings and to the experience of conducting a didactic unit as a trainer.

Data analysis

The whole corpus of 59 journals was subjected to software (T-Lab; Lancia 2004) aidedcontent analysis: We chose single mental words as the analysis unit.

The decision to use single words as the unit was based on the fact that the aim of suchanalysis is not to describe each student’s level of reflection but to detect the characteristicsand the changes of reflexive thinking during different phases of the programme.

Firstly, in order to reveal the presence and features of students’mental language, the mentalwords were selected and counted. The mental words were selected by three judges whoanalysed the whole dictionary of the corpus. The judges’ evaluations were compared, and foreach word, agreement was achieved through negotiation. Each mental word was then verifiedwithin its context to eliminate the words that had no sense in relation to their use in the sentence.

Once the list and the frequencies of each mental word were defined, their density (thenumber of mental words divided by the whole number of words in each subtext) in thewhole corpus was calculated.

Then, in order to analyse the mental vocabulary in more depth, the mental words wererated according to the following categories:

& Cognitive words: all the verbs, nouns and adjectives describing cognitive processes (e.g.think, mind, clever, etc.)

& Emotive words: all the verbs, nouns and adjectives referring to emotions and feelings(e.g. happiness, scared, etc.)

& Volitive words: all the verbs, nouns and adjectives indicating motivation, volition anddesire (e.g. hope, will, etc.)

After they had been coded, the density of cognitive, emotive and volitive words was alsocalculated in the whole corpus.

Secondly, each journal was divided into three parts, each corresponding to a specificphase of the educational process:

& The first part (codified as time 1, t1) corresponded to the first five meetings of thecourse. During this phase, students started getting to know each other, were divided intogroups, and the main concepts for organizing business training settings wereintroduced.

& The second part (codified as time 2, t2) coincided with the students’ descriptions of thefollowing ten meetings, when they were involved in conducting their training units andin reflecting on them together.

& The third phase (codified as time 3, t3) corresponded to the last five meetings in whichstudents were helped to conceptualize and synthesize what they had learned.

The whole corpus of texts was then subdivided into three subsets: The first subset,corresponding to time 1 (t1) consisted of 143,432 words; the second subset, corresponding totime 2 (t 2), of 192,884 words; and the third subset, corresponding to time 3 (t3), of 68,065words.

A. Bruno et al.

In order to compare the presence and the features of the mental words throughout thethree phases, their density was compared through a t test analysis. The density of cognitive,emotive and volitive words was compared as well.

Finally, in order to analyse the specific contents of the cognitive, emotive and volitivewords in the three phases, a chi-square test was conducted. The test showed which mentallexical units were typical (overused) for each phase and allowed us to compare themqualitatively.

Results

Mental vocabulary and its changes throughout the three phases

As described above, in order to evaluate how mental language is used by students todescribe their learning experience, the vocabulary of the journals was analysed by selectingthe mental terms and comparing their density in the three subsets.

The analysis shows that out of all words in the whole corpus, 6.49% are mental words.More specifically, in t1, the density of the mental words is 6.47%, in t2 6.36% and in t36.89% (Fig. 1).

Moreover, whereas the t test (Table 2) reveals no significative difference between t1 andt2, there was a significative difference both between t2 and t3 (p<0.01) and between t1 andt3 (p<0.01). According to the test, the density of the mental words increases from thebeginning to the end of the course, specifically between the second and third phases.

In order to analyse the characteristics of students’ mental language more deeply, themental words were then subdivided into the three categories described above (cognitivewords, emotive words and volitive words), whose density was finally calculated (Table 1).

Cognitive words appear to be more frequent (density=4.81%) than emotive (density=1.06%) and volitive words (density=0.63%). Moreover, emotive words were alsosubdivided into positive (density=0.60%) and negative (density=0.46%).

Then, the corpus was analysed by comparing the density of the three categories in thethree subsets (Fig. 2).

According to the t test (see Table 2), the density of cognitive words increasessignificatively from t1 to t3 (p<0.01) and also from t2 to t3 (p<0.01). The density ofvolitive words diminishes significatively from t1 to t2 (p<0.01) and increases again from t2to t3 (p<0.01). The density of emotive words, finally, does not show any significative

Fig. 1 Density of mental wordsin the three phases

Evaluating the reflexive practices in a learning experience

difference between the three phases. Nevertheless, after dividing the emotive words intopositive and negative words, the t test reveals several differences (Fig. 3).

Whereas the density of positive and negative emotive words does not vary between t1and t3, negative emotive words increase from t1 to t2 (p<0.05) and diminish from t2 to t3(p<0.01), and positive emotive words diminish from t1 to t2 (p<0.05) and increase from t2to t3 (p<0.1) significatively (Table 2).

Mental vocabulary and its characterization throughout the three phases

In order to analyse the specific features of mental language and their changes in relation tothe time variable, the chi-square test indicates the most typical lexical units (overusedwords, p<0.01) in each subset, allowing us to underline the semantic features thatdifferentiate a text’s subset from the rest of the corpus. The analysis shows differences interms of characterization of mental vocabulary in the three phases of the course.

In Table 3, with respect to cognitive vocabulary, the typical words of the three phasesshow references to very different mental processes.

Within t1, there are cognitive typical words referring to learning, choosing andremembering (knowing, learning, deciding, remembering).Within t2, cognitive vocabulary seems richer and more specific, referring in particularto perceptive processes (attention, impression, to observe, to notice), to complexlearning processes such as analysis and interpretation (analysis, to interpret, to

Table 1 Density of mental words

Mental words Subset time 1 (%) Subset time 2 (%) Subset time 3 (%) Tot corpus (%)

Cognitive 4.74 4.74 5.12 4.81

Emotive 1.06 1.06 1.03 1.06

Emotive + 0.63 0.57 0.63 0.60

Emotive − 0.43 0.49 0.40 0.46

Volitive 0.68 0.55 0.73 0.63

Tot. mental words 6.47 6.36 6.89 6.49

Fig. 2 Density of cognitive,emotive and volitive words inthe three phases

A. Bruno et al.

understand, to focus on), to planning (to foresee, unpredictable) and to judging(judgment/prejudice).Within t3, finally, there are strong references to knowledge and reflexivity. Knowledge,on the one hand, goes hand in hand with concepts such as concept, discovery,confirmation, clarification, explicitation. Reflexivity, on the other hand, is associatedwith terms like re-reading, thinking back, recalling, evaluating.

Table 4 shows examples of the use of cognitive typical words from the students’ withjournals.

Referring to emotive vocabulary, as shown in Table 5, the characteristics of mentalwords vary within a significant range of feelings and experiences.

Within t1, the themes of pleasantness, friendship, trust, curiosity, fascination, fear andboredom stand out.Within t2, the vocabulary referring to emotional states is enriched mainly throughterms indicating negative emotions. Amusement and satisfaction, but mostlydiscomfort, anxiety, stress, confusion appear.Within t3, finally, emotional mental states are again mainly positive. The theme ofgratitude, along with happiness, passion and trust, appears. The only negative state issadness.

Table 6 shows examples of the use of typical emotive words from the students’ journals.With regard to volitive vocabulary, as shown in Table 7, the characteristics of words used

vary throughout the three phases. Most of the words are concentrated within time t1.

Within t1, there is a strong reference to the themes of desire and expectation, with thepresence of terms such as expectation, need, motivation, interest, to desire.

Table 2 T test results

t values Cognitive Emotive Emotive + Emotive − Volitive Tot inenta1

t1 vs. t2 0.00 0.00 2.22** 2.55** 4.73*** 1.28

t2 vs. t3 3.90*** 0.66 3.10* 1.71** 4.90*** 4.22***

t1 vs. t3 3.75*** 0.63 0.00 1.00 1.27 3.09***

***p<0.01; **p<0.05; *p<0.1

Fig. 3 Density of positive andnegative emotive words in thethree phases

Evaluating the reflexive practices in a learning experience

Within t2, the term attempt appears.Within t3, finally, the term expectation is back and the term goal appears.

Table 8 shows some examples of the use of typical volitive words from the students’journals.

Discussion

In order to understand the characteristics of reflexive thinking during a training experiencein higher education, this study has analysed the journals written by the participants,particularly considering their mental vocabulary.

The data show that students’ mental language includes both cognitive words andemotive and volitive words. Cognitive terms in particular are used more frequently. Theseterms, however, are not the only ones used to refer to students’ own and others’ mentalstates. The presence of emotive and volitive terms confirms, as various authors haveclaimed (Boud and Walker 1998), that the reflexive process does not just include cognitiveaspects but also aspects connected with the dimensions of feeling and wishing.

Table 3 Typical cognitive words

t1 To know (χ2=140.54); certainty (χ2=38.98); to decide (χ2=38.26), to learn (χ2=33.21); concept(χ2=30.70); to choose (χ2=24.49); dilemma (χ2=18.01); choice (χ2 = 8.05); to reflect (χ2=7.16); toconvince (χ2=6.88); memory (χ2=6.48); mental (χ2=5.87); to make curious (χ2=5.82); to agree (χ2=5.43); to remember (χ2=5.36); certain (χ2=4.53); knowledge (χ2=4.31); reflexivity (χ2=4.43); tointernalize (χ2=4.07)

t2 Reflection (χ2=31.36); analysis (χ2=29.81); to foresee (χ2=23.30); to search (χ2=25.72); to interpret(χ2=25.16); to observe (χ2=21.19); attention (χ2=17.72); unpredictable (χ2=15.89); to distract (χ2=11.86); vision (χ2=11.44); decision (χ2=17.72); unpredictable (χ2=15.89); to distract (χ2=11.86);vision (χ2=11.44); decision (χ2=10.88); doubt (χ2=15.89); to distract (χ2=11.86, vision (χ2=11.44);decision (χ2=10.88); doubt (χ2=10.85); clearness (χ2=10.07); personally (χ2=9.83); to represent(χ2=8.53); to understand (χ2=7.51); to concentrate on (χ2=6.02); prejudice (χ2=5.84); to focus on(χ2=5.82); certain (χ2=5.69); to notice (χ2=5.09); interpretation (χ2=4.02)

t3 Evaluation (χ2=55.75); concept (χ2=56.10); to re-read (χ2=36.80); to clarify (χ2=23.19); to learn (χ2=23.03); conviction (χ2=21.18); knowledge (χ2=18.25); to analyse (χ2=14.74); discovery (χ2=14.64);perspective (χ2=14.64); explicit (χ2=14.00); doubt (χ2=12.73); discovery (χ2=12.19); to confirm (χ2

=11.45); to think back (χ2=10.96); to define (χ2=10.86); know (χ2=10.41); reflexive (χ2=9.63); tojudge (χ2=9.62); to evaluate (χ2=9.20); to imply (χ2=8.95); thought (χ2=8.56); point of view (χ2=8.42); to recall (χ2=7.62); clear (χ2=4.38).

Table 4 Typical cognitive words: examples from the students’ journals

I didn’t believe that a Psychologist, in the area of training, could work being strongly in touch with hospitalsor clients and 1 have to admit that this possibility fascinates me.… I have learnt that the role of the teacherdelivering a lesson to employees is not the only one.

One important thing I understood in the plenary discussion is that the trainer is not the one who knowsanything but the one who succeeds in integrating the theme of the day with the needs of the participants.

I discovered that I had in mind a too pretentious image of the trainer I began to see it as being more humanwhen I’ve played this role personally.

A. Bruno et al.

The data also show that the mental language used by the students to relate and analysetheir experiences during the course changes over time both from a quantitative and from aqualitative point of view.

From a quantitative point of view, it is possible to see how the course progressivelysustains the use of reflexive thinking.

The data highlight phase 3 as the point in the educational process characterized byintense references to cognitive states of mind. This could be interpreted as a sign of theincrease in the use of reflexivity by the end of the course, just when the students arerequired to think back on their experiences in order to evaluate their learning. Moreover,references to positive emotions and wishes are also frequent. This may be because they areassociated with starting up new projects for the future. These emotive and volitivedimensions also lead us to conclude that the learning experience has been particularlyintense and engaging.

The increase of reflexive practice at the end of the course also seems to be closelylinked with the transformation of vocabulary that characterizes the central phase of theeducational process. Indeed, this is represented as an emotionally draining phase,characterized by active sessions in which the students try out professional practice,planning and conducting a didactic unit. These action sessions are alternated by sessionsof discussing the action and of connecting theories and practical experiences. Giventhese alternating action and reflection sessions, we expected a progressive increase ofemotive and cognitive vocabulary. The data, however, show a growth only in thenegative emotive language together with a decrease in the positive emotive and volitivedimensions and not an increase in cognitive language. The phase of action seemstherefore to create anxiety, doubts about oneself and one’s own effectiveness. Yet thisphase seems to be the prelude to the possibility of using one’s own reflexive thinking

Table 5 Typical emotive words

t1 Pleasant (χ2=134.43); friend (χ2=32.15); friendship (χ2=18.09); to trust (χ2=16.72); fear (χ2=15.83);curious (χ2=6.45); to fascinate (χ2=5.55); boredom (χ2=5.32); courage (χ2=4.43); to frighten (χ2=4.25); embarrassment (χ2=3.97).

t2 Stress (χ2=16.45); to assume (χ2=10.64); discomfort (χ2=10.36); satisfaction (χ2=9.56); nervous (χ2=7.90); aggressive (χ2=7.64); anxiety (χ2=5.97); amusement (χ2=5.95); to puzzle (χ2=5.58); panic(χ2=5.53); annoyance (χ2=5.17); emotion (χ2 = 4.73); feeling (χ2=4.81); confusion (χ2=4.31);frustration (χ2=3.87).

t3 To be grateful (χ2=74.57); sadness (χ2=42.88); happiness (χ2=14.09); to move (χ2=8.68); trust (χ2=5.50); passion (χ2=3.92).

Table 6 Typical emotive words: examples from the students’ journals

Today I experienced fear. I was afraid of talking and, at the same time, I felt nervous for not having told…Maybe I haven’t already seen the group as a place where I can bring part of my life.…

The fact I can’t predict how the others will react is a source of anxiety. I believe, however, that accepting thisuncertainty is one of the competencies of a trainer.

Most of my happiness is that I also realized that I can succeed in fulfilling my dream to become a goodprofessional.

Evaluating the reflexive practices in a learning experience

to a greater extent in the closing phase of the course. Pain or surprise (Schön 1983)connected with inevitable critical incidents that occur during action have been proven to bethe factors that enable a turning point. ‘Wrong footing’ caused by actions, followed by thepossibility of reflecting on it in phase 2 and going back to re-conceptualizing the experiencein phase 3, in a continuous cycle of action, reflection and return to the theory to orient newactions, represents indeed the educational condition that allows reflexive practice to beintensified at the end of the course.

Analysing the contents of the typical mental terms from a qualitative point of view,the data show that in the initial phase, the cognitive language used is connoted mainlyby terms referring to the theme of knowledge and memory/remembering. By using thesecognitive words, students are supposed to relate their present situation to previousknowledge and learning experiences. We assume that these connections between thehere and now of the course and previous knowledge are stimulated by a specific aspect ofthe educational programme: the request to organize small groups in order to plan and toconduct a didactic unit, taking complete responsibility for this process. Such a taskstimulates students to put their effort into integrating and connecting the present and thepast, a past made of other courses, other people, other knowledge. According to theliterature, these results may support the hypothesis that in reflection, past experience andunderstandings are linked with present experience to lead to new understandings andappreciation (Fellows and Zimpher 1988).

Emotive language, moreover, is characterized by reference to trust, curiosity andfear, whereas from a volitive point of view, words such as desire, motivation andexpectation emerge. In this phase, the main focus is to construct with learners a workalliance based on the contents of the course and on its aims. We assume that in thisphase, the students wonder about the possibility to feel like the author, not only apassive actor, of their learning experience. The presence of terms such as desire andtrust indicates that the reflexive process can start when the educational context encouragesstudents to think of themselves as people who can give voice to their desires, needs andexpectations.

Table 7 Typical volitive words

t1 Expectation (χ2=45.70); motivation (χ2=24.44); to desire (χ2=15.43); need (χ2=10.14); interest (χ2=8.87); to hope (χ2=7.22); will (χ2=5.68); available (χ2=3.89).

t2 Attempt (χ2=18.18).

t3 Goal (χ2=72.17); expectation (χ2=3.88).

Table 8 Typical volitive words: examples from the students’ journals

I think we’ve made a good start and 1 hope this will continue. I have found that I have high expectations ofthis course which seems to be (finally!) a chance to be able to do something “concrete” that brings mecloser to the job I think I want to do.…

Human unpredictability, that is what has come out more strongly than we expected from this experience.Unpredictability that we ingenuously thought we could dominate and that shattered everything we hadunsuccessfully attempted to plan in the smallest detail.

I started off with great expectations for this course and if sometimes I have had my doubts about what I wasdoing, about where we were going there is now a clear meaning to it all.

A. Bruno et al.

During the second phase of the course, the encounter with the action appears, asmentioned above, disorienting and the source of emotions such as embarrassment,discomfort and sometimes panic. Moreover, cognitive language does not increase herebut changes qualitatively: Expressions referring to perceptive processes appear (observe,attention), as do those linked with complex analytical processes (interpret, analyse) and tothe theme of planning (to foresee). The action phase, according to the students’ reports, ischaracterized by coming up against unexpected situations. From an emotional point ofview, the contact with the unexpected is linked with the dilemma of power/powerlessness.We hypothesize that the painful clash with the unexpected puts a strain on the idealizedimage of a trainer who is able to plan every single action and is therefore powerful andimmune to error, and creates a new representation of useful skills in a trainer who is ‘goodenough’. The presence of terms that express listening, observation and interpretationactivities could indicate that students recognize that not everything can be foreseen andcontrolled and that this impossibility is not a sign of weakness or inability.

The third phase is the time to conceptualize and synthesize the contents of the course. Inthis phase, cognitive vocabulary is mainly focused on the theme of knowledge, perceived asa process of discovery, thinking back, evaluating. It is interesting to note also the wordconcept: This combination refers to the possibility, at the end of the course, to turn thepractice into an experience that can be thought out and discussed (Kaneklin and OlivettiManoukian 1990). Finally, from the point of view of emotive and volitive terms, the end ofthe course is once again characterized by a return to words indicating trust and gratitude,which seem to indicate that the learning process has been a satisfying experience for thestudents compared with their initial expectations and from a relational perspective.

We can hypothesize that in this course, the opportunity for assessing the wholeeducational process, after the virtual professional task action and the feedback, has resultedfrom the specific condition that allowed the students to trust in their ability to co-buildknowledge and to give value to their learning.

Limitation, directions for future research and implications

One limitation of this study is that it includes a single case group with young psychologistsundergoing training. In order to generalize the results, more groups from different academicand extra-academic scenarios should be studied.

Another limitation concerns the methods of collecting information on reflexivity. Theuse of the journal in academic contexts presents some problems. In an environment basedon competitive, cognitively oriented examinations, students can launder diaries and writedown whatever they think it is that teaching staff expect rather than truly exploring andreflecting on their own experience (Boud and Walker 1998). To overcome this limitation,we have taken into consideration not predicative propositions but the mental language thatoffers the advantage of not being completely controlled by the writer and not being sociallydesirable.

Finally, we have seen that it may be a limitation to take into consideration single wordsisolated from the phrase context. To reduce mechanistic risks, each occurrence of themental words was checked in terms of its use within the sentence context. In order toevaluate the level of reflexive thinking performed by the students, it will nevertheless benecessary to integrate the analysis of the single words with a qualitative analysis of thewhole text body by using the sentence as an analysis unit. As some researchers havepointed out (Astington and Jenkins 1999; De Villiers and Pyers 2002), the mentalizing level

Evaluating the reflexive practices in a learning experience

is investigated by the integrated analysis of language’s semantic and syntactic dimensions(i.e. the subordinate clauses, the use of contrastives). The paper, focused only on theanalysis of the mental words, even with limitations, may offer a first methodologicalindication: The presence of a mental word can be used as an indicator to select the sentences tobe analysed. As a second step, we will develop a qualitative protocol to analyse different typesof reflexive practice. Such a protocol will consider as an analysis unit a sentence containing atleast one mental word and will be structured through integrating different coding systemschosen from those most often used in the educational field.

The paper finally allows us to describe several practical implications for educationalpractices. More specifically, these concern the tools and the activities suitable for promotingreflexive thinking in young adults.

With regard to the educational tools, the proposed methodology of analysing thestudents’ mental language in written journals would allow us to evaluate whether aneducational setting sustains students’ reflexive practice. If mentalizing is not to be intendedas a stable individual feature, but rather as a ‘warmly situated’ ability, sensible to contextualand cultural factors, always changing and never completely stabilized, then it is importantto develop a methodology that can explain if and how an educational process facilitates thepractice of reflexivity.

The analysis carried out in the present contribution also gives some first suggestions asto which activities and setting conditions can activate young adults’ mentalizing ability.

Educational processes seem to prompt reflexive practices when they promote ‘thecapacity to think one’s own and others’ thoughts, emotions, assumptions and to think aboutinternal and external conditions of one’s own professional life and on them, throughlanguage, keep up a dialogue’ (Kaneklin and Olivetti Manoukian 1990, p. 155). In theanalysed case, the data showed that solely active methodologies (e.g. role playing, businessgame, case study) were not enough to sustain reflexivity. Rather, the process of designingand implementing actions, integrated with a collective and dialogical analysis of such aprocess, was what best encouraged deeper exploration of the self in conversation with thesituation and with others. Such an inquiry-based teaching approach (Gilardi and Lozza2009; Justice et al. 2007) therefore revealed itself as strategic in order to prompt a morearticulated and differentiated use of students’ reflexive thinking.

Conclusions

Reflexive practice has a primary significance in professional work and requires to be clearlydefined and evaluated. Several models of evaluating reflexivity could be detected. In thispaper, we have proposed a definition of reflexive practice, meaning the process ofattributing mental states—cognitive, emotive, volitive—to oneself and to others and ofbeing able to explain one’s own and others’ behaviour with reference to those mental states.

In light of this definition, we have presented a methodology to explore the use and thecharacteristics of the reflexive practice of participants in a training experience using theanalysis of the mental language in written texts.

According to our results, and as suggested by other authors (Boud and Walker 1998), wehave highlighted that reflexive practices include cognitive, emotional and volitivedimensions. Moreover, these different dimensions can vary over time according to differentorganizational and educational conditions. The data analysis shows in particular thatinquiry-based teaching encouraged students to consider the events in which they wereactors and authors, paying constant attention to their own and others’ mental states. Indeed,

A. Bruno et al.

it stimulated them to express and to recognize increasingly and deeply, along the way, thethoughts, emotions and beliefs used to ‘see’ situations and to set and solve problems.

References

Allen, J. G. (2006). Mentalizing in practice. In J. G. Allen & P. Fonagy (Eds.), Handbook of mentalization-based treatment (pp. 1–30). Chichester: Wiley.

Antonietti, A., Liverta Sempio, O., & Marchetti, A. (2006). Theory of mind and language in differentdevelopmental context. New York: Springer.

Astington, J. W., & Jenkins, J. M. (1999). A longitudinal study of the relation between language and theoryof mind development. Developmental Psychology, 35, 1311–1320.

Barnett, B. G., & O’Mahony, G. R. (2006). Developing a culture of reflection: Implication for schoolimprovement. Reflective Practice, 7, 499–523.

Biggs, J., Kember, D., & Leung, D. (2001). The revised two factor study process questionnaire: R-SPQ-2F.The British Journal of Educational Psychology, 71, 133–149.

Boud, D., & Walker, D. (1998). Promoting reflection in professional courses. The challenge of context.Studies in Higher Education, 23, 191–206.

Bretherton, I., & Beeghly, M. (1982). Talking about internal states: The acquisition of an explicit theory ofmind. Developmental Psychology, 18(6), 906–921.

Cunliffe, A. L. (2004). On becoming a critically reflexive practitioner. Journal of Management Education,28, 407–426.

De Villiers, J. C., & Pyers, J. E. (2002). Complements to cognition: A longitudinal study of the relationshipbetween complex syntax and false-believe understanding. Cognitive Development, 17, 1037–1060.

Dewey, J. (1933). How we think, revised edition. Boston: D.C. Heath.Fagnant, A., & Crahay, M. (2010). Theories of mind and personal epistemology: Their interrelation and

connection with the concept of metacognition. European Journal of Psychology of Education, 26(2),257–271.

Fellows, K., & Zimpher, N. L. (1988). Reflectivity and the instructional process: A definitional comparisonbetween theory and practice. In H. C. Waxman, H. J. Freiburg, J. C. Vaughn, & M. Weil (Eds.), Imagesof reflection in teacher education (pp. 1–19). Reston: Association of Teacher Educators.

Fonagy, P., & Target, M. (1996). Playing with reality: Theory of mind and the normal development ofpsychic reality. International Journal of Psycho-Analysis, 77, 217–233.

Fonagy, P., & Target, M. (1997). Attachment and reflective function: Their role in self-organization.Development and Psychopathology, 9, 679–700.

Gilardi, S., & Lozza, E. (2009). Inquiry-based learning and undergraduates’ professional identitydevelopment: Assessment of a field research-based course. Innovative Higher Education, 34, 245–256.

Gleaves, A., Walker, C., & Grey, J. (2008). Using digital and paper diaries for assessment and learningpurposes in higher education: A case of critical reflection or constrained compliance? Assessment &Evaluation in Higher Education, 33, 219–231.

Hubbs, D. L., & Brand, C. F. (2005). The paper mirror: Understanding reflective journaling. Journal ofExperiential Education, 28, 60–71.

Hughes, C., Lecce, S., & Wilson, A. (2007). “Do you know what I want?” Preschoolers’ talk aboutdesires, thoughts and feelings in their conversations with sibs and friends. Cognition & Emotion, 21,330–350.

Justice, C., Rice, J., Warry, W., Inglis, S., Miller, S., & Sammon, S. (2007). Inquiry in higher education:Reflections and directions on course design and teaching methods. Innovative Higher Education, 31,201–214.

Kaneklin, C., & Olivetti Manoukian, F. (1990). Conoscere l’organizzazione. Milan: Carocci.Kember, D., Jones, A., Loke, J., McKay, K., Sinclair, H., Tse, C., et al. (1999). Determining the level of

reflective thinking from student’s written journals using a coding scheme based on work of Mezirow.International Journal of Lifelong Education, 18, 18–30.

Kember, D., Leung, D., Jones, A., Loke, A., McKay, J., Sinclair, K., et al. (2000). Development of aquestionnaire to measure the level of reflective thinking. Assessment and Evaluation in HigherEducation, 25, 381–395.

Kember, D., McKay, J., Sinclair, K., & Wong, F. K. Y. (2008). A four-category scheme for coding andassessing the level of reflection in written work. Assessment and Evaluation in Higher Education, 33,369–379.

Evaluating the reflexive practices in a learning experience

Kreber, C. (2005). Reflection on teaching and the scholarship of teaching: focus on science instructors.Higher Education, 50, 323–359.

Lalonde, C. E., & Chandler, M. J. (2002). Children’s understanding of interpretation. New Ideas inPsychology, 20(2–3), 163–198.

Lancia, F. (2004). Strumenti per l’analisi dei testi. Milan: FrancoAngeli.Lecce, S. (2009). Theory of mind within real life context: The mental state talk. Giornale Italiano di

Psicologia, XXXVI(1), 21–50.Mamede, S., & Schmidt, H. (2004). The structure of reflective practice in medicine. Medical Education, 38,

1302–1306.Mann, K., Gordon, J., & MacLeod, A. (2007). Reflection and reflective practice in health professions

education: A systematic review. Advances in Health Sciences Education, 23, 1–27.Mayo, J. A. (2004). Using mini autobiographical narration in applied psychology to personalize course

content and improve conceptual application. Journal of Constructivist Psychology, 17, 237–246.Mezirow, J. (1991). Transformative dimension of adult learning. San Francisco: Jossey-Bass.Milligan, K., Astington, J. W., & Dack, L. A. (2007). Language and theory of mind: Meta-analysis of

the relation between language ability and false-believe understanding. Child Development, 78(2),622–646.

Moon, J. (1999). A handbook of reflective and experiential learning. San Francisco: Jossey-Bass.Reynolds, M., & Vince, R. (Eds.). (2004). Organizing reflection. Aldershot: Ashgate.Sax, P. (2006). Developing preferred stories of identity as reflective practitioners. Journal of Systemic

Therapies, 25, 59–72.Schön, D. A. (1983). The reflective practitioner. New York: Basic Books.Shatz, M., Wellman, H. M., & Silber, S. (1983). The acquisition of mental verbs: A systematic investigation

of the first reference to mental state. Cognition, 14, 301–321.Sobral, D. T. (2001). Medical students reflection-in-learning in relation to approaches to study and academic

achievement. Medical Teacher, 23, 508–513.Symons, D. K. (2004). Mental state discourse, theory of mind, and the internalization of self–other

understanding. Developmental Review, 24, 159–188.Warin, J., Maddock, M., Pell, A., & Hargreaves, L. (2006). Resolving identity dissonance through reflective

and reflexive practice in teaching. Reflective Practice, 7, 233–245.Wong, K., Kember, D., Chung, L., & Yan, L. (1995). Assessing the level of student reflection from reflective

journals. Journal of Advanced Nursing, 22, 48–57.Zanobini, M., Scopesi, A., & Cattani, A. (1998). L’uso dei termini mentali in differenti contesti interattivi. In

A. Scopesi & M. Zanobini (Eds.), Processi comunicativi e linguistici nei bambini e negli adulti:Prospettive evolutive e sociali. Milano: Franco Angeli.

Andreina Bruno, PhD, is Assistant Professor in Work and Organizational Psychology at Università deglistudi di Genova, Genoa, Italy.

Current themes of research:

Professional identity development. Reflexivity and higher education. Health and safety at work. Action-research and organizational learning.

Bruno A. and Vecchio L. (2010). Condurre la ricerca azione nell’università, in C. Kaneklin, C. Piccardo, G.Scaratti (eds.). La ricercaazione. Cambiare per conoscere nei contesti organizzativi. Raffaello CortinaEditore, Milano.

Kaneklin C., Scaratti G., Bruno A. (2006). La Formazione universitaria. Pratiche possibili. Carocci, Roma.Gilardi S., Bruno A. e Pezzotta C. (2006). Discursive Practices and Mentalization Ability in Adult at Work,

in A. Antonietti, O. Liverta Sempio e A. Marchetti (eds). Theory of Mind and Language in DifferentDevelopmental Contexts (pp. 173-191). New York: Springer.

Kaneklin C. and Bruno A. (2000). La consulenza al ruolo, in S. Soresi (ed.). Orientamenti perl’orientamento. Ricerche ed applicazioni dell’orientamento scolastico-professionale. Giunti Organizza-zioni Speciali, Firenze. andreina.bruno@unige.it

A. Bruno et al.

Laura Galuppo is PhD in Work and Organizational Psychology and Post-doc scholar at the Department ofPsychology, Università Cattolica del S. Cuore of Milan.

Current themes of research:

Organizational social sustainability. Health and safety at work. Action-research and organizational learning.

Silvia Gilardi is Assistant Professor in Work and Organizational Psychology at Università degli studi diMilano, Italy. Current themes of research are: professional identity development; reflexivity and learning inadult and higher education; action-research and organizational learning.

Gilardi, S., & Guglielmetti, C. (2011). University life of non-traditional students: Engagement styles andimpact on attrition. The Journal of Higher Education, 82(1), 33–53.

Gilardi, S., & Lozza, E. (2009). Inquiry-based learning and undergraduates’ professional identitydevelopment: Assessment of a field research-based course. Innovative Higher Education, 34(4).

Kaneklin, C., & Gilardi, S. (2007). Formare una pratica professionale competente in ambito psicologico: ilruolo dell’università. Psicologia Sociale, 3, 389–408.

Gilardi, S., Bruno, A., & Pezzotta, C. (2006). Discursive Practices and Mentalization Ability in Adult atWork. In A. Antonietti, O. Liverta Sempio & A. Marchetti (Eds.), Theory of Mind and Language inDifferent Developmental Contexts (pp. 173–191). New York: Springer. E-mail: silvia.gilardi@unimi.it.

Evaluating the reflexive practices in a learning experience